Escalator child independent taking detection method and system

Abstract

The invention discloses a method and a system for detecting independent taking of a child on an escalator, which can improve the detection accuracy and give a proper alarm, and the method comprises the following steps: step 1, setting an ROI (Region of Interest) area at an entrance of the escalator; step 2, marking pedestrians in the ROI on bbox frames, judging whether the highest points of human skeleton heads of all the bbox frames in the ROI are lower than a preset value or not, and if not, judging that a child does not take the elevator alone; if all the values are lower than the preset value, judging that the suspected child takes the elevator alone, and executing the next step; 3, behavior recognition is conducted on pedestrians in the ROI, whether the pedestrians have knee bending actions and / or hip bending actions or not is judged, and if yes, dangerous elevator taking is judged; if not, it is judged that the child takes the elevator alone.

Description

Technical Field

[0001] The present invention belongs to the technical field of elevator detection, and more specifically, relates to a method and system for detecting a child riding alone on an escalator. Background Art

[0002] Escalators are a convenient means of transportation. They can quickly transport people in shopping malls, subway stations, airports and other places, saving time and energy, improving the efficiency of public places, and bringing convenience to people's travel and shopping. However, it is very dangerous for children to ride escalators alone. Their bodies and minds are not fully mature, and their ability to recognize and respond to dangers is weak. When riding the escalator, they may touch the handrails or the edge of the steps out of curiosity and get pinched, or run and play on the escalator and fall or roll down. They will panic in the event of a malfunction or emergency.

[0003] In view of this situation, patent CN201610414372.8 discloses an escalator protection system for protecting children's safety based on an image recognition algorithm, including a control system for controlling the operation of the escalator, an image collector for collecting image information, and an alarm device for warning prompts; the image collector collects image information of the escalator entrance and exit, and transmits the collected analog signal of the current image to the control system, the control system converts the analog data into digital data, calculates the image information according to the image recognition algorithm, obtains whether the object in the current image has a tendency to enter or leave the escalator, and calculates the image information of the current entry or exit of the escalator through the image processing algorithm, determines whether there is a child entering alone, and if so, slowly stops the escalator, and warns with sound or light to remind children not to play on the escalator or enter the escalator without the company of an adult, thereby effectively protecting the life safety of children;

[0004] The escalator protection system analyzes the image information according to the image recognition algorithm, obtains the human body contour data in the image information, and judges whether the contour is an adult or a child by comparing the width and height data of the contour with the preset data, that is, judging whether it is an adult or a child based on image comparison;

[0005] However, in practice, cameras are usually installed on the ceiling of elevator entrances and exits. They are not aimed at shooting faces. There is a big difference between the side and front views, and there will be a big difference in contour detection, which may lead to misjudgment. The posture of the human body is ever-changing. During movement, such as squatting, bending over, etc., the body contour will be distorted. If an adult squats, bends over, etc., they may be easily misjudged as a child. Squatting, bending over, etc. are dangerous elevator rides. If they are mistakenly reported as a child riding the elevator alone, it will affect the accuracy of the judgment and the degree of alarm. The contour algorithm is easily affected by the environment, such as clothing and clothing obstructions. When detecting human contours, strong light may cause overexposure and blur the contour boundary. Weak light may increase image noise and affect the accurate extraction of contours. Summary of the invention

[0006] The main purpose of the present invention is to provide a method and system for detecting a child riding alone on an escalator, which can improve the accuracy of detection and make appropriate warnings.

[0007] According to a first aspect of the present invention, a method for detecting a child riding alone on an escalator is provided, comprising the following steps:

[0008] Step 1: Set an ROI area at the entrance and exit of the escalator;

[0009] Step 2: Mark the pedestrians in the ROI area with a bbox frame, and determine whether the highest point of the human skeleton head in all bbox frames in the ROI area is lower than the preset value. If not, it is determined that the child is not riding the elevator alone; if all are lower than the preset value, it is determined that the child is suspected to be riding the elevator alone, and the next step is executed;

[0010] Step 3: Perform behavior recognition on pedestrians in the ROI area to determine whether the pedestrian has bent knees and / or hips. If so, it is determined that the pedestrian is riding the elevator in a dangerous manner; if not, it is determined that the pedestrian is riding the elevator alone.

[0011] In the above-mentioned method for detecting a child riding alone on an escalator, in step 1, a real-time image of the entrance and exit of the escalator is obtained by a camera, and the ROI area is determined by combining the optical flow method and image semantic segmentation.

[0012] In the above-mentioned method for detecting a child riding alone on an escalator, in step 2, each bbox box in the ROI area has a unique ID, and the movement trajectory of each bbox box is monitored according to the ID.

[0013] In the above-mentioned escalator child riding alone detection method, after completing the movement trajectory monitoring of the bbox frame, it is then determined whether the highest point of the human skeleton head in the bbox frame is lower than a preset value;

[0014] When monitoring the moving trajectory of the bbox frame, compare in real time whether the highest point of the human skeleton head in the bbox frame is lower than the preset value. When the highest point of the human skeleton head in the bbox frame is lower than the preset value throughout the entire moving trajectory, it is determined that the highest point of the human skeleton head in the bbox frame is lower than the preset value.

[0015] According to a second aspect of the present invention, there is provided an escalator child alone riding detection system for implementing the method as described in the first aspect, comprising the following units:

[0016] Area definition unit: set an ROI area at the entrance and exit of the escalator;

[0017] Labeling unit: label the pedestrians in the ROI area with a bbox frame;

[0018] Judgment unit: judge whether the highest points of the human skeleton heads of all bbox boxes in the ROI area are lower than the preset value. If not, it is judged that the child is not riding alone; if all are lower than the preset value, it is judged that the child is suspected to be riding alone.

[0019] Behavior recognition unit: When a child is suspected of riding the elevator alone, the pedestrian in the ROI area is identified for behavior to determine whether the pedestrian has bent knees and / or hips. If so, it is determined to be dangerous; if not, it is determined to be a child riding the elevator alone.

[0020] In the above-mentioned escalator child alone riding detection system, the area definition unit includes a camera, which obtains real-time images of the entrance and exit of the escalator and determines the ROI area by combining optical flow method and image semantic segmentation.

[0021] In the above-mentioned escalator child riding alone detection system, the judgment unit includes a monitoring module, and the monitoring module assigns a unique ID to each bbox frame, and monitors the movement trajectory of each bbox frame according to the ID.

[0022] In the above-mentioned escalator child riding alone detection system, after completing the movement trajectory monitoring of the bbox frame, it is then determined whether the highest point of the human skeleton head in the bbox frame is lower than a preset value;

[0023] When monitoring the moving trajectory of the bbox frame, compare in real time whether the highest point of the human skeleton head in the bbox frame is lower than the preset value. When the highest point of the human skeleton head in the bbox frame is lower than the preset value throughout the entire moving trajectory, it is determined that the highest point of the human skeleton head in the bbox frame is lower than the preset value.

[0024] One of the above technical solutions of the present invention has at least one of the following advantages or beneficial effects:

[0025] In the present invention, it is determined in two steps whether a child is riding the elevator alone. The first step is based on height judgment, which determines whether the highest points of the human skeleton heads of all bbox boxes in the ROI area are lower than the preset value. If they are all lower than the preset value, it is determined that a child is suspected to be riding the elevator alone, and behavior recognition is performed. If they are not all lower than the preset value, it is determined that the child is not riding the elevator alone, and no alarm is issued. The second step is behavior judgment, which determines whether the pedestrian is in the state of bending over, squatting, etc. If not, it is determined that the child is riding the elevator alone, and an alarm is issued. If the pedestrian is in the state of bending over or squatting, it is determined that the child is riding the elevator alone, and an alarm is issued. In this way, the detection accuracy can be improved to ensure that the degree of the alarm issued is appropriate. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention is further described below in conjunction with the accompanying drawings and embodiments;

[0027] Figure 1 is a flow chart of a method for detecting a child riding alone on an escalator according to the present invention;

[0028] Figure 2 It is a schematic diagram of a bbox frame of a method for detecting a child riding alone on an escalator according to the present invention;

[0029] Figure 3 It is another schematic diagram of the bbox frame of the escalator child riding alone detection method of the present invention;

[0030] Figure 4 It is a schematic diagram of a detection system for a child riding alone on an escalator according to the present invention. DETAILED DESCRIPTION

[0031] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0032] It is very dangerous for children to ride escalators alone. Their bodies and minds are not fully mature, and their ability to recognize and respond to danger is weak. They may be pinched by the handrail or the edge of the steps out of curiosity, or fall or roll down while running and playing on the escalator. They may panic in the event of a malfunction or emergency. Therefore, it is necessary to check whether there are children riding alone at the entrance and exit of the escalator;

[0033] The existing escalator protection system analyzes the image information based on the image recognition algorithm, obtains the human body contour data in the image information, and judges whether the contour is an adult or a child by comparing the width and height data of the contour with the preset data. In other words, it judges whether it is an adult or a child based on image comparison; however, in practice, if an adult squats, bends over, etc., they may be easily misjudged as a child. Squatting, bending over, etc. are dangerous escalator riding. If it is mistakenly reported as a child riding the escalator alone, it will affect the accuracy of the judgment and the degree of alarm.

[0034] The present application proposes a method for detecting a child riding alone on an escalator. An ROI area is set at the entrance and exit of the escalator. When a pedestrian walks into the ROI area, a bbox frame will be automatically marked for the pedestrian, and it is determined whether the highest points of the human skeleton heads in all bbox frames in the ROI area are lower than a preset value. If they are all lower than the preset value, it is determined that a child is suspected to be riding alone, and the next step is to perform behavior recognition; if they are not all lower than the preset value, it is determined that the child is not riding alone, and no alarm is issued; during behavior recognition, it is determined whether the pedestrian is in the state of bending over, squatting, etc. If not, it is determined that the child is riding alone and an alarm is issued; if the pedestrian is in the state of bending over or squatting, it is determined that the child is riding alone and an alarm is issued. In this way, the detection accuracy can be improved to ensure that the level of the alarm issued is appropriate.

[0035] Reference Figures 1 to 3 In one embodiment of the present invention, a method for detecting a child riding alone on an escalator comprises the following steps:

[0036] Step 1: Set an ROI area at the entrance and exit of the escalator;

[0037] The ROI area is generally defined by a camera. After the camera is installed, its position is fixed. The camera will face the entrance and exit of the escalator. The camera obtains real-time images of the entrance and exit, and combines the optical flow method and image semantic segmentation to determine the ROI area to be detected.

[0038] Step 2: Mark the pedestrians in the ROI area with a bbox frame, and determine whether the highest point of the human skeleton head in all bbox frames in the ROI area is lower than the preset value. If not, it is determined that the child is not riding the elevator alone; if all are lower than the preset value, it is determined that the child is suspected to be riding the elevator alone, and the next step is executed;

[0039] After the ROI area is determined, the ROI area will be monitored in real time. When a pedestrian enters the ROI area, the pedestrian will be marked with a bbox frame. The purpose of marking the bbox frame is to facilitate the confirmation of the highest point of the pedestrian's human skeleton head. Combined with human skeleton detection, the highest point of the human skeleton head can be determined. The reason for not directly using the highest point of the top of the bbox frame to determine is that some pedestrians will raise their hands. At this time, the highest point of the top of the bbox frame is far from the height of the pedestrian, while the highest point of the human skeleton head is close to the height of the pedestrian;

[0040] At the same time, a preset value is pre-configured. When the highest point of the human skeleton head in the bbox frame is lower than the preset value, the pedestrian is judged to be of the height of a child and needs further confirmation. When the highest point of the human skeleton head in the bbox frame is higher than the preset value, the pedestrian is judged not to be of the height of a child and is not a child riding the escalator alone. At the entrance and exit of the escalator, there is a height limit prompt for children riding the escalator alone, for example, a limit of 120cm, indicating that children under 120cm in height cannot ride the escalator alone. After the staff arranges the camera and calibrates it, they will configure the preset value of 120cm in the image.

[0041] Step 3: Perform behavior recognition on pedestrians in the ROI area to determine whether the pedestrian has bent knees and / or hips. If so, it is determined that the pedestrian is riding the elevator in a dangerous manner; if not, it is determined that the pedestrian is riding the elevator alone;

[0042] Specifically, even if the highest point of the human skeleton head in the bbox frame is lower than the preset value, it cannot be guaranteed that it is a child, because some people happen to be bending over or squatting. Therefore, when it is detected that the highest points of the human skeleton heads in all bbox frames in the ROI area are lower than the preset value, further judgment is required. The human skeleton detection algorithm yolopose and the ST-GCN algorithm are used to detect the behavior of pedestrians in the ROI area to determine whether the pedestrian is bending over or squatting. Bending over is a hip flexion action, and squatting is a knee flexion action, so when judging, it is generally determined whether it is a knee flexion action or a hip flexion action. If it is bending over or squatting, it is a dangerous elevator behavior and a high-level alarm. It is necessary to issue a higher-level alarm than a child riding the elevator alone. If it is neither bending over nor squatting, then it is determined that the child is riding the elevator alone.

[0043] In this embodiment, in the ROI area, each bbox frame has an exclusive ID, and the moving trajectory of each bbox frame is monitored according to the ID. After completing the moving trajectory monitoring of the bbox frame, it is judged whether the highest point of the human skeleton head of the bbox frame is lower than the preset value. When monitoring the moving trajectory of the bbox frame, it is compared in real time whether the highest point of the human skeleton head of the bbox frame is lower than the preset value. When the highest point of the human skeleton head of the bbox frame is lower than the preset value in the entire moving trajectory, it is judged that the highest point of the human skeleton head of the bbox frame is lower than the preset value. Because pedestrians are moving in the ROI area and may have various actions, it is necessary to detect the moving trajectory of pedestrians to ensure the judgment of the highest point of the pedestrian's human skeleton head. If the highest point at a certain moment of the moving trajectory is higher than the preset value, it can also be ruled out that the child is riding the elevator alone;

[0044] When moving, pedestrians will block each other. Using the top center point of the bbox frame to determine the movement trajectory can more clearly determine the movement of pedestrians;

[0045] The mobile trajectory can be monitored using the NVDCF tracking algorithm.

[0046] Reference Figure 4 According to a second aspect of the present invention, there is provided an escalator child alone riding detection system, comprising the following units:

[0047] Region definition unit 1: Set a ROI region at the entrance and exit of the escalator;

[0048] The area definition unit 1 includes a camera, which obtains real-time images of the entrance and exit of the escalator and determines the ROI area by combining the optical flow method and image semantic segmentation;

[0049] Labeling unit 2: label the pedestrians in the ROI area with a bbox frame; after determining the ROI area, the ROI area will be monitored in real time. When a pedestrian enters the ROI area, the pedestrian will be labeled with a bbox frame. The purpose of labeling the bbox frame is to facilitate the confirmation of the highest point of the pedestrian's human skeleton head;

[0050] Judgment unit 3: judge whether the highest points of the human skeleton heads of all bbox boxes in the ROI area are lower than the preset value. If not, it is judged that the child is not riding alone in the elevator; if all are lower than the preset value, it is judged that the child is suspected to be riding alone in the elevator;

[0051] Behavior recognition unit 4: When it is suspected that a child is riding the elevator alone, the pedestrian in the ROI area is subjected to behavior recognition to determine whether the pedestrian has bent knees and / or hips. If so, it is determined to be dangerous; if not, it is determined to be a child riding the elevator alone.

[0052] In this embodiment, the judgment unit 3 includes a monitoring module, which assigns a unique ID to each bbox frame and monitors the movement trajectory of each bbox frame according to the ID;

[0053] After completing the movement trajectory monitoring of the bbox frame, determine whether the highest point of the human skeleton head in the bbox frame is lower than the preset value;

[0054] When monitoring the moving trajectory of the bbox frame, compare in real time whether the highest point of the human skeleton head in the bbox frame is lower than the preset value. When the highest point of the human skeleton head in the bbox frame is lower than the preset value throughout the entire moving trajectory, it is determined that the highest point of the human skeleton head in the bbox frame is lower than the preset value.

[0055] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for detecting a child riding alone on an escalator, characterized in that: The following steps are involved: Step 1: Set an ROI area at the entrance and exit of the escalator; Step 2: Mark the pedestrians in the ROI area with a bbox frame, and determine whether the highest point of the human skeleton head in all bbox frames in the ROI area is lower than the preset value. If not, it is determined that the child is not riding the elevator alone; if all are lower than the preset value, it is determined that the child is suspected to be riding the elevator alone, and the next step is executed; Step 3: Perform behavior recognition on pedestrians in the ROI area to determine whether the pedestrian has bent knees and / or hips. If so, it is determined that the pedestrian is riding the elevator in a dangerous manner; if not, it is determined that the pedestrian is riding the elevator alone.

2. The method for detecting a child riding alone on an escalator according to claim 1, characterized in that: In step 1, a real-time image of the entrance and exit of the escalator is obtained through a camera, and the ROI area is determined by combining the optical flow method and image semantic segmentation.

3. The method for detecting a child riding alone on an escalator according to claim 1, characterized in that: In step 2, each bbox box in the ROI area has a unique ID, and the movement trajectory of each bbox box is monitored according to the ID.

4. The method for detecting a child riding alone on an escalator according to claim 3, characterized in that: After completing the movement trajectory monitoring of the bbox frame, determine whether the highest point of the human skeleton head in the bbox frame is lower than the preset value; When monitoring the moving trajectory of the bbox frame, compare in real time whether the highest point of the human skeleton head in the bbox frame is lower than the preset value. When the highest point of the human skeleton head in the bbox frame is lower than the preset value throughout the entire moving trajectory, it is determined that the highest point of the human skeleton head in the bbox frame is lower than the preset value.

5. An escalator child alone riding detection system for implementing the method as claimed in any one of claims 1 to 4, characterized in that: The following units are included: Area definition unit: set an ROI area at the entrance and exit of the escalator; Labeling unit: label the pedestrians in the ROI area with a bbox frame; Judgment unit: judge whether the highest points of the human skeleton heads of all bbox boxes in the ROI area are lower than the preset value. If not, it is judged that the child is not riding alone; if all are lower than the preset value, it is judged that the child is suspected to be riding alone. Behavior recognition unit: When a child is suspected of riding the elevator alone, the pedestrian in the ROI area is identified for behavior to determine whether the pedestrian has bent knees and / or hips. If so, it is determined to be dangerous; if not, it is determined to be a child riding the elevator alone.

6. The escalator child alone riding detection system according to claim 5 is characterized in that: The area definition unit includes a camera, which obtains real-time images of the entrance and exit of the escalator and determines the ROI area by combining the optical flow method and image semantic segmentation.

7. The escalator child alone riding detection system according to claim 5 is characterized in that: The judgment unit includes a monitoring module, which assigns a unique ID to each bbox frame and monitors the movement trajectory of each bbox frame according to the ID.

8. The escalator child alone riding detection system according to claim 7 is characterized in that: After completing the movement trajectory monitoring of the bbox frame, determine whether the highest point of the human skeleton head in the bbox frame is lower than the preset value; When monitoring the moving trajectory of the bbox frame, compare in real time whether the highest point of the human skeleton head in the bbox frame is lower than the preset value. When the highest point of the human skeleton head in the bbox frame is lower than the preset value throughout the entire moving trajectory, it is determined that the highest point of the human skeleton head in the bbox frame is lower than the preset value.

Images