Method and system for judging platform pedestrian overstepping based on linkage of train and crowd positions

Abstract

The present invention relates to a method and system for judging whether a platform pedestrian crosses the line based on the linkage of the train and crowd positions, which is characterized by including the following steps: Step 1, obtain a preliminary detection image according to the platform warning line and the position outside the railway track; Step 2, determine the defense area according to the preliminary detection image to form a detection image that needs to be detected and analyzed; Step 3, calibrate the position of the train head and the crowd as detection targets, and train a pre-built convolutional neural network for train head detection and crowd detection; Step 4, input the detection image into the trained convolutional neural network to obtain the central positions of the train head and the crowd in the detection image; Step 5, judge whether the platform pedestrian crosses the line according to the detected central positions of the train head and the crowd, and give a crossing-the-line warning according to the judgment result. The present invention can be widely applied to the field of judging whether a platform pedestrian crosses the line.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to a method and system for judging whether a platform pedestrian crosses a line based on the linkage of train and crowd positions. Background Art

[0002] In recent years, with the popularization of high-speed railways, more and more passengers choose to travel by train. However, with the increase in the number of passengers, especially during peak travel periods such as the Spring Festival travel rush, some passengers may not abide by the waiting rules and cross the warning line of the platform due to crowding or being in a hurry, which poses potential safety hazards to passenger safety and station protection, and the related safety requirements are increasing day by day.

[0003] With the rapid development of the field of artificial intelligence, currently, there are algorithms for detecting line crossing. There are two common algorithms:

[0004] The first one is based on multi-frame motion information and judges according to whether the warning line is blocked. First, draw a line on the platform as the warning line, and then extract the moving area. If a moving object blocks the warning line, it means that an object has crossed the line. The advantage of this method is its fast speed, but the disadvantage is that if a pedestrian stands still, even if they cross the warning line, an alarm cannot be triggered, and there will be a large number of false alarms for the situation where a train approaches and a large number of passengers get off the train (this situation cannot be alarmed).

[0005] The second one is based on single-frame motion information. With the development of deep learning, the detection of pedestrians is becoming more and more accurate. Therefore, first use a convolutional neural network to detect pedestrians, and then judge whether the pedestrians have crossed the line. The advantage of this method is that it can solve the situation of static line crossing, but when there are a large number of passengers, there is a lot of occlusion between pedestrians, and it is impossible to accurately detect each pedestrian. This method also cannot solve the false alarm problem when a train enters and passengers get off the train. Summary of the Invention

[0006] Aiming at the above problems, the purpose of the present invention is to provide a method and system for judging whether a platform pedestrian crosses a line based on the linkage of train and crowd positions. After preprocessing the platform image, the method jointly uses the train position and the crowd position to judge whether a pedestrian crosses the platform line, and can be applied to the monitoring system of railway platforms to realize the functions of automatically monitoring and alarming platform-crossing passengers.

[0007] To achieve the above purpose, the present invention adopts the following technical solutions:

[0008] In the first aspect of the present invention, a method for judging whether a platform pedestrian crosses a line based on the linkage of train and crowd positions is provided, which includes the following steps:

[0009] Step 1: Obtain a preliminary detection image according to the platform warning line and the position outside the railway track.

[0010] Step 2: Determine the defense area based on the preliminary detection image to form the image to be detected;

[0011] Step 3: Calibrate the positions of the train head and the crowd in all the training image data as the detection targets for algorithm training, and train the pre-built convolutional neural network for train head detection and crowd detection;

[0012] Step 4: Input the image to be detected into the trained convolutional neural network to obtain the central positions of the train head and the crowd in the image to be detected;

[0013] Step 5: Determine whether the platform pedestrians cross the line according to the obtained central positions of the train head and the crowd, and give a cross-line warning according to the judgment result.

[0014] Furthermore, in the above Step 1, the method for obtaining the preliminary detection image includes the following steps:

[0015] Step 1.1: Draw two straight lines according to the platform warning line and the position outside the railway track to obtain the original image;

[0016] Step 1.2: Rotate the original image counterclockwise so that the warning line in the original image is perpendicular to the bottom edge of the rotated image to obtain the preliminary detection image.

[0017] Furthermore, in the above Step 1.2, the coordinates of the points in the preliminary detection image are:

[0018] x′ = xcosθ - ysinθ

[0019] y′ = xsinθ - ycosθ

[0020] where (x, y) are the coordinates of the points in the original image, (x′, y′) are the coordinates of the points in the preliminary detection image, and θ is the rotation angle of the original image.

[0021] Furthermore, in the above Step 2, the method for obtaining the image to be detected is: Connect the upper and lower ends of the straight lines where the railway track and the warning line are located from left to right as the defense area, retain the image within the defense area, and set all the pixel values of the image outside the defense area to black to obtain the image to be detected.

[0022] Furthermore, in the above Step 4, the calculation formula for the central position of the train head or the crowd in the image to be detected is:

[0023]

[0024]

[0025] Among them, (x1, y1) and (x2, y2) are the coordinate values of the upper left corner and the lower right corner of the train head or the rectangular frame of the crowd respectively; (x c , y c ) is the coordinate value of the center position.

[0026] Furthermore, in the step 5, a method for judging whether the platform pedestrians cross the line according to the detected center positions of the train head and the crowd and giving a cross-line warning according to the judgment result includes the following steps:

[0027] If no train head is detected in the image to be detected, when the bounding box of the crowd exceeds the warning line, an alarm is triggered;

[0028] If a train head is detected in the image to be detected, and it is assumed that in the adjacent detection interval Δt, the center point position (x c , y c ) of the train head moves more than the preset fixed distance Δs, then when the bounding box of the crowd exceeds the warning line, an alarm is triggered;

[0029] If a train head is detected in the image to be detected, but in the adjacent detection interval Δt, the center point position (x c , y c ) of the train head does not move more than the preset fixed distance Δs, then there is no need to trigger an alarm.

[0030] The second aspect of the present invention is to provide a system for jointly judging whether platform pedestrians cross the line based on the positions of the train and the crowd, which includes: a preliminary detection image acquisition module for acquiring a preliminary detection image according to the platform warning line and the position outside the railway track; a detection image acquisition module for determining a defense area according to the preliminary detection image to form an image to be detected; a network training module for calibrating the positions of the train head and the crowd in all training images as the detection targets for algorithm training, and training a pre-built convolutional neural network for train head detection and crowd detection; a crowd and head detection module for obtaining the center positions of the train head and the crowd in the image to be detected according to the image to be detected and the trained convolutional neural network; a cross-line judgment and warning module for judging whether the platform pedestrians cross the line according to the obtained center positions of the train head and the crowd, and giving a cross-line warning according to the judgment result.

[0031] Furthermore, the preliminary detection image acquisition module includes an original image acquisition module for drawing two straight lines according to the platform warning line and the position outside the railway track to obtain an original image; an image rotation module for rotating the original image counterclockwise so that the warning line in the image is perpendicular to the bottom edge of the rotated image to obtain a preliminary detection image.

[0032] Further, the calculation formula for the central position of the train head or crowd in the image to be detected is as follows:

[0033]

[0034]

[0035] where (x1, y1) and (x2, y2) are the coordinate values of the upper left corner and the lower right corner of the rectangular frame of the train head or crowd respectively; (x c , y c ) are the coordinate values of the central position.

[0036] Further, the crossing line judgment and warning module includes: a first judgment module, which is used to judge whether there is a train head in the data output by the crowd and train head detection module. If not, the judgment result is sent to the second judgment module; otherwise, the judgment result is sent to the third judgment module; a second judgment module, which is used to judge whether the bounding box of the crowd exceeds the warning line after receiving the judgment result sent by the first judgment module. If it exceeds, an alarm signal is sent to the warning module; a third judgment module, which is used to judge whether the central point position of the train head exceeds a preset fixed distance Δs within a preset detection interval. If it exceeds, an alarm signal is sent to the warning module when the bounding box of the crowd exceeds the warning line; otherwise, it does not act; the warning module is used to give an alarm according to the received alarm signal.

[0037] Since the present invention adopts the above technical solutions, it has the following advantages: This solution mainly solves the detection and alarm of crossing line behavior in a complex platform environment. This method solves the problem of a large number of false alarms when the train enters the station, and the problem of being unable to effectively detect the target when pedestrians are crowded. This method has the following characteristics:

[0038] 1. Prevent false detection outside the defense area. After directly blackening the area outside the defense area in the present invention and then performing detection, the detection amount is reduced while the detection result is more accurate.

[0039] 2. Can still accurately predict in crowded situations. The present invention proposes to use the crowd as the detection target instead of individual pedestrians, and judge through the central position of the crowd. In this way, even during the peak passenger flow period, it is still possible to accurately judge whether someone crosses the line.

[0040] 3. Will not cause false detection when the train arrives at the station or is about to depart. When judging whether a pedestrian crosses the line in the present invention, by designing the interaction strategy between the crowd and the train, whether there is a train and whether the train is moving are fully considered, so as to avoid false detection when people get on and off the train.

[0041] Therefore, the present invention can be widely applied to the field of computer vision. Brief Description of the Drawings

[0042] FIG. 1(a) and FIG. 1(b) are the original image and the rotated image in an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of a defense area in an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of a crowd and train head detection network in an embodiment of the present invention. Detailed Description of the Invention

[0045] The present invention will be described in detail below with reference to the drawings and embodiments.

[0046] After analysis, it can be seen that there are two problems in the prior art. The first is that when the passenger flow is crowded, it is impossible to accurately judge whether someone crosses the line. The second is that when the train enters the station, the normal getting off of passengers will cause many false alarms. Therefore, the present invention provides a method for judging the crossing of platform pedestrians based on the linkage of the positions of the train and the crowd to solve the problem of pedestrian crossing detection in complex scenarios such as crowded passenger flow and train entering the station. Specifically, it includes the following steps:

[0047] Step 1: Obtain a preliminary detection image according to the platform warning line and the position outside the railway track.

[0048] Specifically, the method for obtaining the preliminary detection image includes the following steps:

[0049] Step 1.1: Draw two straight lines according to the platform warning line and the position outside the railway track to obtain the original image.

[0050] As shown in FIG. 1, draw a line along the straight line where the platform warning line is located, that is, L1 marked in FIG. 1(a); draw a line along the straight line outside the railway track and mark it as L2 to obtain the original image. Since the distance scales of the cameras on the platform are different, L1 and L2 appear to be slightly inclined.

[0051] Step 1.2: Rotate the original image counterclockwise so that the warning line (i.e., L1) in the original image is perpendicular to the bottom edge of the rotated image to obtain the preliminary detection image.

[0052] Assume that the original image needs to be rotated counterclockwise by θ°. Then, for the point (x, y) in the original image, its rotated coordinates are (x′, y′), and the rotation formula is:

[0053] x′ = xcosθ - ysinθ

[0054] y′ = xsinθ - ycosθ

[0055] Rotate the original image so that the warning line in the image is perpendicular to the bottom edge of the image, in order to offset the deviation caused by the angle of the platform camera. Since the detected target box is rectangular, if the warning line is not perpendicular to the bottom edge of the image, then a part of the target box will be the black background.

[0056] Step 2: Determine the defense area based on the preliminary detected image to form the image to be detected.

[0057] As Figure 2 shown, connect the upper and lower ends of the straight lines where the railway track and the warning line are located from left to right as the defense area, retain the image within the defense area, and set all the pixels of the image outside the defense area to black, then the image to be detected can be obtained.

[0058] Step 3: Calibrate the positions of the train head and the crowd in all the training image data as the detection targets for algorithm training, and train the pre-built convolutional neural network for train head detection and crowd detection.

[0059] Considering that the crowd on the platform is relatively crowded and there is serious occlusion among the crowd, it will be very difficult and inaccurate to calibrate pedestrians individually. Therefore, the present invention proposes crowd detection (Crowd Detection), and the crowd (Crowd) is defined as an area composed of a group of people who cannot be segmented in the image, which can be one person or multiple people. In addition, the present invention also calibrates the train head (TrainHead) as the rectangular area of the front face of the train. Calibrate all the images that need to be trained according to this method.

[0060] As Figure 3 shown, build a convolutional neural network (3 types of classifications, including background, train head and crowd), detect the crowd and the train head at the same time, and existing network structures such as SSD, YOLO, Faster RCNN, etc. can be used for detection. The network structure and its related loss function and training method are all well-known technologies in the art.

[0061] Step 4: Input the image to be detected into the trained convolutional neural network for train head detection and crowd detection to obtain the central positions of the train head or the crowd in the image to be detected.

[0062] If there is a crowd or a train head in the image to be detected, after passing through the convolutional neural network, the positions of the rectangular boxes corresponding to the train head or the crowd can be obtained, and the coordinate values of the upper left corner and the lower right corner of the rectangular box are respectively recorded, then the central positions of the train head or the crowd can be obtained. Among them, the calculation method of the central position of the rectangular box is as follows:

[0063]

[0064]

[0065] Among them, (x1, y1) and (x2, y2) are the coordinate values of the upper left corner and the lower right corner of the rectangular frame respectively; (x c , y c ) is the coordinate value of the center position.

[0066] Step 5: According to the obtained center position of the train head or the crowd, judge whether the platform pedestrians cross the line, and give a crossing-the-line warning according to the judgment result.

[0067] The specific judgment method is as follows:

[0068] If no train head is detected in the image to be detected, it means that it is currently in the waiting state. Once the bounding box of the crowd exceeds the warning line, it means there is a behavior of crossing the line, and an alarm is triggered;

[0069] If a train head is detected in the image to be detected, assuming that in the adjacent detection interval Δt, the center point position (x c , y c ) of the train moves more than the preset fixed distance Δs, then it means that the train is in motion. At this time, if there is a behavior of crossing the line, it is very dangerous. Therefore, once the bounding box of the crowd exceeds the warning line, an alarm is triggered;

[0070] If a train head is detected in the image to be detected, but in the adjacent detection interval Δt, the center point position (x c , y c ) of the train does not move more than the preset fixed distance Δs, it means that the train has stopped and is waiting for passengers to get on and off. At this time, it is normal to have a behavior of crossing the line and no alarm needs to be triggered.

[0071] The present invention also provides a system for jointly judging the crossing of platform pedestrians based on the positions of the train and the crowd, which includes: a preliminary detection image acquisition module for acquiring a preliminary detection image according to the platform warning line and the position outside the railway track; a detection image acquisition module for determining a defense area according to the preliminary detection image to form an image to be detected; a network training module for calibrating the positions of the train head and the crowd in all training images as the detection targets for algorithm training, and training a pre-built convolutional neural network for train head detection and crowd detection; a crowd and head detection module for obtaining the center positions of the train head and the crowd in the image to be detected according to the image to be detected and the trained convolutional neural network; a crossing-the-line judgment and warning module for judging whether the platform pedestrians cross the line according to the obtained center positions of the train head and the crowd, and giving a crossing-the-line warning according to the judgment result.

[0072] Further, the preliminary detection image acquisition module includes an original image acquisition module, which is used to draw two straight lines according to the platform warning line and the position outside the railway track to obtain the original image; an image rotation module, which is used to rotate the original image counterclockwise so that the warning line in the image is perpendicular to the bottom edge of the rotated image, and obtain the preliminary detection image.

[0073] Further, in the preliminary detection image, the coordinates of a point are:

[0074] x′ = xcosθ - ysinθ

[0075] y′ = xsinθ - ycosθ

[0076] where (x, y) are the coordinates of a point in the original image, (x′, y′) are the coordinates of a point in the preliminary detection image, and θ is the rotation angle of the original image.

[0077] Further, the calculation formula for the central position of the train head or the crowd in the image to be detected is:

[0078]

[0079]

[0080] where (x1, y1) and (x2, y2) are the coordinate values of the upper left corner and the lower right corner of the rectangular frame of the train head or the crowd respectively; (x c , y c ) are the coordinate values of the central position.

[0081] Further, the crossing line judgment and warning module includes: a first judgment module, which is used to judge whether there is a train head according to the data output by the crowd and train head detection module. If not, it sends the judgment result to the second judgment module, otherwise it sends the judgment result to the third judgment module; the second judgment module is used to judge whether the bounding box of the crowd exceeds the warning line after receiving the judgment result sent by the first judgment module. If it exceeds, it sends an alarm signal to the warning module; the third judgment module is used to judge whether the central point position of the train head exceeds a preset fixed distance Δs within a preset detection interval. If it exceeds, when the bounding box of the crowd exceeds the warning line, it sends an alarm signal to the warning module, otherwise it does not act; the warning module is used to give an alarm according to the received alarm signal.

[0082] The above embodiments are only used to illustrate the present invention. The structures, connection methods, manufacturing processes, etc. of each component can be changed. Any equivalent transformation and improvement based on the technical solution of the present invention should not be excluded from the protection scope of the present invention.

Claims

1. A method for judging whether a platform pedestrian crosses the line based on the linkage of train and crowd positions, characterized in that Including the following steps: Step 1: Obtain a preliminary detection image according to the platform warning line and the position outside the railway track; Step 2: Determine the detection area based on the preliminary detection image to form an image to be detected; Step 3: Calibrate the positions of the train heads and the crowds in all the training image data as the detection targets for algorithm training, and train the pre-built convolutional neural network for train head detection and crowd detection; Step 4: Input the image to be detected into the trained convolutional neural network to obtain the central positions of the train heads and the crowds in the image to be detected; Step 5: Judge whether the platform pedestrians cross the line according to the obtained central positions of the train heads and the crowds, and give a crossing-line warning according to the judgment result; In the said Step 1, the method for obtaining the preliminary detection image includes the following steps: Step 1.1: Draw two straight lines according to the platform warning line and the position outside the railway track to obtain an original image; Step 1.2: Rotate the original image counterclockwise so that the warning line in the original image is perpendicular to the bottom edge of the rotated image to obtain the preliminary detection image; In the said Step 2, the method for obtaining the image to be detected is: Connect the upper and lower ends of the straight lines where the railway track and the warning line are located from left to right as the detection area, retain the image within the detection area, and set all the pixel values of the image outside the detection area to black to obtain the image to be detected; In the said Step 4, the calculation formula for the central position of the train head or the crowd in the image to be detected is: Among them, are respectively the coordinate values of the upper left corner and the lower right corner of the train head or crowd rectangle; is the coordinate value of the central position; In the said Step 5, the method for judging whether the platform pedestrians cross the line according to the obtained central positions of the train heads and the crowds and giving a crossing-line warning according to the judgment result includes the following steps: If no train head is detected in the image to be detected, when the bounding box of the crowd exceeds the warning line, trigger an alarm; If the train head is detected in the image to be detected, and assuming that within the adjacent detection intervals the center point position of the train head moves more than a preset fixed distance then when the bounding box of the crowd exceeds the warning line, an alarm is triggered; If the train head is detected in the image to be detected, but within the adjacent detection intervals , the center point position of the train head does not move more than a preset fixed distance , then there is no need to trigger an alarm.

2. The method for judging platform pedestrian overstepping based on the linkage of train and crowd positions according to claim 1, characterized in that: In the said Step 1.2, the coordinates of the points in the preliminary detection image are: Among them, (x, y) are the coordinates of points in the original image, , ) are the coordinates of points in the preliminary detection image, is the rotation angle of the original image.

3. A system for judging whether a platform pedestrian crosses the line based on the linkage of train and crowd positions, which is applicable to the method described in any one of claims 1 to 2, characterized in that, Including: A preliminary detection image acquisition module, which is used to obtain a preliminary detection image according to the platform warning line and the position outside the railway track; A detection image acquisition module, which is used to determine the detection area based on the preliminary detection image to form an image to be detected; A network training module, which is used to calibrate the positions of the train heads and the crowds in all the training images as the detection targets for algorithm training, and train the pre-built convolutional neural network for train head detection and crowd detection; A crowd and train head detection module, which is used to obtain the central positions of the train heads and the crowds in the image to be detected according to the image to be detected and the trained convolutional neural network; A crossing-line judgment and warning module, which judges whether the platform pedestrians cross the line according to the obtained central positions of the train heads and the crowds, and gives a crossing-line warning according to the judgment result.

4. The system for judging the overstepping of platform pedestrians based on the linkage of train and crowd positions as claimed in claim 3, wherein, The said preliminary detection image acquisition module includes an original image acquisition module, which is used to draw two straight lines according to the platform warning line and the position outside the railway track to obtain an original image; and an image rotation module, which is used to rotate the original image counterclockwise so that the warning line in the image is perpendicular to the bottom edge of the rotated image to obtain the preliminary detection image.

5. The system for judging platform pedestrian overstepping based on the linkage of train and crowd positions according to claim 3, characterized in that, The calculation formula for the central position of the train head or the crowd in the image to be detected is: Among them, are respectively the coordinate values of the upper left corner and the lower right corner of the train head or crowd rectangular frame; is the coordinate value of the central position.

6. The system for judging platform pedestrian overstepping based on the linkage of train and crowd positions according to claim 3, wherein The said crossing-line judgment and warning module includes: The first judgment module is used to judge whether there is a train head in the data output by the crowd and train head detection module. If not, the judgment result is sent to the second judgment module; otherwise, the judgment result is sent to the third judgment module. The second judgment module is used to judge whether the bounding box of the crowd exceeds the warning line after receiving the judgment result sent by the first judgment module. If it exceeds, an alarm signal is sent to the alarm module. The third judgment module is used to judge whether the center point position of the train head exceeds a preset fixed distance within a preset detection interval If it exceeds, when the bounding box of the crowd exceeds the warning line, an alarm signal is sent to the alarm module, otherwise it does not act; the alarm module is used to give an alarm according to the received alarm signal.

Images