A method for detecting abnormal events in crowd flow of surveillance video in crowded scenes

Abstract

The present invention relates to the field of video surveillance and analysis technology, and specifically to a method for detecting abnormal events in crowd flow monitoring videos in crowded scenes, so as to detect whether there are safety hazards such as excessive flow, slow walking, and inability to pass all the people when the crowd flows across the road, so as to ensure the safety of the crowd flowing across the road at intersections with large pedestrian flow. A method for detecting abnormal events in crowd flow monitoring videos in crowded scenes is applied to detect abnormal events when pedestrians cross the road at intersections with large pedestrian flow, including real-time collection of monitoring videos of pedestrians crossing the road by a first camera at the intersection, analysis of each video clip, and estimation of whether the number of pedestrians crossing the road within the existing green light time is greater than an upper limit value. If so, further analysis of the actual average speed of all pedestrians crossing the road, if the actual average speed is less than the set average speed, it is judged that the pedestrian crossing the road is an abnormal event, and a signal to extend the green light time is sent to the traffic light at the intersection.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance analysis, and in particular to a method for detecting abnormal events in crowd flow of surveillance videos in crowded scenes. Background Art

[0002] With the development of urbanization, the level of urbanization has significantly improved, and the urban population has become more and more dense. Therefore, crowds are crowded in some public places, such as large shopping malls, stadiums, exhibition halls, and urban road intersections. Once the crowd is crowded, it is easy to cause stampedes, thus threatening people's lives and property.

[0003] At present, at some intersections with heavy traffic, manual assistance is usually used to guide the crowd to cross the road in an orderly manner. Manual assistance at intersections has problems such as high labor costs, low efficiency, and randomness, and it cannot fundamentally solve the problem. At present, there is a lack of effective methods to automatically monitor the state of crowd flow and take effective measures to ensure that the crowd flows safely across the road. At intersections with heavy traffic, there is no method to predict abnormal events in the crowd flow and prevent stampedes. Summary of the invention

[0004] The present invention aims to provide a method for detecting abnormal events in crowd flow monitoring videos in crowded scenes, so as to detect whether there are safety hazards such as excessive flow, slow walking, and inability of all people to pass through when the crowd crosses the road, so as to ensure the safety of the crowd crossing the road at intersections with large pedestrian flow.

[0005] A method for detecting abnormal crowd flow events in surveillance videos in crowded scenes is applied to detect abnormal events when pedestrians cross the road at intersections with large pedestrian flow. The method includes collecting surveillance videos of pedestrians crossing the road in real time through a first camera at the intersection, analyzing each video clip, estimating whether the number of pedestrians crossing the road within the existing green light time is greater than an upper limit value, and if so, further analyzing the actual average speed of all pedestrians crossing the road. If the actual average speed is less than the set average speed, it is determined that the pedestrian crossing is an abnormal event, and a signal to extend the green light time is sent to the traffic light at the intersection.

[0006] The principle and beneficial effect of the present invention are as follows: by collecting monitoring videos of pedestrians crossing the road in real time through the first camera at the intersection, the state of the crowd flow crossing the road at the intersection can be understood in real time; in the present invention, each video clip is analyzed to estimate whether the number of pedestrians crossing the road within the existing green light time is greater than the upper limit value, the upper limit value mentioned here is set according to different intersections, according to the distance between the starting point and the end point of the intersection, the crowd flow walking in one direction is in the form of a matrix, starting from the first row (at this time, the green light allowing pedestrians to cross the road has just been lit) to cross the road until the green light allowing pedestrians to pass in the last row is just turned off, the width of the matrix is ​​the width of the crosswalk, the upper limit value of the crowd flow allowed to pass at the intersection calculated under the above situation, the calculation method for the opposite direction is the same as above; if the estimated existing green light time is less than If the number of pedestrians crossing the road within a certain period of time is greater than the upper limit, it means that the crowd flow at the intersection is very large and has exceeded the upper limit. Although it has exceeded the upper limit, if the crowd flows at a faster speed, they may still pass through the intersection. Therefore, it is necessary to further analyze the actual average speed of all pedestrians crossing the road. If the actual average speed is less than the set average speed, the set average speed is the speed at which pedestrians cross the road under normal circumstances. If the actual average speed is less than the set average speed, it means that the crowd still has a problem of walking slowly. Within the existing green light time allowing pedestrians to pass, all the people cannot safely pass through the intersection. Therefore, it is judged that this pedestrian crossing is an abnormal event. In this case, in order to ensure the safety of pedestrians crossing the road, a signal to extend the green light time is sent to the traffic light at the intersection.

[0007] Furthermore, the method also includes analyzing each video clip to determine whether the number of pedestrians crossing the road within a specified time is greater than a reference value. If so, the analysis frequency of the video frames is increased, and on the basis that the existing green light is on and the road is passable at a set average walking speed, it is determined whether there is a continuous gathering of pedestrians crossing the road. If so, the number of pedestrians who will pass within the existing green light time is estimated to determine whether the estimated number of pedestrians is greater than an upper limit.

[0008] Principle and beneficial effects: Although the flow of people at the intersection is large, usually not all people who want to cross the road will wait at the intersection to cross the road at once. Therefore, this method uses an estimated way to count the number of pedestrians in the crowd flow who want to cross the road. Specifically, each video clip is analyzed to determine whether the number of pedestrians crossing the road within a specified time is greater than a reference value. The specified time mentioned in the present invention is a time within a set time period. For example, within 2 minutes before the green light allowing pedestrians to pass is turned on, the number of pedestrians who want to cross the road has exceeded the reference value, indicating that the flow of people crossing the road this time is large, and it is necessary to strengthen monitoring and analysis and to estimate the number of pedestrians who want to cross the road. Determine whether the number of pedestrians crossing the road within a specified time is greater than a reference value. If so, increase the analysis frequency of the video frames. On the basis that the existing green light is on and pedestrians can pass at the set average speed, that is, pedestrians who arrive within the green light time but cannot pass through the intersection at the set average speed are not included in the statistics. Determine whether there is a continuous gathering of pedestrians crossing the road. If so, estimate the number of pedestrians who will pass within the existing green light time to determine whether the estimated number of pedestrians is greater than the upper limit.

[0009] Furthermore, it also includes analyzing the gathering speed of the continuously gathering pedestrians. When the gathering speed of the pedestrians is greater than the set speed, it is judged that the pedestrian crossing is an abnormal event, and a signal to extend the green light time is sent to the traffic light at the intersection. If the number of pedestrians crossing the road within the specified time is greater than the reference value, and the gathering speed of the pedestrians is too fast in the case of continuous gathering, then the number of pedestrians about to cross the road will increase rapidly in a short period of time, and it is very likely to exceed the set upper limit value. Therefore, it is judged that the pedestrian crossing is an abnormal event, and a signal to extend the green light time is sent to the traffic light at the intersection.

[0010] Furthermore, each video clip is analyzed to see whether there is a collision between the crowds going in opposite directions. If so, it is determined that the pedestrian crossing is an abnormal event, and a first warning message is issued to the pedestrians at the intersection.

[0011] Due to the large flow of people crossing the road, people walking in opposite directions will inevitably use other roads, collide with each other in order to pass through the intersection quickly, which will lead to the collision of people flow. Once the collision occurs, it will be very dangerous. It will be judged as an abnormal event for pedestrians crossing the road. The first warning information will be sent to pedestrians at the intersection in time to remind pedestrians of possible dangers and enable them to walk in an orderly manner. Pedestrians can also be informed that the green light time has been extended, so there is no need to panic and to prevent trampling incidents.

[0012] Furthermore, each video clip is analyzed to see whether there is any irregular surge in the crowd flow. If so, it is determined that the pedestrian crossing is an abnormal event, and a second warning message is issued to pedestrians at the intersection.

[0013] Since the flow of people crossing the road is large, whether it is the same direction or the opposite direction, there will inevitably be crowding, queue cutting and other behaviors, which will lead to irregular surge of the crowd and easy to cause trampling incidents. Therefore, when there is irregular surge of the crowd, it is judged that the pedestrian crossing the road is an abnormal event, and a second warning message is issued to pedestrians at the intersection to remind pedestrians that irregular surge and trampling may occur, so that pedestrians can walk in an orderly manner, and pedestrians can be informed that the green light time has been extended.

[0014] Furthermore, each video clip is analyzed to determine whether there are pedestrians in the crowd whose walking speed is lower than the set minimum speed. If so, a signal is sent to the traffic light at the intersection to extend the green light time.

[0015] If there are pedestrians in the crowd whose walking speed is lower than the set minimum speed, it means that there are special groups such as the elderly, the blind or children who have difficulty walking in the crowd. Therefore, in order to ensure safety, a signal is sent to the traffic light at the intersection to extend the green light time.

[0016] Furthermore, it also includes collecting traffic conditions through a second camera at the intersection. If the traffic volume is less than a set lower limit, and the number of pedestrians crossing the road within the estimated existing green light time is greater than an upper limit, the second camera at the intersection is also used to collect information on special groups of people, and estimate whether special groups of people can pass based on the set walking speed of special groups of people in crowded scenes and the existing green light time. If so, a prompt message is issued, if not, a waiting message is issued.

[0017] The traffic situation is collected by the second camera at the intersection. If the traffic volume is less than the set lower limit, it means that there are fewer or no vehicles at the intersection at this time. The second camera does not need to be used frequently to collect traffic information. Instead, it can also be used to collect information on special groups when the number of pedestrians crossing the road is estimated to be greater than the upper limit within the existing green light time. The special group of people is estimated to pass according to the set walking speed of the special group of people in the crowded scene and the existing green light time. If so, a prompt message is issued, if not, a waiting message is issued, that is, providing an accurate reference for whether the special group of people can pass through the intersection smoothly. First, it increases the safety of special groups of people passing through the intersection, and second, it avoids too many special groups of people blindly joining the crowd flow, resulting in the crowd flow being unable to pass through the intersection smoothly within the green light time allowed, resulting in extended green light time, thereby reducing the adjustment frequency of traffic lights and minimizing the impact on normal traffic.

[0018] Furthermore, in order to strengthen the control of intersections and improve management efficiency and safety, statistics are collected on abnormal pedestrian crossings within a specified time period, and management demand information is issued for intersections where the number of statistical events is greater than the normal value. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1The present invention is a flow chart of an embodiment of a method for detecting abnormal events in crowd flow of monitored video in a crowded scene. DETAILED DESCRIPTION

[0020] The following is further described in detail through specific implementation methods:

[0021] As attached Figure 1 As shown, in this embodiment, a method for detecting abnormal events in crowd flow monitoring video in a crowded scene is applied to detect abnormal events when pedestrians cross the road at intersections with heavy pedestrian flow, so as to detect whether there are safety hazards such as excessive flow, slow walking, and inability of all pedestrians to pass through when the crowd crosses the road, so as to ensure that the crowd passes safely across the road at intersections with heavy pedestrian flow.

[0022] A method for detecting abnormal crowd flow events in surveillance videos in crowded scenes includes collecting surveillance videos of pedestrians crossing the road in real time through a first camera at an intersection, analyzing each video clip, specifically extracting video clips from the surveillance video, dividing the video clips into several continuous frames, and analyzing each frame.

[0023] The analysis includes: estimating whether the number of pedestrians crossing the road within the existing green light time is greater than the upper limit value, and the specific estimation method is: judging whether the number of pedestrians crossing the road within the specified time is greater than the reference value, if so, increasing the analysis frequency of the video frame, judging whether there is a continuous gathering of pedestrians crossing the road on the basis that the existing green light is on and the pedestrians can pass at the set average speed, if so, estimating the number of pedestrians who will pass within the existing green light time, and estimating whether the number of pedestrians is greater than the upper limit value. The upper limit value mentioned in this embodiment is set according to different intersections. According to the distance between the starting point and the end point of the intersection, the crowd flow walking in one direction is in the form of a matrix, starting from the first row (at this time, the green light allowing pedestrians to cross the road has just been lit) to cross the road until the green light allowing pedestrians to pass in the last row is just turned off. The width of the matrix is ​​the width of the crosswalk. The upper limit value of the intersection allowed for the crowd flow to pass calculated under the above situation is the same as above in the opposite direction; the reference value described in this embodiment can be half of the upper limit value, or one-third of the upper limit value, and this embodiment is preferably half of the upper limit value.

[0024] By extracting the number of particles in each frame of video, the number of pedestrians contained in each frame of video is calculated. This is a prior art and will not be described in detail here. In this way, the number of pedestrians crossing the road within a specified time can be calculated. In the past, by comparing the previous and next frames of video, it was possible to determine whether there was a continuous gathering of pedestrians crossing the road, and based on the gathering speed and the time period when the existing green light is on and the pedestrians can walk at the set average speed, it can be estimated whether the number of pedestrians crossing the road is greater than the upper limit.

[0025] In this embodiment, if the number of pedestrians crossing the road is greater than the upper limit, the actual average speed of all pedestrians crossing the road is further analyzed. If the actual average speed is less than the set average speed, it is determined that the pedestrian crossing the road is an abnormal event, and a signal to extend the green light time is sent to the traffic light at the intersection. The actual average speed of all pedestrians can be calculated based on the moving speed of the same particle in different consecutive frames of video.

[0026] In this embodiment, on the basis that the estimated number of pedestrians crossing the road is greater than the upper limit value and the actual average speed of all pedestrians crossing the road is less than the set average speed, it also includes analyzing the gathering speed of pedestrians who continue to gather. When the pedestrian gathering speed is greater than the set speed, it is judged that the pedestrian crossing this time is an abnormal event, and a signal to extend the green light time is sent to the traffic light at the intersection.

[0027] In this embodiment, in order to manage risky intersections more efficiently and safely, it also includes counting abnormal events of pedestrians crossing the road within a specified time, and issuing management demand information for intersections where the number of statistical times is greater than the normal value.

[0028] In this embodiment, each video clip is also analyzed to determine whether there are pedestrians in the crowd whose walking speed is lower than the set minimum speed. If so, a signal to extend the green light time is sent to the traffic light at the intersection. This indicates that there are special groups such as elderly people or children with limited mobility in the crowd, and this measure is adopted to ensure safety.

[0029] In this embodiment, each video clip is analyzed to analyze whether there is a hedging phenomenon in the crowd flow going in opposite directions. If so, it is judged that the pedestrian crossing this time is an abnormal event, and the first warning information is issued to the pedestrians at the intersection. The way to analyze whether there is a hedging phenomenon in the crowd flow is: each particle in the video clip has its own motion trajectory, and the particles can be horizontally and vertically matched between adjacent frames. The similarity between the trajectory segments is measured using a partial shape matching strategy, and candidate trajectory segment pairs are extracted from the trajectory segments. The trajectory features of the candidate trajectory segment pairs are extracted, and the matching cost is calculated. The minimum matching cost is used as the similarity of the two trajectory segments. The trajectory segments are hierarchically clustered, and the particle trajectory segments are clustered using a hierarchical clustering algorithm to extract motion features, including particle size, average speed of particle movement, and main direction of particle movement. Whether there is a hedging movement is determined based on the merged direction of the main direction of the overlapping part of the particle activity range.

[0030] Each video clip is analyzed to determine whether there is an irregular surging phenomenon in the crowd flow. If so, the pedestrian crossing is judged to be an abnormal event, and a second warning message is issued to pedestrians at the intersection. The method for analyzing whether there is a surging phenomenon in the crowd flow is as follows: based on the above-disclosed method for analyzing the hedging phenomenon, the crowd flow state is judged according to the number of particles in the hedging area and the total number of particles, thereby determining whether the crowd flow is abnormal, that is, whether there is a surging phenomenon.

[0031] This embodiment also includes collecting traffic flow conditions through the second camera at the intersection. If the traffic flow is less than the set lower limit, when the number of pedestrians crossing the road within the estimated current green light time is greater than the upper limit, the second camera at the intersection is also used to collect information on special groups, and estimate whether the special groups can pass according to the set walking speed of the special groups in the crowded scene and the current green light time. If so, a prompt message is issued, and if not, a waiting message is issued. In this way, prompts are given to whether the special groups cross the road at the appropriate time.

[0032] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A method for detecting abnormal events in crowd flow in surveillance video in crowded scenes, which is applied to detect abnormal events when pedestrians cross the road at intersections with large pedestrian flow, and is characterized by: It includes collecting surveillance videos of pedestrians crossing the road in real time through the first camera at the intersection, analyzing each video clip, estimating whether the number of pedestrians crossing the road within the existing green light time is greater than the upper limit value, and if so, further analyzing the actual average speed of all pedestrians crossing the road. If the actual average speed is less than the set average speed, it is judged that the pedestrian crossing this time is an abnormal event, and a signal is sent to the traffic light at the intersection to extend the green light time; It also includes analyzing each video clip to determine whether the number of pedestrians crossing the road within a specified time is greater than a reference value. If so, the analysis frequency of the video frame is increased, and on the basis that the existing green light is on and the pedestrians can pass at a set average speed, it is determined whether there is a continuous gathering of pedestrians crossing the road. If so, the number of pedestrians that will pass within the existing green light time is estimated to determine whether the estimated number of pedestrians is greater than an upper limit value. It also includes analyzing the gathering speed of pedestrians who are continuously gathering. When the gathering speed of pedestrians is greater than the set speed, it is determined that the pedestrian crossing is an abnormal event, and a signal is sent to the traffic light at the intersection to extend the green light time; It also includes collecting traffic conditions through the second camera at the intersection. If the traffic volume is less than the set lower limit, and the number of pedestrians crossing the road within the estimated existing green light time is greater than the upper limit, the second camera at the intersection is also used to collect information on special groups of people, and estimate whether special groups of people can pass based on the set walking speed of special groups of people in crowded scenes and the existing green light time. If so, a prompt message is issued, if not, a waiting message is issued.

2. The method for detecting abnormal events in crowd flow of surveillance video in crowded scenes according to claim 1 is characterized by: Each video clip is analyzed to see whether there is any collision between the crowds walking in opposite directions. If so, it is judged that the pedestrian crossing is an abnormal event and the first warning information is issued to the pedestrians at the intersection.

3. The method for detecting abnormal events in crowd flow of surveillance video in crowded scenes according to claim 1, characterized in that: Each video clip is analyzed to see whether there is any irregular surge in the crowd flow. If so, it is determined that the pedestrian crossing is an abnormal event and a second warning message is issued to pedestrians at the intersection.

4. The method for detecting abnormal events in crowd flow of surveillance video in crowded scenes according to claim 1, characterized in that: Each video clip is analyzed to determine whether there are pedestrians in the crowd whose walking speed is lower than the set minimum speed. If so, a signal is sent to the traffic light at the intersection to extend the green light time.

5. The method for detecting abnormal events in crowd flow of surveillance video in crowded scenes according to claim 1, characterized in that: Abnormal pedestrian crossing events are counted within a specified time period, and management demand information is issued for intersections where the number of counts is greater than the normal value.

Images