Method and system for recognizing panicked pedestrian loitering behavior based on motion trajectory

Abstract

The present invention relates to a method and system for recognizing a panicked pedestrian loitering behavior based on a motion trajectory. The method comprises the following steps: acquiring a motion image of a pedestrian, recognizing the moving center of mass of the pedestrian on the basis of the image, and constructing a motion trajectory of the pedestrian; and performing loitering behavior recognition on the motion trajectory of the pedestrian on the basis of a pedestrian loitering behavior criterion to obtain a recognition result, wherein the pedestrian loitering behavior criterion is established through the following steps: with reference to common loitering behaviors in public places, drawing loitering motion trajectories that may occur in different scenarios; and analyzing the loitering motion trajectories of pedestrians to obtain the pedestrian loitering behavior criterion. Compared with the prior art, the present invention takes into account different loitering motion patterns in different scenarios, improving the targeted recognition of a pedestrian loitering behavior.

Description

A panic pedestrian wandering behavior recognition method and system based on motion trajectory TECHNICAL FIELD

[0001] The present application relates to the field of crowd stability analysis, in particular to a panic pedestrian wandering behavior recognition method and system based on motion trajectory. BACKGROUND

[0002] With the improvement of network monitoring camera coverage in public places, the video and image data are rich, and the human behavior recognition technology of computer vision has been fully developed and widely applied. Domestic and foreign scholars have begun to study crowd evacuation and emergency events since the early 20th century. In various public places, pedestrians are easily affected by the environment and fall into a panic state, which may further lead to various panic behaviors.

[0003] There are still some deficiencies in the current research: 1) At present, the research on pedestrian wandering motion trajectory does not consider different wandering motion patterns in different scenes. For example, patent application CN 114662521 A discloses a pedestrian wandering behavior detection method and system, which includes: acquiring video compression code stream of a detection area, extracting multiple frames of image data contained in the video compression code stream; acquiring a pre-trained neural network model, inputting the image data into the neural network model frame by frame to generate a coordinate set for representing the target moving path composed of multiple target detection frame coordinates; based on the coordinate set, the back detection of the target in a single direction is performed, and if the back number exceeds the threshold, the detection result of the target with wandering behavior is output. This method does not consider different wandering motion patterns in different scenes, and lacks more targeted wandering behavior criteria based on motion trajectory. 2) Most of the existing panic crowd disturbance propagation research based on computer vision is based on crowd flow or density information, and there are few researches on disturbance propagation analysis by extracting motion trajectory, speed or acceleration and other dynamic characteristics from video image data. SUMMARY

[0004] The purpose of the present application is to overcome the defects of the prior art and provide a panic pedestrian wandering behavior recognition method and system based on motion trajectory, which realizes accurate recognition of pedestrian wandering behavior.

[0005] The purpose of the present application can be achieved by the following technical solutions:

[0006] A panic pedestrian wandering behavior recognition method based on motion trajectory, comprising the following steps:

[0007] Acquiring the motion image of the pedestrian, identifying the motion centroid of the pedestrian based on the image, and constructing the pedestrian motion trajectory;

[0008] According to the pedestrian wandering behavior criterion, the motion trajectory of the pedestrian is identified, and an identification result is obtained;

[0009] The pedestrian wandering behavior criterion is established according to the following steps:

[0010] Referring to the common wandering behavior in public places, the wandering motion trajectory that may appear in different scenes is drawn;

[0011] The wandering motion trajectory of the pedestrian is analyzed, and the pedestrian wandering behavior criterion is obtained.

[0012] Further, the pedestrian motion trajectory is constructed according to the following steps:

[0013] Based on the motion image of the pedestrian, the key nodes of the human body are obtained, different body segments are constituted, and the centroid positions of the body segments are calculated;

[0014] The centroid positions of the body segments of the pedestrian are used to calculate the total centroid position of the human body;

[0015] After fitting the coordinates of the pedestrian centroid in time sequence, the motion trajectory of the pedestrian centroid in a period of time can be obtained.

[0016] Further, the key nodes include: head top, left ear, right ear, left eye, right eye, nose, left corner of mouth, right corner of mouth, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

[0017] Further, the calculation formula of the body segment centroid position is as follows: x k =x p l p +x d l d y k =y p l p +y d l d z k =z p l p +z d l d

[0018] In the formula, (x k , y k , z k ) is the segment centroid coordinate, (x p , y p , z p ) is the coordinate of the proximal end of the segment, (x d , y d , z d ) is the coordinate of the distal end of the segment, and lp is the percentage of the segment length from the proximal end to the distal end, l d is the percentage of the segment length from the distal end to the proximal end.

[0019] Further, the formula for calculating the total centroid position of the human body is as follows:

[0020] In the formula, (x c , y c , z c ) is the coordinate of the centroid of the human body; (x k , y k , z k ) is the coordinate of the kth segment; m k is the mass of the kth segment; and M is the total mass of the body segments.

[0021] Further, the wandering motion trajectory includes multiple elliptical motion trajectories, spiral motion trajectories, snake motion trajectories, and Brownian motion trajectories.

[0022] Further, the criterion for the wandering behavior of the pedestrian is as follows:

[0023] In the formula, Oval.p. is the criterion for the elliptical motion trajectory, Spiral.p. is the criterion for the spiral motion trajectory, Snake.p. is the criterion for the snake motion trajectory, Brown.p. is the criterion for the Brownian motion trajectory, N count is the threshold value for the number of inflection points in the pedestrian motion trajectory.

[0024] Further, when the pedestrian motion appears to be reciprocating, the inflection point appears in the pedestrian motion trajectory, and whether the pedestrian behavior is a wandering behavior is determined according to whether the number of occurrences of the inflection point exceeds a threshold value.

[0025] Further, the inflection point is a change in the direction angle of the trajectory point of the pedestrian motion trajectory that exceeds 90 degrees, and the formula is as follows:

[0026] In the formula, θ is the direction angle of the trajectory point, P t-1 is the pedestrian trajectory coordinate point at the t-1th moment, P t is the pedestrian trajectory coordinate point at the tth moment, and P t+1 is the pedestrian trajectory coordinate point at the t+1th moment.

[0027] According to one aspect of the present application, a panic pedestrian wandering behavior recognition system based on a motion trajectory is provided, which comprises:

[0028] The pedestrian motion trajectory construction module is configured to acquire a motion image of the pedestrian, identify a motion centroid of the pedestrian based on the image, and construct a motion trajectory of the pedestrian.

[0029] The pedestrian loitering behavior identification module is configured to identify a loitering behavior of the pedestrian according to a pedestrian loitering behavior criterion, and obtain an identification result.

[0030] The pedestrian loitering behavior criterion is established by the following steps:

[0031] The pedestrian loitering behavior criterion is established by the following steps:

[0032] The pedestrian loitering behavior criterion is established by the following steps:

[0033] Compared with the prior art, the present application has the following beneficial effects:

[0034] 1. The present application analyzes the loitering motion trajectory characteristics according to different loitering motion modes of the pedestrian in different environments, and proposes a targeted loitering behavior identification criterion for each motion mode, thereby improving the targeting of the pedestrian loitering behavior identification.

[0035] 2. The present application extracts the motion centroid and motion trajectory information of the object from the video or image information by using the computer vision technology, and uses the obtained data for loitering behavior identification and crowd stability analysis research, thereby improving the accuracy of the pedestrian loitering behavior identification. BRIEF DESCRIPTION OF DRAWINGS

[0036] Fig. 1 is a flowchart of a panic pedestrian loitering behavior identification method based on a motion trajectory according to the present application;

[0037] Fig. 2 is a schematic diagram of possible loitering trajectories in different scenes, wherein (2a) is a multiple-ellipse motion trajectory, (2b) is a spiral motion trajectory, (2c) is a snake motion trajectory, and (2d) is a Brownian motion trajectory;

[0038] Fig. 3 is a schematic diagram of a turning point formed in a pedestrian motion trajectory;

[0039] Fig. 4 is a schematic diagram of an experimental scene in an embodiment, wherein (4a) is a physical schematic diagram of an experimental scene, and (4b) is a virtual schematic diagram of an experimental scene;

[0040] Fig. 5 is a schematic diagram of a motion trajectory of a pedestrian centroid in a loitering behavior. DETAILED DESCRIPTION

[0041] The application will be described in detail below in conjunction with the drawings and specific embodiments. The embodiments are implemented on the premise of the technical solutions of the application, and detailed implementation modes and specific operation processes are given, but the protection scope of the application is not limited to the following embodiments.

[0042] Embodiment 1

[0043] The embodiment provides a panic pedestrian wandering behavior recognition method based on a motion trajectory, as shown in FIG. 1, including the following steps.

[0044] S1, acquiring a motion image of a pedestrian, recognizing a motion centroid of the pedestrian based on the image, and constructing a pedestrian motion trajectory.

[0045] In the embodiment, a pedestrian falling video is shot and observed in a first-floor hall of a Zhixin Hall office building of Tongji University as an experimental scene, as shown in FIG. 4 (4a). The camera position is on the right side of the door, the height is 1.6 meters, and the position and site information are as shown in FIG. 4 (4b).

[0046] The construction steps of the pedestrian motion trajectory are as follows:

[0047] Based on the motion image of the pedestrian, a human body key node is acquired, different body segments are constituted, and the centroid positions of the body segments are calculated; in the embodiment, 21 key nodes are adopted to constitute 13 segments, and the nodes are as follows: head top, left ear, right ear, left eye, right eye, nose, left mouth corner, right mouth corner, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0048] A calculation formula of the centroid position of a body segment is defined as shown in the following formula: x k =x p l p +x d l d y k =y p l p +y d l d z k =z p l p +z d l d

[0049] In the formula, (x k , y k , z k ) is the centroid coordinate of the segment, (x p , y p , z p ) is the coordinate of the proximal end of the segment, (x d , y d , zd is the coordinate of the segment distal end, l p is the percentage of the segment length from the proximal end to the distal end, l d is the percentage of the segment length from the proximal end to the distal end.

[0050] The total centroid position of the pedestrian is calculated using the centroid positions of each body segment, and the calculation formula is as follows:

[0051] In the formula, (x c , y c , z c ) are the coordinates of the body centroid; (x k , y k , z k ) are the coordinates of the kth segment; m k is the mass of the kth segment; and M is the total mass of the body segments.

[0052] After fitting the coordinates of the pedestrian centroid in time sequence, the motion trajectory of the pedestrian centroid in a period of time can be obtained. In this embodiment, video information is collected according to the schematic diagram of FIG. 4, the video collection frequency f = 29.97 fps (Frames Per Second), the video single-frame resolution is 1920*1080 pixels, the video frame image is intercepted using matlab, and the interception frequency FrameRate = 30. The upper left corner position of the video frame image is defined as the coordinate origin, the horizontal direction is the x-axis, the horizontal coordinate value represents 1920 pixels, and the vertical direction is the y-axis, the vertical coordinate value represents 1080 pixels; then the key points of the pedestrian body are identified, and the centroid trajectory of the pedestrian loitering behavior is obtained as shown in FIG. 5, the pedestrian falling process lasts for 10 seconds, and a total of 30 frames of images are extracted.

[0053] S2, identifying the loitering behavior of the pedestrian according to the pedestrian loitering behavior criterion to obtain an identification result.

[0054] The establishment steps of the pedestrian loitering behavior criterion are as follows:

[0055] Referring to the common loitering behavior in public places, the loitering dynamic trajectory graph that can appear in different scenes is drawn as shown in FIG. 2, including four kinds of loitering trajectories, multiple elliptical motion trajectories as shown in (2a) of FIG. 2, spiral motion trajectories as shown in (2b) of FIG. 2, snake-shaped motion trajectories as shown in (2c) of FIG. 2, and Brown motion trajectories as shown in (2d) of FIG. 2

[0056] The wandering process of pedestrians in public places is often affected by scene factors, and there are many obstacles or small pedestrian activity ranges in the scene, so the wandering range of the pedestrian is small, and the trajectory will be reciprocating. The trajectory of the wandering motion of the pedestrian is analyzed to determine whether the pedestrian behavior is wandering. When the target motion appears reciprocating, a turning point will be formed, as shown by the black solid point in FIG. 3. If the range of the direction angle change of the trajectory points of the continuous frames of the pedestrian motion trajectory in the wandering process exceeds 90 degrees, the point is determined to be a turning point, and the calculation formula is as follows:

[0057] In the formula, θ is the direction angle of the trajectory point, P t-1 is the pedestrian trajectory coordinate point at the t-1 time, P t is the pedestrian trajectory coordinate point at the t time, and P t+1 is the pedestrian trajectory coordinate point at the t+1 time.

[0058] By counting the number of turning points in the pedestrian motion trajectory, if the number exceeds the threshold N count , it can be determined that the motion trajectory is a wandering behavior. The wandering behavior criteria of the four kinds of wandering trajectories are shown in Case 1 to Case 4.

[0059] Case 1 (elliptical motion trajectory):

[0060] Case 2 (spiral motion trajectory):

[0061] Case 3 (snake motion trajectory):

[0062] Case 4 (Brown motion trajectory):

[0063] The turning direction can be directly observed through the centroid trajectory. According to the wandering trajectory criterion, the trajectory turning frame time satisfies cosθ<0. And because there are two trajectory turning times, i.e., N count ≥2, it can be obtained that the pedestrian wandering behavior occurs in the video case of the embodiment.

[0064] In order to facilitate the visualization of the motion trajectory recognition model, a character η is used to represent the recognition result of the pedestrian motion trajectory. When the pedestrian motion trajectory is judged to be a wandering trajectory, η=1; otherwise, η=0. In summary, the recognition result of the embodiment is η=1, i.e., there is a wandering behavior.

[0065] Embodiment 2

[0066] The embodiment provides a panic pedestrian wandering behavior recognition system based on a motion trajectory, which comprises:

[0067] The pedestrian motion trajectory construction module is configured to acquire a motion image of the pedestrian, identify a motion center of the pedestrian based on the image, and construct a motion trajectory of the pedestrian.

[0068] The pedestrian loitering behavior identification module is configured to identify the motion trajectory of the pedestrian according to a pedestrian loitering behavior criterion, and obtain an identification result.

[0069] The pedestrian loitering behavior criterion is established by the following steps:

[0070] The pedestrian loitering behavior criterion is established by the following steps:

[0071] The pedestrian loitering behavior criterion is established by the following steps:

[0072] The rest is the same as in Embodiment 1.

[0073] The above detailed description of the preferred embodiments of the present application. It should be understood that those skilled in the art without the need for creative labor can be made according to the concept of the present application, many modifications and changes. Therefore, any person skilled in the art based on the prior art on the basis of the logical analysis, reasoning or limited experiments can be obtained by the concept of the present application, the technical solutions determined by the claims should be within the scope of protection.

Claims

1.A method for recognizing a panic pedestrian loitering behavior based on a motion trajectory, characterized in that, The method comprises the following steps: Obtaining a moving image of a pedestrian, identifying a motion center of the pedestrian based on the image, and constructing a motion trajectory of the pedestrian; Identifying a loitering behavior of the pedestrian based on a loitering behavior criterion of the pedestrian, and obtaining an identification result; The establishment of the loitering behavior criterion of the pedestrian comprises the following steps: Referring to common loitering behaviors in public places, and drawing possible loitering motion trajectories in different scenes; Analyzing the loitering motion trajectories of the pedestrian, and obtaining the loitering behavior criterion of the pedestrian. 2.The method of claim 1, wherein, The construction of the motion trajectory of the pedestrian comprises the following steps: Obtaining key nodes of the pedestrian based on the moving image of the pedestrian, forming different body segments, and calculating the center of mass positions of the body segments; Calculating the total center of mass position of the pedestrian by using the center of mass positions of the body segments of the pedestrian; Fitting the coordinates of the center of mass of the pedestrian according to time sequence, and obtaining a motion trajectory of the center of mass of the pedestrian within a period of time. 3.The method of claim 2, wherein, The key nodes comprise a head top, a left ear, a right ear, a left eye, a right eye, a nose, a left corner of the mouth, a right corner of the mouth, a neck, a left shoulder, a right shoulder, a left elbow, a right elbow, a left wrist, a right wrist, a left hip, a right hip, a left knee, a right knee, a left ankle, and a right ankle. 4.The method of claim 2, wherein, The formula for calculating the body segment centroid position is as follows: x k = x p l p + x d l d y k = y p l p + y d l d z k = z p l p + z d l d where (x k , y k , z k ) are the segment centroid coordinates, (x p , y p , z p ) are the coordinates of the proximal end of the segment, (x d , y d , z d ) are the coordinates of the distal end of the segment, l p is the percentage of the segment length from the proximal end to the midpoint, and l d is the percentage of the segment length from the distal end to the midpoint. 5.The method of claim 2, wherein, The formula for calculating the total center of mass position of the human body is as follows: where (x c , y c , z c ) are the coordinates of the center of mass of the body; (x k , y k , z k ) are the coordinates of the kth segment; m k is the mass of the kth segment; and M is the total mass of the body segments. 6.The method of claim 1, wherein, The loitering motion trajectory comprises multiple elliptical motion trajectories, spiral motion trajectories, snake-shaped motion trajectories, and Brownian motion trajectories. 7.The method of claim 6, wherein, The pedestrian loitering behavior criterion is as follows: In the formula, Oval.p. is an elliptical motion trajectory criterion, Spiral.p. is a spiral motion trajectory criterion, Snake.p. is a snake motion trajectory criterion, Brown.p. is a Brownian motion trajectory criterion, N count is a threshold value of the number of inflection points in the motion trajectory of the pedestrian. 8.The method of claim 1, wherein, When the motion of the pedestrian appears to be reciprocating, an inflection point appears in the motion trajectory of the pedestrian, and whether the behavior of the pedestrian is a loitering behavior is determined according to whether the number of occurrences of the inflection point exceeds a threshold value. 9.The method of claim 8, wherein, The inflection point is a change in the direction angle of the trajectory point of the continuous frames in the pedestrian motion trajectory exceeding 90 degrees, and the calculation formula is as follows: where θ is the direction angle of the trajectory point, P t-1 is the pedestrian trajectory coordinate point at the t-1 time, P t is the pedestrian trajectory coordinate point at the t time, P t+1 is the pedestrian trajectory coordinate point at the t+1 time. 10.A system for recognizing panic pedestrian loitering behavior based on a motion trajectory, the system comprising: The method comprises the following steps: A pedestrian motion trajectory construction module is configured to obtain a moving image of a pedestrian, identify a motion center of the pedestrian based on the image, and construct a motion trajectory of the pedestrian; A pedestrian loitering behavior identification module is configured to identify a loitering behavior of the pedestrian based on a loitering behavior criterion of the pedestrian, and obtain an identification result; The establishment of the loitering behavior criterion of the pedestrian comprises the following steps: Referring to common loitering behaviors in public places, and drawing possible loitering motion trajectories in different scenes; Analyzing the loitering motion trajectories of the pedestrian, and obtaining the loitering behavior criterion of the pedestrian.

Images