Method and system for recognizing wandering behavior of pedestrian based on motion trajectory

Abstract

Method for recognizing a pedestrian’s wandering behaviour (pattern), comprising: acquiring motion images of the pedestrian (Fig.4a); recognizing motion centroids of the pedestrian in the motion images; constructing the motion trajectory of the pedestrian using the motion centroids (Fig.5); and performing recognition on the motion trajectory of the centroid to obtain a wandering pattern result; wherein recognition on the motion trajectory comprises analysing the wandering motion trajectories relative to known common wandering behaviours in public places (Fig.2). The pedestrian trajectory may be obtained by: determining key nodes of the pedestrian e.g. head, ears, eyes, nose, mouth, neck, shoulders, elbows, wrists, hips, knees, and ankles; determining body segments based on the nodes, and calculating a centroid of each body segment; calculating total centroid from each body segment centroid; and obtaining a motion trajectory of the total centroids of the pedestrian. The known wandering motion trajectories may be determined from inflection point so the trajectory (Fig.3) and comprise an: oval (Fig.2a); spiral (Fig.2b); snake (Fig.2c); and Brownian motion (Fig.2d).

Description

[0001] The disclosure relates to the field of population stability analysis, and more particularly to a method and a system for recognizing a wandering behavior of a pedestrian based on a motion trajectory. BACKGROUND

[0002] With the improvement of the coverage of network surveillance cameras in public places, video and image data have become abundant, and computer vision technology for human behavior recognition has also been fully developed and widely applied. Scholars at home and abroad have been studying crowd evacuation and emergency events since the early 20th century. In various sudden accidents in the public places, pedestrians are easily influenced by environment and fall into a state of panic, which can then trigger various panic behaviors.

[0003] At present, there are still some deficiencies in related research. On one hand, at present, research on wandering motion trajectories of the pedestrians does not consider different wandering motion modes in different scenes. For example, a patent application CN114662521A discloses a pedestrian wandering behavior detection method and system, including: obtaining a video compressed code stream of a detection region, and extracting the multi-frame image data contained in the video compressed code stream; obtaining a pre-trained neural network model, and inputting the image data into the neural network model frame by frame to generate a coordinate set which is composed of multiple target detection frame coordinates and is used for representing a target moving path; and performing rollback detection of the target in a single direction based on the coordinate set, and if a number of rollback times exceeds a threshold value, outputting a detection result that the target has a wandering behavior. This method does not consider the different wandering motion modes in the different scenes, and lacks a more targeted wandering behavior criterion based on a motion trajectory. On the other hand, the existing researches in the related art on disturbance propagation of panic crowd based on computer vision are mostly based on crowd flow or density information. There is less research that extracts dynamic characteristics such as the motion trajectory, velocity, or acceleration from video image data for disturbance propagation analysis. SUMMARY

[0004] In order to overcome the deficiencies of the existing technology in the related art, the disclosure provides a method and a system for recognizing a wandering behavior of a pedestrian based on a motion trajectory thereby achieving accurate recognition of the wandering behavior of the pedestrian.

[0005] A purpose of the disclosure can be achieved through the following technical solutions.

[0006] A method for recognizing a wandering behavior of a pedestrian based on a motion trajectory, including:

[0007] acquiring motion images of the pedestrian, recognizing motion centroids of the pedestrian based on the motion images, and constructing the motion trajectory of the pedestrian according to the motion centroids; and

[0008] performing, according to pedestrian wandering behavior criteria, a wandering behavior recognition on the motion trajectory of the pedestrian to obtain a recognition result;

[0009] where construction steps of the pedestrian wandering behavior criteria include:

[0010] drawing, according to common wandering behaviors in public places, wandering motion trajectories appeared in different scenes; and

[0011] analyzing the wandering motion trajectories appeared in the different scenes to obtain the pedestrian wandering behavior criteria.

[0012] In an embodiment, the recognizing motion centroids of the pedestrian based on the motion images, and constructing the motion trajectory of the pedestrian according to the motion centroids includes:

[0013] obtaining, based on the motion images of the pedestrian, key nodes of the pedestrian to form different body segments, and calculating a centroid of each body segment of the different body segments;

[0014] calculating total centroids of the pedestrian as the motion centroids of the pedestrian by using the centroid of each body segment of the different body segments; and

[0015] fitting coordinates of the total centroids of the pedestrian in a period time according to a time sequence, to thereby obtain a motion trajectory of the total centroids of the pedestrian as the motion trajectory of the pedestrian in the period of time.

[0016] In an embodiment, the key nodes of the pedestrian include: a head, a left ear, a right ear, a left eye, a right eye, a nose, a left mouth comer, a right mouth corner, a neck, a left shoulder, a right shoulder, a left elbow, a right elbow, a left wrist, a right wrist, a left hip, a right hip, a left knee, a right knee, a left ankle and a right ankle.

[0017] In an embodiment, a calculation formula of the centroid of each body segment of the different body segments is as follows: %k + Xdld Vk yplp d” Vd^-d zk zp^p + zd^d

[0018] where (xk,yk>zk) represents coordinates of the centroid of each body segment of the different body segments, (xp,yp,zp) represents coordinates of a proximal end of each body segment of the different body segments, (%d, yd, zd~) represents coordinates of a distal end of each body segment of the different body segments, lp represents a weighting coefficient for the coordinates of the proximal end of each body segment of the different body segments, and ld represents a weighting coefficient for the coordinates of the distal end of each body segment of the different body segments.

[0019] In an embodiment, a calculation formula of each of the total centroids of the pedestrian is as follows: xc =---n--- M ’ 1 3 'I'Vl 'I f Zjk=l mkVk v_ =----------

[0020] where (xc, yc, zc) represents coordinates of each of the total centroids of the pedestrian, (xk, Vk> zk) represents coordinates of a k-th body segment of the different body segments, mk represents a mass of the k-th body segment of the different body segments, and M represents a total mass of the different body segments.

[0021] In an embodiment, the wandering motion trajectories appeared in the different scenes comprise: a multiple oval motion trajectory, a spiral motion trajectory, a snake-like motion trajectory and a Brownian motion trajectory.

[0022] In an embodiment, the pedestrian wandering behavior criteria are as follows: Oval.p. = cos 0 <0 - ^count — 2 Spiral.p. cos 0 <0 8 >Ncount - Snake, p. = t cos 0 <0 hv >2 v* count — Brown, p. = t COS 0 <0 IN , >2 v*count —

[0023] where Oval.p. represents a multiple oval motion trajectory criterion, Spiral.p. represents a spiral motion trajectory criterion, Snake.p. represents a snake motion trajectory criterion, Brown.p. represents a Brownian motion trajectory criterion, 0 represents an angle between trajectory vectors of each two frames in a wandering process, and Ncount represents a threshold of a number of inflection points in the motion trajectory of the pedestrian.

[0024] In an embodiment, when there is a back-and-froth motion of the pedestrian, there is at least one inflection point in the motion trajectory of the pedestrian; and

[0025] the performing, according to the pedestrian wandering behavior criteria, a wandering behavior recognition on the motion trajectory of the pedestrian to obtain a recognition result includes:

[0026] determining, according to a determining result corresponding to whether a number of the at least one inflection point exceeds a threshold, whether a behavior of the pedestrian is the wandering behavior.

[0027] In an embodiment, each of the at least one inflection point is configured to indicate that a change range of a direction angle of a trajectory point in continuous frames of the motion trajectory of the pedestrian exceeds 90 degrees, and a calculation formula for the direction angle of the trajectory point is as follows: n Pt-iPt ’ PfPt+i cos P = . —, \Pt-i^t| ’ |

[0028] where 0 represents the direction angle of the trajectory point, P^ represents a coordinate point of the motion trajectory of the pedestrian at a (t-l)-th time, Pt represents a coordinate point of the motion trajectory of the pedestrian at a t-th time, and Pt+1 represents a coordinate point of the motion trajectory of the pedestrian at a (t-l)-th time.

[0029] According to two aspects of the disclosure, the disclosure provides a system for recognizing a wandering behavior of a pedestrian based on a motion trajectory, including:

[0030] a pedestrian motion trajectory construction module, configured to acquire motion images of the pedestrian, recognize motion centroids of the pedestrian based on the motion images, and construct the motion trajectory of the pedestrian according to the motion centroids; and

[0031] a pedestrian wandering behavior recognition module, configured to perform a wandering behavior recognition on the motion trajectory of the pedestrian according to pedestrian wandering behavior criteria, to obtain a recognition result;

[0032] where construction steps of the pedestrian wandering behavior criteria include:

[0033] drawing, according to common wandering behaviors in public places, wandering motion trajectories appeared in different scenes; and

[0034] analyzing the wandering motion trajectories appeared in the different scenes to obtain the pedestrian wandering behavior criteria.

[0035] Compared to the related art, the disclosure has the following beneficial effects.

[0036] 1. According to different wandering motion modes that pedestrians may exhibit in various environments, characteristics of the pedestrian wandering motion trajectories are analyzed. A targeted wandering behavior recognition criterion is proposed according to each wandering motion mode, which improves pertinence of recognizing the wandering behavior of the pedestrian.

[0037] 2. Motion centroid and motion trajectory information of an object from video or image information is extracted through the computer vision technology, and the obtained data is used for recognizing the wandering behavior and analyzing crowd stability, thereby improve accuracy of recognizing the wandering behavior of the pedestrian. BRIEF DESCRIPTION OF DRAWINGS

[0038] FIG. 1 illustrates a flowchart of a method for recognizing a wandering behavior of a pedestrian based on a motion trajectory provided by the disclosure.

[0039] FIG. 2 illustrates a schematic diagram of wandering trajectories that may exist in different scenes, where (2a) represents a multiple elliptical motion trajectory, (2b) represents a spiral motion trajectory, (2c) represents a snake motion trajectory, and (2d) represents a Brownian motion trajectory.

[0040] FIG. 3 illustrates a schematic diagram of inflection points formed in the motion trajectory of the pedestrian.

[0041] FIG. 4 illustrates a schematic diagram of an experimental scene in an embodiment, where (4a) represents a physical schematic diagram of the experimental scene, and (4b) represents a virtual schematic diagram of the experimental scene.

[0042] FIG. 5 illustrates a schematic diagram of a motion trajectory of total centroids of the pedestrian in the wandering behavior. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The disclosure will be described in detail below in combination with the accompanying drawings and specific embodiments. The embodiments are implemented based on the technical scheme of the disclosure, providing a detailed implementation mode and a specific operation process, but the protection scope of the disclosure is not limited to the following embodiments.

[0044] First embodiment

[0045] The first embodiment provides a method for recognizing a wandering behavior of a pedestrian based on a motion trajectory, as shown in FIG. 1, which includes the following steps SI and S2.

[0046] In the step SI, motion images of the pedestrian are acquired, based on the motion images, motion centroids of the pedestrian are recognized, and the motion trajectory of the pedestrian is constructed according the motion centroids.

[0047] In the first embodiment, a pedestrian falling video is captured and observed in a hall on a first floor of Tongji University Information Center office building as an experimental scene, as shown in (4a) of FIG. 4. A location of a camera is on a right side of a gate, a height of the camera is 1.6 meters (m), and location and site information are shown in (4b) of FIG. 4.

[0048] Construction steps of the motion trajectory of the pedestrian are as follows.

[0049] Based on the motion images of the pedestrian, key nodes of the pedestrian are obtained to form different body segments, and a centroid of each body segment of the different body is calculated. In the first embodiment, 21 key nodes are used to form 13 body segments, and the 21 key nodes are as follows: a head, a left ear, a right ear, a left eye, a right eye, a nose, a left mouth corner, a right mouth comer, a neck, a left shoulder, a right shoulder, a left elbow, a right elbow, a left wrist, a right wrist, a left hip, a right hip, a left knee, a right knee, a left ankle and a right ankle.

[0050] A calculation formula of the centroid of each body segment of the different body segments is as follows: + xdJd + d” xdld xk = Xplp yk = yPiP

[0051] where (xk,yk, zk) represents coordinates of the centroid of each body segment of the different body segments, (xp,yp,zp) represents coordinates of a proximal end of each body segment of the different body segments, (xd, yd, zd) represents coordinates of a distal end of each body segment of the different body segments, lp represents a weighting coefficient for the coordinates of the proximal end of each body segment of the different body segments, and ld represents a weighting coefficient for the coordinates of the distal end of each body segment of the different body segments.

[0052] Total centroids of the pedestrian (i.e., motion centroids of the pedestrian) are calculated by using the centroid of each body segment of the different body segments, and a calculation formula of each of the total centroids of the pedestrian is as follows: xc =---n--- M _lkii^kyk yc^ M mhmkzk zr =--------- c M

[0053] where (xc,yc,zc) represents coordinates of each of the total centroid of the pedestrian, (Xk>yk>zk) represents coordinates of a k-th body segment of the different body segment, mk represents a mass of the k-th body segment of the different body segments, and M represents a total mass of the different body segments.

[0054] Coordinates of the total centroid of the pedestrian in a period of time are fitted according to a time sequence, thus a motion trajectory of the total centroid of the pedestrian (i.e., motion trajectory of the pedestrian) in the period of time is obtained. In the first embodiment, video information is collected according to a schematic diagram of FIG. 4. A video capture frequency is f =29.97 frames per second (fps), and a resolution of each video frame is 1920*1080 pixels. MATLAB is used to extract video frame images, and an extract frequency as FrameRate is 30. A top-left corner of each video frame image is defined as a coordinate origin, a horizontal direction is an x-axis, where a x-coordinate value represents 1920 pixels, and a vertical direction is a y-axis, where a y-coordinate value represents 1080 pixels. Then, the key nodes of the pedestrian are recognized to obtain a motion trajectory of the total centroid of the pedestrian in the wandering behavior, as shown in FIG. 5. Apedestrian fall process lasts for 10 seconds, and atotal of 30 frames of images are extracted.

[0055] In the step S2, according to pedestrian wandering behavior criteria, a wandering behavior recognition is performed on the motion trajectory of the pedestrian to obtain a recognition result.

[0056] Construction steps of the pedestrian wandering behavior criteria are as follows.

[0057] According to common wandering behaviors in public places, wandering motion trajectories appeared in different scenes are drawn, as shown in FIG. 2. The wandering motion trajectories appeared in the different scenes include four types of the wandering motion trajectories: a multiple oval motion trajectory as shown in (2a) of FIG. 2, a spiral motion trajectory as shown (2b) of FIG. 2, a snake-like motion trajectory as shown (2c) of FIG. 2 and a Brownian motion trajectory as shown (2d) of FIG. 2.

[0058] A wandering process of the pedestrian in the public places is often influenced by environmental factors, such as a large number of obstacles or a small range of pedestrian activities. As a result, a range of pedestrian wandering is relatively small, and the motion trajectory of the pedestrian tends to be repetitive. By analyzing the motion trajectory of the pedestrian, whether a pedestrian behavior belongs to the wandering behavior is determined. When the motion trajectory of the pedestrian exhibits repetition, it forms an inflection point, as shown by a black solid dot in FIG. 3. If a change range of a direction angle of a trajectory point in continuous frames of the motion trajectory of the pedestrian exceeds 90 degrees, the trajectory point is recognized as the inflection point. A calculation formula for the direction angle of the trajectory point is as follows: n Pt-iPt' PfPt+i cos d = , —, \pt-lpt\-\ptpt+l\

[0059] where 0 represents the direction angle of the trajectory point, Pt_1 represents a coordinate point of the motion trajectory of the pedestrian at a (t-l)-th time, Pt represents a coordinate point of the motion trajectory of the pedestrian at a t-th time, and Pt+1 represents a coordinate point of the motion trajectory of the pedestrian at a (t-l)-th time. Spiral, p. = jg Snake, p. = Brown, p. =

[0060] By counting a number of the inflection point in the motion trajectory of the pedestrian, if the number exceeds a threshold Ncount, the motion trajectory of the pedestrian can be determined as the wandering behavior. The pedestrian wandering criteria of the four types of the wandering motion trajectories are shown in Cases 1 to 4.

[0061] Case 1, the multiple oval motion trajectory is expressed as follows: , ( cos 0 <0 Oval.p.= 4 2

[0062] Case 2, the spiral motion trajectory is expressed as follows: cos 6 <0 > N >2 — ''count —

[0063] Case 3, the snake-like motion trajectory is expressed as follows: cos 6 <0 N , >2 1 'count —

[0064] Case 4, the Brownian motion trajectory is expressed as follows: cos 6 <0 N , >2 ' 'count —

[0065] A trajectory direction change can be seen visually through the motion trajectory of the total centroid of the pedestrian. According to the pedestrian wandering trajectory criteria, a frame moment of the trajectory direction change can satisfy cos 0 <0. And since there are two frame moments of the trajectory direction change, that is, Ncount >2, it can be concluded that the wandering behavior of the pedestrian occurs in a video case (i.e., the pedestrian falling video) of the first embodiment.

[0066] In order to facilitate visualization of a motion trajectory recognition model, a character p is used to represent the recognition result of the motion trajectory of the pedestrian. When the motion trajectory of the pedestrian is recognized as a wandering trajectory, ?]=1; otherwise, p=0. In summary, the recognition result of the first embodiment is p=l, that is, there is the wandering behavior.

[0067] Second embodiment

[0068] A system for recognizing a wandering behavior of a pedestrian based on a motion trajectory is provided in the second embodiment, including a pedestrian motion trajectory construction module and a pedestrian wandering behavior recognition module.

[0069] The pedestrian motion trajectory construction module is configured to acquire motion images of the pedestrian, recognize motion centroids of the pedestrian based on the motion images, and construct the motion trajectory of the pedestrian according to the motion centroids.

[0070] The pedestrian wandering behavior recognition module is configured to perform a wandering behavior recognition on the motion trajectory of the pedestrian according to the pedestrian wandering behavior criteria, to obtain the recognition result.

[0071] Construction steps of the pedestrian wandering behavior criteria are as follows.

[0072] According to the common wandering behaviors in the public places, the wandering motion trajectories appeared in the different scenes are drawn.

[0073] The wandering motion trajectories appeared in the different scenes are analyzed to obtain the pedestrian wandering behavior criteria.

[0074] The rest is the same as the first embodiment.

[0075] The above provides a detailed description of the specific embodiments of the disclosure. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the disclosure without the need for inventive effort. Therefore, any technical solutions that can be obtained by those skilled in the art based on the concept of the disclosure, through logical analysis, reasoning, or limited experimentation, should fall within the scope of protection defined by the claims.

Claims

1. A method for recognizing a wandering behavior of a pedestrian based on a motion trajectory, comprising:acquiring motion images of the pedestrian, recognizing motion centroids of the pedestrian based on the motion images, and constructing the motion trajectory of the pedestrian according to the motion centroids; andperforming, according to pedestrian wandering behavior criteria, a wandering behavior recognition on the motion trajectory of the pedestrian to obtain a recognition result;wherein construction steps of the pedestrian wandering behavior criteria comprise:drawing, according to common wandering behaviors in public places, wandering motion trajectories appeared in different scenes; andanalyzing the wandering motion trajectories appeared in the different scenes to obtain the pedestrian wandering behavior criteria.

2. The method for recognizing the wandering behavior of the pedestrian based on the motion trajectory as claimed in claim 1, wherein the recognizing motion centroids of the pedestrian based on the motion images, and constructing the motion trajectory of the pedestrian according to the motion centroids comprises:obtaining, based on the motion images of the pedestrian, key nodes of the pedestrian to form different body segments, and calculating a centroid of each body segment of the different body segments;calculating total centroids of the pedestrian as the motion centroids of the pedestrian by using the centroid of each body segment of the different body segments; andfitting coordinates of the total centroids of the pedestrian in a period time according to a time sequence, to thereby obtain a motion trajectory of the total centroids of the pedestrian as the motion trajectory of the pedestrian in the period of time.

3. The method for recognizing the wandering behavior of the pedestrian based on the motion trajectory as claimed in claim 2, wherein the key nodes of the pedestrian comprise: a head, a left ear, a right ear, a left eye, a right eye, a nose, a left mouth comer, a right mouth corner, a neck, a left shoulder, a right shoulder, a left elbow, a right elbow, a left wrist, a right wrist, a left hip, a right hip, a left knee, a right knee, a left ankle and a right ankle.

4. The method for recognizing the wandering behavior of the pedestrian based on the motion trajectory as claimed in claim 2, wherein a calculation formula of the centroid of each body segment of the different body segments is as follows:xk Xplp + Xdld yk = yPiP+ydid ^k ^p^p + ^d^d where represents coordinates of the centroid of each body segment of thedifferent body segments, (xp,yp,zp) represents coordinates of a proximal end of each body segment of the different body segments, (xd, yd, zd) represents coordinates of a distal end of each body segment of the different body segments, lp represents a percentage of a length of each body segment of the different body segments from the proximal end to a tail end, and ld represents a percentage of a length of each body segment of the different body segments from the distal end to the tail end.

5. The method for recognizing the wandering behavior of the pedestrian based on the motion trajectory as claimed in claim 2, wherein a calculation formula of each of the total centroids of the pedestrian is as follows:Xc =----77----MSkU mkyk yc ~ M _ Lk=l zc -where (xc,yc, zc) represents coordinates of each of the total centroid of the pedestrian, (xk, Vk> zk) represents coordinates of a £-th body segment of the different body segments, mk represents a mass of the £-th body segment of the different body segments, and M represents a total mass of the different body segments.

6. The method for recognizing the wandering behavior of the pedestrian based on the motion trajectory as claimed in claim 1, wherein the wandering motion trajectories appeared in the different scenes comprise: a multiple oval motion trajectory, a spiral motion trajectory, a snakelike motion trajectory and a Brownian motion trajectory.

7. The method for recognizing the wandering behavior of the pedestrian based on the motion trajectory as claimed in claim 6, wherein the pedestrian wandering behavior criteria are as follows:Oval.p.cos 0 <04 > / V >7 count —Spiral.p. =cos 0 <08 — count — 2cos 0 <0count —where Oval.p. represents a multiple oval motion trajectory criterion, Spiral, p. represents a spiral motion trajectory criterion, Snake.p. represents a snake motion trajectory criterion, Brown.p. represents a Brownian motion trajectory criterion, and Ncount represents a threshold of a number of inflection points in the motion trajectory of the pedestrian.

8. The method for recognizing the wandering behavior of the pedestrian based on the motion trajectory as claimed in claim 1, wherein when there is a back-and-forth motion of the pedestrian, there is at least one inflection point in the motion trajectory of the pedestrian; andwherein the performing, according to the pedestrian wandering behavior criteria, a wandering behavior recognition on the motion trajectory of the pedestrian to obtain a recognition result comprises:determining, according to a determining result corresponding to whether a number of the at least one inflection point exceeds a threshold, whether a behavior of the pedestrian is the wandering behavior.

9. The method for recognizing the wandering behavior of the pedestrian based on the motion trajectory as claimed in claim 8, wherein each of the at least one inflection point is configured to indicate that a change range of a direction angle of a trajectory point in continuous frames of the motion trajectory of the pedestrian exceeds 90 degrees, and a calculation formula for the direction angle of the trajectory point is as follows:where 0 represents the direction angle of the trajectory point, Pt_1 represents a coordinate point of the motion trajectory of the pedestrian at a (M)-th time, Pt represents a coordinate point of the motion trajectory of the pedestrian at a / -th time, and Pt+1 represents a coordinate point of the motion trajectory of the pedestrian at a (M)-th time.

10. A system for recognizing a wandering behavior of a pedestrian based on a motiontrajectory, comprising:a pedestrian motion trajectory construction module, configured to acquire motion images of the pedestrian, recognize motion centroids of the pedestrian based on the motion images, and construct the motion trajectory of the pedestrian according to the motion centroids; anda pedestrian wandering behavior recognition module, configured to perform a wandering behavior recognition on the motion trajectory of the pedestrian according to pedestrian wandering behavior criteria, to obtain a recognition result;wherein construction steps of the pedestrian wandering behavior criterion, comprise:drawing, according to common wandering behaviors in public places, wandering motion trajectories appeared in different scenes; andanalyzing the wandering motion trajectories appeared in the different scenes to obtain the pedestrian wandering behavior criteria.

Images