Crowd anomaly detection method based on pedestrian group ring block feature extraction

Abstract

The present invention belongs to the field related to video analysis and image processing, and discloses a crowd anomaly detection method based on pedestrian group ring block feature extraction. The optical flow field is preprocessed by the molecular diffusion idea to obtain a motion field that can better reflect the movement of the crowd, and a progressive strategy is adopted to group pedestrians. First, a direction group is constructed according to the direction of crowd movement, and then the direction group is clustered again in combination with time and space information to obtain a more detailed pedestrian group. For each pedestrian group, the crowd energy characteristics and the ring block energy histogram characteristics constructed based on the energy field are used to characterize the pedestrian movement information, and finally a class of SVM is used to perform crowd anomaly detection. The present invention has good applicability and can be used in a variety of public places. Since the crowd has been carefully grouped, the positioning of abnormal positions is also very accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of video analysis and image processing, and in particular relates to a crowd anomaly detection method based on pedestrian group ring block feature extraction. Background Art

[0002] As crowd safety issues in public places have received more and more attention, crowd anomaly detection research has become a hot topic in the field of video processing. As more and more crowd activities are organized in public places, crowd safety issues have also become prominent, while traditional video surveillance mainly relies on manual monitoring video analysis, and it is difficult for humans to work for a long time and at a high intensity. In this case, the accuracy of detecting abnormal events will be greatly reduced, so it is very necessary to improve the efficiency and intelligence of monitoring. By automatically analyzing the surveillance video, the abnormal events in the scene can be continuously tracked and detected, so as to judge the behavior of the target and effectively avoid possible dangerous situations. Therefore, abnormal detection of crowds in the scene also has great practical significance.

[0003] Since there are many types of abnormal behaviors of pedestrians in real scenes, and they are prone to mutual occlusion, it is also a great challenge to detect abnormal behaviors in scenes in real time. From the current research, the focus of crowd anomaly detection is on feature extraction. In order to extract features, the effective area in the image is usually obtained first. Most of them divide the continuous image frames into regional blocks. This block segmentation divides the image into equal-sized and shaped areas on average, without considering the distribution of the crowd in the scene. It will separate the overall pedestrians and cannot guarantee the integrity of individual pedestrians. This method will cause the extracted features to fail to truly reflect the movement status of pedestrians, affecting the effectiveness of the detection results.

[0004] By grouping the crowd in the scene and extracting relevant features while ensuring the integrity of the pedestrians, the pedestrian movement information can be more accurately expressed. When extracting features, the influence of the pedestrian's limb movement is considered while considering the pedestrian's movement state. In this way, the extracted features will be more reasonable and effective when used for crowd anomaly detection. Summary of the invention

[0005] The present invention aims to provide a crowd anomaly detection method based on pedestrian group ring block feature extraction to solve the crowd safety problem in public places.

[0006] In order to solve the above technical problems, the specific technical solution of the crowd anomaly detection method based on pedestrian group ring block feature extraction of the present invention is as follows:

[0007] A crowd anomaly detection method based on pedestrian group ring block feature extraction includes the following steps:

[0008] Step 1: Preprocess the vector field using molecular diffusion theory;

[0009] Step 2: Pedestrians are grouped based on spatiotemporal information using a progressive strategy;

[0010] Step 3: Extract features for each pedestrian group, extract pedestrian group energy features and ring block energy histogram features to represent motion information, and extract appearance features based on the gray level co-occurrence matrix;

[0011] Step 4: Anomaly detection.

[0012] Furthermore, the step 1 includes the following specific steps:

[0013] Step 1.1: Consider each pixel in the vector field as a particle; let F a =(f x ,f y ) and F b =(f x ,f y ) are vector particles a and b in the field. After diffusion, vector particle a can be expressed as:

[0014]

[0015] Where I is a neighborhood of point a in the field, n represents the number of particles in the neighborhood whose diffusion amount to particle a is greater than 0, and the vector diffusion formula of points a and b is defined as:

[0016]

[0017] Where g (x,y) represents the gradient of particles a and b in the horizontal and vertical directions, θ a,b represents the cosine value of two vector particles, D a,b represents the Euclidean distance between two particles.

[0018] Furthermore, the step 2 includes the following specific steps:

[0019] Step 2.1: First, divide the direction groups according to the main movement direction of the crowd. The calculation method is as follows:

[0020]

[0021] Where F is the movement direction of the particles inside the pedestrian, d is the direction vector of the scene road extension, and the crowd is divided into four main movement directions according to the cosine value of the angle with the road;

[0022] Step 2.2: After grouping the crowd directions, a rough group is obtained, and then the pedestrian speed in the crowd time information and the pedestrian speed in the spatial information are used to refine the direction group; an elliptical structure is used for offset; the similarity calculation formula is as follows:

[0023]

[0024] Where a and b are the minor and major axes of the ellipse, respectively, and r p Represents the Euclidean distance between two pixel coordinates, r q represents the Euclidean distance between the particle velocities within the pedestrian, t 1 and t 2 For its weight.

[0025] Furthermore, the step 3 includes the following specific steps:

[0026] Step 3.1: Extract the minimum horizontal circumscribed rectangle of each group, and divide the pedestrian group into four blocks in a circular shape with the center of gravity as the center point. The width of each circular block from the inside to the outside is set to 0.4, 0.3, 0.2, and 0.1 of the length from the center of gravity to the side of the rectangle respectively;

[0027] Step 3.2: The energy of particles in each block after segmentation is calculated as follows:

[0028]

[0029] Where u is the horizontal velocity of the particle, v is the vertical velocity of the particle, and the mass m of each particle is considered to be 1;

[0030] The particle energy is divided into eight ranges and a histogram is drawn. The formula for calculating the characteristic value of the bth block after the division is:

[0031]

[0032] where ε i is the weight of different bins of the histogram, N i Represents the frequency of different bins, and O represents the bin threshold.

[0033] Step 3.3: For each pedestrian group, the feature values ​​of the four small blocks outward from the center of gravity are weighted by 0.45, 0.25, 0.2, and 0.1, and then the ring block energy histogram features are obtained.

[0034] Furthermore, the step 4 includes the following specific steps:

[0035] Step 4.1: The three features obtained in step 3 are classified by a class SVM to detect crowd anomalies.

[0036] The crowd anomaly detection method based on pedestrian group ring block feature extraction of the present invention has the following beneficial effects: after the present invention adopts a progressive strategy to group pedestrians, the integrity of pedestrians is ensured, and the accuracy of anomaly detection and positioning is improved. The extracted features can also be applied to a variety of public places and have good applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the crowd anomaly detection method based on pedestrian group ring block feature extraction of the present invention;

[0038] Figure 2a-2c The pre-processed optical flow map and crowd segmentation map of the present invention;

[0039] Figure 3a-3c A pedestrian grouping diagram of the present invention;

[0040] Figure 4a , Figure 4b It is a schematic diagram of the energy field of the present invention;

[0041] Figure 5a , Figure 5b It is a schematic diagram of ring block segmentation of the present invention;

[0042] Figure 6a-6c This is an example diagram of anomaly detection results of the present invention;

[0043] Figure 7a-7c Another example diagram of the abnormality detection result of the present invention; DETAILED DESCRIPTION

[0044] In order to better understand the purpose, structure and function of the present invention, the following further describes in detail a crowd anomaly detection method based on pedestrian group ring block feature extraction of the present invention in conjunction with the accompanying drawings.

[0045] The present invention firstly uses the molecular diffusion concept in physics to preprocess the optical flow field, and then groups pedestrians with a progressive strategy. Pedestrians are first grouped according to their main movement direction, and then clustered according to their spatial position and speed to obtain more detailed grouping. The pedestrian group energy characteristics and ring block energy histogram characteristics are extracted for each pedestrian group to characterize the motion information, which is then combined with the appearance characteristics for crowd anomaly detection.

[0046] like Figure 1 As shown, the crowd detection method based on pedestrian group ring block feature extraction of the present invention includes the following steps:

[0047] Step 1: Preprocess the optical flow field using molecular diffusion theory;

[0048] Step 1.1: Calculate the optical flow field using the Horn-Schunck optical flow algorithm;

[0049] In the input video sequence, two consecutive frames are extracted, and the motion field of the particle is extracted by the position change of the same pixel point (a point with unchanged pixel value in the local neighborhood) between the two frames. Let c i is a point in two-dimensional motion, its coordinates are (x, y), and the optical flow size is (u, v), then c i The optical flow at the point is represented as (x, y, u, v), and finally the set of optical flows of all points is called the optical flow field, which can provide the motion information of the pixels in the video.

[0050] Step 1.2: From step 1.1, we can get the optical flow field corresponding to the image, and regard each pixel in the field as a particle. Let F a =(f x ,f y ) and F b =(f x ,f y ) is the optical flow vector of particles a and b in the field, where f x and f y Represent the horizontal velocity and vertical velocity in the vector respectively. After diffusion, the vector particle a can be expressed as:

[0051]

[0052] Where I is a neighborhood of point a in the field, n represents the number of particles in the neighborhood whose diffusion amount to particle a is greater than 0, and the vector diffusion formula of points a and b is defined as:

[0053]

[0054] Where g (x,y) represents the gradient of particles a and b in the horizontal and vertical directions, θ a,b represents the cosine value of two vector particles, D a,b Represents the Euclidean distance between two particles. After processing all particles in the field, a new optical flow field S is obtained.

[0055] Step 2: After preprocessing the optical flow field, a progressive strategy is used to group the crowd according to spatial position, speed, and direction.

[0056] Step 2.1: First, divide the direction groups according to the main movement direction of the crowd. The calculation method is as follows:

[0057]

[0058] Where F is the optical flow vector of a point in the vector field S, d is the direction vector of the scene road extension, and the crowd is divided into four main movement directions according to the cosine value of the angle with the road. noise, merge it with the normal group G normal The group with the longest correlation part and the individual noise are directly discarded.

[0059] Step 2.2: After grouping the crowd direction, a rough group is obtained, and then the pedestrian speed in the crowd time information and the pedestrian speed in the spatial information are used to refine the direction group. With the help of these two types of information, classification is performed based on the mean shift clustering algorithm, and the calculation formula for the shift mean is:

[0060]

[0061] where g 1 (x)=-k(x), k(x) is the cross-sectional function of the kernel function K(x), h is the bandwidth, and X is the pixel coordinates in the segmented crowd image.

[0062] Since pedestrians visually look more like ellipses that are wider at the top and bottom and narrower at the left and right than circles, the elliptical structure is used for offset. The similarity calculation formula is as follows:

[0063]

[0064] Where a and b are the minor and major axes of the ellipse, respectively, and r p Represents the Euclidean distance between two pixel coordinates, r q It represents the Euclidean distance between two vector points in the optical flow field S corresponding to two pixels, t 1 and t 2 For its weight.

[0065] Step 3: Extract features for each pedestrian group, extract pedestrian group energy features and ring block energy histogram features to represent motion information, and extract appearance features based on the gray-level co-occurrence matrix;

[0066] Step 3.1: Extract the appearance features that reflect the image texture differences through the gray-level co-occurrence matrix. Construct a gray-level co-occurrence matrix for each pedestrian group and extract the appearance features. The construction formula of the gray-level co-occurrence matrix is ​​as follows:

[0067]

[0068] Where K i,j (d,θ) represents the number of pixel pairs with grayscale i and j, distance d, and angle θ, and n is the total number of pixel pairs. Based on the grayscale co-occurrence matrix, three eigenvalues, energy, entropy, and contrast, are calculated.

[0069] Step 3.2: Extract the energy features of the pedestrian group, and regard the points in the optical flow field S as particles. The energy of each particle can be expressed as:

[0070]

[0071] Where u represents the horizontal optical flow, v represents the vertical optical flow, and the mass m of each particle is considered to be 1. For each pedestrian group, the mean E of the energy of all points in the group is taken as its energy feature.

[0072] Step 3.3: Extract the energy histogram features of the ring block, extract the horizontal minimum circumscribed rectangle of each group, and divide the pedestrian group into four blocks in a ring shape with the center of gravity as the center point. In order to make the area difference between the ring blocks as small as possible, the width of each ring block from the inside to the outside is set to 0.4, 0.3, 0.2, and 0.1 of the length from the center of gravity to the rectangular side.

[0073] The particle energy is divided into eight ranges and a histogram is drawn. The formula for calculating the characteristic value of the bth block after the division is:

[0074]

[0075] where ε i is the weight of different bins of the histogram, N i Represents the frequency of different bins, and O represents the bin threshold. For each pedestrian group, the feature values ​​of the four small blocks outward from the center of gravity are weighted by 0.45, 0.25, 0.2, and 0.1, and then the ring block energy histogram features of each pedestrian group are obtained.

[0076] Step 4: Anomaly detection;

[0077] Step 4.1: The three features obtained in step 3 are classified by a class SVM to detect crowd anomalies.

Claims

1. A crowd anomaly detection method based on pedestrian group ring block feature extraction, characterized in that: The steps include: Step 1: Preprocess the vector field using molecular diffusion theory; Step 2: Pedestrians are grouped based on spatiotemporal information using a progressive strategy; Step 3: Extract features for each pedestrian group, extract pedestrian group energy features and ring block energy histogram features to represent motion information, and extract appearance features based on the gray level co-occurrence matrix; Step 4: abnormality detection; Step 1 includes the following specific steps: Step 1.1: Consider each pixel in the vector field as a particle; let F a =(f x ,f y ) and F b =(f x ,f y ) are vector particles a and b in the field. After diffusion, vector particle a can be expressed as: Where I is a neighborhood of point a in the field, n represents the number of particles in the neighborhood whose diffusion amount to particle a is greater than 0, and the vector diffusion formula of points a and b is defined as: Where g (x,y) represents the gradient of particles a and b in the horizontal and vertical directions, θ a,b represents the cosine value of two vector particles, D a,b represents the Euclidean distance between two particles; The step 2 comprises the following specific steps: Step 2.1: First, divide the direction groups according to the main movement direction of the crowd. The calculation method is as follows: Where F is the movement direction of the particles inside the pedestrian, d is the direction vector of the scene road extension, and the crowd is divided into four main movement directions according to the cosine value of the angle with the road; Step 2.2: After grouping the crowd directions, a rough group is obtained, and then the pedestrian speed in the crowd time information and the pedestrian speed in the spatial information are used to refine the direction group; an elliptical structure is used for offset; the similarity calculation formula is as follows: Where a and b are the minor and major axes of the ellipse, respectively, and r p Represents the Euclidean distance between the coordinates of two pixels, r q represents the Euclidean distance between the particle velocities within the pedestrian, and t1 and t2 are their weights; The step 3 comprises the following specific steps: Step 3.1: Extract the minimum horizontal circumscribed rectangle of each group, and divide the pedestrian group into four blocks in a circular shape with the center of gravity as the center point. The width of each circular block from the inside to the outside is set to 0.4, 0.3, 0.2, and 0.1 of the length from the center of gravity to the side of the rectangle respectively; Step 3.2: The energy of particles in each block after segmentation is calculated as follows: Where u is the horizontal velocity of the particle, v is the vertical velocity of the particle, and the mass m of each particle is considered to be 1; The particle energy is divided into eight ranges and a histogram is drawn. The formula for calculating the characteristic value of the bth block after the division is: where ε i is the weight of different bins of the histogram, N i Represents the frequency of different bins, and O represents the bin threshold; Step 3.3: For each pedestrian group, the feature values ​​of the four small blocks outward from the center of gravity are weighted by 0.45, 0.25, 0.2, and 0.1, and then the ring block energy histogram features are obtained.

2. The method for detecting anomalies in crowds based on pedestrian group ring block feature extraction according to claim 1 is characterized in that: The step 4 comprises the following specific steps: Step 4.1: The three features obtained in step 3 are classified by a class SVM to detect crowd anomalies.

Images