Multi-feature and multi-resolution trajectory anomaly detection method

Abstract

The multi-feature and multi-resolution trajectory anomaly detection method includes the following steps: 1) trajectory extraction of moving targets; 2) grouping of trajectory movement directions; 3) multi-resolution trajectory segmentation; 4) calculation of multi-dimensional features of trajectories; 5) feature clustering anomaly detection of trajectories; by combining the multi-resolution segmentation method, the multi-dimensional feature method and the clustering method, the contextual features of the trajectory are integrated to avoid missed detection caused by trajectory segmentation, and the trajectory with abnormal position, speed and direction can be detected, and abnormal trajectories can be adaptively identified, which is suitable for various fields of trajectory anomaly detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of video motion trajectory anomaly detection, and in particular relates to a multi-feature and multi-resolution trajectory anomaly detection method. Background Art

[0002] With the widespread use of surveillance equipment, a large number of surveillance videos are recorded. How to accurately detect abnormal trajectories in videos is a difficult problem in public safety. On the one hand, the determination of trajectory anomalies is closely related to the environment and requires manual judgment. On the other hand, trajectories usually contain multiple motion modes, and the same trajectory may have local anomalies or multiple abnormal behaviors. Therefore, it is necessary to deeply mine the local motion characteristics of the trajectory.

[0003] There are currently three methods for detecting abnormal pedestrian trajectories in surveillance videos: the first is to train a model using normal target trajectories, and a trajectory is considered abnormal when it does not match the model. The second is to train a model using normal trajectories and pre-defined abnormal trajectories. The third is to detect traditional features of the trajectory, extract low-level and high-level features of the trajectory to represent the target's motion trajectory, and define outlier trajectories and trajectories with a small number of occurrences as abnormal trajectories. Existing anomaly detection methods extract features of the entire trajectory or extract sub-trajectory features in segments. The selection of segmentation points will cause abnormal trajectory interruptions. The trajectory after the interruption may not have abnormal behavior, but it has abnormal behavior when it is not segmented. Therefore, the selection of segmentation points greatly affects the detection results. Simple segmentation is not reasonable. On the other hand, simple segmentation processing is difficult to combine trajectory context motion features, which causes deviations in pedestrian abnormality detection results. Summary of the invention

[0004] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a multi-feature and multi-resolution trajectory anomaly detection method. The present invention uses global features and local features of the trajectory, combines a multi-resolution segmentation method, a multi-dimensional feature method and a clustering method, integrates the contextual features of the trajectory, avoids the missed detection phenomenon caused by trajectory segmentation, can detect trajectories with abnormal trajectory position, speed, and direction, can adaptively identify abnormal trajectories, and is suitable for various fields of trajectory anomaly detection.

[0005] To achieve the above object, the technical solution adopted by the present invention is: a multi-feature multi-resolution trajectory anomaly detection method, characterized in that it includes the following steps:

[0006] Step 1, extracting the trajectory of the moving target, specifically, collecting the time information and position information of the moving target, and defining the position coordinates of the moving target in a continuous sequence within a period of time as the moving trajectory of the target;

[0007] Step 2: Group the trajectories by direction of movement. The specific steps are:

[0008] The trajectory coordinates are used to calculate the trajectory movement direction, and the trajectories are grouped according to the trajectory movement direction. After grouping, a fitting trajectory is generated for each grouped trajectory;

[0009] Step 3: Multi-resolution trajectory segmentation. The specific steps are:

[0010] Use multiple different resolutions to segment the trajectory to obtain sub-trajectories. Sub-trajectories with the same resolution have the same calculation dimension.

[0011] Step 4: Calculate the multi-dimensional features of the trajectory. The specific steps are:

[0012] According to the sub-trajectories with different resolutions in step 3, the position, speed, and direction distance of the real trajectory and the fitted trajectory are calculated, and the position, speed, and direction distance are used as the multi-dimensional features of the trajectory;

[0013] Step 5: Trajectory feature clustering anomaly detection. The specific steps are:

[0014] According to the multi-dimensional features in step 4, the trajectories are clustered and the trajectories in the same direction group are divided into two groups. The anomaly score is calculated for each group obtained by clustering. The trajectories are merged according to the anomaly score, and finally the abnormal trajectories and normal trajectories are obtained.

[0015] The step 2 specifically includes the following steps:

[0016] Step 2.1, group the trajectories by direction of movement. The specific steps are:

[0017] Step 2.1.1, collect the position coordinates of each trajectory at the starting time and the disappearing time, the position coordinates are represented by abscissa and ordinate, and the movement direction of the trajectory is calculated by the change between the starting position coordinates and the disappearing position coordinates of the trajectory;

[0018] Step 2.1.2, using the movement direction calculated in step 2.1.1 as the feature of the trajectory, clustering the movement direction of the trajectory using a clustering algorithm to obtain multiple groups of the trajectory;

[0019] Step 2.2, generate a fitting trajectory for each group obtained in step 2.1.2. The coordinates of the fitting trajectory are determined by the trajectory within the group. The specific steps are:

[0020] Step 2.2.1, set the number of trajectory sampling points, divide the spatial position distance between the starting points of the trajectory into equal intervals, and calculate the coordinates of the division points;

[0021] Step 2.2.2, take the point in the real trajectory with the smallest spatial distance to the segmentation point as the sampling point, and record the coordinates of the sampling point;

[0022] Step 2.2.3, calculate the average coordinates of all sampling points in the group, and connect the average coordinate positions of adjacent sampling points as the position coordinates of the fitting trajectory.

[0023] In step 4, the calculation of the multi-dimensional features of the trajectory includes the following steps:

[0024] Step 4.1, calculate the multi-dimensional local position features and global position features of the trajectory. The specific steps are as follows:

[0025] Step 4.1.1, select the sub-trajectories obtained with the same resolution;

[0026] Step 4.1.2, calculate the spatial distance between the real sub-trajectory and the sub-trajectory corresponding to the fitted trajectory, and use the spatial distance difference between the next moment and the previous moment to represent the local motion change of the sub-trajectory in the current dimension;

[0027] Step 4.1.3, calculate the average value of the spatial distance difference of all sub-trajectories of the trajectory, and use the average value to represent the position change characteristics of the trajectory in the current dimension, then return to step 4.1.1 to select sub-trajectories of other resolutions and repeat until the calculation of the position change characteristics of all dimensions is completed. The calculation method is shown in formula (1):

[0028]

[0029] in Indicates the resolution is ε l The position offset feature of trajectory i, k represents the sequence number of the sub-trajectory, M(ε l ) indicates the resolution is ε l The number of sub-trajectory segments at the time, d i,k represents the offset distance between the actual position of the kth sub-trajectory of target tube i and the fitted trajectory, d i,k-1 represents the offset distance between the actual position of the k-1th sub-trajectory of target tube i and the fitted trajectory, ε l Indicates resolution;

[0030] Step 4.2, calculate the multi-dimensional local velocity characteristics and global velocity characteristics of the trajectory. The specific steps are as follows:

[0031] Step 4.2.1, select the sub-trajectories obtained by segmentation with the same resolution;

[0032] Step 4.2.2, respectively calculate the speed of the real sub-trajectory segment and the corresponding fitted sub-trajectory segment, and use the speed difference between the two to represent the speed change characteristics of the sub-trajectory in the current dimension;

[0033] In step 4.2.3, the speed change characteristics of all sub-trajectories are averaged, and the average value is used to represent the speed change characteristics of the trajectory in the current dimension. Then, return to step 4.2.1 to select sub-trajectories of other resolutions and repeat until the calculation of the speed change characteristics of all dimensions is completed. The calculation method is shown in formula (2):

[0034]

[0035] in Indicates the resolution is ε l The velocity deviation characteristic of trajectory i, v i,k represents the average speed of the kth sub-trajectory of target tube i, represents the average speed of the kth sub-trajectory of the fitted target tube i;

[0036] Step 4.3, calculate the multi-dimensional local directional features and global directional features of the trajectory. The specific steps are as follows:

[0037] Step 4.3.1, select the sub-trajectories obtained with the same resolution;

[0038] Step 4.3.2, respectively calculate the directions of the real sub-trajectory segment and the corresponding fitted sub-trajectory segment, and use the direction angle difference between the two to represent the direction change characteristics of the sub-trajectory in the current dimension;

[0039] In step 4.3.3, the directional change characteristics of all sub-trajectories of the trajectory are averaged, and the average value is used to represent the directional change characteristics of the trajectory in the current dimension. Then, return to step 4.3.1 to select sub-trajectories of other resolutions and repeat until the calculation of the directional change characteristics of all dimensions is completed. The calculation method is shown in formula (3):

[0040]

[0041] in Indicates the resolution is ε l The directional characteristics of trajectory i, a i,k represents the motion direction of the kth sub-trajectory of the real target tube i, represents the motion direction of the kth sub-trajectory of the fitting target tube i, M(ε l ) indicates the resolution is ε l The number of segments of the sub-track is , and k represents the sequence number of the sub-track.

[0042] In step 5, the feature clustering anomaly detection of the trajectory includes the following steps:

[0043] Step 5.1, multi-feature trajectory clustering, the specific steps are:

[0044] Step 5.1.1, select the position features of the trajectory in different dimensions for clustering;

[0045] Step 5.1.2, select the velocity features of the trajectory in different dimensions for clustering;

[0046] Step 5.1.3, select the directional features of the trajectory in different dimensions for clustering;

[0047] Step 5.2, merge the trajectory groups. The specific steps are:

[0048] For each clustering in step 5.1, the intra-group anomaly score must be calculated for the obtained group. The anomaly score is the sum of the offset distances in different dimensions. If the difference in the anomaly scores of two groups of trajectories is less than the threshold, the two groups are merged and the trajectories within the groups are regarded as normal trajectories. If the difference in the anomaly scores of two groups of trajectories is greater than the threshold, the group with the larger anomaly score is regarded as the abnormal trajectory group.

[0049] The beneficial effects of the present invention are:

[0050] 1) The present invention adopts a multi-resolution segmentation method, which integrates the contextual features of the trajectory while segmenting, thus avoiding the problem of missed detection caused by unreasonable selection of trajectory segmentation points;

[0051] 2) The present invention adopts a multi-feature detection method, which can detect the position anomaly, speed anomaly, and direction anomaly of the trajectory. Compared with the single-feature anomaly detection method, the detected anomaly types are richer and the detection range is more comprehensive;

[0052] 3) The present invention adopts a trajectory clustering method, which does not require pre-definition of trajectory anomaly types and is applicable to a variety of scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A fitted trajectory graph is generated for the trajectory group of the present invention.

[0054] Figure 2 It is a schematic diagram of different resolution segmentation of the trajectory of the present invention.

[0055] Figure 3 Schematic diagram of the local offset distance of the sub-track of the present invention.

[0056] Figure 4 It is a schematic diagram of the speed offset between the real trajectory and the fitted trajectory of the present invention.

[0057] Figure 5 It is a schematic diagram of the direction deviation between the real trajectory and the fitted trajectory of the present invention.

[0058] Figure 6 Schematic diagram of anomaly detection results of single-feature and multi-feature methods in the present invention.

[0059] Figure 7 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0060] The present invention is further described in detail below in conjunction with specific implementations and accompanying drawings.

[0061] See also Figure 7 ,The multi-feature multi-resolution trajectory anomaly detection method includes the following steps:

[0062] Step 1, extracting the trajectory of the moving target. The specific method is: collecting the time information and position information of the moving target, and defining the position coordinates of the moving target in a continuous sequence within a period of time as the moving trajectory of the target.

[0063] Step 2: Group the movement directions of the trajectories. The specific steps are:

[0064] Step 2.1, group the trajectories by direction of movement. The specific steps are:

[0065] Step 2.1.1, collect the position coordinates of each trajectory at the time of appearance and the time of disappearance, the coordinates are expressed as horizontal and vertical coordinates, and the movement direction of the trajectory is calculated by the change between the starting coordinates of the trajectory and the end target.

[0066] Step 2.1.2: Use the movement direction calculated in 2.1 as the feature of the trajectory, use a clustering algorithm to cluster the movement direction of the trajectory, and obtain multiple groups of the trajectory.

[0067] Step 2.2, generate a fitting trajectory for each group obtained in step 2.1, and the coordinates of the fitting trajectory are determined by the trajectory within the group, such as Figure 1 The specific steps are:

[0068] Step 2.2.1, set the number of trajectory sampling points, divide the trajectory into equal intervals according to the spatial position distance between the starting points of the trajectory, and calculate the coordinates of the division points.

[0069] Step 2.2.2, take the real trajectory point with the smallest spatial distance from the segmentation point as the sampling point, and record the coordinates of the sampling point;

[0070] Step 2.2.3, calculate the average coordinates of all sampling points in the group, and connect the average coordinates of adjacent sampling points as the position coordinates of the fitting trajectory;

[0071] Step 3: Multi-resolution trajectory segmentation. The specific steps are:

[0072] Use several different resolutions to segment the trajectory to obtain sub-trajectories. Sub-trajectories with the same resolution are one dimension. The multi-resolution segmentation method is as follows: Figure 2 As shown, three different resolutions are used for illustration;

[0073] Step 4: Calculate the multi-dimensional features of the trajectory. The specific steps are:

[0074] For sub-trajectories with different resolutions, the position, speed, and direction distance of the real trajectory and the fitted trajectory are calculated, and the position, speed, and direction distance are used as the multi-dimensional features of the trajectory;

[0075] Step 4.1, calculate the multi-dimensional local position features and global position features of the trajectory. The specific steps are as follows:

[0076] Step 4.1.1, select the sub-trajectories obtained with the same resolution;

[0077] Step 4.1.2, calculate the spatial distance between the real sub-trajectory and the sub-trajectory corresponding to the fitted trajectory, and use the spatial distance difference between the next moment and the previous moment to represent the local motion change of the sub-trajectory in the current dimension, such as Figure 3 As shown, the sub-trajectory at time t∈n-2n is selected to illustrate the local motion change;

[0078] Step 4.1.3, calculate the average of the spatial distance differences of all sub-trajectories of the trajectory, use the average value to represent the position change characteristics of the trajectory in the current dimension, and then return to 4.1.1 to select sub-trajectories of other resolutions and loop until the calculation of position change characteristics of all dimensions is completed.

[0079] Step 4.2, calculate the multi-dimensional local velocity characteristics and global velocity characteristics of the trajectory. The specific steps are as follows:

[0080] Step 4.2.1, select the sub-trajectories obtained by segmentation with the same resolution;

[0081] Step 4.2.2, calculate the speed of the real sub-trajectory segment and the corresponding fitted sub-trajectory segment respectively, and use the speed difference between the two to represent the speed change characteristics of the sub-trajectory in the current dimension, such as Figure 4 As shown in the figure, the target tube is divided into three segments, and the sub-trajectory at time t0-t1 is selected for illustration, where t represents time;

[0082] Step 4.2.3, calculate the average value of the speed change characteristics of all sub-trajectories, use the average value to represent the speed change characteristics of the trajectory in the current dimension, and then return to step 4.2.1 to select sub-trajectories of other resolutions to loop until the calculation of speed change characteristics of all dimensions is completed;

[0083] Step 4.3, calculate the multi-dimensional local directional features and global directional features of the trajectory. The specific steps are as follows:

[0084] Step 4.3.1, select the sub-trajectories obtained with the same resolution;

[0085] Step 4.3.2, calculate the directions of the real sub-trajectory segment and the corresponding fitted sub-trajectory segment respectively, and use the direction angle difference between the two to represent the direction change characteristics of the sub-trajectory in the current dimension, such as Figure 5 As shown in the figure, the first and second sub-trajectories are used to illustrate the calculation of the direction difference between the real sub-trajectory and the fitted sub-trajectory as the direction offset of the sub-trajectory. The trajectory direction difference range is (-π, π);

[0086] In step 4.3.3, the directional change characteristics of all sub-trajectories of the trajectory are averaged, and the average value is used to represent the directional change characteristics of the trajectory in the current dimension. Then, return to step 4.3.1 to select sub-trajectories of other resolutions and repeat until the calculation of the directional change characteristics of all dimensions is completed.

[0087] Step 5: Trajectory feature clustering anomaly detection. The specific steps are:

[0088] Step 5.1, multi-feature trajectory clustering, the specific steps are:

[0089] Step 5.1.1, select the position features of the trajectory in different dimensions for clustering;

[0090] Step 5.1.2, select the velocity features of the trajectory in different dimensions for clustering;

[0091] Step 5.1.2, select the directional features of the trajectory in different dimensions for clustering;

[0092] Step 5.2, merge the trajectory groups. The specific steps are:

[0093] The intra-group anomaly score is calculated for each clustering group. If the difference in anomaly scores between two groups of trajectories is less than the threshold, the two groups are merged and the intra-group trajectories are taken as normal trajectories. If the difference in anomaly scores between two groups of trajectories is greater than the threshold, the group with the larger anomaly score is taken as the abnormal trajectory group.

[0094] By applying the method proposed in the present invention to the surveillance video, the abnormal trajectory in the surveillance video can be well detected. The detection results are as follows: Figure 6 As shown in the figure, the darker tracks are abnormal tracks. The first three pictures are single-feature anomaly detection results, which detect fewer abnormal tracks and are not comprehensive. The last abnormal picture is the multi-resolution multi-feature anomaly detection method of this method, which detects more abnormal tracks and a more comprehensive range of abnormalities. The abnormal behaviors include abnormal location, abnormal speed, abnormal direction, etc.

Claims

1. A multi-feature and multi-resolution trajectory anomaly detection method, characterized in that: The following steps are involved: Step 1, extracting the trajectory of the moving target, specifically, collecting the time information and position information of the moving target, and defining the position coordinates of the moving target in a continuous sequence within a period of time as the moving trajectory of the target; Step 2: Group the trajectories by direction of movement. The specific steps are: The trajectory coordinates are used to calculate the trajectory movement direction, and the trajectories are grouped according to the trajectory movement direction. After grouping, a fitting trajectory is generated for each grouped trajectory; Step 3: Multi-resolution trajectory segmentation. The specific steps are: Use multiple different resolutions to segment the trajectory to obtain sub-trajectories. Sub-trajectories with the same resolution have the same calculation dimension. Step 4: Calculate the multi-dimensional features of the trajectory. The specific steps are: According to the sub-trajectories with different resolutions in step 3, the position, speed, and direction distance of the real trajectory and the fitted trajectory are calculated, and the position, speed, and direction distance are used as the multi-dimensional features of the trajectory; In step 4, the calculation of the multi-dimensional features of the trajectory includes the following steps: Step 4.1, calculate the multi-dimensional local position features and global position features of the trajectory. The specific steps are as follows: Step 4.1.1, select the sub-trajectories obtained with the same resolution; Step 4.1.2, calculate the spatial distance between the real sub-trajectory and the sub-trajectory corresponding to the fitted trajectory, and use the spatial distance difference between the next moment and the previous moment to represent the local motion change of the sub-trajectory in the current dimension; Step 4.1.3, calculate the average value of the spatial distance difference of all sub-trajectories of the trajectory, and use the average value to represent the position change characteristics of the trajectory in the current dimension, then return to step 4.1.1 to select sub-trajectories of other resolutions and repeat until the calculation of the position change characteristics of all dimensions is completed. The calculation method is shown in formula (1): where represents the position offset feature of trajectory i when the resolution is εl, k represents the sequence number of the sub-trajectory, M(εl) represents the number of sub-trajectory segments when the resolution is εl, di,k represents the offset distance between the real position of the kth sub-trajectory of target tube i and the fitted trajectory, di,k-1 represents the offset distance between the real position of the k-1th sub-trajectory of target tube i and the fitted trajectory, and εl represents the resolution; Step 4.2, calculate the multi-dimensional local velocity characteristics and global velocity characteristics of the trajectory. The specific steps are as follows: Step 4.2.1, select the sub-trajectories obtained by segmentation with the same resolution; Step 4.2.2, respectively calculate the speed of the real sub-trajectory segment and the corresponding fitted sub-trajectory segment, and use the speed difference between the two to represent the speed change characteristics of the sub-trajectory in the current dimension; Step 4.2.3, calculate the average value of the speed change characteristics of all sub-trajectories, and use the average value to represent the speed change characteristics of the trajectory in the current dimension, then return to step 4.2.1 to select sub-trajectories of other resolutions and loop until the calculation of the speed change characteristics of all dimensions is completed. The calculation method is shown in formula (2): Wherein, represents the velocity offset characteristic of trajectory i when the resolution is εl, vi,k represents the average velocity of the k-th sub-trajectory of target tube i, and represents the average velocity of the k-th sub-trajectory of fitted target tube i; Step 4.3, calculate the multi-dimensional local directional features and global directional features of the trajectory. The specific steps are as follows: Step 4.3.1, select the sub-trajectories obtained with the same resolution; Step 4.3.2, respectively calculate the directions of the real sub-trajectory segment and the corresponding fitted sub-trajectory segment, and use the direction angle difference between the two to represent the direction change characteristics of the sub-trajectory in the current dimension; In step 4.3.3, the directional change characteristics of all sub-trajectories of the trajectory are averaged, and the average value is used to represent the directional change characteristics of the trajectory in the current dimension. Then, return to step 4.3.1 to select sub-trajectories of other resolutions and repeat until the calculation of the directional change characteristics of all dimensions is completed. The calculation method is shown in formula (3): Wherein, represents the directional feature of trajectory i when the resolution is εl, ai,k represents the motion direction of the kth sub-trajectory of the real target tube i, represents the motion direction of the kth sub-trajectory of the fitted target tube i, M(εl) represents the number of sub-trajectory segments when the resolution is εl, and k represents the sequence number of the sub-trajectory; Step 5: Trajectory feature clustering anomaly detection. The specific steps are: According to the multi-dimensional features in step 4, the trajectories in the same direction group are clustered into two groups. The anomaly score is calculated for each group, and the trajectories are merged according to the anomaly score. Finally, the abnormal trajectories and normal trajectories are obtained. The feature clustering anomaly detection of the trajectory includes the following steps: Step 5.1, multi-feature trajectory clustering, the specific steps are: Step 5.1.1, select the position features of the trajectory in different dimensions for clustering; Step 5.1.2, select the velocity features of the trajectory in different dimensions for clustering; Step 5.1.3, select the directional features of the trajectory in different dimensions for clustering; Step 5.2, merge the trajectory groups. The specific steps are: For each clustering in step 5.1, the intra-group anomaly score must be calculated for the obtained group. The anomaly score is the sum of the offset distances in different dimensions. If the difference in the anomaly scores of two groups of trajectories is less than the threshold, the two groups are merged and the trajectories within the groups are regarded as normal trajectories. If the difference in the anomaly scores of two groups of trajectories is greater than the threshold, the group with the larger anomaly score is regarded as the abnormal trajectory group.

2. The multi-feature multi-resolution trajectory anomaly detection method according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step 2.1, group the trajectories by direction of movement. The specific steps are: Step 2.1.1, collect the position coordinates of each trajectory at the starting time and the disappearing time, the position coordinates are represented by abscissa and ordinate, and the movement direction of the trajectory is calculated by the change between the starting position coordinates and the disappearing position coordinates of the trajectory; Step 2.1.2, using the movement direction calculated in step 2.1.1 as the feature of the trajectory, clustering the movement direction of the trajectory using a clustering algorithm to obtain multiple groups of the trajectory; Step 2.2, generate a fitting trajectory for each group obtained in step 2.1.

2. The coordinates of the fitting trajectory are determined by the trajectory within the group. The specific steps are: Step 2.2.1, set the number of trajectory sampling points, divide the spatial position distance between the starting points of the trajectory into equal intervals, and calculate the coordinates of the division points; Step 2.2.2, take the point in the real trajectory with the smallest spatial distance to the segmentation point as the sampling point, and record the coordinates of the sampling point; Step 2.2.3, calculate the average coordinates of all sampling points in the group, and connect the average coordinate positions of adjacent sampling points as the position coordinates of the fitting trajectory.

Images