A method for detecting abnormal behavior of moving targets in traffic scenarios

Abstract

This invention belongs to the field of road traffic safety detection technology and discloses a method for detecting abnormal behavior of moving targets in traffic scenarios. This method extracts trajectory orientation information based on the topological relationship of the road network and introduces a clustering algorithm for historical target trajectories from a spatial perspective, taking into account the orientation information of multiple frames of historical trajectories to mine behavioral patterns. Secondly, to achieve intelligent and automated abnormal behavior detection and to complete refined analysis of real-time and historical trajectories, a comprehensive detection method for target violation behavior based on the "3W1H" framework is designed. Finally, based on actual road rules, abnormal behavior is set according to traffic rules. Combining the behavioral patterns of historical trajectories and traffic rules, rule-based abnormal behavior detection and behavior pattern-based abnormal behavior detection algorithms determine when, where, and what kind of abnormal behavior occurred in the target.

Description

Technical Field

[0001] This invention relates to the field of road traffic safety detection technology, specifically to a method for detecting abnormal behavior of moving targets in traffic scenarios. Background Technology

[0002] With the acceleration of urbanization, urban road networks are becoming increasingly intricate, and human activities are becoming increasingly complex. Traffic monitoring systems, with their real-time high-definition video, increasingly sophisticated intelligent analysis functions, and low cost, are playing an increasingly important role in safety-related fields. This study focuses on moving targets captured by fixed cameras in keyframe images, aiming at the analysis and understanding of their behavior. It detects, tracks, and analyzes the behavior of moving targets in keyframe images to intelligently acquire their motion characteristics and behavioral patterns, which is of significant practical importance for ensuring public safety. Among these, abnormal behavior detection of moving targets is one of the fundamental goals of computer vision, a crucial approach to enabling computers to "see the world." Especially in traffic scenarios, abnormal behavior detection of moving targets has extremely high research value, both from the perspective of computer vision theory and artificial intelligence.

[0003] Anomaly detection of moving targets involves judging the abnormal behavior of different types of dynamic targets, such as vehicles and pedestrians, when they enter or leave the field of view of a specific camera. It is an important means of achieving "video-based security." Traditional video surveillance data can often only be used as "evidence for post-incident investigations." To achieve "prevention before the event" or "resolving issues during the event," it is necessary to utilize the relevant theories and methods of anomaly detection of moving targets. With almost no human intervention, the trajectory of moving targets is automatically extracted from the image frame sequence captured by the camera. Based on this, the relevant behavior of the target is analyzed and judged, ultimately forming an interpretation of the behavior of moving targets in traffic scenes. The extraction of moving target trajectories in traffic scenes includes two types: historical trajectories and real-time trajectories. Historical trajectories are sets of coordinate sequences formed when moving targets enter or leave the field of view in multiple frames. Real-time trajectories are coordinate sequences formed when moving targets enter or leave the field of view in any frame. Fully mining historical trajectories can reveal the movement patterns and behavioral patterns of targets, providing more refined analysis of anomalies. Combining historical and real-time trajectories can provide more intelligent analysis of anomalies. Therefore, combining historical and real-time trajectories for intelligent video analysis can provide regulatory authorities with more intelligent information on abnormal behavior analysis and a more refined observation perspective, reducing the difficulty of intelligent analysis of abnormal target behavior in traffic scenarios.

[0004] However, most existing trajectory orientation segmentation methods, which rely on trajectory similarity, cannot meet the refined requirements of abnormal behavior analysis. Furthermore, to further obtain behavioral patterns from trajectories with similar routes, it is necessary to mine these patterns based on the trajectory's orientation information. Current trajectory behavior pattern mining methods primarily employ trajectory clustering algorithms, treating each clustered trajectory sequence as a behavioral pattern. Traditional target trajectory clustering algorithms treat multiple trajectories as a whole, forming a complex trajectory set, and then divide this set into multiple trajectory clusters according to similarity algorithms. This approach fails to meet the refined requirements of abnormal behavior analysis for behavioral pattern segmentation. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method for detecting abnormal behavior of moving targets in traffic scenarios, in order to solve the problem that existing methods for classifying existing trajectories based on similarity and the process of mining trajectory behavior patterns using existing trajectory clustering algorithms cannot meet the requirements of the fine-grained classification of behavior patterns in abnormal behavior analysis.

[0006] To achieve the above objectives, the method of the present invention extracts trajectory orientation information based on the topological relationship of the road network, and takes into account the orientation information of multiple frames of historical trajectories to mine their behavioral patterns. The method includes the following steps:

[0007] 1) Obtain historical trajectories of different target types in the area to be detected, including vehicles and pedestrians;

[0008] 2) Obtain the directional routes of the video scenes corresponding to the segmented detection area based on the topological relationships between road networks. Determine the directional information of each historical trajectory and directional route. Divide historical trajectories with the same directional information under the same target type into groups with the same directional information. Group the multi-frame trajectory sequences of historical trajectories in each group with the same directional information. Divide the multi-frame trajectory sequences with the same position information into groups with the same position information. Calculate the sequence average vector of the multi-frame trajectory sequences in the groups with the same position information and obtain the angle θ between the sequence average vector and the horizontal axis. n And rotate all multi-frame trajectory sequences in the same position information group by θ according to the direction of the sequence average vector. n The angle is used to obtain the rotated trajectory. The rotated trajectory is scanned at equal intervals according to the direction of the sequence average vector to obtain the set of intersection points between the scan line and each trajectory. The set of intersection points of each trajectory is clustered to obtain the set of clustered trajectory coordinates. The clustered trajectory is rotated to the original coordinates to obtain the result set of target trajectory clustering. The result set of clustering is used as the normal behavior pattern of the target's historical trajectory.

[0009] According to traffic rules, trajectory rules are set at specific locations in video frames of the area to be detected. The trajectory rules include time period information, target type information, and image spatial information. The image spatial information includes surface rules and line rules.

[0010] 3) Obtain the motion trajectory information of real-time moving targets in the area to be detected, compare it with the normal behavior pattern, and determine whether the real-time moving targets deviate from the normal operating state;

[0011] Based on whether the real-time moving target's trajectory information meets the time period and target type information, as well as the geometric relationship between the real-time moving target's trajectory information and surface and line rules, it is determined whether the real-time moving target violates traffic rules.

[0012] The method of this invention has the following advantages: To obtain more refined behavioral patterns of the target, it is necessary to combine road network information to finely divide historical trajectories. First, the method of this invention considers that moving targets in videos usually move along specific routes or are greatly constrained by the surrounding environment during movement. That is, they are restricted moving objects that move along specific routes, such as rail transit equipment, highway traffic equipment, and other moving equipment with fixed routes. Therefore, the orientation of historical trajectories can be divided according to the road network topology that restricts the target's movement. Based on this, this invention uses the road network topology to determine the correct driving trajectory of a certain road segment. That is, the orientation route of the video scene corresponding to the segmented detection area is obtained according to the topology relationship between the road network. Based on the orientation information of each historical trajectory and the orientation route, multiple historical trajectories are divided into multiple groups. The historical trajectories in each group are along the same direction. Therefore, the road network topology-based method provided in this invention can accurately realize the orientation division of historical trajectories.

[0013] Secondly, the method of this invention achieves the orientation division of historical trajectories by comparing each historical trajectory with a directional route. It utilizes a precise directional route as a benchmark for orientation division, thus ensuring the accuracy of the method and meeting the refined requirements of abnormal behavior analysis. Furthermore, existing trajectory similarity-based division processes suffer from the problem of abnormal driving patterns due to the large number of historical trajectories. These abnormal historical trajectories can affect the accuracy of existing similarity-based division results to varying degrees. The precise directional route-based division process in this invention avoids the influence of abnormal historical trajectories on the division results, thereby improving the accuracy of the method. Therefore, the orientation division of historical trajectories based on road network topology in this invention meets the refined requirements of abnormal behavior analysis.

[0014] In addition to the location segmentation of historical trajectories, to further obtain behavioral patterns with similar driving routes, it is necessary to mine behavioral patterns based on the location information of the trajectories. The key to behavioral pattern mining based on multi-frame historical trajectories lies in discovering the trajectories frequently traversed by moving objects. This invention has discovered that the behavior of moving objects often exhibits a certain periodicity, that is, moving objects always follow the same movement trajectory to a greater or lesser extent within a specific time interval. Therefore, obtaining the periodic behavioral patterns of moving targets has important application value for behavior analysis. Thus, this invention proposes a clustering algorithm based on trajectory location information, taking into account the location information of multi-frame historical trajectories to mine behavioral patterns. Specifically, in order to obtain the periodicity of moving targets, this invention... Averaging multiple historical trajectories (averaging multiple trajectory sequences with the same location information) is performed to obtain the trajectory pattern at that location. After calculating the trajectory average vector of multiple historical trajectories within the same category, further steps such as trajectory rotation and trajectory clustering are taken to obtain the historical trajectory behavior pattern mining result. This result represents the correct driving direction of the historical trajectory on a certain road segment. This process fully utilizes the directional information of multiple historical trajectories to obtain the periodic behavior pattern of the moving target, i.e., the correct driving direction of the multiple historical trajectories on a certain road segment, ensuring the fine-grained requirements of behavior pattern segmentation. Furthermore, based on the behavior patterns obtained that meet the fine-grained requirements, behavior detection in real-time operation can be performed, effectively improving the accuracy of abnormal behavior detection.

[0015] In this invention, trajectory rules are also set in the area corresponding to traffic rules (i.e., trajectory rules are set at specific positions in the video frames of the area to be detected according to traffic rules). These rules include time period information, target type information, and information that visualizes the specific rules. Thus, when making judgments based on the judgment rules of this invention, accuracy is ensured, and the type, time, and location of the abnormal behavior can be determined.

[0016] In summary, the behavior patterns of historical trajectories obtained by the method of this invention meet the requirements of refinement. Furthermore, by combining the behavior patterns of historical trajectories with traffic rules, rule-based abnormal behavior detection and behavior pattern-based abnormal behavior detection algorithms determine when, where, and what kind of abnormal behavior occurred in the target.

[0017] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1a This is a schematic diagram of the geographic spatial distribution pattern of the target trajectory.

[0019] Figure 1b This is a schematic diagram of the spatial distribution pattern of the target trajectory image.

[0020] Figure 2 This is a schematic diagram of trajectory orientation determination according to the present invention.

[0021] Figure 3 This is a schematic diagram of the clustering algorithm process based on trajectory orientation information of the present invention.

[0022] Figure 4 This is a diagram showing the correspondence between the rule image space and geographic space of this invention.

[0023] Figure 5 This is a multi-level discrimination relationship diagram of the rules and trajectories of the present invention.

[0024] Figure 6 This is a flowchart of the violation analysis method of the present invention.

[0025] Figure 7 This is a schematic diagram of the test scenario in this embodiment.

[0026] Figure 8 This is a schematic diagram of the trajectory behavior pattern mining process in this embodiment.

[0027] Figure 9a This is a schematic diagram of the image space for judging abnormal behavior in this embodiment.

[0028] Figure 9b This is a geospatial diagram illustrating the abnormal behavior judgment in this embodiment.

[0029] Figure 10a This is the spatiotemporal trajectory diagram of abnormal behavior detected by the pressure line in this embodiment.

[0030] Figure 10b This is the spatiotemporal trajectory diagram of the abnormal behavior detected by reverse navigation in this embodiment.

[0031] Figure 10c This is the spatiotemporal trajectory diagram of prohibited abnormal behavior in this embodiment.

[0032] Figure 10d This is a spatiotemporal trajectory diagram of the deviated behavior pattern in this embodiment.

[0033] Figure 11a This is a comparison chart of the overall accuracy verification of the method in this embodiment with the three existing methods, TraM and LowF, based on accuracy.

[0034] Figure 11b This is a comparison chart of the overall accuracy verification of the method in this embodiment with the three existing methods, TraM and LowF, based on recall.

[0035] Figure 11c This is a comparison chart of the overall accuracy verification of the method in this embodiment with the three existing methods, TraM and LowF, based on F1 scores.

[0036] Figure 12a This is a comparison chart of the absolute values ​​of detection errors of the method in this embodiment with those of the three existing algorithms: TraM and LowF.

[0037] Figure 12b This is a comparison diagram of the error direction line segments of the method in this embodiment with the three existing algorithms: TraM and LowF.

[0038] Figure 13a This is a comparison chart of the target anomaly detection accuracy of the method in this embodiment with the three existing algorithms TraM and LowF based on precision.

[0039] Figure 13b This is a comparison chart of the target anomaly detection accuracy of the method in this embodiment with the three existing algorithms TraM and LowF based on recall.

[0040] Figure 13c This is a comparison chart of the target anomaly detection accuracy of the method in this embodiment with the three existing algorithms TraM and LowF based on F1 score.

[0041] Figure 14 This is a flowchart of the abnormal behavior detection method for moving targets according to the present invention.

[0042] Figure 15a This is a schematic diagram of rule-based reverse detection in this invention.

[0043] Figure 15b This is a schematic diagram of the rule-based west-to-east trajectory line detection in this invention.

[0044] Figure 15c This is a schematic diagram of the rule-based east-west trajectory line detection in this invention.

[0045] Figure 15d This is a schematic diagram of the rule-based west-to-east trajectory prohibition detection in this invention.

[0046] Figure 15e This is a schematic diagram of the rule-based east-to-west trajectory prohibition detection in this invention.

[0047] Figure 16a This is a schematic diagram of abnormal behavior detected based on behavioral patterns in this invention.

[0048] Figure 16b This is a schematic diagram of normal behavior detected based on behavioral patterns in this invention.

[0049] Figure 17a This is a schematic diagram of the heterogeneity detection rules in this embodiment.

[0050] Figure 17b This is a schematic diagram of the wire clamping detection rules in this embodiment.

[0051] Figure 17c This is a schematic diagram of the entry prohibition detection rules in this embodiment.

[0052] Figure 17d This is a schematic diagram of the rules for detecting deviations from normal behavior patterns in this embodiment. Detailed Implementation

[0053] The technical solution of the present invention will be clearly and completely described below with reference to specific embodiments. However, those skilled in the art should understand that the embodiments described below are only for illustrating the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example of a method for detecting abnormal behavior of moving targets in traffic scenarios

[0055] The mobile target abnormal behavior detection method in this embodiment first extracts trajectory orientation information based on the road network topology and introduces a clustering algorithm for historical target trajectories from a spatial perspective, taking into account the orientation information of multiple frames of historical trajectories to mine behavioral patterns. Second, to achieve intelligent and automated abnormal behavior detection and to complete refined analysis of real-time and historical trajectories, a comprehensive detection method for target violation behavior based on the "3W1H" framework is designed. Finally, based on actual road rules, abnormal behavior is set according to traffic rules. Combining the behavioral patterns of historical trajectories and traffic rules, rule-based abnormal behavior detection and behavior pattern-based abnormal behavior detection algorithms determine when, where, and what kind of abnormal behavior the target has committed.

[0056] The key to behavior pattern mining based on multi-frame historical trajectories lies in discovering the frequently traversed routes of moving objects. However, the behavior of moving objects often exhibits a certain periodicity, meaning that they tend to follow the same motion trajectory to varying degrees within specific time intervals. Obtaining the periodic behavioral patterns of moving targets has significant application value for behavior analysis. To obtain more refined behavior patterns, it is necessary to finely segment the historical trajectories using road network information. This embodiment considers that moving targets in videos typically move along specific routes or are heavily constrained by their surrounding environment during movement; they are restricted moving objects that move along specific routes, such as rail transit equipment, highway transportation equipment, and other moving equipment with fixed routes. Therefore, the orientation of historical trajectories can be segmented based on the road network topology that restricts the target's movement. Based on this, a trajectory orientation segmentation method based on road network topology is proposed, which obtains the orientation routes of segmented video scenes based on the road network topology. i Roadline i The spatial distribution of the trajectory sequence in image space and geographic space is as follows: Figure 1a as well as Figure 1b As shown.

[0057] Directions and routes i The video area is divided into multiple regions according to the road network. Assuming there are two-way roads in the video scene, then the Roadline... i The system needs to be able to divide the two-way road to meet the granular requirements of abnormal behavior analysis, with different roads... i The video scene's roads will be divided into multiple regions, and different division methods will produce different results. Furthermore, Roadline... i The historical trajectory sequence of multiple video frames is divided into different trajectory categories, such as Figure 2 By calculating the relationship between each trajectory point and the roadline i The starting point T1(x1,y1) and the ending point T n (x n ,y n The cross product value O n Determine the relationship between multiple historical trajectory sequences and the Roadline i The orientation relationship is calculated as shown in Equation (1), and the algorithm flow is shown in Table 1.

[0058]

[0059] Table 1

[0060]

[0061] Based on the road network topology, the relationship between each historical trajectory and the roadline can ultimately be obtained. i The location information of the trajectory is used to identify historical trajectories with similar location information. These trajectories can be classified as a unified category, i.e., moving targets with similar routes traveling in the same direction. To further obtain behavioral patterns of trajectories with similar routes, it is necessary to mine these behavioral patterns based on the location information of the trajectory. Therefore, this embodiment proposes a clustering algorithm (OtraClus) based on trajectory location information. The algorithm obtains the results of historical trajectory behavioral pattern mining through steps such as trajectory average vector calculation, trajectory rotation, and trajectory clustering. The process is as follows: Figure 3 .

[0062] The trajectory average vector calculation process in this embodiment is as follows:

[0063] Assuming the multi-frame trajectory sequence is The vectors of the trajectory sequence are respectively And for Tra i Find the average vector of the vectors As shown in equation (2).

[0064]

[0065] Among them, |V n | is a vector The model, (V) xn V yn ) is the trajectory vector of the nth line. The coordinates.

[0066] The calculation process for trajectory coordinate rotation is as follows:

[0067] calculate The angle θ between the vector and the x-axis unit vector n As shown in formula (3). The trajectory sequence is arranged according to... Rotation θ n The angles are as shown in formulas (4) and (5).

[0068]

[0069] x'=xcosθ n +ysinθ n (4)

[0070] y'=ycosθ n -xsinθ n (5)

[0071] Where (x,y) are the coordinates of the multi-frame trajectory sequence before rotation, and (x',y') are the coordinates of the trajectory after rotation.

[0072] The process of trajectory clustering and rotation to the original coordinates is as follows:

[0073] The rotated trajectory is scanned at equal intervals along the average vector direction, and the set Inter of n intersection points between the scan line and each trajectory is obtained. i ={(sx1',sy1'),(sx2',sy2'),…,(sx n ',sy n According to formulas (6) and (7), the coordinate set of the clustered trajectories is obtained.

[0074]

[0075] Rotate the clustered trajectories back to their original coordinates to obtain the result set of clustered target trajectories. There are multiple, such as equations (8) and (9), which represent the behavior pattern of the target's historical trajectory at a certain location, i.e., the correct driving direction of the multi-frame historical trajectory on a certain road segment.

[0076] cx n =cx' n ×cos(π+θ)-cy n '×sin(π+θ) (8)

[0077] cy n =cy' n ×sin(π+θ)+cy n '×cos(π+θ) (9)

[0078] To obtain high-precision target abnormal behavior detection results, this embodiment proposes a comprehensive detection of target violation behavior based on "3W1H" on the basis of behavior pattern mining. Abnormal behavior is set according to actual traffic rules, and rule-based abnormal behavior analysis and behavior pattern-based abnormal behavior analysis are realized by combining the set rules.

[0079] Setting abnormal behaviors based on traffic rules should comprehensively consider the needs of the traffic scene within the camera's field of view, defining the time period (when), target type (who), rule area (where), and violation behavior setting (how). First, establish the mapping relationship between traffic rules in image space and geographic space; second, set surface rules P at specific locations in the image space of the video frame. r =P r {(Px1,Py1),(Px2,Py2),(Px3,Py3),(Px4,Py4)} and line rule L r ={(Lx s ,Ly s ),(Lx e ,Lye )}, P r and L r The corresponding rule start time ST start and termination time ET start The rule targets the type Type; finally, P is determined according to formula (10). r and L r The image space is converted to geographic space to obtain the surface rule P. gr ={(PX1,PY1),…,(PX n ,PY n )} and line rule L gr ={(LX s ,LY s ),(LX e ,LY e The behavior patterns of the historical trajectories of multiple targets. Convert to geospatial Where λ is a non-zero factor. Taking the setting of three rules—prohibiting wrong-way driving, prohibiting crossing the line, and prohibiting entry—as an example, the rules are set and mapped to the geographic space, as shown below. Figure 4 .

[0080]

[0081] Based on trajectory behavior pattern mining and traffic rule-based settings, anomaly behavior detection surface rules P can be obtained. r Line rule L r and behavioral patterns Based on this, this embodiment designs a comprehensive detection method for violations based on the "when-where-who-how" 3W1H violation analysis. The essence of "3W1H" is to determine abnormal behavior based on the multi-level discriminative relationship between the target trajectory and the multi-layered semantic information of the trajectory rules, such as... Figure 5 Among them, P r and L r The corresponding rule start time ST start and termination time ET end The target type Type and P that the rules are aimed at r L r and Corresponding geographic space P gr L gr and Assuming a camera N Each frame N Contains multiple target trajectories and trajectory type The time point at which the abnormal behavior occurred was The trajectory type is obtained in real time by training a YOLOv5 model.

[0082] Based on the above rules, the 3W1H method is used to comprehensively determine whether a violation occurred within a given time frame, the target type of the violation, and its traffic violation type. First, video frames are acquired through video. Second, considering the needs of the traffic scene within the camera's field of view, the time period (when), target type (who), rule area (where), and violation rule (how) are set. Third, three types of abnormal traffic behaviors—crossing the line, driving against traffic, prohibited entry, and deviation from normal patterns—are detected in the geographic space. Finally, information on abnormal traffic behaviors of the target in the video is obtained, as follows: Figure 6 .

[0083] The rule-based abnormal behavior analysis process in this embodiment includes: assumptions and For any two trajectories that do not conform to each other, the trajectory type is: L r To prohibit driving against traffic or crossing the line, P r To enforce the prohibition of entry rule, a three-level rule is applied to any trajectory in the video. The first and second levels of rules are necessary for judging different abnormal behaviors. The third level of rules requires different methods to be used depending on the geometric relationship between the rule and the trajectory. The reverse movement detection process is shown in Table 2, the line crossing detection process is shown in Table 3, and the prohibition of entry detection process is shown in Table 4.

[0084] The primary discrimination rule is: the time of the latest point in the trajectory. Whether it is within the scope of the rules, i.e.

[0085] The secondary discrimination rule is: target trajectory type Whether it is consistent with the target type of the set rule, i.e.

[0086] The three-level discrimination rule includes the following three cases:

[0087] 1) Reverse rule.

[0088] Determine the geometric relationship between the trajectory and the rules: Assume the starting point of the trajectory is... <x s ,y s >, the termination point is <x e ,y e >, Rule L r The starting point is <Ly s ,Ly s >, the termination point is <Lx e ,Ly eThe angle between the trajectory and the rule, Angle, is calculated as follows (11).

[0089]

[0090] If Angle < 90, the trajectory follows the same direction as the rule; otherwise, the trajectory follows the opposite direction as the rule, as shown in the table below. With L r Reverse, With L r Same direction. If the target trajectory simultaneously satisfies the first-level and second-level discrimination rules, and the result of the reverse driving rule detection in the third-level discrimination rule is Angle≥90, then the target violates the traffic rule prohibiting reverse driving and is judged as abnormal behavior.

[0091] Table 2 Reverse Detection

[0092]

[0093] 2) Line pressing rules.

[0094] Determine the geometric relationship between the trajectory and the rules: Assume the starting point of the trajectory is... <x s ,y s >, the termination point is <x e ,y e >, Rule L r The starting point is <Lx s ,Ly s >, the termination point is <Ly e ,Ly e If the trajectory intersects with the rule, i.e., Intersect>0, then the trajectory intersects with the rule, as shown in equation (12). At the same time, if the target trajectory satisfies both the first-level and second-level discrimination rules, and the detection result of the line-crossing rule in the third-level discrimination rule is Intersect>0, then the target violates the rule prohibiting line crossing, and the target is judged as abnormal behavior.

[0095] Intersect=((Ly s -y s )*(x e -x s )-(Lx s -x s )*(y e -y s ))*((Ly e -y s )*(x e -x s )-(Lx e -x s )*(y e -y s (12)

[0096] Table 3 Wire Clamping Inspection

[0097]

[0098] 3) No entry rules.

[0099] Determine the geometric relationship between the trajectory and the rules:

[0100] Let the latest point of the trajectory be... <x e ,y e > Irregular quadrilateral P r The coordinates of the four vertices are<P1(Px1,Py1),P2(Px2,Py2),P3(Px3,Py3),P4(Px4,Py4)> If any two points form a straight line P1P2, P2P3, P3P4, P1P4, then the cross product of the latest point of the trajectory and the four straight lines is cP1P2, cP2P3, cP3P4, cP1P4. The calculation formulas are as follows (13), (14), (15), (16).

[0101] cP1P2=(Px2-Px1)*(y e -Py1)-(Py2-Py1)*(x e -Px1) (13)

[0102] cP2P3=(Px3-Px2)*(y e -Py2)-(Py3-Py2)*(x e -Px2) (14)

[0103] cP3P4=(Px4-Px3)*(y e -Py3)-(Py4-Py3)*(x e -Px3) (15)

[0104] cP1P4=(Px1-Px4)*(y e -Py4)-(Py1-Py4)*(x e -Px4) (16)

[0105] If cP1P2>0&&cP2P3>0&&cP3P4>0&&cP1P4>0 or cP1P2<0&&cP2P3<0&&cP3P4<0&&cP1P4<0, then the latest point of the trajectory is located on the regular surface P. rInside. Simultaneously, if the target trajectory simultaneously satisfies both the first-level and second-level discrimination rules, and the detection result of the prohibition-entry rule in the third-level discrimination rules is cP1P2>0&&cP2P3>0&&cP3P4>0&&cP1P4>0 or cP1P2<0&&cP2P3<0&&cP3P4<0&&cP1P4<0, then the target violates the prohibition-entry rule and is judged as exhibiting abnormal behavior.

[0106] Table 4 Prohibited Entry for Testing

[0107]

[0108] The rule-based abnormal behavior analysis described in this embodiment can detect abnormal behaviors such as going against the flow, crossing lines, and violating no-entry rules. This embodiment also performs abnormal behavior analysis based on behavior pattern mining. The abnormal behavior analysis process based on behavior patterns in this embodiment also includes a three-level rule judgment on any trajectory in the video. The first and second level discrimination rules are the same as in the rule-based abnormal behavior analysis process, while the third level discrimination rule is based on the similarity between the behavior pattern and the trajectory, as shown in Table 5.

[0109] The process for determining the similarity between behavioral patterns and trajectories in this embodiment is as follows:

[0110] First, the behavioral patterns of the target historical trajectory are obtained using the OtraClus clustering algorithm in this embodiment, i.e., the results of trajectory sequence clustering. The real-time trajectory sequence of the moving target is

[0111] Assuming that turning is prohibited under normal driving conditions on a certain road section, the real-time trajectory of a certain target can be used as a reference. With behavioral patterns The similarity is used to determine whether it is abnormal behavior. If the similarity is low, the target is not traveling along the normal route. Considering that the Hausdorff distance algorithm has better real-time performance, this embodiment uses the Hausdorff distance algorithm to measure... and The similarity of trajectories between them, among which and There exist two one-way Hausdorff distances and one absolute Hausdorff distance, the one-way ones being... arrive as well as arrive The one-way Hausdorff distance is calculated using formulas (17) and (18), and the absolute Hausdorff distance is the final similarity metric, as shown in formula (19). Where H(Tra icTra i The smaller the value, the less similar the two trajectory sequences are, meaning the real-time trajectory is less likely to resemble a normal behavior pattern, thus indicating abnormal behavior. Furthermore, if the target trajectory simultaneously satisfies both the first-level and second-level discrimination rules, and the third-level discrimination rule indicates that the similarity between the real-time trajectory and the behavior pattern's arc is less than the set standard, then the target deviates from the normal behavior pattern and exhibits abnormal behavior.

[0112]

[0113] H(Tra i cTra i )=max{h(cTra i Tra i ),h(Tra i cTra i )} (19)

[0114] Where h(cTra) i Tra i )≠h(Tra i cTra i That is, it does not satisfy the symmetry of the distance metric, d(a n ,b n ) is usually defined as a n and b n The Euclidean distance between them.

[0115] Table 5. Trajectory Deviation from Normal Pattern

[0116]

[0117] To address the problem that existing traffic anomaly detection methods do not fully utilize real-time and historical trajectories, and that most of these trajectories are located in image space, resulting in a lack of real-world location and low accuracy, this embodiment proposes a moving target anomaly behavior detection method (TraB) in traffic scenarios. By introducing historical trajectory location segmentation and behavior pattern mining algorithms from a geospatial perspective, more refined historical trajectory behavior pattern mining can be achieved. Based on trajectory behavior patterns and established traffic rules, anomaly behavior detection from a geospatial perspective is achieved using a comprehensive detection method based on the "3W1H" target violation behavior framework. The "3W1H" stands for "when-where-who-how". "When" represents the time period during which the abnormal behavior occurs, and the start and end times of the rule-based video analysis validity period. "Where" represents the location on the map, such as the area where a line, point, or polygon element is located. "Who" represents the target type, mainly including motor vehicles and pedestrians (including pedestrians on foot and those riding non-motorized vehicles). "How" represents the type of abnormal behavior, including normal behavior patterns obtained by setting traffic rules and mining moving target behavior patterns. The "3W1H" method focuses on defining four traffic behavior detection rules: Line pressure detection (target trajectory intersects with the warning line / area) (LD), Retrograde detection (target direction is opposite to the warning line direction) (RD), No-entry detection (target is within the warning area) (ND), and Deviation from normal mode detection (vehicle deviates from the normal driving trajectory) (DD).

[0118] TraB (Traffic Target Abnormal Behavior Detection Method) mainly consists of two parts: behavior pattern mining based on trajectory clustering and comprehensive detection of target violation behavior based on the "3W1H" framework. The process is as follows: Figure 14 .

[0119] (1) Behavioral pattern mining based on trajectory clustering.

[0120] In addressing the issue that existing abnormal behavior analysis algorithms rarely consider the collaborative analysis of historical and real-time trajectories in geographic space, this embodiment extracts trajectory orientation information based on the topological relationship of the road network and introduces a clustering algorithm for target historical trajectories from a spatial perspective, taking into account the orientation information of multiple frames of historical trajectories to mine their behavioral patterns.

[0121] (2) Comprehensive detection of target violations based on “3W1H”.

[0122] To achieve intelligent and automated anomaly detection and to perform refined analysis of real-time and historical trajectories, a comprehensive detection method for target violation behavior based on the "3W1H" framework was designed. Based on actual road rules, abnormal behavior is defined according to traffic rules. Combining historical trajectory behavior patterns with traffic rules, rule-based and behavior pattern-based anomaly detection algorithms determine when, where, and what type of abnormal behavior occurred in the target.

[0123] To verify the accuracy and effectiveness of the method in this embodiment, the following verification process is performed on the method in this embodiment:

[0124] 1) Experimental design.

[0125] This experiment uses Visual Studio and PyCharm as the development environment, C# and Python as the development languages, and relies on the Emcuv visual image processing library. The experimental setting is a traffic scene in an urban area; all videos are large outdoor scenes with an image resolution of 1280×720 and a height of 20 meters. Figure 7 .

[0126] Precision, recall, and F1 score were used as evaluation metrics to verify the accuracy of object detection. The calculations were performed as shown in formulas (20), (21), and (22). The results of the four object detection algorithms were recorded every 30 frames. The object detection results were compared on two scales: single-frame accuracy and overall accuracy. The robustness of the algorithms was evaluated using the mean Mp, Mr, MF, variance Vp, Vr, VF, and mean error Mn of the three evaluation metrics.

[0127]

[0128] Where TP represents the number of correctly identified abnormal behaviors, FP represents the number of incorrectly identified abnormal behaviors, FN represents the number of abnormal behaviors identified as normal behaviors, and TN represents the number of normal abnormal behaviors in the experimental scenario.

[0129] 2) Trajectory behavior pattern mining.

[0130] Trajectory data was randomly acquired every 50 frames of experimental data, comprising 40 frames containing 652 trajectories. Trajectories with fewer than 100 points were removed. The algorithm of this embodiment was used to mine the behavioral patterns of historical trajectories. Taking historical trajectories on a two-way road in the experimental scenario as an example, the trajectory orientation was divided to obtain the behavioral patterns of the trajectories. The results are shown in Table 6.

[0131] Table 6 Trajectory Behavior Patterns

[0132]

[0133] Based on the topological relationships between road networks, i.e., the dividing lines of two-way lanes in the experimental data, the directional information of the trajectory sequences is obtained, and different colors are used to indicate whether the trajectory sequence belongs to the left or right lane. In this embodiment, yellow and blue are used to mark whether the trajectory sequence belongs to the left or right lane and whether the direction is from west to east or from east to west. Using the spatial clustering algorithm of this embodiment, trajectories with similar driving routes are clustered to further obtain the behavior patterns of different roads, where the blue line segment represents the behavior pattern of two-way lanes, such as... Figure 8 .

[0134] 3) Abnormal behavior detection.

[0135] Abnormal behavior detection rules are set for video scenes, including rule-based abnormal behavior rules (reverse driving detection, line crossing detection, and entry prohibition detection) and behavior pattern-based abnormal behavior rules (behavior patterns of historical trajectories), as shown in Table 7. Combined with the real-time trajectory of the target, abnormal behavior of the target in the video is detected in real time.

[0136] Table 7. Exception Rule Settings

[0137]

[0138] Select any frame from the video, and based on the two main categories of rules mentioned above, determine any abnormal behavior in the real-time trajectory, and obtain the time, location, and event of the abnormal behavior. For example... Figure 9a as well as Figure 9b Targets with IDs 24 and 56 violated the rule prohibiting entry at time 2022-11-01 12:55:02 and coordinates (12698321.5380338,3779564.25452946), and were judged as abnormal behavior.

[0139] 4) Single-frame accuracy evaluation of abnormal behavior.

[0140] The spatiotemporal trajectory and rules of any target anomaly within a video frame are selected and displayed in the same coordinate system. This provides a more intuitive representation of the timing of the anomalous behavior in geographic space, as well as the geometric and topological relationships between the trajectory and rules at any given time. Figure 10a , Figure 10b , Figure 10c as well as Figure 10d .

[0141] To verify the accuracy of the TraB (TraTarget Abnormal Behavior Detection) method for detecting abnormal behavior in traffic scenarios, precision (P), recall (R), and F1 score (F1) were used as evaluation metrics. The accuracy of TraB, TraM, and LowF algorithms for detecting four types of abnormal behavior—line crossing (LD), wrong-way driving (RD), prohibited entry (ND), and deviation from normal behavior patterns (DD)—was obtained. Compared to the TraM algorithm, TraB improved the precision, recall, and F1 score for LD, RD, ND, and DD by 4.461%-14.285%, 10.952%-18.367%, and 8.755%-13.157%, respectively. Compared to the LowF algorithm, TraB improved the precision, recall, and F1 score for LD, RD, ND, and DD by 21.428%-25%, 12.698%-30.672%, and 15.151%-27.472%, respectively. Figure 11a , Figure 11b as well as Figure 11c .

[0142] Anomaly detection results were recorded every 50 frames of the video, resulting in 1000 data points. 100 data points were randomly selected for single-frame accuracy verification. The number of actual anomalies was randomly counted for every 10 data points, and this was combined with the sum of the anomaly detection counts from the TraB method, the moving target trajectory-based method (TraM) (Chen 2022), and the low-level video feature-based method (LowF) (Chandrakala2022) of this embodiment. The 10 results from the randomized count of the 100 data points were labeled [1, 10]. The results are as follows: Figure 12a TN represents the actual number of anomalies that occurred in the experimental scenario. It can be seen that the result of TraB algorithm's persistence of abnormal behavior, TP, is closer to TN. Figure 12b The box plot compares the average difference in the number of anomaly detections between TraB, TraM, and LowF algorithms and TN. It can be seen that: (1) the median and mean difference between TP and TN for TraB algorithm are smaller than those for TraM and LowF; (2) the mean error of TraB algorithm is reduced by 20%-66.67% and 20%-83.33% respectively compared with TraM and LowF. This indicates that the TraB algorithm in this embodiment has better accuracy in single frame accuracy.

[0143] 5) Overall robustness assessment of abnormal behavior.

[0144] Anomaly detection was performed on the real experimental data collected in the above experimental design of this embodiment. The anomaly detection results of three methods—TraB (Tra-based), TraM (Tra-based), and Low-level video feature-based (LowF)—were statistically analyzed every 10 minutes. Precision (P), recall (R), and F1 score (F1) were used as evaluation metrics. Figure 13a , Figure 13b as well as Figure 13c The results show that the average precision (P) of the TraB, TraM, and LowF algorithms are 83.28%, 75.83%, and 65.47%, respectively; the average recall (R) is 84.41%, 74.8%, and 67.01%, respectively; and the average F1 score (F1) is 83.72%, 75.25%, and 66.01%, respectively. Compared with the TraM and LowF algorithms, the TraB algorithm increases the mean precision (MP) by 11.385%-21.383%, the mean recall (MR) by 14.085%-20.615%, and the mean F1 score (MF1) by 12.817%-21.147%, demonstrating the robustness of the TraB algorithm in this embodiment.

[0145] The variances VP, VR, and VF, and the standard deviations SP, SR, and SF of the multi-frame calculation results for each indicator were obtained to further verify the robustness of the TraB algorithm in this embodiment. The results are shown in Table 8. It can be seen that the variances VP, VR, and VF, and the standard deviations SP, SR, and SF of the precision (P), recall (R), and F1 score (F1) of the abnormal behavior detection extracted by the TraB algorithm are all smaller than those of the moving target trajectory-based method (TraM) and the low-level video feature-based method (LowF), indicating that the TraB algorithm has high robustness. Specifically, the variances VP, VR, and VF of the TraB algorithm are reduced by 6.47%-52.82%, 48.27%-55.95%, and 51.99%-84.71% respectively, and the standard deviations SP, SR, and SF are reduced by 14.26%-28.26%, 28.09%-51.73%, and 30.72%-60.9% respectively, compared to TraM and LowF.

[0146] Table 8 Comparison of anomaly detection variances VP, VR, VF and standard deviations SP, SR, SF

[0147]

[0148] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for detecting abnormal behavior of a moving target in a traffic scene, characterized in that, The method comprises the following steps: 1) obtaining historical trajectories of different target types in a detection area, the target types including vehicles and pedestrians; 2) According to the topological relationship between the road network, the orientation route of the video scene corresponding to the segmented detection area is obtained, the cross product value of the trajectory point on each historical trajectory and the vector formed by the starting point to the terminal point of the orientation route is calculated, the orientation information of each historical trajectory and the orientation route is judged according to whether the number of cross product values greater than 0 is more than the number of cross product values less than 0, the historical trajectories with the same orientation information under the same target type are divided into the same group of orientation information, the multi-frame trajectory sequence in each group of orientation information is grouped, the multi-frame trajectory sequence with the same position information is divided into the same group of position information, the sequence average vector of the multi-frame trajectory sequence in the same group of position information is obtained, and the angle between the sequence average vector and the horizontal axis direction is obtained , and all the multi-frame trajectory sequences in the same group of position information are rotated by an angle of the direction of the sequence average vector to obtain the rotated trajectory, the rotated trajectory is scanned according to the direction of the sequence average vector, the intersection set of the scanning line and each trajectory is obtained, the intersection set of each trajectory is clustered to obtain the clustered trajectory coordinate set, the clustered trajectory is rotated to the original coordinate to obtain the clustered result set of the target trajectory, and the clustered result set is taken as the normal behavior mode of the historical trajectory of the target. ​ setting trajectory rules at specific positions of video frames in the detection area according to traffic rules, the trajectory rules including time period information, target type information and image space information, the image space information including face rules and line rules; 3) obtaining motion trajectory information of real-time moving targets in the detection area, and comparing the motion trajectory information with normal behavior patterns to determine whether the real-time moving targets deviate from normal operation states; judging whether the real-time moving targets violate traffic rules according to whether the motion trajectory information of the real-time moving targets meets the time period information and the target type information, and a geometric relationship between the motion trajectory information of the real-time moving targets and the face rules and the line rules.

2. The method of claim 1, wherein, In step 3), when the line rule is a line pressing prohibition rule, if the motion trajectory information of the real-time moving targets meets the time period information and the target type information, and there is an intersection between the motion trajectory information and the line rule, the real-time moving targets violate the line pressing prohibition rule.

3. The method of claim 1, wherein, In step 3), when the line rule is a reverse direction prohibition rule, if the motion trajectory information of the real-time moving targets meets the time period information and the target type information, and the direction of the motion trajectory information is opposite to the direction of the line rule, the real-time moving targets violate the reverse direction prohibition rule.

4. The method of claim 1, wherein, In step 3), when the face rule is an entry prohibition rule, if the motion trajectory information of the real-time moving targets meets the time period information and the target type information, and the real-time moving targets are in the range of the face rule, the real-time moving targets violate the entry prohibition rule.

5. The method of claim 1, wherein, In step 3), the Hausdorff distance algorithm is used to compare the similarity with the normal behavior patterns.

6. The method of claim 1, wherein, In step 2), the sequence mean vector is calculated The calculation formula is: ； in, For vectors The model, For the nth trajectory vector The coordinates.

7. The method of claim 6, wherein, In step 2), the multi-frame trajectory sequence is rotated by an angle of The calculation formula of the rotated trajectory is: ； ； where (x, y) is the coordinate of the multi-frame trajectory sequence before rotation, and (x', y') is the coordinate of the trajectory after rotation. , ) is the coordinate of the trajectory after rotation.

8. The method of claim 7, wherein, In step 2), the intersection set of each trajectory is clustered to obtain a clustered trajectory coordinate set The calculation formula is: ； ； wherein is the set of intersection points of the scan line with each trajectory.

9. The method of claim 8, wherein, In step 2), the clustered trajectories are rotated to the original coordinates to obtain a result set of the clustered target trajectories The calculation formula is: ； 。

Images