Method for detecting abnormal behavior of moving target in traffic scene

Abstract

The invention belongs to the technical field of road traffic safety detection, and discloses a method for detecting abnormal behaviors of a moving target in a traffic scene, which is characterized in that track azimuth information extraction is completed based on a topological relation of a road network, and a clustering algorithm of a target historical track in a spatial view angle is introduced. Mining a behavior mode by considering azimuth information of a multi-frame historical track; secondly, in order to realize intelligence and automation of abnormal behavior detection and complete fine analysis of cooperation of a real-time track and a historical track, a target illegal behavior comprehensive detection method based on 3W1H is designed. And finally, abnormal behavior setting based on traffic rules is completed according to actual road rules, and the time, the place and the abnormal behavior of the target are judged by combining a behavior mode of a historical track and the traffic rules, abnormal behavior detection based on the rules and an abnormal behavior detection algorithm based on the behavior mode.

Description

Technical Field

[0001] The present invention relates to the technical field of road traffic safety detection, and in particular to a method for detecting abnormal behavior of a moving target in a traffic scene. Background Art

[0002] With the acceleration of urbanization, urban road networks are crisscrossed, and human behavior is becoming increasingly complex. Traffic monitoring systems are playing an increasingly important role in security-related fields with real-time high-definition video, increasingly sophisticated intelligent analysis functions, and low price advantages. Taking the moving targets contained in the key frame images taken by fixed cameras as the research object, and the behavioral analysis and understanding of the moving targets as the research purpose, the moving targets in the key frame images are detected, tracked, and analyzed, and the motion characteristics and behavior patterns of the moving targets are intelligently obtained, which has important practical significance for ensuring public safety. Among them, abnormal behavior detection of moving targets is one of the fundamental purposes of the field of computer vision, and it is an important way to try to make computers "open their eyes to see the world". In particular, abnormal behavior detection of moving targets in traffic scenes has extremely high research value both from the perspective of computer vision theory research and the level of artificial intelligence.

[0003] Abnormal behavior detection of mobile targets is to judge the abnormal behavior of different types of dynamic targets such as vehicles and pedestrians on the road when they enter and exit the spatiotemporal field of view of a specific camera. It is an important means to achieve "video security". Traditional video surveillance data can often only be used as "evidence of post-investigation". In order to achieve "pre-prevention" or "problem solving during the event", it is necessary to use the relevant theories and methods of abnormal behavior detection of mobile targets. In the case of almost no human intervention, the trajectory of the mobile target is automatically extracted through the image frame sequence collected by the camera, and the relevant behavior of the target is analyzed and judged on this basis, and finally an explanation of the behavior of the mobile target in the traffic scene is formed. The extraction of mobile target trajectories in traffic scenes includes two types: historical trajectories and real-time trajectories. The historical trajectory is a set of coordinate sequences formed when multiple frames of mobile targets enter and exit the field of view; the real-time trajectory is a coordinate sequence formed when any frame of mobile targets enter and exit the field of view. Fully mining the historical trajectory can obtain the movement law and behavior pattern of the target, which can provide a more refined analysis of abnormal behavior, and combining the historical trajectory and real-time trajectory can provide a more intelligent analysis of abnormal behavior. Therefore, how to combine historical and real-time trajectories for intelligent video analysis can provide regulatory authorities with more intelligent abnormal behavior analysis information and a more refined observation perspective, reducing the difficulty of intelligent analysis of abnormal behavior of targets in traffic scenarios.

[0004] However, most of the existing trajectory orientation division methods focus on trajectory similarity and cannot meet the refined requirements of abnormal behavior analysis. In order to further obtain the behavior patterns of trajectories with similar driving routes, it is necessary to mine their behavior patterns based on the trajectory orientation information. The existing trajectory behavior pattern mining mostly uses trajectory clustering algorithms, that is, each type of trajectory sequence obtained by clustering is regarded as the behavior pattern of the trajectory. The traditional target trajectory clustering algorithm treats multiple trajectories as a whole to form a complex trajectory set, and divides the trajectory set into multiple trajectory clusters according to the similarity algorithm, which cannot meet the refined requirements of abnormal behavior analysis for behavior pattern division. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method for detecting abnormal behavior of mobile targets in traffic scenarios, so as to solve the problem that the existing method of dividing the existing trajectory orientation according to similarity and the process of mining trajectory behavior patterns using the existing trajectory clustering algorithm cannot meet the requirements of abnormal behavior analysis for refined behavior pattern division.

[0006] To achieve the above purpose, the method of the present invention completes the extraction of trajectory orientation information based on the topological relationship of the road network, and takes into account the orientation information of multiple frames of historical trajectories to mine their behavior patterns. The method includes the following steps:

[0007] 1) Obtaining historical trajectories of different target types in the area to be detected, wherein the target types include vehicles and pedestrians;

[0008] 2) According to the topological relationship between the road networks, the orientation route of the video scene corresponding to the segmented detection area is obtained, the orientation information of each historical trajectory and the orientation route is determined, and the historical trajectories with the same orientation information under the same target type are divided into groups with the same orientation information. The multi-frame trajectory sequences of the historical trajectories in each group with the same orientation information are grouped, and the multi-frame trajectory sequences with the same position information are divided into groups with the same position information. The average vector of the sequence is obtained for the multi-frame trajectory sequences in the group with the same position information, and the angle θ between the average vector of the sequence and the horizontal axis direction is obtained. n And rotate all multi-frame trajectory sequences in the same group of position information according to the direction of the sequence average vector θ n The rotated trajectory is obtained by an angle, and the rotated trajectory is scanned equidistantly in the direction of the sequence average vector to obtain a set of intersections between the scan line and each trajectory. The intersection set of each trajectory is clustered to obtain a set of clustered trajectory coordinates. The clustered trajectory is rotated to the original coordinates to obtain a set of clustered target trajectory results, and the clustered result set is used as the normal behavior mode of the target's historical trajectory.

[0009] Setting a trajectory rule at a specific position of a video frame in the area to be detected according to traffic rules, wherein the trajectory rule includes time period information, target type information, and image space information, wherein the image space information includes surface rules and line rules;

[0010] 3) Obtain the motion trajectory information of the real-time moving target in the area to be detected, compare the similarity with the normal behavior pattern, and determine whether the real-time moving target deviates from the normal operating state;

[0011] Whether the real-time moving target violates traffic rules is determined based on whether the moving trajectory information of the real-time moving target satisfies the time period information and the target type information, as well as the geometric relationship between the moving trajectory information of the real-time moving target and the surface rules and line rules.

[0012] The method of the present invention has the following advantages: in order to obtain a more detailed behavior pattern of the target, it is necessary to combine the road network information to finely divide the historical trajectory. First, the method of the present invention takes into account that the moving target in the video usually moves along a specific route or is greatly constrained by the surrounding environment during the movement, that is, it is a restricted mobile object that moves along a specific route, such as rail transportation equipment, road transportation equipment, and other sports equipment with fixed routes. Therefore, the orientation division of the historical trajectory can be completed according to the road network topological relationship that limits the moving orientation of the target. Based on this, the present invention determines the correct driving trajectory of a certain section of road by adopting the topological relationship based on the road network, that is, the orientation route of the video scene corresponding to the segmented area to be detected is obtained according to the topological relationship between the road networks, and multiple historical trajectories are divided into multiple groups according to the orientation information of each historical trajectory and the orientation route. The historical trajectories in each group have the same direction. Therefore, the road network topological relationship provided in the present invention can accurately realize the orientation division of historical trajectories.

[0013] Secondly, the method of the present invention realizes the azimuth division of historical trajectories by comparing each historical trajectory with the azimuth route. It uses the accurate azimuth route as a benchmark to divide the azimuth of historical trajectories. Therefore, the accuracy of the division method is guaranteed, and it can meet the refined requirements of abnormal behavior analysis. In addition, in the existing division process for trajectory similarity, there is a situation where the historical trajectory has abnormal driving due to the large number of historical trajectories. The abnormal driving historical trajectory will more or less lead to the accuracy of the results of the existing similarity-based division process. The process of division based on accurate azimuth routes in the present invention avoids the influence of abnormal driving historical trajectories on the division results, and thus the accuracy of the division method in the present invention is improved. Therefore, the process of completing the azimuth division of historical trajectories based on the road network topology relationship in the present invention meets the refined requirements of abnormal behavior analysis.

[0014] In addition, after completing the azimuth division of historical trajectories, in order to further obtain the behavior pattern of trajectories with similar driving routes, it is necessary to mine the behavior pattern based on the trajectory azimuth information, and the key to mining the behavior pattern based on multi-frame historical trajectories is to find the trajectory routes that the mobile object frequently passes through. The present invention finds that the behavior of the mobile object often shows a certain periodic law, that is, the mobile object always follows the same motion trajectory more or less within a specific time interval. Therefore, obtaining the periodic behavior law of the mobile target has important application value for behavior analysis. Therefore, the present invention proposes a clustering algorithm based on trajectory azimuth information, taking into account the azimuth information of multi-frame historical trajectories to mine its behavior pattern. Specifically, in order to obtain the periodic law of the mobile target, the present invention mines the behavior pattern of the azimuth information of the multi-frame historical trajectories. The multi-frame historical trajectory is averaged (i.e., the multi-frame trajectory sequence with the same position information is averaged) to obtain the running trajectory law at the position, and after calculating the trajectory average vector of the multi-frame historical trajectory under the same category, the trajectory rotation and trajectory clustering steps are continued to obtain the historical trajectory behavior pattern mining results, which is the correct driving direction of the historical trajectory on a certain road section. This process first averages the multi-frame trajectory information, and then rotates, clusters and rotates to the original coordinates based on the average processing result after the average processing, making full use of the orientation information of the multi-frame historical trajectory, and obtains the periodic behavior law of the mobile target, that is, the correct driving direction of the multi-frame historical trajectory on a certain road section, ensuring the refinement requirements for the behavior pattern division. Then, based on the behavior pattern that meets the refinement requirements, the real-time operation process is detected, which can effectively improve the accuracy of abnormal behavior detection.

[0015] The present invention also sets trajectory rules on the area corresponding to the traffic rules (i.e., sets trajectory rules at specific positions of the video frame of the area to be detected according to the traffic rules), and the rules include time period information, target type information, and information of the specific rules in the form of images. Therefore, when making judgments based on the judgment rules of the present invention, it ensures accuracy while also being able to obtain what type of mobile target the abnormal behavior is, when and where the abnormal behavior occurs.

[0016] In summary, the behavior pattern of the historical trajectory obtained by the method of the present invention meets the refinement requirements, and by combining the behavior pattern of the historical trajectory with traffic rules, the rule-based abnormal behavior detection and the behavior pattern-based abnormal behavior detection algorithm determine when and where the target has what kind of abnormal behavior.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites a preferred embodiment and describes it in detail with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1a It is a schematic diagram of the geographic spatial distribution pattern of target trajectories.

[0019] Figure 1b It is a schematic diagram of the spatial distribution pattern of the target trajectory image.

[0020] Figure 2 It is a schematic diagram of trajectory orientation judgment of the present invention.

[0021] Figure 3 It is a schematic diagram of the clustering algorithm process based on trajectory orientation information of the present invention.

[0022] Figure 4 It is a corresponding relationship diagram between the regular image space and the geographic space of the present invention.

[0023] Figure 5 It is a multi-level discriminant relationship diagram of the rules and trajectories of the present invention.

[0024] Figure 6 It is a flow chart of the violation analysis method of the present invention.

[0025] Figure 7 Schematic diagram of the test scenario of this embodiment.

[0026] Figure 8 Schematic diagram of the trajectory behavior pattern mining process of this embodiment.

[0027] Figure 9a Schematic diagram of image space for abnormal behavior judgment in this embodiment.

[0028] Figure 9b It is a geographic space schematic diagram of abnormal behavior judgment in this embodiment.

[0029] Fig.10a This is a spatiotemporal trajectory diagram of abnormal behavior of line pressing detection in this embodiment.

[0030] Fig.10b This is a spatiotemporal trajectory diagram of abnormal behavior detected in retrograde direction in this embodiment.

[0031] Fig.10c This embodiment prohibits entry into the abnormal behavior time-space trajectory diagram.

[0032] Fig.10d It is a spatiotemporal trajectory diagram of the behavior pattern that deviates from the normal in this embodiment.

[0033] Fig.11a This is a comparison chart of the overall accuracy verification based on the accuracy rate of the method in this embodiment and the three existing TraM and LowF methods.

[0034] Fig.11b This is a comparison chart of the overall accuracy verification based on the recall rate of the method in this embodiment and the three existing TraM and LowF methods.

[0035] Fig.11c This is a comparison chart of the overall accuracy verification based on the F1 score of the method in this embodiment and the three existing TraM and LowF methods.

[0036] Fig.12a 3 is a comparison chart of the absolute values ​​of detection errors of the method of this embodiment and three existing algorithms of TraM and LowF.

[0037] Figure 12b It is a comparison diagram of error line segment diagrams of the method of this embodiment and three existing TraM and LowF algorithms.

[0038] Fig.13a This is a comparison chart of target anomaly detection accuracy based on precision rate between the method of this embodiment and the three existing TraM and LowF algorithms.

[0039] Fig.13b This is a comparison chart of target anomaly detection accuracy based on recall rate between the method of this embodiment and three existing algorithms of TraM and LowF.

[0040] Fig.13c This is a comparison chart of target anomaly detection accuracy based on F1 score between the method of this embodiment and three existing algorithms of TraM and LowF.

[0041] Fig.14 It is a flow chart of the abnormal behavior detection method of a mobile target of the present invention.

[0042] Fig.15a It is a schematic diagram of rule-based retrograde detection in the present invention.

[0043] Fig.15b It is a schematic diagram of line pressure detection of a west-to-east trajectory based on rules in the present invention.

[0044] Fig.15c It is a schematic diagram of line pressure detection of the east-to-west trajectory based on rules in the present invention.

[0045] Fig.15d It is a schematic diagram of the no-entry detection based on the rule of the west-to-east trajectory in the present invention.

[0046] Fig.15e It is a schematic diagram of the no-entry detection of the east-to-west trajectory based on the rule in the present invention.

[0047] Fig.16a It is a schematic diagram of detecting abnormal behavior based on the behavior pattern in the present invention.

[0048] Fig.16b It is a schematic diagram when normal behavior is detected based on the behavior pattern in the present invention.

[0049] Fig.17a Schematic diagram of the heterosexuality detection rule of this embodiment.

[0050] Fig.17b Schematic diagram of the wire pressing detection rule of this embodiment.

[0051] Fig.17c It is a schematic diagram of the prohibited entry detection rule of this embodiment.

[0052] Fig.17d It is a schematic diagram of the detection rules of the behavior pattern that deviates from the normal state in this embodiment. DETAILED DESCRIPTION

[0053] The technical scheme of the present invention will be clearly and completely described below in conjunction with specific embodiments, but those skilled in the art should understand that the embodiments described below are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Embodiment of a method for detecting abnormal behavior of a moving target in a traffic scenario

[0055] The method for detecting abnormal behavior of mobile targets in this embodiment first completes the extraction of trajectory orientation information based on the topological relationship of the road network, and introduces a clustering algorithm for the target's historical trajectory from a spatial perspective, taking into account the orientation information of multiple frames of historical trajectories to mine its behavior pattern; secondly, in order to realize the intelligence and automation of abnormal behavior detection and complete the refined analysis of the coordination of real-time trajectory and historical trajectory, a comprehensive detection method for target violation behavior based on "3W1H" is designed. Finally, according to the actual road rules, the abnormal behavior setting based on traffic rules is completed, and combined with the behavior pattern of historical trajectory and traffic rules, the rule-based abnormal behavior detection and the abnormal behavior detection algorithm based on behavior pattern determine when and where the target has what kind of abnormal behavior.

[0056] Among them, the key to mining behavioral patterns based on multi-frame historical trajectories is to discover the trajectory routes that mobile objects frequently pass through. However, the behavior of mobile objects often shows certain periodic laws, that is, mobile objects always follow the same motion trajectory more or less within a specific time interval. Obtaining the periodic behavior laws of mobile targets has important application value for behavioral analysis. In order to obtain more detailed behavioral patterns of the target, it is necessary to finely divide the historical trajectories in combination with road network information. This embodiment takes into account that the mobile targets in the video usually move along a specific route or are greatly constrained by the surrounding environment during the movement, that is, they are restricted mobile objects that move along a specific route, such as rail transit equipment, highway traffic equipment, and other sports equipment with fixed routes. Therefore, the azimuth division of the historical trajectory can be completed according to the road network topological relationship that restricts the target's moving orientation. Based on this, a trajectory azimuth division method based on the road network topological relationship is proposed, and the azimuth route Roadline of the segmented video scene is obtained according to the topological relationship of the road network. i , where Roadline i The spatial distribution of trajectory sequences in image space and geographic space is as follows Figure 1a as well as Figure 1b shown.

[0057] DirectionRoadline i The video area is divided into multiple areas according to the road network. Assuming there is a two-way road in the video scene, Roadline i The two-way road must be divided to meet the refined requirements of abnormal behavior analysis. i The road in the video scene will be divided into multiple areas, and different division methods will produce different results. i The historical trajectory sequences of multiple video frames are divided into different trajectory categories, such as Figure 2 By calculating the relationship between each trajectory point and Roadline i The starting point T1(x1,y1) and the ending point T n (x n ,y n )'s cross product value O n , determine the relative position of multiple historical trajectory sequences to Roadline i The calculation method is as shown in formula (1), and the algorithm flow is shown in Table 1.

[0058]

[0059] Table 1

[0060]

[0061] Based on the road network topology, each historical trajectory can be obtained relative to the Roadline i The historical trajectories with the same azimuth information can be regarded as trajectories of the same category, that is, moving targets with similar driving routes along the same direction. In order to further obtain the behavior patterns of trajectories with similar driving routes, it is necessary to mine their behavior patterns based on the azimuth information of the trajectories. Therefore, this embodiment proposes a clustering algorithm (OtraClus) based on trajectory azimuth information. The results of historical trajectory behavior pattern mining are obtained through the steps of trajectory average vector calculation, trajectory rotation, and trajectory clustering. The process is as follows: Figure 3 .

[0062] The trajectory average vector calculation process in this embodiment is:

[0063] Assume that the multi-frame trajectory sequence is Then the vectors of the trajectory sequence are And to Tra i Find the average vector of the vector As shown in formula (2).

[0064]

[0065] Among them, |V n | is a vector The module (V xn ,V yn ) is the nth trajectory vector The coordinates of .

[0066] The calculation process of trajectory coordinate rotation is:

[0067] calculate The angle θ with the x-axis unit vector n , as shown in formula (3). The direction of rotation is θ n angle, as shown in formulas (4) and (5).

[0068]

[0069] x'=xcosθ n +ysinθ n (4)

[0070] y'=ycosθ n -xsinθ n (5)

[0071] Where (x, y) is the coordinate of the multi-frame trajectory sequence before rotation, and (x', y') is the coordinate of the trajectory after rotation.

[0072] The process of trajectory clustering and rotation to the original coordinates is:

[0073] Scan the rotated trajectory equidistantly in the direction of the average vector and obtain the set of n intersections between the scan line and each trajectory Inter i ={(sx1',sy1'),(sx2',sy2'),…,(sx n ',sy n ')}, according to formulas (6) and (7), we get the coordinate set of the clustered trajectory

[0074]

[0075] Rotate the clustered trajectories to the original coordinates to obtain the result set of target trajectory clustering For multiple, such as equations (8) and (9), the behavior pattern of the historical trajectory of the target at a certain position, that is, the correct driving position of multiple frames of historical trajectories on a certain road section.

[0076] xx n =cx' n ×cos(π+θ)-cy n '×sin(π+θ) (8)

[0077] cy n =cy' n ×sin(π+θ)+cy n '×cos(π+θ) (9)

[0078] In order to obtain high-precision target abnormal behavior detection results, based on behavior pattern mining, this embodiment proposes a comprehensive detection of target violation behaviors based on "3W1H", completes the abnormal behavior setting according to actual traffic rules, and combines the set rules to realize rule-based abnormal behavior analysis and behavior pattern-based abnormal behavior analysis.

[0079] The abnormal behavior setting based on traffic rules should comprehensively consider the needs of traffic scenes within the camera's field of view, set the time period (when), target type (who), rule area (where), and violation setting (how). First, establish the mapping relationship between traffic rules in image space and geographic space; second, set the surface rule P in the image space at a specific position of the video frame. r =P r {(Px1,Py1),(Px2,Py2),(Px3,Py3),(Px4,Py4)} and line rule L r ={(Lx s ,Ly s ),(Lx e ,Lye )},P r and L r The corresponding rule start time ST start and end time ET start , the target type Type of the rule; finally, according to formula (10), P r and L r Convert image space to geographic space to get the surface rule P gr ={(PX1,PY1),…,(PX n ,PY n )} and line rule L gr ={(LX s ,LY s ),(LX e ,LY e )}. The behavior patterns of the historical trajectories of multiple targets Convert to Geospatial Among them, λ is a non-zero factor. Taking the three rules of prohibiting reverse driving, prohibiting crossing the line and prohibiting entry as an example, the rules are set and mapped to the geographic space, as shown in the figure. Figure 4 .

[0080]

[0081] According to trajectory behavior pattern mining and traffic rules, the abnormal behavior detection surface rule P can be obtained. r , Line Rule L r and behavioral patterns On this basis, this embodiment designs a comprehensive detection method based on the "when-where-who-how" 3W1H violation analysis. The essence of "3W1H" is to judge abnormal behavior based on the multi-level discriminant relationship between the multi-layer semantic information of the target trajectory and the trajectory rules, such as Figure 5 Among them, P r and L r The corresponding rule start time ST start and end time ET end , the target type Type of the rule, and P r , L r and The corresponding geographic space P gr , L gr and Assume that the camera N Every frame N Contains multiple target tracks And the trajectory type The time when the abnormal behavior occurs is The trajectory type is obtained in real time through the YOLOV5 training model.

[0082] Based on the above rules, the 3W1H method is combined to comprehensively judge whether the violation occurs within a given time range, the target type of the violation and the type of traffic violation. First, the video frame is obtained through the video Video; secondly, the needs of the traffic scene within the camera's field of view are comprehensively considered, and the time period (when), target type (who), rule area (where), and violation rules (how) are set; thirdly, three types of traffic abnormal behaviors, such as crossing the line, driving in the opposite direction, prohibited entry, and deviation from the normal mode, are detected in the geographic space; finally, the abnormal behavior information of the traffic target in the video is obtained. The process is as follows Figure 6 .

[0083] The rule-based abnormal behavior analysis process of this embodiment includes: assuming and For any two trajectories with inconsistent directions, the trajectory type is L r To prohibit the reverse or prohibit the line rule, P r For the no-entry rule, any track in the video is judged by three-level rules. The first-level and second-level rules are necessary processes for judging different abnormal behaviors. The third-level rules need to be judged by different methods according to the different geometric relationships between the rules and the tracks. The retrograde detection process is shown in Table 2, the line-pressing detection process is shown in Table 3, and the no-entry detection process is shown in Table 4.

[0084] The first-level judgment rule is: the time of the latest point of the trajectory Is it within the scope of the rules?

[0085] The secondary discrimination rules are: target trajectory type Is it consistent with the target type of the set rule, that is,

[0086] The three-level discrimination rules include the following three situations:

[0087] 1) Retrograde rules.

[0088] Determine the geometric relationship between the trajectory and the rule: Assume that the starting point of the trajectory is <x s ,y s >, the end point is <x e ,y e >, Rule L r The starting point is <Ly s ,Ly s >, the end point is <Lx e ,Ly e>, the angle Angle between the trajectory and the rule is calculated as follows (11).

[0089]

[0090] If Angle < 90, the trajectory is in the same direction as the rule, otherwise the trajectory is in the opposite direction of the rule, then the following table With L r Reverse, With L r Same direction. If the target trajectory satisfies both the first-level and second-level discrimination rules, and the detection result of the reverse rule in the third-level discrimination rule is Angle ≥ 90, then the target violates the traffic rule prohibiting reverse driving and is judged as abnormal behavior.

[0091] Table 2 Retrograde detection

[0092]

[0093] 2) Line pressing rules.

[0094] Determine the geometric relationship between the trajectory and the rule: Assume that the starting point of the trajectory is <x s ,y s >, the end point is <x e ,y e >, Rule L r The starting point is <Lx s ,Ly s >, the end point is <Ly e ,Ly e >, if the trajectory and the rule have an intersection, that is, Intersect>0, then the trajectory intersects with the rule, as shown in formula (12). At the same time, if the target trajectory satisfies both the first-level discrimination rule and the second-level discrimination rule, and the result of the line-pressing rule detection in the third-level discrimination rule is Intersect>0, then the target violates the rule of prohibiting line-pressing, and the target is judged to be abnormal behavior.

[0095] Intersect=((Ly s -y s )*(x e -x s )-(Lx s -x s )*(y e -y s ))*((Ly e -y s )*(x e -x s )-(Lx e -x s )*(y e -y s )) (12)

[0096] Table 3 Wire pressing detection

[0097]

[0098] 3) No entry rules.

[0099] Determine the geometric relationship between the trajectory and the rule:

[0100] Let the latest point of the trajectory be <x e ,y e >, irregular trapezoid P r The coordinates of the four vertices are<P1(Px1,Py1),P2(Px2,Py2),P3(Px3,Py3),P4(Px4,Py4)> . If any two points form a straight line P1P2, P2P3, P3P4, P1P4, then the cross product of the latest point of the trajectory and the four straight lines is cP1P2, cP2P3, cP3P4, cP1P4, and the calculation formulas are as follows (13), (14), (15), (16).

[0101] cP1P2=(Px2-Px1)*(y e -Py1)-(Py2-Py1)*(x e -Px1) (13)

[0102] cP2P3=(Px3-Px2)*(y e -Py2)-(Py3-Py2)*(x e -Px2) (14)

[0103] cP3P4=(Px4-Px3)*(y e -Py3)-(Py4-Py3)*(x e -Px3) (15)

[0104] cP1P4=(Px1-Px4)*(y e -Py4)-(Py1-Py4)*(x e -Px4) (16)

[0105] If cP1P2>0&&cP2P3>0&&cP3P4>0&&cP1P4>0 or cP1P2<0&&cP2P3<0&&cP3P4<0&&cP1P4<0, then the latest point of the trajectory is located on the regular surface P rAt the same time, if the target trajectory satisfies both the first-level discrimination rule and the second-level discrimination rule, and the detection result of the no-entry rule in the third-level discrimination rule is cP1P2>0&&cP2P3>0&&cP3P4>0&&cP1P4>0 or cP1P2<0&&cP2P3<0&&cP3P4<0&&cP1P4<0, then the target violates the no-entry rule and is judged to be an abnormal behavior.

[0106] Table 4 Prohibited entry detection

[0107]

[0108] The above rule-based abnormal behavior analysis of this embodiment can detect whether there are abnormal behaviors such as driving against traffic, crossing the line, and violating the no-entry rule. This embodiment can also analyze abnormal behaviors based on behavior patterns based on behavior pattern mining. The abnormal behavior analysis process based on behavior patterns in this embodiment also includes the determination of three-level rules for any trajectory in the video, where the first-level discrimination rules and the second-level discrimination rules are the same as the rule-based abnormal behavior analysis process, and the third-level discrimination rules need to be determined based on the similarity between the behavior pattern and the trajectory, as shown in Table 5.

[0109] The process of determining the similarity between the behavior pattern and the trajectory in this embodiment is as follows:

[0110] First, the OtraClus clustering algorithm of this embodiment is used to obtain the behavior pattern of the target historical trajectory, that is, the trajectory sequence clustering The real-time trajectory sequence of the moving target is

[0111] Assuming that a certain road section is in a normal driving state and turning is prohibited, the real-time trajectory of a target can be and behavior patterns The similarity is used to determine whether it is an abnormal behavior. If the similarity is small, the target does not follow the normal route. Considering that the Hausdorff distance algorithm has good real-time performance, this embodiment uses the Hausdorff distance algorithm to measure and The trajectory similarity between and There are two one-way Hausdorff distances and one absolute Hausdorff distance. The one-way ones are arrive as well as arrive The one-way Hausdorff distance of is calculated as follows (17) and (18), and the absolute Hausdorff distance is the final similarity metric, as shown in formula (19). i,cTra i ) is smaller, indicating that the two trajectory sequences are less similar, that is, the less the real-time trajectory is close to the normal behavior pattern, it can be judged as abnormal behavior. At the same time, if the target trajectory satisfies both the first-level discrimination rule and the second-level discrimination rule, and the result of the third-level discrimination rule is that the similarity between the real-time trajectory and the behavior pattern arc is less than the set standard, then the target deviates from the normal behavior pattern and has abnormal behavior.

[0112]

[0113] H(Tra i ,cTra i )=max{h(cTra i ,Tra i ),h(Tra i ,cTra i )} (19)

[0114] Where h(cTra i ,Tra i )≠h(Tra i ,cTra i ), that is, it does not satisfy the symmetry of the distance metric, d(a n ,b n ) is usually defined as a n and b n The Euclidean distance between .

[0115] Table 5 Trajectory deviation from normal mode

[0116]

[0117] In view of the problem that the existing abnormal traffic behavior detection does not make full use of real-time and historical trajectories, and most of them are in image space, resulting in the lack of real orientation and low accuracy of abnormal traffic behavior detection, this embodiment proposes a method for abnormal behavior detection of mobile targets in traffic scenarios (TraB), which introduces the historical trajectory orientation division and behavior pattern mining algorithm from a geographic space perspective to complete more refined historical trajectory behavior pattern mining; and based on the trajectory behavior pattern and the set traffic rules, with the help of a comprehensive detection method for target violations based on "3W1H", completes abnormal behavior detection from a geographic space perspective. The meaning of "3W1H" is "when-where-who-how", that is, when represents the time period when the abnormal behavior occurs, the start time and end time of the validity period of the rule video analysis; where represents the location in the map, such as the area where the line, point or surface element is located; who represents the target type, mainly including motor vehicles and pedestrians (including walking pedestrians and pedestrians riding non-motor vehicles); how represents the type of abnormal behavior, which includes normal behavior patterns set according to traffic rules and mined through mobile target behavior patterns. The "3W1H" method focuses on limiting four traffic behavior detection rules, namely line pressure detection (target trajectory intersects with the warning line / surface) (LD, Line pressure detection), retrograde detection (target direction is opposite to the warning line direction) (RD, Retrograde detection), no-entry detection (target is within the warning surface) (ND, No-entry detection) and deviation from normal mode detection (vehicle deviates from the normal driving track) (DD, Deviation from normal mode detection).

[0118] TraB (abnormal behavior detection method for mobile targets in traffic scenarios) mainly includes two parts, namely, behavior pattern mining based on trajectory clustering and comprehensive detection of target violations based on "3W1H". The process is as follows: Fig.14 .

[0119] (1) Behavioral pattern mining based on trajectory clustering.

[0120] In the existing abnormal behavior analysis algorithm, the problem of collaborative analysis of historical trajectories and real-time trajectories in geographic space is rarely considered. The present embodiment extracts trajectory orientation information based on the topological relationship of the road network, introduces a clustering algorithm for the target historical trajectory from a spatial perspective, and takes into account the orientation information of multiple frames of historical trajectories to mine their behavior patterns.

[0121] (2) Comprehensive detection of target violations based on “3W1H”.

[0122] In order to realize the intelligent and automated detection of abnormal behavior and complete the refined analysis of the real-time trajectory and historical trajectory collaboration, a comprehensive detection method for target violation behavior based on "3W1H" is designed. According to the actual road rules, the abnormal behavior setting based on traffic rules is completed. Combined with the behavior pattern of historical trajectory and traffic rules, the rule-based abnormal behavior detection and behavior pattern-based abnormal behavior detection algorithms determine when and where the target has behaved abnormally.

[0123] In order to verify the accuracy and effectiveness of the method of this embodiment, the method of this embodiment is subjected to the following inspection process:

[0124] 1) Experimental design.

[0125] This experiment uses Visual studio and pycharm software as the development environment, C# and python as the development language, and relies on the Emgucv visual image processing library. A certain urban traffic road scene is used as the experimental site. The videos are all large outdoor scenes with an image resolution of 1280×720 and a height of 20 meters. Figure 7 .

[0126] The precision, recall and F1 score are used as evaluation indicators to verify the accuracy of target detection. The calculations are as shown in formulas (20), (21) and (22). The results of the four target detection algorithms are recorded every 30 frames, and the target detection results are compared from the two scales of single-frame accuracy and overall accuracy. At the same time, the mean Mp, Mr, MF, variance Vp, Vr, VF and error mean Mn of the three evaluation indicators are used to evaluate the robustness of the algorithm.

[0127]

[0128] Among them, TP represents the correct identification of abnormal behaviors, FP represents the number of abnormal behaviors identified incorrectly, FN represents the number of abnormal behaviors identified as normal behaviors, and TN represents the number of abnormal behaviors identified as normal in the experimental scenario.

[0129] 2) Trajectory behavior pattern mining.

[0130] Randomly obtain 40 frames of trajectory data every 50 frames of experimental data, and remove the trajectory data of 40 frames, where 40 frames contain 652 trajectories, and remove the trajectories with less than 100 trajectory points. The algorithm of this embodiment is used to mine the behavior pattern of historical trajectories. Taking the historical trajectory in the two-way road in the experimental scenario as an example, the trajectory orientation is divided to obtain the behavior pattern of the trajectory. The results are shown in Table 6.

[0131] Table 6 Trajectory behavior patterns

[0132]

[0133] According to the topological relationship between the road networks, that is, the dividing line of the two-way lanes in the experimental data, the orientation information of the trajectory sequence is obtained, and different colors are used to edit the trajectory sequence to indicate whether it belongs to the left or right side. In this embodiment, yellow and blue are used to mark the trajectory sequence as belonging to the left or right lane, and the direction is from west to east or from east to west. The clustering algorithm under the spatial perspective of this embodiment is used to cluster the trajectories with similar driving routes, and further obtain the behavior patterns of different roads, where the blue line segment is the behavior pattern of the two-way lane, such as Figure 8 .

[0134] 3) Abnormal behavior detection.

[0135] Abnormal behavior detection rules are set for video scenes, including rule-based abnormal behavior rules (reverse detection, line-crossing detection, and no-entry detection) and behavior pattern-based abnormal behavior rules (behavior patterns of historical trajectories), as shown in Table 7. Combined with the real-time trajectory of the target, the abnormal behavior of the target in the video is detected in real time.

[0136] Table 7 Exception rule settings

[0137]

[0138] Select any frame in the video, and judge the abnormal behavior of the real-time trajectory according to the above two categories of rules, and obtain the time, location and event of the target abnormal behavior. Figure 9a as well as Figure 9b , the targets with target IDs 24 and 56 violated the no-entry rule at time 2022-11-0112:55:02 and coordinate point (12698321.5380338,3779564.25452946) and were judged as abnormal behavior.

[0139] 4) Single-frame accuracy evaluation of abnormal behavior.

[0140] Select the spatiotemporal trajectory and rules of any target in the video frame, and display the trajectory and rules in the same coordinate system to more intuitively show the time when the abnormal behavior occurs in the geographic space, as well as the geometric topological relationship between the moment trajectory and the rules, such as Fig.10a , Fig.10b , Fig.10c as well as Fig.10d .

[0141] In order to verify the accuracy of abnormal behavior detection method (TraB) for each type of abnormal behavior in traffic scenarios, the precision (P), recall (R) and F1 score (F1) are used as evaluation indicators to obtain the accuracy of TraB, TraM and LowF algorithms for four types of abnormal behavior detection: line crossing detection (LD), reverse detection (RD), no entry detection (ND) and deviation from normal behavior pattern (DD). Compared with the TraM algorithm, the precision, recall and F1 scores of TraB for LD, RD, ND and DD are improved by 4.461%-14.285%, 10.952%-18.367% and 8.755%-13.157%, respectively. Compared with the LowF algorithm, the precision, recall and F1 scores of TraB for LD, RD, ND and DD are improved by 21.428%-25%, 12.698%-30.672% and 15.151%-27.472%, respectively. Fig.11a , Fig.11b as well as Fig.11c .

[0142] The abnormal behavior detection results of the video were recorded every 50 frames, and a total of 1000 data were obtained. 100 data were randomly selected for single-frame accuracy verification, and the number of real abnormal behaviors of every 10 data were randomly counted, as well as the sum of the number of abnormal behavior detections of the TraB method of this embodiment, the method based on moving target trajectory (TraM) (Chen 2022) and the method based on low-level video features (LowF) (Chandrakala 2022). The 10 results after random statistics of 100 data were marked as [1,10]. The results are as follows Fig.12a ,TN is the actual number of anomalies that occur in the experimental scenario. It can be seen that the result TP of the abnormal behavior persistence of the TraB algorithm is closer to TN. Figure 12b The box plot is a comparison of the average differences in the number of abnormal detections between the three algorithms TraB, TraM, and LowF and TN. It can be seen that: (1) the median and mean of the differences between TP and TN of the TraB algorithm are smaller than those of TraM and LowF; (2) the mean error of the TraB algorithm is reduced by 20%-66.67% and 20%-83.33% relative to that of TraM and LowF, respectively. This shows that the TraB algorithm of this embodiment has better accuracy in single-frame accuracy.

[0143] 5) Overall robustness evaluation of abnormal behavior.

[0144] The real experimental data collected in the above experimental design of this embodiment are used for abnormal behavior detection. The abnormal behavior detection results of the mobile target abnormal behavior detection method (TraB) in traffic scenes, the method based on mobile target trajectory (TraM) and the method based on low-level video features (LowF) are counted every 10 minutes, and the precision rate (P), recall rate (R) and F1 score (F1) are used as evaluation indicators. Fig.13a , Fig.13b as well as Fig.13c The results show that the average precision (P) of the TraB algorithm, TraM algorithm and LowF algorithm are 83.28%, 75.83% and 65.47% respectively, the average recall (R) is 84.41%, 74.8% and 67.01% respectively, and the average F1 score (F1) is 83.72%, 75.25% and 66.01% respectively. Compared with the TraM and LowF algorithms, the average precision (MP) of the TraB algorithm increases by 11.385%-21.383%, the average recall (MR) increases by 14.085-20.615%, and the average F1 score (MF1) increases by 12.817%-21.147%, indicating the robustness of the TraB algorithm of this embodiment.

[0145] The calculation results of multiple frames of various indicators are used to obtain the variances VP, VR, VF and the standard deviations SP, SR, SF to further verify the robustness of the TraB algorithm of this embodiment. The results are shown in Table 8. It can be seen that the variances VP, VR, VF and the standard deviations SP, SR, SF of the precision (P), recall (R) and F1 score (F1) of abnormal behavior detection extracted by the TraB algorithm are smaller than those of the method based on moving target trajectory (TraM) and the method based on low-level video features (LowF), indicating that the TraB algorithm has high robustness. Among them, the variances VP, VR, VF of the TraB algorithm are reduced by 6.47%-52.82%, 48.27%-55.95% and 51.99%-84.71% respectively compared with the variances VP, VR, VF of TraM and LowF, and the standard deviations SP, SR, SF are reduced by 14.26%-28.26%, 28.09%-51.73% and 30.72%-60.9% respectively.

[0146] Table 8 Comparison of anomaly detection variance VP, VR, VF and standard deviation SP, SR, SF

[0147]

[0148] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for detecting abnormal behavior of a moving target in a traffic scenario, characterized in that: The steps include: 1) Obtaining historical trajectories of different target types in the area to be detected, wherein the target types include vehicles and pedestrians; 2) According to the topological relationship between the road networks, the orientation route of the video scene corresponding to the segmented detection area is obtained, the orientation information of each historical trajectory and the orientation route is determined, and the historical trajectories with the same orientation information under the same target type are divided into groups with the same orientation information. The multi-frame trajectory sequences of the historical trajectories in each group with the same orientation information are grouped, and the multi-frame trajectory sequences with the same position information are divided into groups with the same position information. The average vector of the sequence is obtained for the multi-frame trajectory sequences in the group with the same position information, and the angle θ between the average vector of the sequence and the horizontal axis direction is obtained. n And rotate all multi-frame trajectory sequences in the same group of position information according to the direction of the sequence average vector θ n The rotated trajectory is obtained by an angle, and the rotated trajectory is scanned equidistantly in the direction of the sequence average vector to obtain a set of intersections between the scan line and each trajectory. The intersection set of each trajectory is clustered to obtain a set of clustered trajectory coordinates. The clustered trajectory is rotated to the original coordinates to obtain a set of clustered target trajectory results, and the clustered result set is used as the normal behavior mode of the target's historical trajectory. Setting a trajectory rule at a specific position of a video frame in the area to be detected according to traffic rules, wherein the trajectory rule includes time period information, target type information, and image space information, wherein the image space information includes surface rules and line rules; 3) Obtain the motion trajectory information of the real-time moving target in the area to be detected, and compare the similarity with the normal behavior pattern to determine whether the real-time moving target deviates from the normal operating state; Whether the real-time moving target violates traffic rules is determined based on whether the moving trajectory information of the real-time moving target satisfies the time period information and the target type information, as well as the geometric relationship between the moving trajectory information of the real-time moving target and the surface rules and line rules.

2. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 1, characterized in that: In step 3), when the line rule is a prohibition line crossing rule, if the motion trajectory information of the real-time moving target satisfies the time period information and the target type information, and there is an intersection with the line rule, the real-time moving target violates the prohibition line crossing rule.

3. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 1, characterized in that: In step 3), when the line rule is a prohibition of retrograde rule, if the motion trajectory information of the real-time moving target satisfies the time period information and the target type information, and is opposite to the direction of the line rule, the real-time moving target violates the prohibition of retrograde rule.

4. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 1, characterized in that: In step 3), when the surface rule is a no-entry rule, if the motion trajectory information of the real-time moving target satisfies the time period information and the target type information and is within the range of the surface rule, the real-time moving target violates the no-entry rule.

5. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 1, characterized in that: In step 3), the similarity is compared with the normal behavior pattern using the Hausdorff distance algorithm.

6. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 1, characterized in that: In step 2), the cross product value of the vector formed by the trajectory point on each historical trajectory and the starting point to the end point of the azimuth route is calculated, and the azimuth information of each historical trajectory and the azimuth route is determined according to whether the number of cross product values ​​greater than 0 is greater than the number of cross product values ​​less than 0.

7. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 1 or 6, characterized in that: In step 2), find the average vector of the sequence The calculation formula is: Among them, |V n | is a vector The module (V xn ,V yn ) is the nth trajectory vector The coordinates of .

8. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 7, characterized in that: In step 2), the multi-frame trajectory sequence is rotated by θ according to the direction of the sequence average vector n The calculation formula of the trajectory after rotation is: x'=xcosθ n +ysinθ n ; y'=ycosθ n -xsinθ n ; Where (x, y) is the coordinate of the multi-frame trajectory sequence before rotation, and (x', y') is the trajectory coordinate after rotation.

9. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 8, characterized in that: In step 2), the intersection point set of each trajectory is clustered to obtain the clustered trajectory coordinate set The calculation formula is: Among them, i ={(sx1',sy1'),(sx2',sy2'),…,(sx n ',sy n ')} is the set of intersections between the scan line and each trajectory.

10. The method for detecting abnormal behavior of a moving target in a traffic scene according to claim 9, characterized in that: In step 2), the clustered trajectories are rotated to the original coordinates to obtain the result set of the target trajectory clustering The calculation formula is: cx n =cx' n ×cos(π+θ n )-cy n '×sin(π+θ n ); cy n =cy' n ×sin(π+θ n )+cy n '×cos(π+θ n )。

Images