Road target abnormal traffic behavior detection method based on road traffic element semantics

By semantic annotation of road traffic elements and generating semantic maps, combined with motion target detection information, the problem of inability to finely detect abnormal traffic behavior in the existing technology is solved, efficient and accurate detection of multiple road targets is achieved, and road traffic is improved Perception ability of situation.

CN119068429APending Publication Date: 2024-12-03WUHAN BOLING TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411148807.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing road traffic behavior detection methods cannot accurately identify and detect abnormal traffic behavior, especially due to the lack of high-precision image semantic segmentation of road traffic elements and the accurate distinction between dotted and solid lines, resulting in the inability to effectively detect abnormal behavior of road targets. .

Method used

By semantic annotation of road traffic elements, a semantic diagram of road traffic elements is generated, and combined with motion target detection information, the motion parameters of road targets and standard motion parameters are compared to abnormal traffic behavior.

Benefits of technology

It realizes efficient and accurate detection of various abnormal traffic behaviors of motor vehicles, non-motor vehicles or pedestrians, and improves the perception ability of road traffic situations and the utilization rate of detection data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068429A_ABST
    Figure CN119068429A_ABST
Patent Text Reader

Abstract

The invention provides a road target abnormal traffic behavior detection method based on road traffic element semantics, and the method gives semantics to selected various road traffic elements, and constructs a road traffic element semantic graph. Due to the fact that road target abnormal traffic behavior detection can be carried out by combining road semantic information and moving target detection dual information, the method overcomes the technical defect that a conventional road traffic abnormal behavior detection method only has moving target detection information and lacks road information. Therefore, various abnormal traffic behaviors of various road targets such as motor vehicles, non-motor vehicles or pedestrians can be efficiently and accurately detected, the utilization rate of detection data generated by the roadside radar and the visual perception equipment is improved, and the perception capability of the road traffic situation is further improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of behavior recognition technology, and more specifically, to a method for detecting abnormal traffic behavior of road targets based on the semantics of road traffic elements. Background Art

[0002] With the development of hardware computing power and computer vision technology, especially the development and deployment of radar and visual perception devices equipped with graphics processing units (GPUs), it is possible to efficiently, accurately and in real time detect the category, longitude and latitude, speed, heading, position in the image and length and width of the target detection box of moving targets on the road. However, relying solely on the detection results of such perception devices is not enough to perceive the traffic behavior of road targets (motor vehicles, non-motor vehicles and pedestrians, etc.), especially to finely identify and detect abnormal traffic behaviors of road targets. Existing related methods generally draw virtual lines on the edge of the lane as the basis for detecting the traffic behaviors of pedestrians, non-motor vehicles and motor vehicles. However, such methods cannot provide a basis for the detection of abnormal traffic behaviors in a fine-grained manner.

[0003] With the development of computer vision, especially deep learning theory and technology, the accuracy of image semantic segmentation technology is getting higher and higher. The IoU (intersection over union) of semantic segmentation of large categories such as streets and skies in street scene images has exceeded 90%. However, even the most advanced image semantic segmentation models cannot make high-precision image semantic segmentation of road traffic elements such as traffic markings and lanes on the road. The existing models of this type cannot even accurately distinguish between dotted lines and solid lines, yellow lines and white lines, so that they cannot further finely identify and detect abnormal traffic behaviors of road targets. Summary of the invention

[0004] In response to at least one defect or improvement need in the prior art, the present application proposes a method for detecting abnormal traffic behavior of road targets based on road traffic element semantics, which is used for more precise identification and detection of abnormal traffic behavior of road targets.

[0005] To achieve the above objectives, the present application provides a method for detecting abnormal traffic behavior of road targets based on road traffic element semantics, comprising:

[0006] Performing semantic annotation on a plurality of pre-selected road traffic elements in the acquired traffic scene image, and performing structural processing on the semantic information of the annotated road traffic elements and the resolution of the traffic scene image to obtain a road annotation structured data file;

[0007] Parsing the road annotation structured data file, obtaining an initialization semantic map based on the obtained resolution, embedding the obtained semantic information of the annotated road traffic elements into the initialization semantic map, and generating a road traffic element semantic map;

[0008] Setting standard motion parameters including maximum and minimum speeds, forward and reverse headings for each motor vehicle lane in the road traffic element semantic graph;

[0009] Acquire motion parameters of each of the road targets in the same traffic scene image, and determine the position of the target detection frame of each of the road targets in the road traffic element semantic map;

[0010] The abnormal traffic behavior detection result of the road target is obtained based on the semantic distribution data of the road traffic elements in the road traffic element semantic graph area covered by the target detection frame where the road target is located and / or the comparison result of the motion parameters of the road target with the standard motion parameters.

[0011] In general, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0012] The present application proposes a method for detecting abnormal traffic behaviors of road targets based on the semantics of road traffic elements. The method assigns semantics to various selected road traffic elements and constructs a semantic graph of road traffic elements. Because the dual information of road semantic information and moving target detection can be combined to detect abnormal traffic behaviors of road targets, the method overcomes the technical defects of conventional road traffic abnormal behavior detection methods that only have moving target detection information but lack road information, thereby being able to efficiently and accurately detect various abnormal traffic behaviors of various road targets such as motor vehicles, non-motor vehicles or pedestrians, thereby improving the utilization rate of detection data generated by roadside radar and visual perception equipment, and further improving the perception ability of road traffic conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 A core flow chart of a method for detecting abnormal traffic behavior of road targets based on road traffic element semantics provided in an embodiment of the present application;

[0015] Figure 2A detailed flow chart of a method for detecting abnormal traffic behavior of road targets based on road traffic element semantics provided in an embodiment of the present application;

[0016] Figure 3 A schematic diagram of the marking method of the first type of road traffic markings and yellow dashed and solid lines in lanes provided in an embodiment of the present application;

[0017] Figure 4 A schematic diagram of the marking method of the second type of road traffic markings and white dotted lines in lanes provided in an embodiment of the present application;

[0018] Figure 5 A schematic diagram of the marking method of the third type of road traffic markings and double yellow solid lines in lanes provided in an embodiment of the present application;

[0019] Figure 6 A schematic diagram of the marking method of the fourth type of road traffic markings and tidal lanes in the lanes provided in the embodiment of the present application;

[0020] Figure 7 A schematic diagram of detecting crossing a line, changing lanes, running a red light and lateral collision of a motor vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0022] The terms "first", "second" or "nth" in the specification, claims or drawings of the present application are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0023] As described in the background technology section of the specification, relying solely on the detection results of such sensing devices is not sufficient to perceive the traffic behavior of road targets, especially to finely identify and detect abnormal traffic behaviors of road targets. Even the most advanced image semantic segmentation models currently cannot perform high-precision image semantic segmentation of road traffic elements such as traffic markings and lanes on the road. Existing models of this type cannot even accurately distinguish between dotted lines and solid lines, yellow lines and white lines, so that they cannot further finely identify and detect abnormal traffic behaviors of road targets. In view of this, the present application proposes a method for detecting abnormal traffic behaviors of road targets based on the semantics of road traffic elements, which is used to more finely identify and detect abnormal traffic behaviors of road targets.

[0024] refer to Figure 1 and Figure 2 An embodiment of the present application proposes a method for detecting abnormal traffic behavior of road targets based on road traffic element semantics. The detection method may mainly include the following five steps.

[0025] Step 1: semantically annotate 22 pre-selected road traffic elements in the acquired traffic scene image, and perform structured processing on the semantic information of the annotated road traffic elements and the resolution of the traffic scene image to obtain a road annotation structured data file.

[0026] Common abnormal traffic behaviors of motor vehicles include: crossing the line, changing lanes, running red lights, speeding, slowing, driving in the wrong direction, illegal parking, driving in the wrong lane, and lateral collisions. Abnormal traffic behaviors of non-motor vehicles or pedestrians include: crossing the road, driving or walking on the motor vehicle lane, and running red lights. These abnormal behaviors that are very easy for the human eye to identify are difficult for machines. Even some of the current high-performance computing terminals deployed with artificial intelligence algorithms can only identify the category, speed, size, heading, position and other information of the moving target, but cannot directly and accurately identify the traffic behavior of each moving target. Therefore, this application proposes to identify the abnormal traffic behavior of road targets based on the semantic information of road traffic elements, as well as the category, speed, heading, size, position and other information of the moving target.

[0027] Before semantic annotation, the corresponding relationship between the label names and semantics of 22 road traffic elements such as traffic markings and lanes on the road must be clarified. In order to facilitate promotion and implementation, an embodiment of the present application defines the label names and semantic categories of 22 traffic markings and lanes, and more road traffic elements can be defined as needed for further refinement and supplementation.

[0028] Table 1 Correspondence and annotation method of traffic markings, lanes and semantics

[0029]

[0030]

[0031] Referring to Table 1, we can get the correspondence between traffic markings and lanes and label names and semantics. The semantics of traffic markings and lanes are integer values ​​between 0 and 255 representing different types of traffic markings and lanes. It is worth noting that for yellow dashed and solid lines and white dashed and solid lines, label names and annotation semantics are not assigned separately. Instead, the yellow dashed and solid lines borrow the label names and semantic values ​​of the yellow solid line and the yellow dashed line, and the white dashed and solid lines borrow the label names and semantic values ​​of the white solid line and the white dashed line. For a yellow dashed and solid line, the label name of the actual line part is s_y_d, the semantic value is 203, and the label name of the dashed line part is b_y_g, and the semantic value is 204. For a white dashed and solid line, the label name of the actual line part is s_w_d, the semantic value is 200, and the label name of the dashed line part is b_w_g, and the semantic value is 201.

[0032] For motor vehicle lanes, in order to better detect abnormal traffic behaviors, different semantics need to be assigned to each motor vehicle lane appearing in the RGB image, considering that the widest lane in China is a two-way ten-lane lane, and the widest lane in the world is a two-way eighteen-lane lane. Therefore, for different motor vehicle lanes in the RGB image of the traffic scene road, their label names are m_v_l01, m_v_l02 to m_v_l40 from left to right and from top to bottom, and their semantic values ​​are 64, 65 to 103 from left to right and from top to bottom, that is, in the same RGB image, up to 40 different motor vehicle lane semantics can be expressed.

[0033] Semantic annotation does not mean converting the road traffic elements in the real world into semantics in the digital world intact, but making the results of semantic annotation more conducive to the identification of traffic behaviors. An embodiment of the present application divides the 22 commonly used traffic markings and lanes (some instances of road traffic elements) into 4 categories. In view of the differentiated characteristics of these 4 categories of road traffic elements, corresponding semantic annotation methods are adopted for each type of road traffic elements in a targeted manner, so that the results of semantic annotation are more conducive to the precise identification of abnormal traffic behaviors. More specifically, the specific traffic markings and / or lanes included in these 4 categories of road traffic elements and the corresponding semantic annotation methods are discussed below.

[0034] The first type of road traffic elements: yellow dashed and solid lines, white dashed and solid lines, etc. Figure 3 As shown in the figure, for the yellow dashed and solid line traffic marking, the semantics of the solid line only covers the area where the solid line is located; in the practice of abnormal traffic behavior detection, it is not considered a violation for a vehicle to run over the dashed line and the blank area between the dashed line and the solid line. Therefore, the area where the dashed line is located and the blank area between the dashed line and the solid line are marked as dashed line semantics. Figure 3 It can also be seen that although the area where the yellow dotted line is located is not continuous, a continuous semantic area is needed to semantically annotate the discontinuous dotted line area. The semantic annotation method of the white dotted and solid line refers to the yellow dotted and solid line, but it has its own label name and semantic value. That is, the solid part of the yellow dotted and solid line is annotated according to the yellow solid line, and its dotted part and the blank area between it and the solid line are annotated according to the yellow dotted line; the solid part of the white dotted and solid line is annotated according to the white solid line, and its dotted part and the blank area between it and the solid line are annotated according to the white dotted line.

[0035] The second type of road traffic elements: lane direction dividing lines, yellow solid lines, yellow dashed lines, white solid lines, white dashed lines, stop lines and road edge lines, etc. Figure 4 As shown in the figure, for the white dotted line, the semantic annotation area is the area covered by the white dotted line. It should be noted that although the area where the white dotted line is located is not continuous, a continuous semantic area is required to semantically annotate the discontinuous dotted line area. The semantic annotation method of the yellow dotted line refers to the white dotted line, but it has its own label name and semantic value. For other categories of line markings in the second category, the semantic annotation area only covers the area where the line is located. Therefore, for the second category of line markings, whether it is a dotted line or a solid line, its semantics are annotated with a solid line, that is, a continuous semantic area is used to annotate the continuous solid line marking or the discontinuous dotted line marking. The semantic annotation area of ​​the line marking should not be larger or smaller than the line marking area, that is, the semantic area just covers the line marking area. It is worth noting that physical isolation devices such as water barriers (plastic shell obstacles used to divide the road surface or form barriers, usually a small top and large bottom structure, with holes on the top for filling water to increase weight, so they are called water barriers, while non-water-filled wooden or iron movable obstacles are called chevaux de frise. Some water barriers also have horizontal through holes to allow them to be connected through rods to form longer blocking chains or blocking walls.) need to be marked with yellow solid lines.

[0036] The third type of road traffic elements: double yellow solid lines, double yellow dashed lines, guide lines, yellow grids, stop and yield areas, detour areas, pedestrian crossings and non-motor vehicle crossings, etc. Figure 5 As shown in the figure, for the double yellow solid line marking, not only the semantics of the solid line part needs to cover the area where the solid line marking is located and be annotated according to Table 1, but also the blank area between the two solid lines needs to be annotated as the semantics of the double yellow solid line according to Table 1. For the double yellow dashed line marking, its semantic annotation method can refer to the double yellow solid line. Although it is a discontinuous dashed line marking, it is necessary to use a continuous semantic area to semantically annotate the discontinuous dashed line marking. The other categories of markings in the third category are similar to the double yellow solid line. Regardless of whether there is a blank space in its internal structure, the entire outline and the interior are semantically annotated according to the overall outline shape of the marking.

[0037] The fourth type of road traffic elements: non-motorized vehicle lanes, motorized vehicle lanes, tidal lanes, deceleration lanes and variable lanes, etc. Figure 6 As shown in the figure, for tidal lanes, only the lane part is marked, and the markings on both sides of the lane are semantically annotated according to the category of the markings: if one side is a double yellow solid line, the semantic annotation is performed according to the marking method of this type of marking; if the other side is a physical isolation device such as a water barrier, the semantic annotation is performed according to the yellow solid line in the second category. The semantic annotation method for non-motorized lanes, motor lanes, deceleration lanes and variable lanes can refer to tidal lanes. When a motor lane acts as a tidal lane, deceleration lane or variable lane, the semantics of the motor lane are not annotated, but only the semantics of the tidal lane, deceleration lane or variable lane are annotated.

[0038] The roadside radar equipment collects an RGB image containing traffic markings and lanes, so that it does not collect images containing moving road targets as much as possible, so that the traffic markings and lanes are not blocked by moving road targets. The specific process of semantic annotation is as follows: select a certain type of traffic markings or lanes in the RGB image, divide the closed area where it is located in the form of polygons, record the label name according to Table 1, and store the closed area in the form of structured data with the label name as the key and the vertex coordinates as the value. Similar data representation and storage are made for all traffic markings and lane areas related to abnormal traffic behavior detection in the RGB image. And the resolution of the RGB image, the vertex coordinates and label names of the closed areas where all traffic markings and lanes related to abnormal traffic behavior detection are located in the RGB image are stored in the same road annotation structured data file. Since the position and angle of the camera on the side of the road generally do not change, the RGB image collected by the roadside radar equipment generally only needs to be annotated once.

[0039] Since the most advanced image semantic segmentation models cannot make high-precision image semantic segmentation of traffic markings and lanes on the road, such models cannot even distinguish between dotted lines and solid lines, yellow lines and white lines. Therefore, it is impossible to achieve semantic annotation of traffic markings and lanes using only the existing deep learning model, and manual annotation is still required. However, the cost of such manual annotation is not too high. For example, the radar vision equipment for collecting road scenes is generally deployed every 100 meters or so. A 10-kilometer road only needs to manually annotate 100 RGB images collected by different devices, and the time to annotate an image is only about 5 minutes. The total time, manpower and economic expenses are not large. After such radar vision equipment is installed, debugged and put into use, its angle and position will no longer change easily. Therefore, the position of static elements such as traffic markings and lanes on the road in the collected RGB images will not change frequently. Using this manual annotation method, once the semantic annotation of traffic markings and lanes is completed, the obtained road semantic information can be used for a long time, which can be said to be once and for all.

[0040] Step 2: parse the road annotation structured data file, obtain an initialization semantic map based on the obtained resolution, embed the obtained semantic information of the annotated road traffic elements into the initialization semantic map, and generate a road traffic element semantic map.

[0041] More specifically, by parsing the road annotation structured data file, the resolution of the image is obtained, and it is initialized according to the resolution to obtain a semantic map of traffic markings and lanes with zero pixel values. By parsing the vertex coordinates and labels of each closed area stored in the structured data file, the vertex coordinates and label names of the closed areas of various traffic markings and lanes in the semantic map are obtained. Then, according to the correspondence between the label name and the semantic value in Table 1, pixel values ​​are assigned to the pixels in each closed area in the semantic map, and the pixel value is the semantic value corresponding to the label name of the closed area. By assigning values ​​to the pixels in each closed area, a semantic map of road traffic elements carrying the semantics of traffic markings and lanes is obtained.

[0042] Step 3, set standard motion parameters including maximum and minimum speeds, forward and reverse directions for each motor vehicle lane in the road traffic element semantic graph. For example, 80km / h is the maximum driving speed, 30km / h is the minimum driving speed, 0 degrees is the forward driving direction, and 180 degrees is the reverse driving direction. The corresponding parameter settings can be made according to the specific situation. This is only an example and is not limited. These pre-set motion parameters are standard motion parameters, which can be used as a reference for judgment and comparison in the following steps.

[0043] Step 4: Obtain motion parameters of each road target in the same traffic scene image, and determine the position of the target detection frame of each road target in the road traffic element semantic map.

[0044] Specifically, the category, longitude and latitude, speed, heading, position in the image, length and width of the target detection box and other moving target information of each road target on the road in the same scene detected by the roadside radar equipment is obtained and saved in the target detection structured data file in the form of structured data.

[0045] Parse the target detection structured data file to obtain the category, longitude and latitude, speed, heading, position in the image, length and width of the target detection box of each road target, and determine the position of the target detection box of each road target in the semantic graph. More specifically, Figure 2As shown in the figure, the target detection structured file is parsed to obtain the category, longitude and latitude, speed, heading, position in the image, length and width of the target detection frame and other information of each road target on the road generated by the roadside radar equipment. According to the position in the image and the length and width of the target detection frame, the position and length and width of the target detection frame in the semantic map are determined, and only the semantic map area where the lower half of the target detection frame is located is selected to reduce the possible misjudgment caused by the upper half of the target detection frame. The pre-selected area range of the target detection frame can be adjusted according to the actual situation.

[0046] Step 5: Obtain abnormal traffic behavior detection results of the road target based on the semantic distribution data of the road traffic elements in the road traffic element semantic map area covered by the target detection frame where the road target is located and / or the comparison results of the motion parameters of the road target with the standard motion parameters.

[0047] Specifically, the distribution of various semantics in the semantic graph covered by the lower half of the target detection box where each road target is located is counted. Figure 7 As shown, if the category of the current road target is a motor vehicle, and the area covered by the lower half of the target detection frame contains the semantics of the lane direction dividing line, double yellow solid line, yellow solid line, yellow dashed line, yellow dashed and solid line, white solid line, white dashed line, etc., it is judged as crossing the line; if the area covered by the lower half of the target detection frame contains the semantics of traffic markings and the semantics of two or more different lanes, it is judged as changing lanes; if the semantic area covered by the lower half of the target detection frame contains the semantics of the stop line and the semantics of the lane or traffic marking below the stop line, and the current traffic light is red, it is judged as running a red light.

[0048] When the angle between the heading of the motor vehicle and the heading of the preset lane is greater than 25 degrees, it is judged that the motor vehicle has the possibility of a lateral collision; if the semantic area covered by the lower half of the target detection frame is mainly lane semantics, the maximum and minimum speeds and headings specified in the lane are obtained according to the relationship between the lane semantics and the lane speed information to determine whether the current motor vehicle is speeding, slowing, driving in the wrong direction and / or illegally parked. If it exceeds the maximum speed, it is judged as speeding; if it is lower than the minimum speed, it is judged as slowing; if the angle between the heading of the moving target and the heading specified in the current lane is greater than 90 degrees, it is judged as driving in the wrong direction; if the speed is 0 and lasts for a certain period of time, it is judged as illegal parking; if the semantic area covered by the lower half of the target detection frame contains the semantics of the non-motor vehicle lane, it is judged as not driving in the prescribed lane.

[0049] If the category of the current road target is a non-motor vehicle or a pedestrian, and the angle between its heading and the prescribed heading of the motor vehicle lane where the road target is located is less than 25 degrees, in order to promptly detect abnormal behavior during the transition of non-motor vehicles or pedestrians from non-motor vehicle lanes to motor vehicle lanes, if the proportion of the semantic area of ​​the motor vehicle lane in the semantic area covered by the lower half of the target detection frame is greater than 50%, the current non-motor vehicle or pedestrian is judged to be driving or walking in the motor vehicle lane; if the category of the current road target is a non-motor vehicle or a pedestrian, the angle between the heading of the moving target and the prescribed heading of the motor vehicle lane is greater than 25 degrees, and the proportion of the semantic area of ​​the motor vehicle lane in the area covered by the lower half of the target detection frame is greater than 50%, the current non-motor vehicle or pedestrian is judged to be crossing the road; if the semantic area covered by the lower half of the target detection frame contains the semantics of a non-motor vehicle crosswalk or a pedestrian crosswalk, and the current traffic light is red, the current non-motor vehicle or pedestrian is judged to be running a red light.

[0050] After obtaining the abnormal traffic behavior detection results of road targets, early warning information of abnormal traffic behaviors of surrounding road targets detected can be sent to the terminal device based on the latitude and longitude information of the current handheld or vehicle-mounted terminal device, so that these road targets in dangerous states can be avoided as much as possible, further improving the safety of road traffic.

[0051] The present application proposes a method for detecting abnormal traffic behaviors of road targets based on the semantics of road traffic elements. The method assigns semantics to various selected road traffic elements and constructs a semantic graph of road traffic elements. Because the dual information of road semantic information and moving target detection can be combined to detect abnormal traffic behaviors of road targets, the method overcomes the technical defects of conventional road traffic abnormal behavior detection methods that only have moving target detection information but lack road information, thereby being able to efficiently and accurately detect various abnormal traffic behaviors of various road targets such as motor vehicles, non-motor vehicles or pedestrians, thereby improving the utilization rate of detection data generated by roadside radar and visual perception equipment. The present application can detect nine types of abnormal traffic behaviors of motor vehicles, including crossing the line, changing lanes, running red lights, speeding, slowing, driving in the wrong direction, illegal parking, lateral collisions and driving in the wrong lane, based on the semantic distribution data of road traffic elements in the road traffic element semantic graph area covered by the target detection frame where the road target is located and / or the comparison results of the motion parameters of the road target with the standard motion parameters. It can also detect three types of abnormal traffic behaviors of non-motor vehicles or pedestrians, including driving or walking in the motor vehicle lane, crossing the road and running red lights, thereby greatly enriching the detection range of various abnormal traffic behaviors of various road targets and further improving the perception ability of road traffic situation.

[0052] The flowchart and / or block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method or computer program product according to various embodiments of the present application. In this regard, each box in the flowchart and / or block diagram can represent a part of a module, program segment or code, and a part of the above-mentioned module, program segment or code includes one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0053] Those skilled in the art will appreciate that the technical features described in the various embodiments and / or claims of the present application may be combined and / or coupled in a variety of ways, even if such combinations and / or couplings are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the technical features described in the various embodiments and / or claims of the present application may be combined and / or coupled in a variety of ways, and all of these combinations and / or couplings fall within the scope of the present application.

[0054] Although the present application has been shown and described with reference to specific exemplary embodiments of the present application, it should be understood by those skilled in the art that various changes in form and details may be made to the present application without departing from the spirit and scope of the present application as defined by the appended claims and their equivalents. Therefore, the scope of the present application should not be limited to the above-mentioned embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A method for detecting abnormal traffic behavior of road targets based on road traffic element semantics, characterized in that: include: Performing semantic annotation on a plurality of pre-selected road traffic elements in the acquired traffic scene image, and performing structural processing on the semantic information of the annotated road traffic elements and the resolution of the traffic scene image to obtain a road annotation structured data file; Parsing the road annotation structured data file, obtaining an initialization semantic map based on the obtained resolution, embedding the obtained semantic information of the annotated road traffic elements into the initialization semantic map, and generating a road traffic element semantic map; Setting standard motion parameters including maximum and minimum speeds, forward and reverse headings for each motor vehicle lane in the road traffic element semantic graph; Acquire motion parameters of each of the road targets in the same traffic scene image, and determine the position of the target detection frame of each of the road targets in the road traffic element semantic map; The abnormal traffic behavior detection result of the road target is obtained based on the semantic distribution data of the road traffic elements in the road traffic element semantic graph area covered by the target detection frame where the road target is located and / or the comparison result of the motion parameters of the road target with the standard motion parameters.

2. The method for detecting abnormal traffic behavior of road targets as claimed in claim 1, characterized in that: The step of semantically annotating a plurality of preselected road traffic elements in the acquired traffic scene image, and structurally processing the semantic information of the annotated road traffic elements and the resolution of the traffic scene image, and obtaining a road annotation structured data file includes: In the collected traffic scene image, the road traffic elements are sequentially constructed into corresponding closed areas, the vertex coordinates of the closed areas are recorded, and the label names of the closed areas are specified; The annotation result of the same traffic scene image is saved in the road annotation structured data file with the tag name as the key and the vertex coordinates of the closed area as the value, and the resolution of the traffic scene image is saved in the file.

3. The method for detecting abnormal traffic behavior of road targets as claimed in claim 1 or 2, characterized in that: The road traffic elements include one or more of the first type of road traffic elements, the second type of road traffic elements, the third type of road traffic elements and the fourth type of road traffic elements, and each type of road traffic element adopts a corresponding semantic annotation method; The first type of road traffic elements includes yellow dashed lines and / or white dashed lines; the first type of semantic labeling method used includes: the solid line part of a dashed line of a certain color is labeled according to the solid line of the color, and the dashed line part and the blank area between it and the solid line are labeled according to the dashed line of the color; The second type of road traffic elements include one or more of lane direction dividing lines, yellow solid lines, yellow dashed lines, white solid lines, white dashed lines, stop lines and road edge lines; the second type of semantic labeling methods used include: using continuous semantic areas to label continuous solid line markings or discontinuous dashed line markings, and the semantic areas just cover the marking area; The third type of road traffic elements include one or more of double yellow solid lines, double yellow dashed lines, guide lines, yellow grids, stop and yield lines, detour areas, pedestrian crossings and non-motor vehicle crossings; the third type of semantic annotation method used includes: regardless of whether there is a blank space inside, semantic annotation is performed on the entire outline and the inside according to the overall outline shape of the marking line; The fourth category of road traffic elements includes one or more of non-motorized vehicle lanes, motorized vehicle lanes, tidal lanes, deceleration lanes and variable lanes; the fourth category of semantic labeling methods adopted include: only semantically labeling the lane part, and the markings on both sides of the lane are semantically labeled according to the category of the markings based on the aforementioned several categories of semantic labeling methods.

4. The method for detecting abnormal traffic behavior of road targets as claimed in claim 2, characterized in that: The parsing of the road annotation structured data file, obtaining an initialization semantic graph based on the obtained resolution, embedding the obtained semantic information of the annotated road traffic elements into the initialization semantic graph, and generating a road traffic element semantic graph includes: Parsing the road annotation structured data file to obtain the resolution and label names of closed areas of various road traffic elements in the traffic scene image; Initialize according to the resolution and obtain an initialized semantic map of road traffic elements with pixel values ​​all being zero; According to the preset correspondence between the label name and the semantic value, the corresponding semantic value is assigned to the pixels in each closed area in the initialization semantic map, and the road traffic element semantic map carrying the road traffic element semantics is generated.

5. The method for detecting abnormal traffic behavior of road targets as claimed in claim 3, characterized in that: The acquiring of motion parameters of each of the road targets in the same traffic scene image and determining the position of the target detection frame of each of the road targets in the road traffic element semantic map comprises: Acquire motion parameters of each of the road targets in the same traffic scene image, and save the motion parameters into a target detection structured data file; the file name of the target detection structured data file includes identification information of radar and visual perception devices and time information of the road target detection moment; Using the identification information of the radar and visual perception device and the time information of the road target detection moment as indexes, reading and parsing the target detection structured data file to obtain the motion parameters of each of the road targets; According to the position of each road target in the traffic scene image and the length and width of the target detection frame, the position and length and width of the target detection frame of each road target in the road traffic element semantic map are determined, and only the road traffic element semantic map area covered by the target detection frame in a preset range is selected to reduce the misjudgment caused by the target detection frame in other ranges; The motion parameters of the road target include one or more of the category, longitude and latitude, speed, heading, position in the traffic scene image, and length and width of the target detection frame.

6. The method for detecting abnormal traffic behavior of road targets as claimed in claim 5, characterized in that: The radar and visual perception device collect data at a fixed collection position and collection angle to acquire the traffic scene image.

7. The method for detecting abnormal traffic behavior of road targets as claimed in claim 5, characterized in that: The obtaining of the abnormal traffic behavior detection result of the road target according to the semantic distribution data of the road traffic element in the road traffic element semantic graph area covered by the target detection frame where the road target is located and / or the comparison result of the motion parameter of the road target with the standard motion parameter comprises: If the category of the current road target is a motor vehicle, and the road traffic element semantic map area covered by the target detection frame of the preset range includes one or more of the semantics of the lane direction dividing line, double yellow solid line, yellow solid line, yellow dashed line, yellow dashed and solid line, white solid line and white dashed line, it is determined that the motor vehicle has crossed the line; and / or If the category of the current road target is a motor vehicle, and the road traffic element semantic map area covered by the target detection frame of the preset range includes the semantics of at least one traffic marking and the semantics of at least two different lanes, it is determined that the motor vehicle changes lanes; and / or If the category of the current road target is a motor vehicle, and the road traffic element semantic map area covered by the target detection frame of the preset range includes the semantics of the stop line, and the semantics of the lane or traffic marking in the preset direction of the stop line, and the current traffic light is red, it is judged that the motor vehicle has run a red light.

8. The method for detecting abnormal traffic behavior of road targets as claimed in claim 5, characterized in that: The obtaining of the abnormal traffic behavior detection result of the road target according to the semantic distribution data of the road traffic element in the road traffic element semantic graph area covered by the target detection frame where the road target is located and / or the comparison result of the motion parameter of the road target with the standard motion parameter comprises: If the category of the current road target is a motor vehicle, and the road traffic element semantic map area covered by the target detection frame of the preset range includes the semantics of the motor vehicle lane exceeding a first preset ratio, then according to the corresponding relationship between the semantics of the motor vehicle lane and the standard motion parameter, the maximum speed, the minimum speed, the forward heading and / or the reverse heading of the current motor vehicle lane are obtained; If the current speed of the motor vehicle exceeds the maximum speed, it is determined that the motor vehicle is speeding; and / or If the current speed of the motor vehicle is lower than the minimum speed, it is determined that the motor vehicle is at a low speed; and / or If the angle between the current heading of the motor vehicle and the forward heading specified by the current lane is greater than a first preset angle, it is determined that the motor vehicle is traveling in the wrong direction; and / or If the current speed of the motor vehicle is zero and continues for a preset time, it is judged that the motor vehicle is illegally parked.

9. The method for detecting abnormal traffic behavior of road targets as claimed in claim 5, characterized in that: The obtaining of the abnormal traffic behavior detection result of the road target according to the semantic distribution data of the road traffic element in the road traffic element semantic graph area covered by the target detection frame where the road target is located and / or the comparison result of the motion parameter of the road target with the standard motion parameter comprises: If the category of the current road target is a motor vehicle, and the road traffic element semantic map area covered by the target detection frame in the preset range includes the semantics of the motor vehicle lane, and the angle between the current heading of the motor vehicle and the forward heading specified by the current lane is greater than a second preset angle, it is determined that the motor vehicle has a possibility of a lateral collision; and / or If the category of the current road target is a motor vehicle, and the road traffic element semantic map area covered by the target detection frame in the preset range includes the semantics of a non-motor vehicle lane, it is determined that the motor vehicle is not traveling in the prescribed lane.

10. The method for detecting abnormal traffic behavior of road targets as claimed in claim 5, characterized in that: The obtaining of the abnormal traffic behavior detection result of the road target according to the semantic distribution data of the road traffic element in the road traffic element semantic graph area covered by the target detection frame where the road target is located and / or the comparison result of the motion parameter of the road target with the standard motion parameter comprises: If the category of the current road target is a non-motor vehicle or a pedestrian, and the angle between the current heading of the road target and the prescribed heading of the current lane is less than the third preset angle, and the proportion of the semantic area of ​​the motor vehicle lane contained in the road traffic element semantic map area covered by the target detection frame in the preset range is greater than the second preset proportion, it is determined that the non-motor vehicle or the pedestrian is driving or walking on the motor vehicle lane; and / or If the category of the current road target is a non-motor vehicle or a pedestrian, and the angle between the current heading of the road target and the prescribed heading of the current lane is greater than the fourth preset angle, and the proportion of the semantic area of ​​the motor vehicle lane contained in the road traffic element semantic map area covered by the target detection frame in the preset range is greater than the third preset proportion, it is determined that the non-motor vehicle or pedestrian is crossing the road; and / or If the category of the current road target is a non-motor vehicle or a pedestrian, and the road traffic element semantic map area covered by the target detection frame of the preset range includes the semantics of the non-motor vehicle crosswalk or the pedestrian crosswalk, and the current traffic light is red, it is judged that the non-motor vehicle or pedestrian ran a red light.