Illegal vehicle identification method at ramp exit based on deep learning and trajectory tracking

Abstract

The present invention discloses a ramp exit vehicle violation identification method based on deep learning and trajectory tracking, including: collecting vehicle information and making the vehicle information into a sample set; inputting the sample set into a network model for training to obtain a trained network model; inputting the information of the vehicle to be tested into the trained network model, and outputting the detection information of the vehicle to be tested; tracking and processing the position of the vehicle to be tested at adjacent time nodes to obtain the driving trajectory of the vehicle to be tested on the time series; determining the length of the driving trajectory, the residence time of the vehicle to be tested, and the driving direction; if the residence time of the vehicle to be tested exceeds the time threshold and the length of the driving trajectory is less than the length threshold, the vehicle to be tested is a parking violation; if the driving direction of the vehicle to be tested is opposite to the lane direction, the vehicle to be tested is a reverse or reverse driving violation. The present invention can realize comprehensive monitoring of vehicles, accurately identify the illegal behavior of vehicles, and improve the recognition efficiency of driving violations.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and in particular to a method for identifying illegal vehicles at ramp exits based on deep learning and trajectory tracking. Background Art

[0002] With the rapid development of my country's expressways and the continuous increase in the number of vehicles passing through, traffic safety has become an important social issue. People, vehicles, roads and additional infrastructure together constitute the main participants of expressways. In recent years, the construction of roads and infrastructure has been continuously strengthened in road traffic management, effectively improving the driving safety of expressways. However, the number of expressway traffic accidents caused by people and vehicles remains high. A comprehensive analysis of traffic accidents over the years shows that the main causes of traffic accidents are speeding, overloading, fatigue driving, and using mobile phones while driving, and illegal driving is also an important cause of traffic accidents.

[0003] Due to human factors such as drivers' unfamiliarity with road conditions, distraction, carelessness or luck, illegal parking at highway ramp exits, reversing or driving in the wrong direction after missing an exit often occur, which poses a major safety hazard to road traffic. The management of highway ramp exits is an important part of the overall management and is also the most easily overlooked node.

[0004] At present, the identification of common vehicle violations is mainly through manual query of surveillance videos or on-site patrol by traffic police, which not only consumes a lot of manpower and time, but also has low efficiency. How to improve the technological management level of highway ramp exits and effectively identify vehicle violations has become a hot topic in the field of intelligent transportation research. Summary of the invention

[0005] In view of this, the purpose of the present invention is to overcome the defects in the prior art and provide a ramp exit vehicle violation identification method based on deep learning and trajectory tracking, which can achieve comprehensive monitoring of vehicles, accurately identify vehicle violations, and improve the efficiency of identifying driving violations.

[0006] The ramp exit vehicle violation identification method based on deep learning and trajectory tracking of the present invention comprises the following steps:

[0007] S1. Collect vehicle information and make the vehicle information into a sample set;

[0008] S2. Input the sample set into the network model based on deep learning for network training to obtain a trained network model;

[0009] S3. Input the information of the vehicle to be tested into the trained network model, and output the detection information of the vehicle to be tested; wherein the detection information includes the position, category and rectangular frame;

[0010] S4. Track the position of the vehicle to be tested at adjacent time nodes to obtain the driving trajectory of the vehicle to be tested in the time series; wherein the time period of the time series is t s ;

[0011] S5. Determine the length of the driving trajectory, the residence time of the vehicle to be tested, and the driving direction;

[0012] S6. If the residence time of the vehicle under test exceeds the threshold ψ and in time period t s If the length of the inner driving trajectory is less than the threshold ζ, the vehicle under test is in violation of parking regulations; if the driving direction of the vehicle under test is opposite to the lane direction, the vehicle under test is in violation of driving in the wrong direction or reversing regulations.

[0013] Further, the sample set is determined according to the following formula:

[0014]

[0015] Where E is the sample set, P(n) is the positive sample image set containing vehicles, n is the number of positive sample images; δ is the set threshold; N(m) is the negative sample image set not containing vehicles, m is the number of negative sample images.

[0016] Furthermore, the proportion of vehicles in the image containing vehicles is greater than λ% of the image.

[0017] Furthermore, step S1 further includes marking the sample set to obtain a sample set containing marking information; wherein the marking information is:

[0018]

[0019] Where F(p) is the label information of image p in the sample set; t n is the category of the nth vehicle in image p; left n is the relative starting position of the nth vehicle in the x direction in image p; top n is the relative starting position of the nth vehicle in the y direction in image p; width n is the width of the nth vehicle relative to image p; height n is the height of the nth vehicle relative to image p.

[0020] Furthermore, the time series is (t1, t2, ..., t i ,...,t k ), where t i is the i-th time point, k is the number of time points; the step S4 specifically includes:

[0021] S41. Extracting the corner points of the vehicle to be tested at time point t1 from the image of the vehicle to be tested;

[0022] S42. Calculate the position of the corner point at time point t2 according to the position and grayscale of the corner point;

[0023] S43. Taking the position at time point t2 as the target position, calculate the position of the target position at time point t3;

[0024] S44. By analogy with step S43, calculate the positions corresponding to other time points in the time series;

[0025] S45. sequentially connect the positions corresponding to each time point in the time series to obtain a line starting from the position of the corner point and ending at time point t k The corresponding position is a line segment of the end point, and the line segment is used as the driving trajectory of the vehicle to be tested in the time series.

[0026] Furthermore, the dwell time of the vehicle to be tested is determined according to the following formula:

[0027]

[0028] Where T(i) is the dwell time of the vehicle under test at point i of the driving trajectory; (x i ,y i ) is the coordinate position of point i in the driving trajectory; (x j ,y j ) is the coordinate position of point j in the driving trajectory; p i,j is a sign, and its value is 1 or 0; d is the distance threshold between point i and point j in the driving trajectory.

[0029] Further, the driving direction of the vehicle to be tested is determined according to the following steps:

[0030] a. Determine the marking amount R of the driving direction of the vehicle to be tested:

[0031]

[0032] Among them, y n is the y-axis coordinate of the end point of the driving track in the coordinate system; y1 is the y-axis coordinate of the starting point of the driving track in the coordinate system; the coordinate system takes the vertex at the lower left corner of the vehicle image as the coordinate origin and the positive direction of the Y axis is upward;

[0033] b. If the value of the marking quantity R is 1, the driving direction of the vehicle to be tested is the same as the lane direction; if the value of the marking quantity R is -1, the driving direction of the vehicle to be tested is opposite to the lane direction; wherein the lane direction is the direction of the oncoming vehicle in the vehicle image.

[0034] Further, the method further comprises the following steps:

[0035] If the vehicle under test is parked illegally, the vehicle under test will be captured according to the following method:

[0036] Taking the rectangular frame where the vehicle to be tested is located as a reference, determining the target close-up position of the vehicle to be tested, and identifying and recording the license plate of the vehicle to be tested at the target close-up position;

[0037] If the vehicle under test is driving against traffic or reversing illegally, the vehicle under test will be captured according to the following method:

[0038] A first preset position and a second preset position are set in advance at a place where the vehicle to be tested must pass through when going in reverse or reversing; wherein the first preset position is set at the main lane, and the second preset position is set at the ramp;

[0039] If the vehicle to be tested is traveling in the main lane, the license plate of the vehicle to be tested is identified and recorded when the vehicle to be tested reaches the first preset position; if the vehicle to be tested is traveling in the ramp, the license plate of the vehicle to be tested is identified and recorded when the vehicle to be tested reaches the second preset position.

[0040] Furthermore, the license plate of the vehicle to be tested is identified by capturing a target image; the proportion of the vehicle to be tested in the target image is α% to β% of the target image.

[0041] Furthermore, it also includes generating a violation record based on the identified license plate of the vehicle to be tested; the violation record includes the violation time, violation type, license plate number and associated pictures; the associated pictures include several pictures recording the violation process of the vehicle to be tested.

[0042] The beneficial effects of the present invention are as follows: the present invention discloses a ramp exit vehicle violation identification method based on deep learning and trajectory tracking, which obtains a vehicle detection network model by performing network training on the collected vehicle data, and uses the detection network model to perform feature detection on the vehicle to be tested, and obtains the driving trajectory and driving direction of the vehicle to be tested by analyzing the feature detection results, thereby identifying the illegal driving state of the vehicle to be tested, and capturing the illegal driving state. The present invention can realize comprehensive monitoring of vehicles, accurately identify vehicle violations, improve the efficiency of identifying driving violations, ensure road traffic safety, and provide data support for the handling of traffic violations. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0044] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0045] Figure 2 A schematic diagram for selecting a monitoring viewing angle of the present invention;

[0046] Figure 3 This is a schematic diagram of image marking of the present invention;

[0047] Figure 4 It is a schematic diagram of the YOLOV3 training process of the present invention;

[0048] Figure 5 The vehicle corner tracking result diagram of the present invention;

[0049] Figure 6 This is a schematic diagram of the parking violation capture processing of the present invention;

[0050] Figure 7 This is a schematic diagram of the reverse or wrong-way illegal capture processing of the present invention;

[0051] Figure 8 It is a schematic diagram of the snapshot effect of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described below in conjunction with the accompanying drawings, as shown in the drawings:

[0053] The ramp exit vehicle violation identification method based on deep learning and trajectory tracking of the present invention comprises the following steps:

[0054] S1. Collect vehicle information and make the vehicle information into a sample set; wherein, Figure 2 As shown, a spherical camera is installed on the green belt between the ramp and the main lane, and illuminates the direction of the oncoming vehicle to collect information such as video or images of vehicles at the ramp exit. By using a spherical camera, the main lane, ramp and triangular diversion area of ​​the highway can be taken into account, with a large-scene monitoring perspective, and comprehensive monitoring of various vehicles in the perspective can be achieved. It can monitor illegally parked vehicles within the range and detect reversing and wrong-way vehicles in the area;

[0055] S2. Input the sample set into the network model based on deep learning for network training to obtain a trained network model; Figure 4 As shown, the YOLOV3 training framework based on Darknet is used to perform network training to generate a trained network weight file; the weight file includes the weights of each layer of the trained network;

[0056] S3. Input the information of the vehicle to be tested into the trained network model, and output the detection information of the vehicle to be tested; wherein the detection information includes the position, category and rectangular frame; by using the YOLOV3 configuration file, C++ calling interface and weight file generated by training provided by Darknet, the input image to be detected is detected, and the position, category, rectangular frame and confidence of each vehicle in each image are obtained;

[0057] S4. Track the position of the vehicle to be tested at adjacent time nodes to obtain the driving trajectory of the vehicle to be tested in the time series; wherein the time period of the time series is t s ;

[0058] S5. Determine the length of the driving trajectory, the residence time of the vehicle to be tested, and the driving direction;

[0059] S6. If the residence time of the vehicle under test exceeds the threshold ψ and in time period t s If the length of the vehicle's inner driving trajectory is less than the threshold value ζ, the vehicle under test is in violation of parking; if the driving direction of the vehicle under test is opposite to the lane direction, the vehicle under test is in violation of driving in the wrong direction or reversing. That is, by analyzing the vehicle's running trajectory for a period of time, calculating the vehicle's moving distance and running direction, and combining the lane direction, it is determined whether the vehicle's behavior violates the "Road Traffic Safety Law", whether there is illegal parking, driving in the wrong direction, reversing, etc. Among them, the threshold values ​​ψ and ζ can be set according to the actual working conditions.

[0060] In this embodiment, in order to make the sample set reflect all the features of various conditions on the scene as much as possible, the sample set includes images of various weather conditions such as cloudy, sunny, rainy, and snowy as much as possible. At the same time, the sample set should also include images under different lighting conditions during the day and at night. The sample set is determined according to the following formula:

[0061]

[0062] Where E is the sample set, P(n) is the positive sample image set containing vehicles, and n is the number of positive sample images; δ is the set threshold. In order to obtain a better detection effect, the threshold δ is set to 5000; N(m) is the negative sample image set without vehicles, and m is the number of negative sample images. The image set is generated by intercepting several color images from the spherical camera surveillance video and saving the images in JPG format.

[0063] In this embodiment, the proportion of vehicles in the image containing vehicles is greater than λ% of the image. In order to ensure effective detection of vehicles, the value of λ is 70.

[0064] In this embodiment, step S1 also includes labeling the images in the sample set using the open source dedicated tool "labelImage.exe" to obtain a sample set containing label information; Figure 3 As shown in the figure, by selecting the detected vehicle with a rectangular frame and adding the corresponding type label, the TXT text corresponding to each image is obtained. Reading the associated information of the image from the TXT text can provide more computing data for subsequent network training, thereby improving the accuracy and reliability of the network model. The label information is:

[0065]

[0066] Where F(p) is the label information of image p in the sample set; t n is the category of the nth vehicle in image p; left n is the relative starting position of the nth vehicle in the x direction in image p; top n is the relative starting position of the nth vehicle in the y direction in image p; width n is the width of the nth vehicle relative to image p; height n is the height of the nth vehicle relative to the image p. A two-dimensional rectangular coordinate system is established with the lower left corner vertex of the image p as the coordinate origin, the upward direction of the image p as the positive direction of the y axis, and the rightward direction of the image p as the positive direction of the x axis. The positive direction of the y axis of the coordinate system is the y direction, and the positive direction of the x axis of the coordinate system is the x direction.

[0067] In this embodiment, the time series is (t1, t2, ..., t i ,...,t k ), where t i is the i-th time point, k is the number of time points; the step S4 specifically includes:

[0068] S41. Extract the corner points of the vehicle to be tested at time point t1 from the image where the vehicle to be tested is located; wherein, corner point extraction is a method of obtaining image features in computer vision, which is widely used in motion detection, image matching, visual tracking, 3D reconstruction and other fields. There are many ways to define corner points, here it is the intersection of two edges, and common detection methods include Harris operator, Shi-Tomasi operator, etc. The extraction algorithm corresponding to the open source machine vision library OpenCV is used to extract the corner points of the vehicle to be tested at time point t1:

[0069] C(p)=goodFeaturesToTrack(p,cout,n,q,d,t,3,0,0.04)

[0070] Among them, C(p) is the corner extraction function; goodFeaturesToTrack is the name of the extraction function; p is the input image, t is the target area, cout is the corner detection result, n is the maximum number of corner points, q is the quality level coefficient, and d is the minimum distance between two feature points. Points smaller than this distance will be ignored.

[0071] S42. Calculate the position of the corner point at time point t2 according to the position and grayscale of the corner point; wherein, the position of the corner point at time point t2 is calculated using a corner point tracking function named calcOpticalFlowPyrLK provided by OpenCV, and T(p) is assumed to be the corner point tracking function. The specific implementation thereof is as follows:

[0072] T(p)=calcOpticalFlowPyrLK(p,pos1,pos2,st,er,size,l)

[0073] Among them, p is the input image, pos1 is the corner point position of the previous frame, pos2 is the new corner point position calculated in the current frame, one frame image corresponds to one time point, st is the output state vector, er is the output error vector, size is the size of the search window, and l is the number of calculation layers.

[0074] S43. Taking the position at time point t2 as the target position, calculate the position of the target position at time point t3;

[0075] S44. By analogy with step S43, calculate the positions corresponding to other time points in the time series;

[0076] S45. sequentially connect the positions corresponding to each time point in the time series to obtain a line starting from the position of the corner point and ending at time point t k The corresponding position is a line segment of the end point, and the line segment is used as the driving trajectory of the vehicle to be tested in the time series.

[0077] In general, the corner points corresponding to each of the multiple vehicles in each frame of the image are calculated for each frame of the image, and then the corner points corresponding to the target vehicle in the multiple frames of the image are distinguished, and finally the positions of the corner points of the target vehicle are connected in chronological order to generate the driving track of the target vehicle. However, step S4 of the present invention adopts the method of only calculating the corner points of the target vehicle in the image corresponding to the starting time point, and then using the corner points as reference points to calculate the target points of the target vehicle in the image corresponding to the next time point, and then using the target points to calculate the next target point, and so on, and finally using the corner point positions as the starting points to connect the positions of the target points in order to generate the driving track of the target vehicle. Compared with the usual situation, on the one hand, it avoids the extraction and tracking of the corner points of the same vehicle for many times, and reduces the calculation consumption. On the other hand, it avoids the interference caused by the corner points corresponding to multiple vehicles in one frame of the image, and ensures the accuracy of the driving track.

[0078] In this embodiment, the driving trajectory of the vehicle to be tested is known, and the trajectory length calculation formula is as follows:

[0079]

[0080] Among them, L is the length of the trajectory, (x1, y1) is the coordinate position of the starting point of the trajectory, (x n ,y n ) is the coordinate position of the end point of the track. By establishing a two-dimensional rectangular coordinate system, the driving track of the vehicle to be tested obtained above is set to the coordinate system, so that the coordinate position of the starting point of the track and the coordinate position of the end point of the track can be obtained; the establishment of the coordinate system is the same as the establishment principle of the above coordinate system, which will not be repeated here.

[0081] The dwell time of the vehicle under test indicates the length of time the vehicle under test stays at the same location. It can be described by the scattered distribution of the target trajectory points in the time series, and the number of adjacent or overlapping trajectory points in the driving trajectory is used to represent the dwell time of the vehicle under test. The dwell time of the vehicle under test is determined according to the following formula:

[0082]

[0083] Where T(i) is the dwell time of the vehicle under test at point i of the driving trajectory; (x i ,y i ) is the coordinate position of point i in the driving trajectory; (x j ,y j ) is the coordinate position of point j in the driving trajectory; p i,j is a flag value, and its value is 1 or 0; d is the distance threshold between the driving trajectory point i and the driving trajectory point j; the distance threshold d can be set according to the actual working conditions.

[0084] In this embodiment, the driving direction of the vehicle to be tested is determined to be forward or reverse by calculating the relationship between the driving trajectory of the vehicle to be tested and the normal driving direction of the lane. Assuming that the direction of the oncoming vehicle in the lane is the normal driving direction, the driving direction of the vehicle to be tested is determined according to the following steps:

[0085] a. Determine the marking amount R of the driving direction of the vehicle to be tested:

[0086]

[0087] Among them, y n is the y-axis coordinate of the end point of the driving track in the coordinate system; y1 is the y-axis coordinate of the starting point of the driving track in the coordinate system; the coordinate system takes the vertex at the lower left corner of the vehicle image as the coordinate origin and the positive direction of the Y axis is upward;

[0088] b. If the value of the marking quantity R is 1, the driving direction of the vehicle to be tested is the same as the lane direction; if the value of the marking quantity R is -1, the driving direction of the vehicle to be tested is opposite to the lane direction; wherein the lane direction is the direction of the vehicle

[0089] In this embodiment, different control strategies are adopted for the camera to realize capturing of different illegal states of the vehicle to be tested; the control strategy is to rotate the camera to a suitable angle and position to identify the license plate, so as to facilitate capturing and collecting evidence.

[0090] like Figure 6 As shown, parking violation is a static violation, and there is enough time for the camera to rotate, identify and capture. That is, if the vehicle to be tested is a parking violation, the vehicle to be tested is captured according to the following method:

[0091] According to the target frame determined by deep learning, the camera is directly controlled to rotate to the target close-up position, and the license plate of the vehicle to be tested is recognized and recorded at the target close-up position;

[0092] like Figure 7 As shown, driving against traffic or reversing is a dynamic violation. Since it takes a certain amount of time for the camera to rotate from the large scene to the capture position, if it is directly rotated to the target position given by the detection, when the camera rotates to the right position and completes the focus, the target has actually left the point. Therefore, for driving against traffic or reversing, the preset position is adopted and the vehicle is intercepted and captured in front of the vehicle's direction of travel. That is, if the vehicle to be tested is a violation of driving against traffic or reversing, the vehicle to be tested is captured according to the following method:

[0093] A first preset position and a second preset position are set in advance at a place where the vehicle to be tested must pass through when going in reverse or reversing; wherein the first preset position is set at the main lane, and the second preset position is set at the ramp;

[0094] If the vehicle to be tested is traveling on the main lane, the camera is turned to the first preset position to wait for the vehicle to pass, and the license plate of the vehicle to be tested is identified and recorded when the vehicle to be tested arrives at the first preset position; if the vehicle to be tested is traveling on the ramp, the camera is turned to the second preset position to wait for the vehicle to pass, and the license plate of the vehicle to be tested is identified and recorded when the vehicle to be tested arrives at the second preset position.

[0095] The license plate recognition adopts a video image recognition algorithm to locate the license plate from the video image and recognize the color, type and specific license plate number of the license plate. For example, mature license plate recognition software is used with corresponding library files for recognition, and the license plate recognition result is the license plate color and license plate number. After completing the capture of the above-mentioned different illegal states, the camera used for capture is pulled back to the panoramic monitoring position, so that the camera is in the normal panoramic monitoring state again.

[0096] In this embodiment, the license plate of the vehicle to be tested is identified by capturing the target image; the proportion of the vehicle to be tested in the target image is α% to β% of the target image, and the values ​​of α% to β% are 2 / 3 to 3 / 4; this ensures that the proportion of the vehicle to be tested is large enough, the license plate can be seen more clearly, and the accuracy of license plate recognition is guaranteed.

[0097] In this embodiment, it also includes generating a violation record according to the identified license plate of the vehicle to be tested; the violation record includes the violation time, violation type, license plate number and associated pictures; the associated pictures include several pictures recording the violation process of the vehicle to be tested. Four pictures with a time span that can describe the violation process are intercepted from the video image as associated pictures, and the four pictures are two foreground pictures and two close-up pictures.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A ramp exit vehicle violation identification method based on deep learning and trajectory tracking, characterized by: The steps include: S1. Collect vehicle information and make the vehicle information into a sample set; S2. Input the sample set into the network model based on deep learning for network training to obtain a trained network model; S3. Input the vehicle information to be tested into the trained network model, and output the detection information of the vehicle to be tested; wherein the detection information includes the position, category and rectangular frame; S4. Track the position of the vehicle to be tested at adjacent time nodes to obtain the driving trajectory of the vehicle to be tested in the time series; wherein the time period of the time series is t s ; The time series is (t1, t2, ..., t i ,...,t k ), where t i is the i-th time point, k is the number of time points; the step S4 specifically includes: S41. Extracting the corner points of the vehicle to be tested at time point t1 from the image of the vehicle to be tested; S42. Calculate the position of the corner point at time point t2 according to the position and grayscale of the corner point; S43. Taking the position at time point t2 as the target position, calculate the position of the target position at time point t3; S44. By analogy with step S43, calculate the positions corresponding to other time points in the time series; S45. sequentially connect the positions corresponding to each time point in the time series to obtain a line starting from the position of the corner point and ending at time point t k The corresponding position is a line segment of the end point, and the line segment is used as the driving trajectory of the vehicle to be tested in the time series; S5. Determine the length of the driving trajectory, the residence time of the vehicle to be tested, and the driving direction; S6. If the residence time of the vehicle under test exceeds the threshold ψ and in time period t s If the length of the inner driving trajectory is less than the threshold ζ, the vehicle under test is in violation of parking regulations; if the driving direction of the vehicle under test is opposite to the lane direction, the vehicle under test is in violation of driving in the wrong direction or reversing regulations.

2. The ramp exit vehicle violation identification method based on deep learning and trajectory tracking according to claim 1 is characterized by: The sample set is determined according to the following formula: Where E is the sample set, P(n) is the positive sample image set containing vehicles, n is the number of positive sample images; δ is the set threshold; N(m) is the negative sample image set not containing vehicles, m is the number of negative sample images.

3. The ramp exit vehicle violation identification method based on deep learning and trajectory tracking according to claim 2 is characterized by: The proportion of vehicles in the image containing vehicles is greater than λ% of the image.

4. The ramp exit vehicle violation identification method based on deep learning and trajectory tracking according to claim 1 is characterized by: Step S1 also includes marking the sample set to obtain a sample set containing marking information; wherein the marking information is: Where F(p) is the label information of image p in the sample set; t n is the category of the nth vehicle in image p; left n is the relative starting position of the nth vehicle in the x direction in image p; top n is the relative starting position of the nth vehicle in the y direction in image p; width n is the width of the nth vehicle relative to image p; height n is the height of the nth vehicle relative to image p.

5. The ramp exit vehicle violation identification method based on deep learning and trajectory tracking according to claim 1 is characterized by: The dwell time of the vehicle to be tested is determined according to the following formula: Where T(i) is the dwell time of the vehicle under test at point i of the driving trajectory; (x i ,y i ) is the coordinate position of point i on the driving trajectory; (x j ,y j ) is the coordinate position of point j in the driving trajectory; p i,j is a sign, and its value is 1 or 0; d is the distance threshold between point i and point j in the driving trajectory.

6. The ramp exit vehicle violation identification method based on deep learning and trajectory tracking according to claim 1 is characterized by: Determine the driving direction of the vehicle to be tested according to the following steps: a. Determine the marking amount R of the driving direction of the vehicle to be tested: Among them, y n is the y-axis coordinate of the end point of the driving trajectory in the coordinate system; y1 is the y-axis coordinate of the starting point of the driving trajectory in the coordinate system; the coordinate system takes the vertex in the lower left corner of the vehicle image as the coordinate origin and the positive direction of the Y axis is upward; b. If the value of the marking quantity R is 1, the driving direction of the vehicle to be tested is the same as the lane direction; if the value of the marking quantity R is -1, the driving direction of the vehicle to be tested is opposite to the lane direction; wherein the lane direction is the direction of the oncoming vehicle in the vehicle image.

7. The ramp exit vehicle violation identification method based on deep learning and trajectory tracking according to claim 1 is characterized by: It also includes the following steps: If the vehicle under test is parked illegally, the vehicle under test will be captured according to the following method: Taking the rectangular frame where the vehicle to be tested is located as a reference, determining the target close-up position of the vehicle to be tested, and identifying and recording the license plate of the vehicle to be tested at the target close-up position; If the vehicle under test is driving against traffic or reversing illegally, the vehicle under test will be captured according to the following method: A first preset position and a second preset position are set in advance at a place where the vehicle to be tested must pass through when going in reverse or reversing; wherein the first preset position is set at the main lane, and the second preset position is set at the ramp; If the vehicle to be tested is traveling in the main lane, the license plate of the vehicle to be tested is identified and recorded when the vehicle to be tested reaches the first preset position; if the vehicle to be tested is traveling in the ramp, the license plate of the vehicle to be tested is identified and recorded when the vehicle to be tested reaches the second preset position.

8. The ramp exit vehicle violation identification method based on deep learning and trajectory tracking according to claim 7 is characterized by: The license plate of the vehicle to be tested is identified by capturing a target image; the proportion of the vehicle to be tested in the target image is α% to β% of the target image.

9. The ramp exit vehicle violation identification method based on deep learning and trajectory tracking according to claim 7 is characterized by: It also includes generating a violation record based on the identified license plate of the vehicle to be tested; the violation record includes the violation time, violation type, license plate number and associated pictures; the associated pictures include several pictures recording the violation process of the vehicle to be tested.

Images