An automatic identification method for illegal vehicles at the airport departure level based on deep learning

Abstract

The present invention discloses a method for identifying illegal vehicles picking up passengers at the airport departure level based on deep learning: based on the surveillance video of the actual fixed position beside the airport departure lane, the YOLO_v4 algorithm is used to identify vehicles and pedestrians, the DeepSORT algorithm is used to track the target, the vehicle's operating status and the passenger boarding and disembarking behavior are judged according to the target category returned by YOLO_v4 and the target position information recorded by DeepSORT, the change in the number of people in the vehicle is counted when the vehicle stops, and whether the vehicle has a passenger picking behavior is judged according to the statistical result, so as to complete the identification of illegal vehicles at the airport departure level. Beneficial effect: The accuracy can reach 83.3%, and the detection speed is greatly improved compared with the traditional manual identification method, which can greatly reduce the workload of monitoring law enforcement personnel, improve the law enforcement efficiency and accuracy of non-site law enforcement, and has broad development prospects in the field of intelligent traffic supervision, which can further strengthen the safety management of airport landside traffic and provide effective guarantee for the long-term development of the transportation industry.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method for automatically identifying illegal vehicles at an airport departure level based on deep learning. Background Art

[0002] In recent years, my country's air transportation has developed rapidly, and the passenger and cargo throughput of major airports has increased rapidly, which has brought certain pressure to the airport's landside traffic. Air transportation problems caused by landside traffic congestion also occur from time to time. In addition to the road network near the airport, the airport's landside traffic also includes the departure lanes of the airport terminal. The departure lanes of the airport are dedicated to passenger drop-off areas. Because the airport cannot guarantee that all vehicles can comply with regulations, many airport departure lanes are mixed with passenger pick-up and drop-off vehicles, which undoubtedly increases the pressure on the departure level traffic.

[0003] To address these behaviors, currently the commonly used methods are manual on-site supervision and camera capture. However, these two technical methods also have the following problems:

[0004] (1) Manual on-site management requires constant supervision, which consumes huge manpower costs;

[0005] (2) The camera usually needs to park for a certain period of time (5-7 minutes) before it can record the video;

[0006] (3) Both methods have low detection efficiency, limited recognition accuracy and cannot fundamentally curb illegal behaviors.

[0007] In summary, in order to improve the existing means of monitoring violations and enhance the efficiency of airport roadside traffic, it is urgent to change the existing detection methods from the source, carry out research on airport road test intelligent control technology, and establish an automatic identification method for airport illegal vehicles with high accuracy, low cost and strong robustness. Summary of the invention

[0008] The present invention aims to solve one of the above technical problems at least to a certain extent.

[0009] To this end, the purpose of the present invention is to provide a method for automatically identifying vehicles that illegally pick up passengers at the airport departure level based on deep learning, so as to solve the problems of high detection cost, limited detection accuracy and weak robustness of the method in the prior art.

[0010] The present invention is achieved through the following technical solutions:

[0011] The invention discloses a method for automatically identifying illegal vehicles at the airport departure level based on deep learning, which is characterized by comprising the following steps:

[0012] S1: Get the video stream, import the target recognition model and the multi-target tracking judgment model, and judge the video stream frame by frame; S2: Target recognition and judgment, take vehicles and pedestrians as recognition targets, and assign IDs respectively. If it is judged to be a vehicle, execute step S3; if it is judged to be a person, execute step S4 until every frame of the video file is judged; S3: Identify whether the vehicle is in a stationary state, and save the vehicle recognition parameter data set to the parameter memory; S4: Identify whether the pedestrian disappears, and save the pedestrian recognition parameter data set to the parameter memory; S5: Calculate the data sets saved in the parameter memory in S3 and S4, The final change in the number of people in the vehicle is calculated by the algebraic difference of the total number of times pedestrians "appear" and "disappear" within the detection range of a stationary vehicle; S6: Based on the final change in the number of people in the vehicle, the vehicle marked as disappeared is identified for violation of traffic rules; when the change in the number of people is greater than 0, it can be determined that the vehicle has picked up passengers, that is, it has violated traffic rules; when the change in the number of people is less than 0, it can be determined that the vehicle has dropped off passengers, that is, it has not violated traffic rules; when the change in the number of people = 0, it can be determined that the vehicle is a passing vehicle; when the data on the change in the number of people is lost, it means that the vehicle has lost part of the data, and its change in the number of people is recorded as 0, and the vehicle is marked as having lost data; S7: Output the result.

[0013] Furthermore, the video stream in S1 is a surveillance video of a fixed aircraft position beside the lane on the departure level of the airport with a duration of T, and t is used to represent any moment of the surveillance video, t∈[0,T];

[0014] The specific steps of S2, S3, S5 and S6 are as follows: when a vehicle is identified by the target recognition model at time t and judged to be "appeared", it is assigned an id of i, i∈N, where N represents the set of ids assigned by the tracking algorithm to all individuals in the video;

[0015] use Represents the set of car IDs marked as "appearing" by the tracker of the multi-target tracking judgment model at time t, and is used Represents the set of car IDs marked as "disappeared" at time t, using represents the set of car IDs that are not marked as "disappeared" by the tracker of the multi-target tracking judgment model at time t, Determined by formula 1:

[0016]

[0017] Whether the vehicle is in a stationary state using parameters Represents the state correction coefficient judgment of the vehicle with id i at time t. When it is 1, it means that the vehicle is stationary and violation identification can be performed. When it is 0, it means that the vehicle is in other states and violation identification cannot be performed. Determined by formula 2:

[0018]

[0019] In formula 2, Represents the state parameter of the vehicle with id i at time t. When it is 2, it means that the vehicle is in an unknown state where it has not yet been identified as moving or stationary. When it is 1, it means that the vehicle is stationary. When it is 0, it means that the vehicle is in motion. When it is -1, it means that the vehicle is in a state where some monitoring data is lost. Only when it is 1 can the violation of vehicle i be identified.

[0020] Furthermore, the specific steps of S2, S4, S5, and S6 are as follows: when a pedestrian is recognized by the target recognition model at time t and judged to be “appearing”, he is assigned an id of j, j∈N, and It represents the set of IDs of people marked as “appearing” by the tracker of the multi-target tracking judgment model at time t, and is used Represents the set of IDs of people marked as "disappeared" at time t, using represents the set of IDs of people who are not marked as "disappeared" by the tracker of the multi-target tracking judgment model at time t, Determined by formula 8:

[0021]

[0022] Within the time range of the surveillance video, whether vehicle i violates traffic rules can be determined by the number of people in the vehicle when it disappears from the screen, using parameter C Nei Indicates the number of people in the car with id i when the screen disappears. represents the number of people in the vehicle with id i at time t, C Nei It can be determined by formula 9:

[0023]

[0024] when When the vehicle with id i does not have any data loss, it can be divided into three cases: Nei When C is less than 0, it is judged that the vehicle with id i has picked up passengers, that is, it is illegal; Nei When C is less than 0, it is judged that the vehicle with id i has a passenger drop-off behavior, that is, it has not violated the traffic rules; Nei = 0, the vehicle with id i is judged as a passing vehicle; when When , it means that the vehicle with id i has lost some data, and the change in the number of people in the vehicle when the screen disappears is recorded as 0, and the vehicle with id i is marked as having lost data.

[0025] Furthermore, the above Determined by formula 3:

[0026]

[0027] In formula 3, ΔC1 represents the displacement threshold, Represents the latest three consecutive displacement records of vehicle id i at time t, Represents the wth displacement in the set of the three most recent consecutive displacement records of the vehicle with id i at time t; is the number of displacement records, Determined by formula 4:

[0028]

[0029] In formula 4, ε is the displacement record update interval, k i represents the displacement record update time closest to the time when vehicle i is marked as “appeared”, k i Determined by formula 5:

[0030]

[0031] In formula 3, A set of three consecutive displacement records Displacement threshold ΔC1, motion state at the previous moment and the current moment t determines;

[0032] The displacement set in Eq. 3 By extracting the vehicle location record The latest 4 position data in the data are obtained by calculating the difference between the two adjacent position records, which contains a total of 3 differences, that is, three displacement amplitudes. Determined by formula 6:

[0033]

[0034] Represents the ordinate of the center point of the vehicle with id i at time t, Determined by formula 7:

[0035]

[0036] Furthermore, the above Determined by formula 10:

[0037]

[0038] In formula 10, It represents the vehicle detection data integrity coefficient of the vehicle with id i at time t. When it is 1, it means that the data is complete and can be correctly detected. When it is 0, it means that the vehicle data is incomplete and cannot be correctly detected due to occlusion. Determined by formula 11:

[0039]

[0040] In formula 11 represents the change in the number of people in the vehicle with id i at time t, Determined by formula 12, that is:

[0041]

[0042] In formula 12, Indicates the status coefficient of the person with id j at time t. The person is judged to be "disappeared" =1, the person is judged as "appeared" is -1, otherwise is 0, Determined by formula 13:

[0043]

[0044] In formula 12, is the position relationship coefficient between car i and person j at time t. It is 1 when the center point of person j is within the detection range of car i at time t, otherwise it is 0. Determined by formula 14; at the same time, the coordinates of the center point of the pedestrian at time t, the coordinates of the center point of the vehicle at time t, and the maximum and minimum values ​​of the horizontal and vertical coordinates of the detection range are calculated and returned by the multi-target tracking judgment model. is the minimum value of the horizontal coordinate of the detection range of the car with id i at time t, is the maximum value of the horizontal coordinate of the detection range of the car with id i at time t, is the minimum value of the vertical coordinate of the detection range of the car with id i at time t, is the maximum value of the vertical coordinate of the detection range of the tracker of the car with id i at time t, is the horizontal coordinate of the center point of the person with id j at time t, The vertical coordinate of the center point of the person with id j at time t;

[0045]

[0046] Furthermore, the above target recognition model is a YOLO_v4 algorithm model; and the above multi-target tracking and judgment model is a DeepSORT algorithm model.

[0047] The beneficial effects of the present invention are: the false recognition rate is very low, which can meet the application requirements, can greatly reduce the workload of monitoring law enforcement personnel, improve the efficiency and accuracy of non-site law enforcement, and have broad development prospects in the field of intelligent traffic supervision. It can further strengthen the safety management of airport landside traffic and provide effective guarantee for the long-term development of the transportation industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 : Flowchart of the present invention; DETAILED DESCRIPTION

[0049] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments:

[0050] Example 1: Figure 1 As shown in Figure 1, YOLO_v4 is used for target recognition and DeepSORT is used for target tracking. A surveillance video of a fixed aircraft stand on the departure lane of an airport with a duration of T is used. t represents any moment in the surveillance video, t∈[0,T]; when a vehicle is identified as "appearing" by YOLO_v4 at time t, it is assigned an id of i, i∈N, and N represents the set of ids assigned by the tracking algorithm to all individuals in the video. represents the set of car IDs marked as "appearing" by DeepSORT's tracker at time t (also called memory, the same below), with Represents the set of car IDs marked as "disappeared" at time t, using represents the set of car IDs that are not marked as "missing" by the DeepSORT tracker at time t, Determined by formula (1), that is:

[0051]

[0052] Whether vehicle i can perform violation identification depends on whether it is stationary, using the parameter Represents the state correction coefficient of the vehicle with id i at time t. When it is 1, it means the vehicle is stationary, that is, violation identification can be performed. When it is 0, it means the vehicle is in other states and violation identification cannot be performed. Determined by formula (2), that is:

[0053]

[0054] In formula (2), Represents the state parameter of the vehicle with id i at time t. When it is 2, it means that the vehicle is in an unknown state where it has not yet been identified as moving or stationary. When it is 1, it means that the vehicle is stationary. When it is 0, it means that the vehicle is in motion. When it is -1, it means that the vehicle is in a state where some monitoring data is lost. Only when it is 1 can the vehicle i be identified for violation. Determined by formula (3), that is:

[0055]

[0056] In formula (3), ΔC1 represents the displacement threshold, Represents the latest three consecutive displacement records of vehicle id i at time t, Represents the wth displacement in the set of the three most recent consecutive displacement records of the vehicle with id i at time t; is the number of displacement records, Determined by formula (4), that is:

[0057]

[0058] In formula (4), ε is the displacement record update interval, k i represents the displacement record update time closest to the time when vehicle i is marked as “appeared”, k i Determined by formula (5), that is:

[0059]

[0060] In formula (3), A set of three consecutive displacement records Displacement threshold ΔC1, motion state at the previous moment and the current time t. and ΔC1, and the relationship between time t, there are four cases in total. When the vehicle is just identified and its motion state cannot be determined ( When the distance from the vehicle is marked as appearing, its motion state is 2; when the distance from the vehicle is marked as appearing, its motion state is 2; When the displacement data is obtained three times, the displacement amplitudes of three consecutive detections exceed the threshold value ΔC1, and the state of the previous moment is neither 2 nor -1, the motion state is determined to be 1; when the displacement data is obtained three times, the displacement amplitudes of three consecutive detections are less than the threshold value ΔC1, and the state of the previous moment is not -1, the motion state is determined to be 1; when the displacement data is obtained three times, the displacement amplitudes of three consecutive detections are less than the threshold value ΔC1, and the state of the previous moment is 2, or as long as the motion state of the previous moment is -1, the motion state is determined to be -1.

[0061] The displacement set in formula (3) By extracting the vehicle location record The latest 4 position data in the data are obtained by calculating the difference between the two adjacent position records, which contains a total of 3 differences, that is, three displacement amplitudes. Determined by formula (6), that is:

[0062]

[0063] in, Represents the ordinate of the center point of the vehicle with id i at time t, Determined by formula (7), that is:

[0064]

[0065] When a pedestrian's head is recognized as "appearing" by YOLO_v4 at time t, it is assigned an id of j.

[0066] j∈N, use represents the set of IDs of people marked as “appearing” by the DeepSORT tracker at time t, Represents the set of IDs of people marked as "disappeared" at time t, using represents the set of IDs of people who are not marked as “missing” by DeepSORT’s tracker at time t, Determined by formula (8), that is:

[0067]

[0068] Within the time range of the surveillance video, whether vehicle i violates traffic rules can be determined by the number of people in the vehicle when it disappears from the screen, using parameter C Nei Indicates the number of people in the car with id i when the screen disappears. represents the number of people in the vehicle with id i at time t, C Nei It can be determined by formula (9), that is:

[0069]

[0070] when When the vehicle with id i does not have any data loss, it can be divided into three cases: Nei > 0, it can be judged that the vehicle with id i has picked up passengers, that is, it is illegal; when C Nei When C is less than 0, it can be judged that the vehicle with id i has a passenger drop-off behavior, that is, it has not violated the traffic rules; Nei = 0, the vehicle with id i can be judged as a passing vehicle; when When , it means that the vehicle with id i has lost some data, and the change in the number of people in the vehicle when the screen disappears is recorded as 0, and the vehicle with id i is marked as data lost. Determined by formula (10), that is:

[0071]

[0072] In formula (10), It represents the vehicle detection data integrity coefficient of the vehicle with id i at time t. When it is 1, it means that the data is complete and can be correctly detected. When it is 0, it means that the vehicle data is incomplete and cannot be correctly detected due to occlusion. Determined by formula (11), that is:

[0073]

[0074] In formula (11) represents the change in the number of people in the vehicle with id i at time t, Determined by formula (12), that is:

[0075]

[0076] In formula (12), Indicates the status coefficient of the person with id j at time t. The person is judged to be "disappeared" =1, the person is judged as "appeared" is -1, otherwise is 0, Determined by formula (13), that is:

[0077]

[0078] In formula (12), is the position relationship coefficient between car i and person j at time t. It is 1 when the center point of person j is within the detection range of car i at time t, otherwise it is 0. Determined by formula (14). At the same time, the coordinates of the center point of the pedestrian at time t, the coordinates of the center point of the vehicle at time t, and the maximum and minimum values ​​of the horizontal and vertical coordinates of the detection range are returned by DeepSORT, where is the minimum value of the horizontal coordinate of the detection range of the car with id i at time t, is the maximum value of the horizontal coordinate of the detection range of the car with id i at time t, is the minimum value of the vertical coordinate of the detection range of the car with id i at time t, is the maximum value of the vertical coordinate of the detection range of the tracker of the car with id i at time t, is the horizontal coordinate of the center point of the person with id j at time t, The vertical coordinate of the center point of the person with id j at time t.

[0079]

[0080] The above-mentioned method for automatically identifying illegal vehicles at the departure level of an airport based on deep learning is implemented in a computer with a graphics card of 10606G, a CPU of i5-8500, and a memory of 8G, using Python language to convert formulas (1) to (14) in the content of the present invention into formulas according to the flow chart of the present invention (such as Figure 1 The method is programmed to detect video clips containing illegal passenger pick-up from the surveillance video of the No. 3 fixed-stand monitor outside the No. 3 gate of the departure level of Kunming Changshui Airport on September 1, 2019 and September 2, 2019. Then, all illegal passenger pick-up information is recorded by manually watching the clips, and the true value is compared with the detection result to obtain the recognition accuracy of the method. The detection results are shown in Tables 2 and 3. The average FPS of the video is 3.875, that is, nearly 4 pictures can be processed per second.

[0081] The speed of traffic violation recognition depends on the following two points: 1. The recognition rate of YOLO_v4; 2. The tracking efficiency of DeepSORT, and the recognition rate increases with the performance of the graphics card.

[0082] Table 2 Recognition results

[0083]

[0084] Table 3 Recognition accuracy

[0085]

[0086] The data in Table 2 were analyzed in detail, and the results are shown in Table 3. A total of 83.3% of the violations were identified, of which 50% were identified as picking up customers, and 33.3% were identified as suspected picking up customers, which required further manual verification. Among the total samples, 16.7% were not successfully identified as picking up customers, which was caused by two reasons: tracking loss or counting errors.

[0087] In actual applications, even if the recognition rate of illegal vehicles does not reach a high accuracy, it can still have a certain impact on the choice of drivers who intend to pick up passengers at the departure level. However, there is a requirement for the recognition rate of illegal behaviors, and it needs to reach a lower level before it can be applied. Because the identification results of illegal behaviors will eventually be uploaded to the system for fines on illegal vehicles, if the probability of non-illegal vehicles being misjudged as illegal vehicles is too high, it is likely to cause a large number of complaints. Therefore, in addition to counting the identification results of illegal pickup vehicles, this article also counts the misidentification of passenger drop-off vehicles and analyzes them.

[0088] In this example, we select a video clip with an effective duration of one and a half hours from a video clip with a total duration of six hours, and use it to further analyze the recognition accuracy. In the selected one and a half hour clip, there are a total of 10 passenger pick-up samples, 150 passenger drop-off samples, and a total of 160 vehicle samples. The data analysis of the recognition results is shown in Tables 4 and 5.

[0089] Table 4 Analysis of misidentification of passenger drop-off vehicles

[0090]

[0091] The output results only include two results: illegal pick-up and suspected pick-up. Illegal pick-up means there is illegal behavior and no manual verification is required. It can be directly uploaded to the system in the application; suspected pick-up means there may be pick-up behavior but it is not certain, and manual verification is required. During the manual verification, the misidentified results can be manually eliminated to ensure that the passenger drop-off vehicle will not be misreported. In actual operation, for the sake of management efficiency, it is also possible to choose not to conduct manual verification and treat it as not violating the law. As shown in Table 4, when uploaded to the system, the probability of the passenger drop-off vehicle being misreported is 0.

[0092] Table 5 Distribution of samples identified as illegal pick-up

[0093]

[0094] As shown in Table 5, among the total 160 samples, there are a total of 6 samples that are identified as illegal pick-up, of which 0 are actually samples of dropping off passengers. That is, in this automatic identification that directly determines it as illegal pick-up and does not require further manual verification, the false recognition rate is 0.

[0095] In summary, compared with the prior art, the present invention introduces an automatic identification method for illegal vehicles at the departure level of an airport based on deep learning. First, the characteristics of illegal vehicle picking-up behavior are proposed, that is, the algebraic difference of the total number of pedestrians "appearing" and "disappearing" in the detection range of a stationary car is recorded to calculate the final number of people in the vehicle, so as to determine whether the vehicle has picked up passengers. Then, based on YOLO_v4 and DeepSORT, a vehicle static state recognition method and a vehicle picking-up behavior recognition method are proposed, and a preliminary method for identifying illegal vehicles picking up passengers at the lane side of the departure level of an airport is constructed. The recognition accuracy of illegal vehicles obtained by example analysis is 50%, and the recognition accuracy of picking-up behavior reaches 83.3%. Although the recognition accuracy needs to be further improved, in practical applications, it is enough to have a certain impact on the choice of drivers who intend to pick up passengers at the departure level; and the algorithm has a very low misrecognition rate, which can meet the requirements of the application, can greatly reduce the workload of monitoring law enforcement personnel, improve the efficiency and accuracy of non-site law enforcement, and has broad development prospects in the field of intelligent traffic supervision, which can further strengthen the safety management of airport landside traffic and provide effective guarantees for the long-term development of the transportation industry.

[0096] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically identifying illegal vehicles at the airport departure level based on deep learning, characterized by: The following steps are involved: S1: Obtain the video stream, import the target recognition model and the multi-target tracking judgment model, and make frame-by-frame judgments on the video stream; S2: Target recognition and judgment: vehicles and pedestrians are taken as recognition targets and assigned IDs respectively. If it is judged to be a vehicle, step S3 is executed; if it is judged to be a person, step S4 is executed until every frame of the video file is judged; S3: Identify whether the vehicle is in a stationary state, and save the vehicle identification parameter data set to a parameter memory; S4: Identify whether the pedestrian disappears, and save the pedestrian recognition parameter data set to the parameter memory; S5: Calculate the data set stored in the parameter memory in S3 and S4, and calculate the final change in the number of people in the vehicle by the algebraic difference of the total number of times pedestrians "appear" and "disappear" within the detection range of a stationary vehicle; S6: Based on the final change in the number of people in the vehicle, identify the violation of the vehicle marked as disappeared; when the change in the number of people is greater than 0, it is judged that the vehicle has picked up passengers, that is, it has violated the traffic rules; when the change in the number of people is less than 0, it is judged that the vehicle has dropped off passengers, that is, it has not violated the traffic rules; when the change in the number of people = 0, it is judged that the vehicle is a passing vehicle; When the data of the number of people change is lost, it means that the vehicle has lost some data, and its number of people change is recorded as 0, and the vehicle is marked as data lost; S7: output results; The video stream in S1 is a surveillance video of a fixed aircraft position beside the lane on the departure level of the airport with a duration of T, and t is used to represent any moment of the surveillance video, t∈[0,T]; The specific steps of S2, S3, S5 and S6 are as follows: when a vehicle is identified by the target recognition model at time t and judged as "appearing", it is assigned an id of i, i∈N, where N represents the set of ids assigned by the tracking algorithm to all individuals in the video; use Represents the set of car IDs marked as "appearing" by the tracker of the multi-target tracking judgment model at time t, using Represents the set of car IDs marked as "disappeared" at time t, using represents the set of car IDs that are not marked as "disappeared" by the tracker of the multi-target tracking judgment model at time t, Determined by formula 1: Whether the vehicle is in a stationary state using parameters Represents the state correction coefficient judgment of the vehicle with id i at time t. When it is 1, it means that the vehicle is stationary and can be identified for violations. When it is 0, it means that the vehicle is in other states and cannot be identified for violations. Determined by formula 2: In formula 2, Represents the state parameter of the vehicle with id i at time t. When it is 2, it means that the vehicle is in an unknown state where it has not yet been identified as moving or stationary. When it is 1, it means that the vehicle is stationary. When it is 0, it means that the vehicle is in motion. When it is -1, it means that the vehicle is in a state where some monitoring data is lost. Only when it is 1 can the violation of vehicle i be identified.

2. The method for automatically identifying illegal vehicles at the airport departure level based on deep learning as claimed in claim 1, characterized in that: The specific steps of S2, S4, S5 and S6 are as follows: when a pedestrian is identified by the target recognition model at time t and judged as "appearing", the pedestrian is assigned an id of j, j∈N, and Represents the set of IDs of people marked as "appearing" by the tracker of the multi-target tracking judgment model at time t, using Represents the set of IDs of people marked as "disappeared" at time t, using Represents the set of IDs of people who are not marked as "disappeared" by the tracker of the multi-target tracking judgment model at time t, Determined by formula 8: Within the time range of the surveillance video, whether vehicle i violates traffic rules is determined by the number of people in the vehicle when it disappears from the screen, using parameter C Nei Indicates the number of people in the car with id i when the screen disappears. represents the number of people in the vehicle with id i at time t, C Nei Determined by formula 9: when When the vehicle with id i does not have any missing data, there are three cases: Nei > 0, it is judged that the vehicle with id i has picked up passengers, that is, it is illegal; when C Nei <0, it is judged that the vehicle with id i has the behavior of dropping off passengers, that is, it has not violated the traffic rules; when C Nei = 0, the vehicle with id i is judged as a passing vehicle; when When , it means that the vehicle with id i has lost some data, and the change in the number of people in the vehicle when the screen disappears is recorded as 0, and the vehicle with id i is marked as having lost data.

3. The method for automatically identifying illegal vehicles at the airport departure level based on deep learning as claimed in claim 1, characterized in that: Said Determined by formula 3: In formula 3, ΔC1 represents the displacement threshold, Represents the latest three consecutive displacement records of the vehicle with id i at time t, Represents the wth displacement in the set of the three most recent consecutive displacement records of the vehicle with id i at time t; is the number of displacement records, Determined by formula 4: In formula 4, ε is the displacement record update interval, k i represents the displacement record update time closest to the time when vehicle i is marked as "appeared", k i Determined by formula 5: In formula 3, A set of three consecutive displacement records Displacement threshold ΔC1, motion state at the previous moment and the current moment t determines; The displacement set in Eq. 3 By extracting the vehicle location record The latest 4 position data in the data are obtained by calculating the difference between the two adjacent position records, which contains a total of 3 differences, that is, three displacement amplitudes. Determined by formula 6: Represents the ordinate of the center point of the vehicle with id i at time t, Determined by formula 7:

4. The method for automatically identifying illegal vehicles at the airport departure level based on deep learning as claimed in claim 2, characterized in that: Said Determined by formula 10, that is: In formula 10, It represents the vehicle detection data integrity coefficient of the vehicle with id i at time t. When it is 1, it means that the data is complete and correctly detected. When it is 0, it means that the vehicle data is incomplete and cannot be correctly detected due to occlusion. Determined by formula 11: In formula 11 represents the change in the number of people in the vehicle with id i at time t, Determined by formula 12, that is: In formula 12, Indicates the status coefficient of the person with id j at time t. The person is judged to be "disappeared" is 1, the person is judged to be "present" is -1, otherwise is 0, Determined by formula 13: In formula 12, is the position relationship coefficient between car i and person j at time t. It is 1 when the center point of person j is within the detection range of car i at time t, otherwise it is 0. Determined by formula 14; at the same time, the coordinates of the center point of the pedestrian at time t, the coordinates of the center point of the vehicle at time t, and the maximum and minimum values ​​of the horizontal and vertical coordinates of the detection range are calculated and returned by the multi-target tracking judgment model. is the minimum value of the horizontal coordinate of the detection range of the car with id i at time t, is the maximum value of the horizontal coordinate of the detection range of the car with id i at time t, is the minimum value of the vertical coordinate of the detection range of the car with id i at time t, is the maximum value of the vertical coordinate of the detection range of the tracker of the car with id i at time t, is the horizontal coordinate of the center point of the person with id j at time t, The vertical coordinate of the center point of the person with id j at time t; 5. The method for automatically identifying illegal vehicles at the airport departure level based on deep learning according to any one of claims 1 to 4, characterized in that: The target recognition model is the YOLO_v4 algorithm model; the multi-target tracking judgment model is the DeepSORT algorithm model.

Images