A Vehicle Gate Passing Judgment Method and Related Device Based on Image Recognition Technology

Abstract

The present application discloses a vehicle passing through gate judgment method and related device based on image recognition technology, which is used to detect and track the driving trajectory of a vehicle by using a deep learning algorithm, and then judge whether the vehicle passes through the gate according to the driving trajectory, thereby improving the accuracy of vehicle passing through gate judgment. The method of the present application includes: obtaining a video of the vehicle yard entrance and exit through a yard camera; extracting target images frame by frame from the video of the vehicle yard entrance and exit; detecting the vehicle and license plate in the target image by using a target detection algorithm; identifying license plate information of the license plate in the detected target image; binding according to the license plate information and the position relationship between the vehicle and the license plate in the target image, and determining the target vehicle according to the binding result; tracking the vehicle trajectory of the target vehicle by using a tracking algorithm; judging whether the target vehicle passes through the gate according to the vehicle trajectory.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and particularly to a vehicle gate passing judgment method and related device based on image recognition technology. Background Art

[0002] With the progress and development of society, the number and scale of parking lots have also increased significantly. Accurately judging whether a vehicle enters the parking lot is of great significance. It not only relates to the counting of remaining parking spaces in the lot, but also to the reasonable charging for vehicle entry and exit and the control of non-motor vehicles.

[0003] In the prior art, there are various ways to judge whether a vehicle passes through the gate: one is the method based on inductive loop, which is installed under the gate and determines the presence of a vehicle by detecting metal objects. This method cannot truly judge that it is a vehicle passing through the gate, and it is difficult to distinguish whether two vehicles or one vehicle passes through the gate when there are other vehicles following. Another is the method based on radar sensors, which places radar sensors on both sides of the road and monitors the electromagnetic waves returned by the vehicle, thereby judging whether the vehicle passes through the gate by measuring the change in distance. This method also cannot accurately distinguish whether the passing object is a non-motor vehicle or a vehicle.

[0004] In summary, the implementation of the method for judging whether a vehicle passes through the gate in the prior art cannot determine whether the passing object is a motor vehicle or a non-motor vehicle, and cannot judge whether the vehicle reverses and leaves, resulting in low accuracy of vehicle gate passing judgment. Summary of the Invention

[0005] This application provides a vehicle gate passing judgment method and related device based on image recognition technology, which is used to detect and track the driving trajectory of the vehicle by using a deep learning algorithm, and then judge whether the vehicle passes through the gate according to the driving trajectory, thereby improving the accuracy of vehicle gate passing judgment.

[0006] The first aspect of this application provides a vehicle gate passing judgment method based on image recognition technology, including:

[0007] Obtain the video of the parking lot entrance and exit through the parking lot camera;

[0008] Extract target images frame by frame from the video of the parking lot entrance and exit;

[0009] Use a target detection algorithm to detect the vehicle and license plate in the target image;

[0010] Identify the license plate information of the license plate in the detected target image;

[0011] Bind according to the license plate information and the positional relationship between the vehicle and the license plate in the target image, and determine the target vehicle according to the binding result;

[0012] Adopt a tracking algorithm to track the vehicle trajectory of the target vehicle;

[0013] Judge whether the target vehicle passes through the gate according to the vehicle trajectory.

[0014] Optionally, the judging whether the target vehicle passes through the gate according to the vehicle trajectory includes:

[0015] Calculate the trajectory direction and trajectory displacement of the target vehicle according to the vehicle trajectory, where the vehicle trajectory is composed of the center points of the vehicle detection frames in several frames of the target image;

[0016] Judge whether the number of frames of the vehicle trajectory is less than a preset number of frames;

[0017] If not, then judge whether the vehicle trajectory meets the first gate-passing condition;

[0018] The first gate-passing condition includes:

[0019] The vehicle trajectory disappears from the lower boundary of the target image, the length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0;

[0020] When it is determined that the vehicle trajectory meets the first gate-passing condition, it is determined that the target vehicle has passed through the gate.

[0021] Optionally, after judging whether the number of frames of the vehicle trajectory is less than the preset number of frames, the method further includes:

[0022] If so, it is determined that the target vehicle has not passed through the gate.

[0023] Optionally, the method further includes:

[0024] When it is determined that the vehicle trajectory does not meet the first gate-passing condition, then judge whether the vehicle trajectory meets the second gate-passing condition;

[0025] The second gate-passing condition is:

[0026] The length of the trajectory displacement is greater than 200 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0.5;

[0027] Or,

[0028] The length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0.9;

[0029] If it is determined that the vehicle trajectory meets the second gate-passing condition, it is determined that the target vehicle has passed through the gate;

[0030] If it is determined that the vehicle trajectory does not meet the second gate passing condition, it is determined that the target vehicle has not passed through the gate.

[0031] Optionally, before calculating the trajectory direction and trajectory displacement of the target vehicle based on the vehicle trajectory, the method further includes:

[0032] Preprocess the vehicle trajectory to remove vehicle trajectories that do not meet the judgment criteria.

[0033] Optionally, after calculating the trajectory direction and trajectory displacement of the target vehicle based on the vehicle trajectory, and before determining whether the number of trajectory frames of the vehicle trajectory is less than a preset number of frames, the method further includes:

[0034] Perform a rating process on the vehicle trajectory;

[0035] Determine whether the number of disappearing frames of the vehicle trajectory is less than a dynamic threshold, where the dynamic threshold is determined by the result of the rating process;

[0036] The determination of whether the number of trajectory frames of the vehicle trajectory is less than a preset number of frames includes:

[0037] If it is determined that the number of disappearing frames of the vehicle trajectory is greater than the dynamic threshold, then determine whether the number of trajectory frames of the vehicle trajectory is less than a preset number of frames.

[0038] The second aspect of the present application provides a vehicle gate passing judgment system based on image recognition technology, including:

[0039] An acquisition unit for acquiring a video of the vehicle yard entrance and exit through a vehicle yard camera;

[0040] An extraction unit for extracting a target image frame by frame from the video of the vehicle yard entrance and exit;

[0041] A detection unit for detecting vehicles and license plates in the target image using a target detection algorithm;

[0042] An identification unit for identifying license plate information of the license plate in the detected target image;

[0043] A binding unit for binding according to the license plate information and the positional relationship between the vehicle and the license plate in the target image, and determining a target vehicle according to the binding result;

[0044] A tracking unit for tracking the vehicle trajectory of the target vehicle using a tracking algorithm;

[0045] A judgment unit for judging whether the target vehicle has passed through the gate according to the vehicle trajectory.

[0046] Optionally, the judgment unit includes:

[0047] A calculation module, configured to calculate the trajectory direction and trajectory displacement of the target vehicle according to the vehicle trajectory, where the vehicle trajectory is composed of the center points of the vehicle detection frames in a plurality of frames of the target image;

[0048] A first judgment module, configured to judge whether the number of frames of the vehicle trajectory is less than a preset number of frames;

[0049] A second judgment module, configured to judge whether the vehicle trajectory meets the first gate-passing condition when the judgment result of the first judgment module is negative;

[0050] The first gate-passing condition includes:

[0051] The vehicle trajectory disappears from the lower boundary of the target image, the length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0;

[0052] A first determination module, configured to determine that the target vehicle has passed through the gate when the second judgment module determines that the vehicle trajectory meets the first gate-passing condition.

[0053] Optionally, the system may further include:

[0054] A second determination module, configured to determine that the target vehicle has not passed through the gate when the judgment result of the first judgment module is positive.

[0055] Optionally, the judgment unit may further include:

[0056] A third judgment module, configured to judge whether the vehicle trajectory meets the second gate-passing condition when the second judgment module determines that the vehicle trajectory does not meet the first gate-passing condition;

[0057] The second gate-passing condition is:

[0058] The length of the trajectory displacement is greater than 200 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0.5;

[0059] Or,

[0060] The length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0.9;

[0061] The first determination module is further configured to determine that the target vehicle has passed through the gate when the judgment result of the third judgment module is positive;

[0062] The second determination module is further configured to determine that the target vehicle has not passed through the gate when the judgment result of the third judgment module is negative.

[0063] Optionally, the determination unit further includes:

[0064] A preprocessing module for preprocessing the vehicle trajectory to remove vehicle trajectories that do not meet the judgment criteria.

[0065] Optionally, the determination unit further includes:

[0066] A rating module for performing rating processing on the vehicle trajectory;

[0067] A fourth determination module for determining whether the disappearance frame number of the vehicle trajectory is less than a dynamic threshold, where the dynamic threshold is determined by the result of the rating processing;

[0068] The first determination module is specifically configured to:

[0069] When the determination result of the fourth determination module is negative, determine whether the trajectory frame number of the vehicle trajectory is less than a preset frame number.

[0070] A third aspect of the present application provides a vehicle passing through a gate determination device based on image recognition technology, and the device includes:

[0071] A processor, a memory, an input / output unit, and a bus;

[0072] The processor is connected to the memory, the input / output unit, and the bus;

[0073] The memory stores a program, and the processor calls the program to execute the vehicle passing through a gate determination method based on image recognition technology in the first aspect and any optional one in the first aspect.

[0074] A fourth aspect of the present application provides a computer-readable storage medium, and a program is stored on the computer-readable storage medium, and when the program is executed on a computer, it executes the vehicle passing through a gate determination method based on image recognition technology in the first aspect and any optional one in the first aspect.

[0075] From the above technical solutions, it can be seen that the present application has the following advantages:

[0076] Aiming at the problem that the current passing through a gate determination method cannot determine whether the object passing through the gate is a motor vehicle or a non-motor vehicle, the present invention can distinguish the category of the target through a target detection algorithm, such as only detecting motor vehicles and license plates, and then binding the license plate to the vehicle and then tracking, so as to filter non-motor vehicles and some unlicensed vehicles.

[0077] The present invention detects and tracks the driving trajectory of a vehicle through the existing entrance and exit cameras in the parking lot, and comprehensively judges whether the vehicle passes through the gate or leaves from other directions through the direction, displacement, etc. of the driving trajectory, thereby improving the accuracy of vehicle passing through a gate determination. Brief Description of the Drawings

[0078] To more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0079] Figure 1 FIG. is a schematic flowchart of an embodiment of a vehicle gate passing judgment method based on image recognition technology provided by the present application;

[0080] Figure 2 FIG. is a schematic flowchart of another embodiment of a vehicle gate passing judgment method based on image recognition technology provided by the present application;

[0081] Figure 3 FIG. is a schematic diagram of a vehicle trajectory in a vehicle gate passing judgment method based on image recognition technology provided by the present application;

[0082] Figure 4 FIG. is a schematic structural diagram of an embodiment of a vehicle gate passing judgment system based on image recognition technology provided by the present application;

[0083] Figure 5 FIG. is a schematic structural diagram of another embodiment of a vehicle gate passing judgment system based on image recognition technology provided by the present application;

[0084] Figure 6 FIG. is a schematic structural diagram of an embodiment of a vehicle gate passing judgment device based on image recognition technology provided by the present application. Detailed Description of the Embodiments

[0085] The present application provides a vehicle gate passing judgment method and related devices based on image recognition technology, which are used to detect and track the driving trajectory of a vehicle using a deep learning algorithm, and then judge whether to pass through the gate according to the driving trajectory, thereby improving the accuracy of vehicle gate passing judgment.

[0086] It should be noted that the vehicle gate passing judgment method based on image recognition technology provided by the present application can be applied to a terminal or a server. For example, the terminal can be a smart phone, a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of description, the terminal is taken as the execution subject in the present application for illustration.

[0087] Please refer to Figure 1 , Figure 1 FIG. is an embodiment of a vehicle gate passing judgment method based on image recognition technology provided by the present application. The method includes:

[0088] 101. Obtain the video of the parking lot entrance and exit through the parking lot camera;

[0089] In this application, through the existing entrance and exit cameras in the parking lot, combined with deep learning algorithms, the judgment of vehicle passing through the gate is realized. First, the terminal needs to obtain the on-site video captured by the parking lot camera. The parking lot camera in this application refers to the camera installed at the entrance and exit (gate) of the parking lot.

[0090] 102. Extract the target images frame by frame from the video of the parking lot entrance and exit;

[0091] The terminal extracts the target images frame by frame from the obtained video of the parking lot entrance and exit. It should be noted that to ensure the accuracy of subsequent detection and recognition, the target image is at least a color image with a resolution of 1920*1080.

[0092] 103. Use the target detection algorithm to detect the vehicles and license plates in the target images;

[0093] The terminal uses the target detection algorithm to detect the vehicles and license plates in the target images. Specifically, the target detection algorithm can adopt the mainstream SSD object detection algorithm. It should be noted that during the detection process, non-motor vehicles in the target images will be regarded as the background and will not be detected. Therefore, the vehicle passing through the gate judgment method provided in this application can filter non-motor vehicles.

[0094] 104. Identify the license plate information of the license plates in the detected target images;

[0095] The terminal extracts the detected license plate parts in the target images and performs license plate information identification. The processing of the license plate recognition part includes license plate correction, license plate type judgment, license plate voting, and multi-recognition filtering, etc., which are not specifically limited here.

[0096] 105. Bind according to the license plate information and the position relationship between the vehicles and license plates in the target images, and determine the target vehicle according to the binding result;

[0097] The terminal completes the binding of the vehicle and the license plate through the license plate information and the position relationship between the detected vehicle and license plate in the target image. Since the parking lot camera is usually installed at a height between 1 meter and 2 meters, multiple license plates and multiple vehicles can be captured in its picture. The terminal can allocate and bind according to the position relationship between the license plate and the vehicle. Specifically, the vehicle detection box and the license plate detection box calculate the IOU with each other, and then use the IOU as the cost matrix. The Hungarian assignment algorithm is adopted to match one license plate with one vehicle. According to the matching result, the license plate and the vehicle are bound, and the binding result is determined as the target vehicle.

[0098] It should be noted that in the step of binding the vehicle and the license plate, vehicles that do not match the license plate in multiple frames will be treated as unlicensed vehicles, that is, binding cannot be achieved, and subsequent gate passing judgment cannot be achieved. Therefore, the vehicle gate passing judgment method provided in this application can filter unlicensed vehicles.

[0099] 106. Use a tracking algorithm to track the vehicle trajectory of the target vehicle;

[0100] After the terminal determines the target vehicle, it uses a tracking algorithm to track the vehicle trajectory of the target vehicle. Specifically, the DeepSort algorithm can be used to track the target vehicle. The DeepSort algorithm depends on object detection and can use the IOU and CNN features between vehicle detection boxes in two frames of target images for object association. When the trajectory is associated with the vehicle detection box for multiple consecutive frames (3 frames), a new vehicle ID is considered to be generated. When the trajectory is not associated with the vehicle detection box for multiple consecutive frames (30 frames), the vehicle is considered to have disappeared.

[0101] 107. Determine whether the target vehicle passes the gate according to the vehicle trajectory.

[0102] The terminal makes a gate passing judgment based on the vehicle trajectory of the target vehicle. This vehicle trajectory is composed of the center points of the vehicle detection boxes corresponding to the target vehicle in several frames of target images.

[0103] It should be added that after tracking the vehicle trajectory, the terminal can also learn the traffic flow direction, that is, learn a "conventional" vehicle traveling direction through the vehicle trajectories of the previous few vehicles (usually 10 vehicles). This traveling direction can more accurately assist in judging vehicle gate passing. Specifically, when the first vehicle appears, the "conventional" vehicle traveling direction is initialized from top to bottom, that is, the direction vector (0, 1). After the subsequent vehicles pass the gate, the terminal will save their trajectories when passing the gate, and then average all the passing trajectories of the previous 10 vehicles to obtain a "conventional" vehicle traveling direction to assist in vehicle gate passing judgment.

[0104] In this embodiment, for the problem that the current gate passing judgment method cannot determine whether the object passing the gate is a motor vehicle or a non-motor vehicle, this embodiment can distinguish the category of the target through the object detection algorithm, only detect the vehicles and license plates of motor vehicles, and then bind the license plate to the vehicle and then track, so as to filter non-motor vehicles and some unlicensed vehicles.

[0105] In this embodiment, through the existing entrance and exit cameras in the parking lot, the vehicle is detected and the driving trajectory of the vehicle is tracked, and through comprehensive judgment of the direction, displacement, etc. of the driving trajectory, it is determined whether the vehicle passes through the gate or leaves from other directions, thereby improving the accuracy of vehicle gate passing judgment.

[0106] The following will describe in detail the gate passing judgment process in the vehicle gate passing judgment method provided by this application based on image recognition technology. Please refer to Figure 2 and Figure 3 , Figure 2 which is another embodiment of the vehicle gate passing judgment method provided by this application based on image recognition technology. Figure 3 which is a schematic diagram of the vehicle trajectory in the vehicle gate passing judgment method provided by this application based on image recognition technology. The method includes:

[0107] 201. Obtain the video of the vehicle yard entrance and exit through the yard camera;

[0108] 202. Extract the target image frame by frame in the video of the vehicle yard entrance and exit;

[0109] 203. Detect the vehicle and license plate in the target image by using the target detection algorithm;

[0110] 204. Identify the license plate information of the license plate in the detected target image;

[0111] 205. Bind according to the license plate information and the position relationship between the vehicle and the license plate in the target image, and determine the target vehicle according to the binding result;

[0112] 206. Track the vehicle trajectory of the target vehicle by using the tracking algorithm;

[0113] In this embodiment, steps 201 to 206 are similar to steps 101 to 106 in the foregoing embodiment, and will not be elaborated here.

[0114] 207. Preprocess the vehicle trajectory to remove the vehicle trajectories that do not meet the judgment criteria;

[0115] After the terminal obtains the vehicle trajectory of the target vehicle, it needs to perform corresponding preprocessing on the vehicle trajectory. The purpose of this preprocessing step is to pre-remove some trajectories that do not meet the judgment criteria, such as trajectories with too few frames (less than 20 frames) or trajectories with too short disappearance frames (less than 6 frames). For these trajectories that do not meet the judgment criteria, the terminal will not make a judgment.

[0116] 208. Calculate the trajectory direction and trajectory displacement of the target vehicle according to the vehicle trajectory. The vehicle trajectory is composed of the center points of the vehicle detection frames in several frames of the target image;

[0117] When the terminal obtains the vehicle trajectory that meets the judgment conditions, it calculates the trajectory direction and trajectory displacement of the target vehicle according to the vehicle trajectory. The vehicle trajectory provided in this embodiment is as Figure 3As shown in the figure, the set of circles is the vehicle driving trajectory (composed of the center points of the vehicle detection frames), and the starting point and the ending point are respectively composed of the average values of the first 5 points at the start and the last 5 points at the end. The trajectory displacement is the vector pointing from the starting point to the ending point, and another straight line is the conventional vehicle driving direction generated through the learning of the traffic flow direction. The terminal calculates the cosine value between the two vectors to indicate whether the vehicle is approaching or moving away from the gate.

[0118] 209. Determine whether the number of frames of the vehicle trajectory is less than the preset number of frames. If not, execute step 210; if so, directly execute step 213.

[0119] The terminal determines whether the number of frames of the vehicle trajectory is less than the preset number of frames. A normal vehicle passing through the gate usually takes about 5s, that is, about 125 frames, from the start to the disappearance in the field of view. Therefore, the preset number of frames can be set to about 40 frames to eliminate some misdetection situations.

[0120] When the terminal determines that the number of frames of the trajectory is greater than the preset number of frames, execute step 210 to enter the further judgment of passing through the gate.

[0121] When the terminal determines that the number of frames of the trajectory is less than the preset number of frames, it means that the vehicle trajectory is very short and may be a background misdetection. At this time, execute step 213 to determine that the target vehicle has not passed through the gate, and then perform the passing-through-gate judgment on the next target vehicle.

[0122] 210. Determine whether the vehicle trajectory meets the first passing-through-gate condition. If not, execute step 211; if so, directly execute step 212.

[0123] After the terminal determines that the number of frames of the vehicle trajectory is greater than the preset number of frames, it makes a judgment on the first passing-through-gate condition. In step 210, the terminal needs to judge 3 conditions simultaneously, which are as follows:

[0124] 1. The vehicle trajectory disappears from the lower boundary of the target image;

[0125] 2. The length of the trajectory displacement is greater than 100 pixels;

[0126] 3. The cosine value between the trajectory direction and the preset direction is greater than 0.

[0127] When the terminal determines that the vehicle trajectory of the target vehicle simultaneously meets the above three conditions, execute step 212 to return the result that the target vehicle has passed through the gate. When any one of the above three conditions is not met, execute step 211 to make a judgment on the second passing-through-gate condition.

[0128] It should be noted that to determine whether the vehicle trajectory disappears from the lower boundary of the image, it is possible to count the lower boundaries of the vehicle detection frames in the latest 15 frames of the trajectory and check whether the lower boundaries of more than 10 frames are close to the lower boundary of the target image. The preset direction in this embodiment refers to the "conventional" vehicle traveling direction learned by the terminal through the vehicle trajectories of the first few vehicles (usually 10 vehicles). This traveling direction can more accurately assist in judging whether the vehicle passes through the gate. Specifically, when the first vehicle appears, the "conventional" vehicle traveling direction is initialized from top to bottom, that is, the direction vector (0, 1). After the subsequent vehicles pass through the gate, the terminal will save their trajectories when passing through the gate, and then average all the passing-through trajectories of the previous 10 vehicles to obtain a "conventional" vehicle traveling direction to assist in judging whether the vehicle passes through the gate.

[0129] 211. Determine whether the vehicle trajectory meets the second gate-passing condition. If so, execute step 212; if not, execute step 213.

[0130] When the terminal determines that the vehicle trajectory does not meet the first gate-passing condition, it continues to determine whether the vehicle trajectory meets the second gate-passing condition. The second gate-passing condition is as follows:

[0131] 1. The length of the trajectory displacement is greater than 200 pixels and the cosine value between the trajectory direction and the preset direction is greater than 0.5.

[0132] Or,

[0133] 2. The length of the trajectory displacement is greater than 100 pixels and the cosine value between the trajectory direction and the preset direction is greater than 0.9.

[0134] Among them, the second gate-passing condition 1 can summarize the situation where the vehicle travels a relatively long distance but the direction is not very straight, which may occur in some large trucks or T-shaped intersections. And the second gate-passing condition 2 can summarize the situation where the vehicle travels a relatively short distance but the direction is relatively straight, which generally occurs during night driving. Since the imaging at night is not as good as that during the day and the detection of vehicles in the distance is relatively poor, the displacement of the vehicle trajectory tracked by the terminal is relatively short.

[0135] Through the supplementary judgment of the second gate-passing condition 1 and the second gate-passing condition 2, it is possible to make a supplementary judgment on the vehicle passing through the gate in some special cases. When the terminal determines that the vehicle trajectory does not meet the first gate-passing condition but meets any one of the second gate-passing conditions, execute step 212 to return the result that the target vehicle has passed through the gate.

[0136] 212. Determine that the target vehicle has passed through the gate;

[0137] When the terminal determines that the vehicle trajectory of the target vehicle meets the first gate-passing condition, or does not meet the first gate-passing condition but meets any one of the second gate-passing conditions, it can be determined that the target vehicle has passed the gate. It should be noted that after the terminal determines that the target vehicle has passed the gate, the gate-passing judgment of the next target vehicle is carried out, and the result of having passed the gate is only returned once for a vehicle trajectory.

[0138] 213. Determine that the target vehicle has not passed the gate.

[0139] In step 209, when the terminal determines that the number of frames of the vehicle trajectory is less than the preset number of frames, it means that this vehicle trajectory is very short and may be a background misdetection. At this time, it can directly return that the target vehicle has not passed the gate to perform the gate-passing judgment of the next target vehicle.

[0140] Another situation is that when the terminal determines that the number of frames of the vehicle trajectory is greater than the preset number of frames, but the vehicle trajectory does not meet the first gate-passing condition and does not meet the second gate-passing condition, it can be determined that the target vehicle has not passed the gate.

[0141] In this embodiment, aiming at the problem that the current gate-passing judgment method cannot judge whether the object passing through the gate is a motor vehicle or a non-motor vehicle, this embodiment can distinguish the category of the target through the target detection algorithm, only detect the vehicle and license plate of the motor vehicle, and then bind the license plate to the vehicle and then track it, so as to filter non-motor vehicles and some unlicensed vehicles.

[0142] In this embodiment, through the existing entrance and exit cameras in the parking lot, the vehicle is detected and the driving trajectory of the vehicle is tracked, and through the comprehensive judgment of the direction, displacement, etc. of the driving trajectory, it is judged whether the vehicle passes through the gate or leaves from other directions. Among them, multiple judgment conditions are included in the judgment process to achieve accurate judgment of vehicle gate-passing and improve the accuracy of vehicle gate-passing judgment.

[0143] It should be added that when using the DeepSort algorithm to track the target vehicle, problems such as the vehicle being blocked by pedestrians or poor detector performance may occur, resulting in the phenomenon of ID switching, that is, the same vehicle may be considered as two vehicles at different times. Therefore, this application also proposes to perform dynamic rating processing on the vehicle trajectory, so as to increase the algorithm's ability to handle the situation of the vehicle being temporarily blocked and enhance the robustness of the algorithm.

[0144] The following describes in detail the application of this dynamic rating method in the vehicle gate-passing judgment method:

[0145] After calculating the trajectory direction and trajectory displacement of the target vehicle in step 208, before determining whether the number of trajectory frames of the vehicle trajectory is less than a preset number of frames in step 209, the terminal first performs a trajectory rating process on the vehicle trajectory, then adjusts the dynamic threshold according to the result of the trajectory rating, and then determines whether the number of disappearing frames of the vehicle trajectory is greater than the dynamic threshold. When the number of disappearing frames of the vehicle trajectory is greater than the dynamic threshold, step 209 and subsequent gate passing judgments are executed. If the number of disappearing frames of the vehicle trajectory is less than the dynamic threshold, the result of not passing the gate is directly returned.

[0146] Specifically, the levels of the vehicle trajectory can be roughly divided into the following three types:

[0147] Level 1: If the number of vehicle trajectory frames is less than 40 frames or the trajectory displacement is less than 40 pixels. Then adjust the dynamic threshold to 125 frames, and change the number of disappearing frames of the vehicle trajectory to 126. This type of trajectory is most likely a trajectory blocked by the vehicle in front or a pedestrian. Adjusting the threshold larger gives the trajectory a looser disappearing condition, so that the trajectory can continue after the vehicle in front or the pedestrian leaves.

[0148] Level 2: If the displacement length of the trajectory is greater than 400 pixels, the cosine value between the trajectory direction and the preset direction is greater than 0.9, and the area of the detection frames in the last 5 frames of the trajectory shows a decreasing trend (that is, it is statistically determined whether there are more than 3 detection frames among the 5 detection frames with an area reduced by 10% compared to the previous detection frame). When these three conditions are met simultaneously, the dynamic threshold is adjusted to 6. This type of trajectory is considered a relatively good trajectory. The vehicle travels a long distance, and the direction is very close to the normal driving direction, and it also conforms to the rule that the detection frame becomes smaller when disappearing in the picture. For such a trajectory, it can be directly determined that the gate is passed.

[0149] Level 3: When the vehicle trajectory does not meet the standards of Level 1 and Level 2, it is considered an ordinary trajectory. At this time, the dynamic threshold is adjusted to 50, and the number of disappearing frames of the trajectory is changed to 51. Such a trajectory may be caused by poor detector performance. Extending the number of disappearing frames can increase the robustness of the algorithm.

[0150] Please refer to Figure 4 , Figure 4 which is an embodiment of the vehicle gate passing judgment system based on image recognition technology provided by this application. The system includes:

[0151] An acquisition unit 401, configured to obtain a video of the vehicle yard entrance and exit through a yard camera;

[0152] An extraction unit 402, configured to extract a target image frame by frame from the video of the vehicle yard entrance and exit;

[0153] A detection unit 403, configured to detect vehicles and license plates in the target image by using a target detection algorithm;

[0154] An identification unit 404 for identifying license plate information of a license plate in the detected target image;

[0155] A binding unit 405 for binding according to the license plate information and the positional relationship between the vehicle and the license plate in the target image, and determining a target vehicle according to the binding result;

[0156] A tracking unit 406 for tracking the vehicle trajectory of the target vehicle by using a tracking algorithm;

[0157] A judging unit 407 for judging whether the target vehicle passes through the gate according to the vehicle trajectory.

[0158] In this embodiment, the detection unit 403 can distinguish the categories of targets through a target detection algorithm, only detect the vehicles and license plates of motor vehicles, and then the binding unit 405 binds the license plate to the vehicle, and the tracking unit 406 performs tracking, so as to filter non-motor vehicles and some vehicles without license plates.

[0159] The present invention detects and tracks the driving trajectory of a vehicle through an existing entrance and exit camera in a parking lot, and comprehensively judges whether the vehicle passes through the gate or leaves from other directions by judging the direction, displacement, etc. of the driving trajectory through the judging unit 407, thereby improving the accuracy of vehicle gate passing judgment.

[0160] The vehicle gate passing judgment system provided by the present application will be described in detail below. Please refer to Figure 5 , Figure 5 which is another embodiment of the vehicle gate passing judgment system based on image recognition technology provided by the present application. The system includes:

[0161] An acquisition unit 501 for acquiring a video of the entrance and exit of the parking lot through a parking lot camera;

[0162] An extraction unit 502 for extracting target images frame by frame from the video of the entrance and exit of the parking lot;

[0163] A detection unit 503 for detecting vehicles and license plates in the target image by using a target detection algorithm;

[0164] An identification unit 504 for identifying license plate information of a license plate in the detected target image;

[0165] A binding unit 505 for binding according to the license plate information and the positional relationship between the vehicle and the license plate in the target image, and determining a target vehicle according to the binding result;

[0166] A tracking unit 506 for tracking the vehicle trajectory of the target vehicle by using a tracking algorithm;

[0167] A determination unit 507, configured to determine whether the target vehicle passes through the gate according to the vehicle trajectory.

[0168] Optionally, the determination unit 507 includes:

[0169] A calculation module 5071, configured to calculate the trajectory direction and trajectory displacement of the target vehicle according to the vehicle trajectory, where the vehicle trajectory is composed of the center points of the vehicle detection frames in several frames of the target images;

[0170] A first determination module 5072, configured to determine whether the number of frames of the vehicle trajectory is less than a preset number of frames;

[0171] A second determination module 5073, configured to determine whether the vehicle trajectory meets the first gate-passing condition when the determination result of the first determination module 5072 is negative;

[0172] The first gate-passing condition includes:

[0173] The vehicle trajectory disappears from the lower boundary of the target image, the length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0;

[0174] A first determination module 5074, configured to determine that the target vehicle has passed through the gate when the second determination module 5073 determines that the vehicle trajectory meets the first gate-passing condition.

[0175] Optionally, the system further includes:

[0176] A second determination module 5075, configured to determine that the target vehicle has not passed through the gate when the determination result of the first determination module 5072 is positive.

[0177] Optionally, the determination unit 507 further includes:

[0178] A third determination module 5076, configured to determine whether the vehicle trajectory meets the second gate-passing condition when the second determination module 5073 determines that the vehicle trajectory does not meet the first gate-passing condition;

[0179] The second gate-passing condition is:

[0180] The length of the trajectory displacement is greater than 200 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0.5;

[0181] Or,

[0182] The length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0.9;

[0183] The first determination module 5074 is further configured to determine that the target vehicle has passed the gate when the determination result of the third determination module 5076 is yes;

[0184] The second determination module 5075 is further configured to determine that the target vehicle has not passed the gate when the determination result of the third determination module 5076 is no.

[0185] Optionally, the determination unit 507 further includes:

[0186] A preprocessing module 5077, configured to preprocess the vehicle trajectory to remove vehicle trajectories that do not meet the judgment criteria.

[0187] Optionally, the determination unit 507 further includes:

[0188] A rating module 5078, configured to perform a rating process on the vehicle trajectory;

[0189] A fourth determination module 5079, configured to determine whether the disappearance frame number of the vehicle trajectory is less than a dynamic threshold, where the dynamic threshold is determined by the result of the rating process;

[0190] The first determination module 5072 is specifically configured to:

[0191] When the determination result of the fourth determination module 5079 is no, determine whether the trajectory frame number of the vehicle trajectory is less than a preset frame number.

[0192] In the system of this embodiment, the functions of each unit correspond to the steps in the foregoing Figure 2 method embodiment shown, and will not be elaborated here.

[0193] This application further provides a vehicle gate passing determination device based on image recognition technology. Please refer to Figure 6 , Figure 6 which is an embodiment of the vehicle gate passing determination device based on image recognition technology provided by this application. The device includes:

[0194] A processor 601, a memory 602, an input / output unit 603, and a bus 604;

[0195] The processor 601 is connected to the memory 602, the input / output unit 603, and the bus 604;

[0196] The memory 602 stores a program, and the processor 601 calls the program to execute any one of the vehicle gate passing determination methods based on image recognition technology as described above.

[0197] The present application also relates to a computer-readable storage medium, on which a program is stored. It is characterized in that when the program runs on a computer, the computer is caused to execute any one of the above vehicle gate passing judgment methods based on image recognition technology.

[0198] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0199] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0200] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0202] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, read-only memory), random access memories (RAM, random access memory), magnetic disks, or optical discs.

Claims

1. A vehicle gate passing judgment method based on image recognition technology, characterized in that, The method includes: Obtaining the video of the parking lot entrance and exit through the parking lot camera; Frame-by-frame extracting the target images from the video of the parking lot entrance and exit; Using an object detection algorithm to detect the vehicles and license plates in the target images, the object detection algorithm being the SSD object detection algorithm, and the vehicles being motor vehicles; Identifying the license plate information of the license plates in the detected target images; Binding according to the license plate information and the positional relationship between the vehicles and license plates in the target images by using the IOU and Hungarian assignment algorithms, and determining the target vehicles according to the binding results; Tracking the vehicle trajectories of the target vehicles by using the DeepSort tracking algorithm; Judging whether the target vehicles pass through the gate according to the vehicle trajectories; The judging whether the target vehicles pass through the gate according to the vehicle trajectories includes: Calculating the trajectory direction and trajectory displacement of the target vehicles according to the vehicle trajectories, the vehicle trajectories being composed of the center points of the vehicle detection frames in a plurality of frames of the target images; Judging whether the number of frames of the vehicle trajectories is less than a preset number of frames; If not, judging whether the vehicle trajectories meet the first gate-passing condition; The first gate-passing condition includes: The vehicle trajectories disappear from the lower boundary of the target images, the length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0; When it is determined that the vehicle trajectories meet the first gate-passing condition, it is determined that the target vehicles have passed through the gate.

2. The method according to claim 1, wherein After judging whether the number of frames of the vehicle trajectories is less than the preset number of frames, the method further includes: If so, determining that the target vehicles have not passed through the gate.

3. The method according to claim 1, wherein The method further includes: When it is determined that the vehicle trajectories do not meet the first gate-passing condition, judging whether the vehicle trajectories meet the second gate-passing condition; The second gate-passing condition is: The length of the trajectory displacement is greater than 200 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0.5; Or, The length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0.9; If it is determined that the vehicle trajectories meet the second gate-passing condition, it is determined that the target vehicles have passed through the gate; If it is determined that the vehicle trajectories do not meet the second gate-passing condition, it is determined that the target vehicles have not passed through the gate.

4. The method according to claim 1, wherein Before calculating the trajectory direction and trajectory displacement of the target vehicles according to the vehicle trajectories, the method further includes: Preprocessing the vehicle trajectories to remove the vehicle trajectories that do not meet the judgment criteria.

5. The method according to claim 2, wherein After calculating the trajectory direction and trajectory displacement of the target vehicles according to the vehicle trajectories and before judging whether the number of frames of the vehicle trajectories is less than the preset number of frames, the method further includes: Performing a rating process on the vehicle trajectories; Judging whether the number of disappearing frames of the vehicle trajectories is less than a dynamic threshold, the dynamic threshold being determined by the result of the rating process; The judging whether the number of frames of the vehicle trajectories is less than the preset number of frames includes: If it is determined that the number of disappearing frames of the vehicle trajectories is greater than the dynamic threshold, judging whether the number of frames of the vehicle trajectories is less than the preset number of frames.

6. A vehicle gate passing judgment system based on image recognition technology, characterized in that, The system includes: An acquisition unit for acquiring the video of the parking lot entrance and exit through a parking lot camera; An extraction unit for extracting target images frame by frame from the video of the parking lot entrance and exit; A detection unit for detecting vehicles and license plates in the target images by using a target detection algorithm, where the target detection algorithm is the SSD object detection algorithm and the vehicle is a motor vehicle; An identification unit for identifying license plate information of the license plates in the detected target images; A binding unit for binding according to the license plate information and the position relationship between the vehicle and the license plate in the target images by using the IOU and Hungarian assignment algorithms, and determining the target vehicle according to the binding result; A tracking unit for tracking the vehicle trajectory of the target vehicle by using the DeepSort tracking algorithm; A judgment unit for judging whether the target vehicle passes through the gate according to the vehicle trajectory; The judgment unit includes: A calculation module for calculating the trajectory direction and trajectory displacement of the target vehicle according to the vehicle trajectory, where the vehicle trajectory is composed of the center points of the vehicle detection frames in a plurality of frames of the target images; A first judgment module for judging whether the number of frames of the vehicle trajectory is less than a preset number of frames; A second judgment module for judging whether the vehicle trajectory meets the first gate-passing condition when the judgment result of the first judgment module is negative; The first gate-passing condition includes: The vehicle trajectory disappears from the lower boundary of the target image, the length of the trajectory displacement is greater than 100 pixels, and the cosine value between the trajectory direction and the preset direction is greater than 0; A first determination module for determining that the target vehicle has passed through the gate when the second judgment module determines that the vehicle trajectory meets the first gate-passing condition.

7. A vehicle gate passing judgment device based on image recognition technology, characterized in that, The device includes: A processor, a memory, an input-output unit, and a bus; The processor is connected to the memory, the input-output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, on which a program is stored, and the program executes the method according to any one of claims 1 to 5 when executed on a computer.

Images