A vehicle illegal parking detection system based on drone patrol shooting

Abstract

The present invention discloses a vehicle illegal parking detection system based on drone patrol shooting, which relates to the technical field of vehicle illegal parking patrol inspection. It includes a vehicle illegal parking detection platform, and the vehicle illegal parking detection platform is communicatively connected with a drone image acquisition module, an image preprocessing module, a vehicle detection and analysis module, an illegal parking determination module, a license plate recognition module, and a warning notification module. Among them, the electrical signals are connected between the modules. Through drone patrol shooting, the present invention can quickly cover a large area, realize real-time monitoring and detection of vehicle illegal parking behaviors. Compared with the traditional manual patrol method, drone patrol not only greatly shortens the detection cycle, but also significantly improves the detection accuracy. Through high-precision image recognition algorithms, the vehicle illegal parking detection system can accurately identify the license plate number, vehicle type, and illegal parking location, effectively avoiding the subjectivity and errors of manual judgment.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle illegal parking inspection, and particularly relates to a vehicle illegal parking detection system based on drone inspection and shooting. Background Art

[0002] With the improvement of people's living standards and the continuous increase in the number of automobiles, the urban traffic pressure has increased. Illegal parking has become one of the important reasons for traffic congestion. Traditional inspection methods, such as manual inspection and ground vehicle inspection, have many limitations. Limited by human, material and environmental factors, they often lead to low inspection efficiency, limited coverage area, and relatively large safety hazards. The development of drone technology, especially the application of high-definition cameras and artificial intelligence algorithms, enables drones to conduct road monitoring efficiently and flexibly.

[0003] For example, in the method, device, storage medium and computer equipment for detecting license plates of illegally parked vehicles with the Chinese patent application publication number CN115457532A, a regional image of an illegally parked area captured by a drone is obtained; and the regional image is subjected to vehicle recognition through a pre-trained vehicle recognition model to obtain a plurality of illegally parked vehicle images parked in the illegally parked area.

[0004] In the prior art, by training three recognition models to sequentially recognize the corresponding image content of the regional image of the illegally parked area captured by the drone, the accuracy of obtaining the license plate number of the illegally parked vehicle is greatly improved. However, for some vehicles, especially those without license plates, temporary license plates or severely damaged license plates, effective recognition cannot be carried out. Moreover, in the case of dense vehicles, the difficulty of judging the vehicle status is further increased. Therefore, how to analyze the vehicle status in combination with the drone inspection images to classify the vehicle illegal parking status is the problem to be solved. For this reason, a vehicle illegal parking detection system based on drone inspection and shooting is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a vehicle illegal parking detection system based on drone inspection and shooting to solve the problems raised in the above background art.

[0006] To solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A vehicle illegal parking detection system based on drone inspection and shooting includes a vehicle illegal parking detection platform. The vehicle illegal parking detection platform is communicatively connected to a drone image acquisition module, an image preprocessing module, a vehicle detection and analysis module, an illegal parking determination module, a license plate recognition module, and a warning notification module. Among them, the modules are electrically connected to each other;

[0008] The UAV image acquisition module is responsible for the takeoff, flight path planning, altitude adjustment, hovering and landing operations of the UAV, ensuring that the UAV can conduct inspections according to the predetermined route and altitude, and capturing the image information of ground vehicles in real time through the high-definition camera carried by the UAV, providing data support for subsequent vehicle status analysis;

[0009] The image preprocessing module preprocesses the collected vehicle image information, including denoising, enhancing contrast, and correcting colors, to improve the image quality, provides clear image input for subsequent vehicle detection and recognition, and extracts vehicle feature information from the preprocessed image;

[0010] The vehicle detection and analysis module combines deep learning algorithms and vehicle feature information to detect the position and contour of the vehicle and identify the vehicles in the image information;

[0011] The illegal parking determination module automatically determines whether a vehicle is illegally parked according to traffic regulations and the vehicles identified from UAV images, reducing the error of human judgment and improving the fairness and accuracy of illegal parking determination;

[0012] The license plate recognition module classifies the illegally parked vehicles into vehicles with license plates and warning vehicles for the identified illegally parked vehicles, recognizes the license plates and extracts vehicle information. Among them, warning vehicles include vehicles without license plates, temporary license plates and vehicles with severely damaged license plates;

[0013] The warning notification module remotely sends text messages to the owners of illegally parked vehicles with license plates, improving the timeliness of illegal parking handling, giving the owners timely warnings and opportunities for correction. For warning vehicles, it triggers an alarm mechanism and sends the illegal parking information to the traffic management department for on-site handling.

[0014] A further improvement of the technical solution of the present invention lies in: in the UAV image acquisition module, the acquisition process of ground vehicle image information includes:

[0015] Configure a task planning system, mark the inspection area on the map, clarify the inspection tasks for vehicle illegal parking detection, and match the inspection area and inspection tasks;

[0016] Build a task planning system and input the geographical information of the inspection area into the task planning system to plan the flight path of the UAV, including setting the takeoff point, inspection route, hovering point and landing point;

[0017] Start the UAV to conduct inspections according to the preset flight path. In the hovering state, use the high-definition camera carried by the UAV to capture the image information of ground vehicles in real time, ensuring that the captured image information has consistent resolution and clarity;

[0018] Configure wireless communication technologies (such as Wi-Fi, 4G / 5G, etc.) to transmit the ground vehicle image information captured by the drone camera to the vehicle illegal parking detection platform in real time, and store the received ground vehicle image information in the cloud storage of the vehicle illegal parking detection platform to ensure that the stored image information is complete, clear and easy to retrieve.

[0019] A further improvement of the technical solution of the present invention lies in: in the image preprocessing module, the process of extracting vehicle feature information includes:

[0020] Read the vehicle image information captured by the drone from the cloud storage of the vehicle illegal parking detection platform, and perform preprocessing operations such as denoising, contrast enhancement and color correction on the vehicle image information.

[0021] Perform normalization processing on the preprocessed vehicle image information, scale the pixel values to a unified range, so that the brightness and contrast of the image are consistent under different conditions. Normalization helps to reduce the impact of illumination changes on image analysis and improve the robustness of vehicle detection and recognition.

[0022] Perform feature analysis on the processed vehicle image information, and comprehensively use methods of edge detection, corner detection and contour extraction to extract feature information for assisting vehicle recognition and license plate recognition. Among them, use the Canny edge detection algorithm to extract the edge information in the vehicle image, use the Harris corner detection algorithm to detect the corners in the image, use the contour tracking algorithm to extract the contour information of the vehicle, and extract the color features and shape features from the vehicle image. The feature data set is obtained by comprehensively extracting the features.

[0023] A further improvement of the technical solution of the present invention lies in: in the vehicle detection and analysis module, the process of vehicle recognition in the image information includes:

[0024] Collect a large amount of vehicle image data from the sources of public data sets (such as KITTI, Pascal VOC, COCO, etc.), covering vehicle categories under different environmental conditions, and ensure that the data set contains vehicle images under different weather, different illumination conditions, different angles and different backgrounds. The vehicle categories include sedans, trucks and motorcycles.

[0025] Use the LabelImg annotation tool to annotate the collected images, determine the position of the vehicle and create a bounding box, and at the same time mark the category of the vehicle, and divide the annotated data set into a training set and a test set.

[0026] Use a convolutional neural network as the infrastructure of the vehicle recognition model, and use the labeled training set data to train the vehicle recognition model in combination with the convolutional neural network. During the training process, adjust the parameters of the model, such as the learning rate, batch size, etc., to achieve better detection results, and use the data of the test set to test the trained vehicle recognition model to ensure the recognition effect of the vehicle recognition model;

[0027] Input the image feature information of the feature data set into the trained vehicle recognition model, perform real-time detection on the input image, use the non-maximum suppression method to eliminate duplicate detection frames, determine the position and contour of the vehicle, output the bounding box coordinates, and after detecting the position and contour of the vehicle, the vehicle recognition model further identifies the vehicle to determine the category of the vehicle (sedan, truck, motorcycle);

[0028] Output the final detection and recognition results, including the position, contour, and category of the vehicle.

[0029] A further improvement of the technical solution of the present invention lies in that: in the illegal parking determination module, the process of determining whether a vehicle is illegally parked includes:

[0030] Integrate relevant traffic regulations, use a geographic information system to mark no-parking areas and time-limited parking spaces on the map of the patrol area, and set the parking time and no-parking signs as the vehicle illegal parking standards. Convert the vehicle illegal parking standards in traffic regulations into a format that can be recognized by a computer, and then import the marked no-parking area information, parking time limit, and vehicle illegal parking standards into the database of the vehicle illegal parking detection platform;

[0031] Based on the set vehicle illegal parking standards, determine the reference stay time and reference distance. For each detected vehicle, extract the associated data of the stay time of the vehicle in the no-parking area or time-limited parking space and the relative distance between the vehicle and the stop line. Among them, the reference stay time is a threshold preset according to traffic regulations and actual situations, which is used as the time benchmark for determining whether a vehicle is illegally parked, and the reference distance is a preset distance threshold, which is used to determine whether the vehicle crosses the line or is not parked according to regulations;

[0032] Based on the vehicle recognition results obtained from the drone images, match the identified vehicle positions with the no-parking areas and parking time limits in the database to determine whether the vehicle is in a no-parking area or exceeds the parking time limit;

[0033] Combine the relative position of the vehicle and the stop line, calculate the illegal parking evaluation index, and quantify the degree of illegal parking of the vehicle;

[0034] Preset a parking violation evaluation threshold. According to the calculated parking violation evaluation index, analyze the position of the vehicle, determine whether the recognized vehicle is parked illegally, and transmit the evaluation result of the illegally parked vehicle to the license plate recognition module. If the result of the parking violation evaluation index exceeds the parking violation evaluation threshold, it is determined that the vehicle is parked illegally.

[0035] A further improvement of the technical solution of the present invention lies in that: the calculation expression of the parking violation evaluation index is:

[0036]

[0037] Where VI is the parking violation evaluation index, T is the parking time of the vehicle in the no-parking area or the parking space with time limit, T base is the reference parking time, which is a preset threshold used as the time benchmark to determine whether the vehicle is likely to be parked illegally, T s is the time scale parameter used to adjust the sensitivity of the exponential function, D is the relative distance between the vehicle and the parking line, D base is the reference distance, which is a preset distance threshold used to determine whether the vehicle crosses the line or is parked in violation of regulations. The value range of VI is between 0 and 1.

[0038] A further improvement of the technical solution of the present invention lies in that: in the license plate recognition module, the recognition process of the license plate of the illegally parked vehicle includes:

[0039] Further process the determined image of the illegally parked vehicle to more accurately locate the license plate area. Use the edge detection algorithm (Canny edge detection) to process the image of the illegally parked vehicle to highlight the edge features in the image;

[0040] According to the shape, size and color features of the license plate, locate the license plate area in the image after edge detection. From the located license plate area, use the image cropping technology to precisely cut out the license plate image so that the cut-out license plate image contains complete license plate characters and is ready for further recognition processing;

[0041] Use the image transformation technology of affine transformation to geometrically correct the cut-out license plate image to correct the tilt or distortion of the license plate and ensure the clarity of the license plate characters. In the corrected license plate image, recognize and segment individual characters, including letters, numbers and special characters, to ensure that each character is accurately segmented and the boundaries between characters are clear;

[0042] Use optical character recognition technology to recognize each segmented character, convert the characters in the image into text information, and match the recognized license plate characters with the registered license plates in the database. Use the string matching algorithm to calculate the license plate matching coefficient;

[0043] A preset matching threshold is used to classify illegally parked vehicles into vehicles with license plates and warning vehicles according to the license plate matching coefficient. Vehicles with a license plate matching coefficient higher than the matching threshold are regarded as vehicles with license plates, and vehicles with a license plate matching coefficient lower than the matching threshold are regarded as warning vehicles. Among them, vehicles with license plates refer to vehicles with complete license plate information that matches the database, and warning vehicles include vehicles without license plates, temporary license plates, and vehicles with severely damaged license plates.

[0044] A further improvement of the technical solution of the present invention lies in that: the calculation process of the license plate matching coefficient is as follows:

[0045] Determine the total number of characters in the license plate and a preset edit distance reference value, and extract the text information after optical character recognition conversion for each character in the license plate image;

[0046] For each character i in the license plate, compare it with the standard characters in the database to obtain a matching score. The matching score is determined by comparing the visual similarity and shape matching of the characters, and calculate the average value and standard deviation of all matching scores;

[0047] For each letter, use the exponential function to calculate its matching score deviation to obtain the matching score deviation function F(m), and sum the values of the matching score deviation functions of all characters;

[0048] Use the string matching algorithm of Levenshtein distance to calculate the edit distance between the recognized license plate and the standard license plate in the database, and calculate the minimum value function of the license plate character distance;

[0049] Multiply the sum of the values of the matching score deviation functions of all characters by the result of the minimum value function to obtain the final license plate matching coefficient, and compare it with the matching threshold to determine whether the vehicle is a vehicle with a license plate or a warning vehicle.

[0050] A further improvement of the technical solution of the present invention lies in that: the calculation expression of the license plate matching coefficient is:

[0051]

[0052] Where PMI is the license plate matching coefficient, F(m) is the matching score deviation function, n is the total number of characters in the license plate, m i is the matching score of the i-th character, obtained by comparing the recognized character with the standard character in the database, b is the average value of all matching scores, used as the reference value, s is the standard deviation of the matching score, used to adjust the sensitivity of the exponential function, d is the edit distance (Levenshtein distance) between the recognized license plate and the standard license plate in the database, d base is the preset edit distance reference value, used to determine the minimum acceptable difference between license plate characters, and the value range of PMI is between 0 and 1.

[0053] A further improvement of the technical solution of the present invention lies in that: in the early warning notification module, the triggering process of the early warning notification includes:

[0054] Receiving the output of the illegal parking determination module to determine the illegally parked vehicle, and classifying the vehicle as a vehicle with a license plate or a warning vehicle according to the result of the license plate recognition module;

[0055] For vehicles with license plates, retrieve the corresponding owner contact information through the vehicle database, and use the SMS gateway to send a parking violation warning message to the owner of the illegally parked vehicle with a license plate. The content of the message includes the location of the illegally parked vehicle, the parking violation time, the preset vehicle relocation time, possible penalty information, and a guide for the owner to take action;

[0056] For warning vehicles (vehicles without license plates, temporary license plates, or severely damaged license plates), trigger the alarm mechanism, send the parking violation information and the description of the vehicle (model, color, location) to the traffic management department, dispatch law enforcement officers to the scene for verification and handling, contact the owner to relocate the vehicle according to the reserved number of the vehicle, and issue a penalty list if the owner cannot be contacted;

[0057] Track the responses of the owner and the traffic management department, update the status of the parking violation handling. For vehicles with license plates, if the owner does not respond to the SMS notification and relocate the vehicle within the preset time (15 minutes), the system will trigger the alarm mechanism and automatically send the detailed information of the illegally parked vehicle (including license plate number, parking violation time, location, photo evidence, etc.) to the traffic management department. After receiving the information, the traffic management department will dispatch law enforcement officers to the scene for verification and handling, and the law enforcement officers will impose penalties such as fines and towing on the illegally parked vehicle according to the on-site situation;

[0058] The system automatically records the key information of each parking violation notice, owner response, and traffic management department handling for subsequent analysis and tracking, and generates a parking violation handling report, including statistical data on parking violation events, handling efficiency, and owner response. According to the feedback and statistical data of the parking violation handling, continuously optimize the early warning notification process and strategy.

[0059] Due to the adoption of the above technical solution, the technical progress achieved by the present invention compared with the prior art is:

[0060] 1. The present invention provides a vehicle illegal parking detection system based on drone patrol shooting. Through drone patrol shooting, a large area can be quickly covered to realize real-time monitoring and detection of vehicle illegal parking behavior. Compared with the traditional manual patrol method, drone patrol not only greatly shortens the detection cycle but also significantly improves the detection accuracy. Through high-precision image recognition algorithms, the vehicle illegal parking detection system can accurately identify license plate numbers, vehicle types, and illegal parking locations, effectively avoiding the subjectivity and errors of manual judgment.

[0061] 2. The present invention provides a vehicle illegal parking detection system based on drone patrol shooting. By using deep learning algorithms to identify vehicles and determine illegal parking, it reduces errors caused by human factors, improves the accuracy of illegal parking determination, and can timely send illegal parking warning messages to vehicle owners, informing them of their illegal parking behavior and possible penalties. The instant feedback mechanism helps vehicle owners correct their illegal parking behavior in a timely manner, avoid further penalties, and enables traffic management departments to more effectively monitor and manage urban traffic, promptly correct illegal parking behavior, thus maintaining traffic order and improving road usage efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0063] Figure 1 It is a block diagram of the modules of the present invention;

[0064] Figure 2 It is a flowchart for determining whether a vehicle is illegally parked in the present invention;

[0065] Figure 3 It is a flowchart for identifying the license plate of an illegally parked vehicle in the present invention;

[0066] Figure 4 It is a flowchart for triggering early warning notifications in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention fall within the scope of protection of the present invention.

[0068] Embodiment 1, as Figure 1 、 Figure 2 shown, the present invention provides a vehicle illegal parking detection system based on drone patrol shooting, including a vehicle illegal parking detection platform. The vehicle illegal parking detection platform is communicatively connected to a drone image acquisition module, an image preprocessing module, a vehicle detection and analysis module, an illegal parking determination module, a license plate recognition module, and an early warning notification module. Among them, the modules are electrically connected to each other;

[0069] The UAV image acquisition module is responsible for the takeoff, flight path planning, altitude adjustment, hovering and landing operations of the UAV, ensuring that the UAV can conduct inspections according to the predetermined route and altitude. Through the high-definition camera carried by the UAV, it can capture the image information of ground vehicles in real time, providing data support for subsequent vehicle status analysis. Configure a task planning system, mark the inspection area on the map, clarify the inspection tasks for vehicle illegal parking detection, match the inspection area and inspection tasks, build a task planning system, and input the geographical information of the inspection area into the task planning system to plan the flight path of the UAV, including setting the takeoff point, inspection route, hovering point and landing point. According to the requirements of the inspection tasks, adjust the altitude of the UAV to ensure that the UAV maintains a constant altitude during flight, so as to ensure that the captured image information of ground vehicles has consistent resolution and clarity. Start the UAV to conduct inspections according to the preset flight path. When the UAV reaches the predetermined hovering point, use the hovering function of the UAV system to keep the UAV in a stable hovering state. When hovering, the UAV maintains a horizontal attitude to avoid tilting or shaking, so as to ensure the accuracy of image capture. In the hovering state, use the high-definition camera carried by the UAV to capture the image information of ground vehicles in real time, ensuring that the captured image information has consistent resolution and clarity. Configure wireless communication technologies (such as Wi-Fi, 4G / 5G, etc.) to transmit the image information of ground vehicles captured by the UAV camera to the vehicle illegal parking detection platform in real time, and store the received image information of ground vehicles in the cloud storage of the vehicle illegal parking detection platform, ensuring that the stored image information is complete, clear and easy to retrieve;

[0070] The image preprocessing module preprocesses the collected vehicle image information, including denoising, enhancing contrast, and correcting color, to improve the image quality, provide a clear image input for subsequent vehicle detection and recognition, and extract vehicle feature information from the preprocessed image. Read the vehicle image information captured by the drone from the cloud storage of the vehicle illegal parking detection platform, and perform preprocessing operations such as denoising, contrast enhancement, and color correction on the vehicle image information. Among them, apply a denoising algorithm to reduce random noise in the image and improve the image clarity. Denoising helps reduce noise introduced by sensor noise and environmental interference factors during image acquisition. Enhance the contrast of the image through histogram equalization and contrast-limited adaptive histogram equalization methods. Contrast enhancement helps highlight details in the image, making the features of the vehicle more obvious. Adjust the color balance of the image according to the environmental light conditions and camera characteristics, including brightness, saturation, and hue. Color correction helps improve the visual quality of the image, making the colors more realistic and conducive to feature extraction. Normalize the preprocessed vehicle image information, scale the pixel values to a unified range, so that the brightness and contrast of the image remain consistent under different conditions. Normalization helps reduce the impact of lighting changes on image analysis and improve the robustness of vehicle detection and recognition. Perform feature analysis on the processed vehicle image information, comprehensively using methods such as edge detection, corner detection, and contour extraction, and extract feature information for assisting vehicle recognition and license plate recognition from it. Among them, use the Canny edge detection algorithm to extract edge information in the vehicle image, use the Harris corner detection algorithm to detect corners in the image, use the contour tracking algorithm to extract the contour information of the vehicle, and extract color features and shape features from the vehicle image. Combine the extracted features to obtain a feature dataset;

[0071] The vehicle detection and analysis module detects the position and contour of vehicles by combining deep learning algorithms and vehicle feature information, identifies the vehicles in the image information, collects a large amount of vehicle image data from the sources of public datasets (such as KITTI, Pascal VOC, COCO, etc.), covering vehicle categories under different environmental conditions, ensuring that the dataset contains vehicle images with different weather, different lighting conditions, different angles and different backgrounds. The vehicle categories include sedans, trucks, and motorcycles. Use the LabelImg annotation tool to annotate the collected images, determine the position of the vehicles and create bounding boxes, while marking the vehicle categories, and divide the annotated dataset into a training set and a test set. Use a convolutional neural network as the basic architecture of the vehicle recognition model, and use the annotated training set data combined with the convolutional neural network to train the vehicle recognition model. During the training process, adjust the parameters of the model, such as the learning rate, batch size, etc., to achieve better detection effects, and use the data of the test set to test the trained vehicle recognition model to ensure the recognition effect of the vehicle recognition model. Input the image feature information of the feature dataset into the trained vehicle recognition model to perform real-time detection on the input images. Use the non-maximum suppression method to eliminate duplicate detection boxes, determine the position and contour of the vehicles, output the bounding box coordinates, and after detecting the position and contour of the vehicles, the vehicle recognition model further identifies the vehicles to determine the vehicle categories (sedans, trucks, motorcycles), and outputs the final detection and recognition results, including the position, contour, and category of the vehicles;

[0072] The illegal parking judgment module automatically determines whether a vehicle is parked illegally based on traffic regulations and the vehicles identified from the drone images, reducing the errors of manual judgment, improving the fairness and accuracy of illegal parking judgment. It integrates relevant traffic regulations, uses a geographic information system to mark no-parking areas and time-limited parking spaces on the map of the inspection area, and sets the parking time and no-parking signs as the vehicle illegal parking criteria. It converts the vehicle illegal parking criteria in traffic regulations into a format recognizable by a computer, and then imports the marked no-parking area information, parking time limit, and vehicle illegal parking criteria into the database of the vehicle illegal parking detection platform. Based on the set vehicle illegal parking criteria, a reference stay time and a reference distance are determined. For each detected vehicle, associated data such as the stay time of the vehicle in the no-parking area or time-limited parking space and the relative distance between the vehicle and the parking line are extracted. Among them, the reference stay time is a threshold preset according to traffic regulations and actual situations, which is used as the time benchmark for determining whether a vehicle is parked illegally, and the reference distance is a preset distance threshold, which is used to determine whether the vehicle crosses the line or is parked in violation of regulations. Based on the vehicle recognition result obtained from the drone image, the identified vehicle position is matched with the no-parking area and parking time limit in the database to determine whether the vehicle is in the no-parking area or exceeds the parking time limit. Combining the relative position of the vehicle and the parking line, an illegal parking evaluation index is calculated to quantify the degree of illegal parking of the vehicle. A preset illegal parking evaluation threshold is set. According to the calculated illegal parking evaluation index, the position of the vehicle is analyzed to determine whether the identified vehicle is parked illegally, and the evaluation result of the illegally parked vehicle is transmitted to the license plate recognition module. If the result of the illegal parking evaluation index exceeds the illegal parking evaluation threshold, it is determined that the vehicle is parked illegally;

[0073] Further, the calculation expression of the illegal parking evaluation index is:

[0074]

[0075] where VI is the illegal parking evaluation index, T is the stay time of the vehicle in the no-parking area or time-limited parking space, T base is the reference stay time, which is a preset threshold used as the time benchmark for determining whether a vehicle may be parked illegally, T s is the time scale parameter used to adjust the sensitivity of the exponential function, D is the relative distance between the vehicle and the parking line, D base is the reference distance, which is a preset distance threshold used to determine whether the vehicle crosses the line or is parked in violation of regulations. The value range of VI is between 0 and 1. When the vehicle stay time T is less than the reference stay time T base , the exponential part tends to 1, and the value of VI is mainly determined by the minimum value function, that is, when the vehicle does not cross the line or is not overtime, VI is close to 1. When the vehicle stay time T exceeds T baseWhen the exponential part tends to 0, it indicates an increase in the possibility of illegal parking, and the value of VI will decrease. When the relative distance D between the vehicle and the stop line exceeds the reference distance D base When the minimum value function part tends to 0, it indicates that the vehicle has crossed the line or is parked in violation of regulations, and the value of VI will also decrease;

[0076] License plate recognition module. For the identified illegally parked vehicles, the illegally parked vehicles are classified into vehicles with license plates and warning vehicles, the license plates are recognized and vehicle information is extracted. Among them, warning vehicles include vehicles without license plates, temporary license plates, and vehicles with severely damaged license plates;

[0077] Early warning notification module. For illegally parked vehicles with license plates, text messages are remotely sent to notify the vehicle owners, improving the timeliness of handling illegal parking, and giving vehicle owners timely warnings and opportunities for correction. For warning vehicles, an alarm mechanism is triggered, and the illegal parking information is sent to the traffic management department for on-site handling.

[0078] Embodiment 2, as Figure 3 、 Figure 4 shown. On the basis of Embodiment 1, the present invention provides a technical solution: Preferably, in the license plate recognition module, the recognition process of the license plate of the illegally parked vehicle includes:

[0079] The determined illegally parked vehicle image is further processed to more accurately locate the license plate area. The Canny edge detection algorithm is used to process the illegally parked vehicle image to highlight the edge features in the image. According to the shape, size, and color features of the license plate, the license plate area is located in the edge-detected image. From the located license plate area, the license plate image is precisely cut out using image cropping technology, so that the cut-out license plate image contains complete license plate characters, and is prepared for further recognition processing. The image transformation technology of affine transformation is used to geometrically correct the cut-out license plate image to correct the tilt or distortion of the license plate and ensure the clarity of the license plate characters. In the corrected license plate image, individual characters, including letters, numbers, and special characters, are recognized and segmented, ensuring that each character is accurately segmented and the boundaries between characters are clear. Each segmented character is recognized using optical character recognition technology, the characters in the image are converted into text information, and the recognized license plate characters are matched with the registered license plates in the database. The string matching algorithm is used to calculate the license plate matching coefficient, and a matching threshold is preset. According to the license plate matching coefficient, the illegally parked vehicles are classified into vehicles with license plates and warning vehicles. Vehicles with a license plate matching coefficient higher than the matching threshold are regarded as vehicles with license plates, and vehicles with a license plate matching coefficient lower than the matching threshold are regarded as warning vehicles. Among them, vehicles with license plates refer to vehicles with complete license plate information and matching the database, and warning vehicles include vehicles without license plates, temporary license plates, and vehicles with severely damaged license plates;

[0080] Furthermore, the calculation process of the license plate matching coefficient is:

[0081] Determine the total number of characters in the license plate and a preset edit distance reference value, and extract the text information after optical character recognition conversion for each character in the license plate image. For each character i in the license plate, compare it with the standard characters in the database to obtain a matching score. The matching score is determined by comparing the visual similarity and shape matching of the characters, and calculate the average value and standard deviation of all matching scores. For each letter, calculate its matching score deviation using the exponential function to obtain the matching score deviation function F(m), and sum the values of the matching score deviation functions for all characters. Use the string matching algorithm of the Levenshtein distance to calculate the edit distance between the recognized license plate and the standard license plate in the database, and calculate the minimum value function of the license plate character distance. Multiply the sum result of the values of the matching score deviation functions for all characters by the result of the minimum value function to obtain the final license plate matching coefficient, and compare it with the matching threshold to determine whether the vehicle is a licensed vehicle or a warning vehicle;

[0082] Furthermore, the calculation expression of the license plate matching coefficient is:

[0083]

[0084] Among them, PMI is the license plate matching coefficient, F(m) is the matching score deviation function, n is the total number of characters in the license plate, m i is the matching score of the i-th character, obtained by comparing the recognized character with the standard character in the database, b is the average value of all matching scores, used as the reference value, s is the standard deviation of the matching score, used to adjust the sensitivity of the exponential function, d is the edit distance (Levenshtein distance) between the recognized license plate and the standard license plate in the database, d base is the preset edit distance reference value, used to determine the minimum acceptable difference between license plate characters. The value range of PMI is between 0 and 1. When the matching score m of each character i is close to or equal to the reference value b, the exponential part tends to 1, indicating good character matching. When the matching score m of the character i is far from the reference value b, the exponential part tends to 0, indicating poor character matching. The minimum value function ensures that if the edit distance d exceeds the reference value d base , the license plate matching coefficient PMI will be restricted, indicating that the overall matching degree is not high;

[0085] In the early warning notification module, the triggering process of the early warning notification includes:

[0086] Receive the output of the illegal parking judgment module, determine the illegally parked vehicles, and classify the vehicles as licensed vehicles or warning vehicles according to the results of the license plate recognition module. For licensed vehicles, retrieve the corresponding owner contact information through the vehicle database, and use the SMS gateway to send an illegal parking warning message to the owner of the licensed illegally parked vehicle. The content of the message includes the location of the illegally parked vehicle, the illegal parking time, the preset vehicle relocation time, possible penalty information, and a guide for the owner to take action. For warning vehicles (vehicles without license plates, temporary license plates, or severely damaged license plates), trigger the alarm mechanism, send the illegal parking information and the description of the vehicle (model, color, location) to the traffic management department, dispatch law enforcement officers to the scene for verification and handling, contact the owner to relocate the vehicle according to the reserved vehicle number. If the owner cannot be contacted, issue a penalty list, track the responses of the owner and the traffic management department, and update the status of the illegal parking handling. For licensed vehicles, if the owner does not respond to the SMS notification and relocate the vehicle within the preset time (15 minutes), the system will trigger the alarm mechanism and automatically send the detailed information of the illegally parked vehicle (including license plate number, illegal parking time, location, photo evidence, etc.) to the traffic management department. After receiving the information, the traffic management department will dispatch law enforcement officers to the scene for verification and handling. The law enforcement officers will impose penalties such as fines and towing on the illegally parked vehicle according to the on-site situation. The system automatically records the key information of each illegal parking notice, owner response, and traffic management department handling for subsequent analysis and tracking, and generates an illegal parking handling report, including statistical data on illegal parking incidents, handling efficiency, and owner responses. Continuously optimize the early warning notification process and strategy based on the feedback and statistical data of the illegal parking handling.

[0087] As described above, it is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claimed rights.

Claims

1. A vehicle illegal parking detection system based on drone inspection and photography, including a vehicle illegal parking detection platform, characterized in that: The vehicle illegal parking detection platform is communicatively connected with a drone image acquisition module, an image preprocessing module, a vehicle detection and analysis module, an illegal parking determination module, a license plate recognition module, and an early warning notification module, wherein electrical signals are connected between the modules; The UAV image acquisition module captures image information of ground vehicles in real time through a high-definition camera carried by the UAV; The image preprocessing module preprocesses the collected vehicle image information and extracts vehicle feature information from the preprocessed image; The vehicle detection and analysis module detects the position and outline of the vehicle by combining the deep learning algorithm and the vehicle feature information, and identifies the vehicle in the image information. In the vehicle detection and analysis module, the process of identifying the vehicle in the image information includes: Collect vehicle image data from public dataset sources, covering vehicle categories in different environmental conditions, including cars, trucks, and motorcycles; Use the LabelImg annotation tool to annotate the collected images, determine the location of the vehicle and create a bounding box, mark the category of the vehicle, and divide the annotated dataset into a training set and a test set; Use convolutional neural network as the basic architecture of the vehicle recognition model, use the labeled training set data combined with the convolutional neural network to train the vehicle recognition model, and use the test set data to test the trained vehicle recognition model; The image feature information of the feature data set is input into the trained vehicle recognition model, the input image is detected in real time, the non-maximum suppression method is used to eliminate duplicate detection frames, the position and outline of the vehicle are determined, the bounding box coordinates are output, and after the position and outline of the vehicle are detected, the vehicle recognition model further identifies the vehicle and determines the category of the vehicle; Output the final detection and recognition results, including the location, outline, and category of the vehicle; The illegal parking determination module automatically determines whether a vehicle is illegally parked based on the vehicle identified from the drone image; The license plate recognition module, for the identified illegally parked vehicles, divides the illegally parked vehicles into licensed vehicles and warning vehicles, recognizes the license plates and extracts vehicle information, wherein the warning vehicles include vehicles without license plates, temporary license plates and vehicles with severely damaged license plates; The early warning notification module triggers corresponding early warning notifications for vehicles with license plates illegally parked and warning vehicles.

2. According to claim 1, a vehicle parking violation detection system based on drone inspection and photography is characterized in that: In the UAV image acquisition module, the process of acquiring ground vehicle image information includes: Configure the task planning system, mark the inspection areas on the map, clarify the inspection tasks for illegal parking detection, and match the inspection areas and inspection tasks; Build a mission planning system, input the geographic information of the inspection area into the mission planning system, and plan the flight trajectory of the drone, including setting the take-off point, inspection route, hovering point, and landing point; Start the drone to conduct inspections along the preset flight path. In the hovering state, use the high-definition camera on the drone to capture image information of ground vehicles in real time; Wireless communication technology is configured to transmit ground vehicle image information captured by the drone camera to the vehicle illegal parking detection platform in real time, and the received ground vehicle image information is stored in the cloud storage of the vehicle illegal parking detection platform.

3. The vehicle parking violation detection system based on drone inspection and photography according to claim 2 is characterized by: In the image preprocessing module, the vehicle feature information extraction process includes: Read the vehicle image information captured by the drone from the cloud storage of the vehicle illegal parking detection platform, and perform pre-processing operations such as denoising, contrast enhancement, and color correction on the vehicle image information; Normalize the preprocessed vehicle image information and scale the pixel values ​​to a uniform range so that the brightness and contrast of the image remain consistent under different conditions; The processed vehicle image information is subjected to feature analysis, and the methods of edge detection, corner detection, and contour extraction are integrated to extract feature information for assisting vehicle identification and license plate recognition. The Canny edge detection algorithm is used to extract edge information in vehicle images, the Harris corner detection algorithm is used to detect corner points in the image, the contour tracking algorithm is used to extract the contour information of the vehicle, and color features and shape features are extracted from the vehicle image. The extracted features are integrated to obtain a feature data set.

4. The vehicle parking violation detection system based on drone inspection and photography according to claim 3 is characterized by: In the illegal parking determination module, the process of determining whether a vehicle is illegally parked includes: Use geographic information systems to mark prohibited parking areas and time-limited parking spaces on the map of the inspection area, set parking times and prohibited parking signs as vehicle illegal parking standards, convert vehicle illegal parking standards into a computer-recognizable format, and then import the marked prohibited parking area information, parking time limits, and vehicle illegal parking standards into the database of the vehicle illegal parking detection platform; Based on the set vehicle illegal parking standards, the benchmark stay time and benchmark distance are determined. For each detected vehicle, the associated data of the vehicle's stay time in the prohibited parking area or time-limited parking space and the relative distance between the vehicle and the parking line are extracted. Among them, the benchmark stay time is a threshold preset according to the actual situation, which is used to determine whether the vehicle is illegally parked. The benchmark distance is a preset distance threshold, which is used to determine whether the vehicle has crossed the line or is not parked according to regulations; Based on the vehicle recognition results obtained from the drone image, the identified vehicle location is matched with the prohibited parking areas and parking time limits in the database to determine whether the vehicle is in the prohibited parking area or exceeds the parking time limit; Combined with the relative position of the vehicle and the parking line, the illegal parking evaluation index is calculated to quantify the degree of illegal parking of the vehicle; A parking violation evaluation threshold is preset, and the position of the vehicle is analyzed based on the calculated parking violation evaluation index to determine whether the identified vehicle is illegally parked. The evaluation result of the illegally parked vehicle is transmitted to the license plate recognition module. If the result of the illegal parking evaluation index exceeds the parking violation evaluation threshold, the vehicle is determined to be illegally parked.

5. The vehicle parking violation detection system based on drone inspection and photography according to claim 4 is characterized by: The calculation expression of the illegal parking evaluation index is: Among them, VI is the illegal parking evaluation index, T is the time the vehicle stays in the prohibited parking area or the time-limited parking space, and T base is the reference residence time, T s is the time scale parameter used to adjust the sensitivity of the exponential function, D is the relative distance between the vehicle and the stop line, and D base is the reference distance, and the value of VI ranges from 0 to 1.

6. The vehicle parking violation detection system based on drone inspection and photography according to claim 5 is characterized by: In the license plate recognition module, the recognition process of the illegally parked vehicle license plate includes: Further processing is performed on the determined illegally parked vehicle images, using edge detection algorithms to process the illegally parked vehicle images and highlight edge features in the images; According to the shape, size and color features of the license plate, the license plate area is located in the image after edge detection, and the license plate image is cut out from the located license plate area using image cropping technology so that the cut license plate image contains the complete license plate characters; The cut license plate image is geometrically corrected using the image transformation technology of affine transformation, and individual characters, including letters, numbers and special characters, are recognized and segmented in the corrected license plate image; Use optical character recognition technology to recognize each segmented character, convert the characters in the image into text information, match the recognized license plate characters with the registered license plates in the database, and use the string matching algorithm to calculate the license plate matching coefficient; A matching threshold is preset, and illegally parked vehicles are divided into licensed vehicles and warning vehicles according to the license plate matching coefficient. Vehicles with license plate matching coefficients higher than the matching threshold are regarded as licensed vehicles, and vehicles with license plate matching coefficients lower than the matching threshold are regarded as warning vehicles. Licensed vehicles refer to vehicles whose license plate information is complete and matches the database, and warning vehicles include vehicles without license plates, temporary license plates, and vehicles with severely damaged license plates.

7. The vehicle parking violation detection system based on drone inspection and photography according to claim 6 is characterized by: The calculation process of the license plate matching coefficient is: Determine the total number of characters in the license plate and a preset edit distance reference value, and extract text information converted by optical character recognition for each character in the license plate image; For each character i in the license plate, compare it with the standard characters in the database to obtain a matching score, which is determined by comparing the visual similarity and shape matching of the characters, and calculating the average and standard deviation of all matching scores; For each letter, use the exponential function to calculate its matching score deviation to obtain the matching score deviation function F(m), and sum the matching score deviation function values ​​of all characters; Use the Levenshtein distance string matching algorithm to calculate the edit distance between the recognized license plate and the standard license plate in the database, and calculate the minimum function of the license plate character distance; The sum of the matching score deviation function values ​​of all characters is multiplied by the result of the minimum function to obtain the final license plate matching coefficient, which is compared with the matching threshold to determine whether the vehicle is a licensed vehicle or a warning vehicle.

8. The vehicle parking violation detection system based on drone inspection and photography according to claim 7 is characterized by: The calculation expression of the license plate matching coefficient is: Among them, PMI is the license plate matching coefficient, F(m) is the matching score deviation function, n is the total number of characters in the license plate, and m is i is the matching score of the ith character, b is the average of all matching scores, which is used as the benchmark value, s is the standard deviation of the matching score, which is used to adjust the sensitivity of the exponential function, and d is the edit distance between the recognized license plate and the standard license plate in the database, d base It is the preset edit distance benchmark value, and the value range of PMI is between 0 and 1.

9. The vehicle parking violation detection system based on drone inspection and photography according to claim 8 is characterized by: In the warning notification module, the triggering process of the warning notification includes: Receive the output of the illegal parking determination module, determine the illegally parked vehicle, and classify the vehicle as a licensed vehicle or a warning vehicle based on the result of the license plate recognition module; For licensed vehicles, the corresponding owner contact information is retrieved through the vehicle database, and a parking violation warning SMS is sent to the owner of the licensed illegally parked vehicle using the SMS gateway. The SMS content includes the location of the illegally parked vehicle, the illegal parking time, the preset moving time, and the owner's guide to take action; For the warning vehicle, the alarm mechanism is triggered, and the illegal parking information and vehicle description are sent to the traffic management department, and law enforcement officers are dispatched to the scene for verification and processing. The owner is contacted according to the vehicle reserved number to move the vehicle. If the owner cannot be contacted, a penalty list is issued; Track the responses of the vehicle owner and the traffic management department, and update the status of illegal parking. For licensed vehicles, if the owner does not respond to the SMS notification and move the vehicle within the preset time, the system will trigger the alarm mechanism and automatically send the detailed information of the illegally parked vehicle to the traffic management department. After receiving the information, the traffic management department will dispatch law enforcement officers to the scene for verification and processing, and the law enforcement officers will punish the illegally parked vehicle according to the on-site situation; The system automatically records key information of each illegal parking notification, the owner's response, and the handling by the traffic management department, and generates a parking violation handling report, including statistics on illegal parking incidents, handling efficiency, and the owner's response. Based on the feedback and statistics of illegal parking handling, the early warning notification process and strategy are continuously optimized.

Images