Vehicle illegal parking detection system based on patrol shooting of unmanned aerial vehicle

Abstract

The invention discloses a vehicle illegal parking detection system based on unmanned aerial vehicle inspection shooting, and relates to the technical field of vehicle illegal parking inspection. The vehicle illegal parking detection platform is in communication connection with an unmanned aerial vehicle image acquisition module, an image preprocessing module, a vehicle detection analysis module, an illegal parking judgment module, a license plate recognition module and an early warning notification module, and all the modules are in electric signal connection. According to the method, a large area can be rapidly covered through unmanned aerial vehicle inspection shooting, real-time monitoring and detection of vehicle illegal parking behaviors are achieved, compared with a traditional manual inspection mode, unmanned aerial vehicle inspection greatly shortens the detection period, the detection accuracy is remarkably improved, and the detection efficiency is improved through a high-precision image recognition algorithm. The vehicle illegal parking detection system can accurately recognize the license plate number, the vehicle type and the illegal parking position, and subjectivity and errors of manual judgment are effectively avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle illegal parking inspection, and in particular to a vehicle illegal parking detection system based on unmanned aerial vehicle inspection and photography. Background Art

[0002] With the improvement of people's living standards and the continuous increase in the number of cars, urban traffic pressure has increased, and illegal parking has become one of the important causes of traffic congestion. Traditional inspection methods, such as manual inspection and ground vehicle inspection, have many limitations. Limited by manpower, material resources and environmental factors, they often lead to low inspection efficiency, limited coverage area, and great safety hazards. The development of drone technology, especially the application of high-definition cameras and artificial intelligence algorithms, enables drones to monitor roads efficiently and flexibly.

[0003] For example, the method, device, storage medium and computer equipment for detecting license plates of illegally parked vehicles disclosed in Chinese patent publication number CN115457532A obtain regional images of illegally parked areas taken by drones; perform vehicle recognition on the regional images through a pre-trained vehicle recognition model to obtain images of several illegally parked vehicles parked in the illegally parked areas.

[0004] In the prior art, three recognition models are trained to recognize the corresponding image content of the regional images of the illegal parking area taken by the drone in turn, which greatly improves the accuracy of obtaining the license plate numbers of illegally parked vehicles. However, for some vehicles, especially those without license plates, temporary license plates or severely damaged license plates, effective recognition cannot be performed, and in the case of dense vehicles, the difficulty of judging the vehicle status is further increased. Therefore, how to analyze the vehicle status in combination with the drone inspection images to classify the vehicle's illegal parking status is a problem we need to solve. To this end, a vehicle illegal parking detection system based on drone inspection photography is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a vehicle illegal parking detection system based on drone inspection and photography to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A vehicle illegal parking detection system based on drone inspection photography includes a vehicle illegal parking detection platform, wherein the vehicle illegal parking detection platform is communicatively connected with a drone image acquisition module, an image preprocessing module, a vehicle detection and analysis module, an illegal parking determination module, a license plate recognition module, and an early warning notification module, wherein electrical signals are connected between the modules;

[0008] The UAV image acquisition module is responsible for the take-off, flight trajectory planning, altitude adjustment, hovering and landing operations of the UAV, ensuring that the UAV can perform inspections according to the predetermined route and altitude, and through the high-definition camera carried by the UAV, it can capture the image information of ground vehicles in real time to provide data support for subsequent vehicle status analysis;

[0009] The image preprocessing module preprocesses the collected vehicle image information, including denoising, contrast enhancement, and color correction, so as to improve the image quality, provide clear image input for subsequent vehicle detection and recognition, and extract vehicle feature information from the preprocessed image;

[0010] The vehicle detection and analysis module detects the position and outline of the vehicle by combining the deep learning algorithm and the vehicle feature information, and identifies the vehicle in the image information;

[0011] The illegal parking determination module automatically determines whether a vehicle is illegally parked based on traffic regulations and the vehicle identified from the drone image, thereby reducing errors in human judgment and improving the fairness and accuracy of illegal parking determination;

[0012] The license plate recognition module, for the identified illegally parked vehicles, divides the illegally parked vehicles into licensed vehicles and warning vehicles, recognizes the license plates and extracts vehicle information, wherein the warning vehicles include vehicles without license plates, temporary license plates and vehicles with severely damaged license plates;

[0013] The early warning notification module remotely sends text messages to the owners of illegally parked vehicles with license plates, thereby improving the timeliness of illegal parking processing and giving the owners timely warnings and correction opportunities. For warning vehicles, the alarm mechanism is triggered and the illegal parking information is sent to the traffic management department for on-site processing.

[0014] A further improvement of the technical solution of the present invention is that in the drone image acquisition module, the process of acquiring ground vehicle image information includes:

[0015] Configure the task planning system, mark the inspection areas on the map, clarify the inspection tasks for illegal parking detection, and match the inspection areas and inspection tasks;

[0016] Build a mission planning system, input the geographic information of the inspection area into the mission planning system, and plan the flight trajectory of the drone, including setting the take-off point, inspection route, hovering point, and landing point;

[0017] Start the drone to conduct inspections along the preset flight path. In the hovering state, use the high-definition camera on the drone to capture image information of ground vehicles in real time, ensuring that the captured image information has consistent resolution and clarity;

[0018] Configure wireless communication technology (such as Wi-Fi, 4G / 5G, etc.) to transmit the ground vehicle image information captured by the drone camera to the vehicle illegal parking detection platform in real time, and store the received ground vehicle image information in the cloud storage of the vehicle illegal parking detection platform to ensure that the stored image information is complete, clear and easy to retrieve.

[0019] A further improvement of the technical solution of the present invention is that in the image preprocessing module, the process of extracting vehicle feature information includes:

[0020] Read the vehicle image information captured by the drone from the cloud storage of the vehicle illegal parking detection platform, and perform pre-processing operations such as denoising, contrast enhancement, and color correction on the vehicle image information;

[0021] Normalize the preprocessed vehicle image information and scale the pixel values ​​to a uniform range so that the brightness and contrast of the image remain consistent under different conditions. Normalization helps reduce the impact of illumination changes on image analysis and improves the robustness of vehicle detection and recognition.

[0022] The processed vehicle image information is subjected to feature analysis, and the methods of edge detection, corner detection, and contour extraction are integrated to extract feature information for assisting vehicle identification and license plate recognition. The Canny edge detection algorithm is used to extract edge information in vehicle images, the Harris corner detection algorithm is used to detect corner points in the image, the contour tracking algorithm is used to extract the contour information of the vehicle, and color features and shape features are extracted from the vehicle image. The extracted features are integrated to obtain a feature data set.

[0023] A further improvement of the technical solution of the present invention is that in the vehicle detection and analysis module, the process of vehicle identification in the image information includes:

[0024] Collect a large amount of vehicle image data from public datasets (KITTI, Pascal VOC, COCO, etc.), covering vehicle categories under different environmental conditions, ensuring that the dataset contains vehicle images in different weather, different lighting conditions, different angles and different backgrounds. Vehicle categories include cars, trucks, and motorcycles.

[0025] Use the LabelImg annotation tool to annotate the collected images, determine the location of the vehicle and create a bounding box, mark the category of the vehicle, and divide the annotated dataset into a training set and a test set;

[0026] Use convolutional neural network as the basic architecture of vehicle recognition model, use labeled training set data combined with convolutional neural network to train vehicle recognition model, adjust model parameters such as learning rate and batch size during training to achieve better detection effect, and use test set data to test the trained vehicle recognition model to ensure the recognition effect of vehicle recognition model;

[0027] The image feature information of the feature data set is input into the trained vehicle recognition model, the input image is detected in real time, the non-maximum suppression method is used to eliminate duplicate detection frames, the position and outline of the vehicle are determined, the bounding box coordinates are output, and after the position and outline of the vehicle are detected, the vehicle recognition model further identifies the vehicle and determines the category of the vehicle (sedan, truck, motorcycle);

[0028] Output the final detection and recognition results, including the location, outline, and category of the vehicle.

[0029] A further improvement of the technical solution of the present invention is that: in the illegal parking determination module, the process of determining whether a vehicle is illegally parked includes:

[0030] Integrate relevant traffic regulations, use geographic information systems to mark prohibited parking areas and time-limited parking spaces on the map of the inspection area, set parking time and prohibited parking signs as vehicle illegal parking standards, convert vehicle illegal parking standards in traffic regulations into a computer-recognizable format, and then import the marked prohibited parking area information, parking time limits and vehicle illegal parking standards into the database of the vehicle illegal parking detection platform;

[0031] Based on the set vehicle illegal parking standards, the benchmark stay time and benchmark distance are determined. For each detected vehicle, the associated data of the vehicle's stay time in the prohibited parking area or time-limited parking space and the relative distance between the vehicle and the parking line are extracted. Among them, the benchmark stay time is a threshold preset according to traffic regulations and actual conditions, which is used to determine whether the vehicle is illegally parked. The benchmark distance is a preset distance threshold, which is used to determine whether the vehicle has crossed the line or is not parked according to regulations;

[0032] Based on the vehicle recognition results obtained from the drone image, the identified vehicle location is matched with the prohibited parking areas and parking time limits in the database to determine whether the vehicle is in the prohibited parking area or exceeds the parking time limit;

[0033] Combined with the relative position of the vehicle and the parking line, the illegal parking evaluation index is calculated to quantify the degree of illegal parking of the vehicle;

[0034] A parking violation evaluation threshold is preset, and the position of the vehicle is analyzed based on the calculated parking violation evaluation index to determine whether the identified vehicle is illegally parked. The evaluation result of the illegally parked vehicle is transmitted to the license plate recognition module. If the result of the illegal parking evaluation index exceeds the parking violation evaluation threshold, the vehicle is determined to be illegally parked.

[0035] A further improvement of the technical solution of the present invention is that the calculation expression of the illegal parking evaluation index is:

[0036]

[0037] Among them, VI is the illegal parking evaluation index, T is the time the vehicle stays in the prohibited parking area or the time-limited parking space, and T base is the benchmark stay time, which is a preset threshold used to determine whether a vehicle may park illegally. s is the time scale parameter used to adjust the sensitivity of the exponential function, D is the relative distance between the vehicle and the stop line, and D base is the reference distance, which is a preset distance threshold used to determine whether the vehicle has crossed the line or is not parked according to regulations. The value range of VI is between 0 and 1.

[0038] A further improvement of the technical solution of the present invention is that: in the license plate recognition module, the recognition process of the license plate of an illegally parked vehicle includes:

[0039] The image of the illegally parked vehicle is further processed to more accurately locate the license plate area. The image of the illegally parked vehicle is processed using an edge detection algorithm (Canny edge detection) to highlight the edge features in the image.

[0040] According to the shape, size and color characteristics of the license plate, the license plate area is located in the image after edge detection. From the located license plate area, the license plate image is accurately cut out using image cropping technology so that the cut license plate image contains the complete license plate characters and is ready for further recognition processing;

[0041] Use affine transformation image transformation technology to perform geometric correction on the cut license plate image to correct the tilt or distortion of the license plate and ensure the clarity of the license plate characters. In the corrected license plate image, identify and segment individual characters, including letters, numbers and special characters, to ensure that each character is accurately segmented and the boundaries between characters are clear;

[0042] Use optical character recognition technology to recognize each segmented character, convert the characters in the image into text information, match the recognized license plate characters with the registered license plates in the database, and use the string matching algorithm to calculate the license plate matching coefficient;

[0043] A matching threshold is preset, and illegally parked vehicles are divided into licensed vehicles and warning vehicles according to the license plate matching coefficient. Vehicles with license plate matching coefficients higher than the matching threshold are regarded as licensed vehicles, and vehicles with license plate matching coefficients lower than the matching threshold are regarded as warning vehicles. Licensed vehicles refer to vehicles whose license plate information is complete and matches the database, and warning vehicles include vehicles without license plates, temporary license plates, and vehicles with severely damaged license plates.

[0044] A further improvement of the technical solution of the present invention is that the calculation process of the license plate matching coefficient is:

[0045] Determine the total number of characters in the license plate and a preset edit distance reference value, and extract text information converted by optical character recognition for each character in the license plate image;

[0046] For each character i in the license plate, compare it with the standard characters in the database to obtain a matching score, which is determined by comparing the visual similarity and shape matching of the characters, and calculating the average and standard deviation of all matching scores;

[0047] For each letter, use the exponential function to calculate its matching score deviation to obtain the matching score deviation function F(m), and sum the matching score deviation function values ​​of all characters;

[0048] Use the Levenshtein distance string matching algorithm to calculate the edit distance between the recognized license plate and the standard license plate in the database, and calculate the minimum function of the license plate character distance;

[0049] The sum of the matching score deviation function values ​​of all characters is multiplied by the result of the minimum function to obtain the final license plate matching coefficient, which is compared with the matching threshold to determine whether the vehicle is a licensed vehicle or a warning vehicle.

[0050] A further improvement of the technical solution of the present invention is that the calculation expression of the license plate matching coefficient is:

[0051]

[0052] Among them, PMI is the license plate matching coefficient, F(m) is the matching score deviation function, n is the total number of characters in the license plate, and m is i is the matching score of the ith character, obtained by comparing the recognized character with the standard character in the database, b is the average of all matching scores, used as a benchmark, s is the standard deviation of the matching scores, used to adjust the sensitivity of the exponential function, d is the edit distance (Levenshtein distance) between the recognized license plate and the standard license plate in the database, and d base It is a preset edit distance benchmark value used to determine the minimum acceptable difference between license plate characters. The PMI value range is between 0 and 1.

[0053] A further improvement of the technical solution of the present invention is that: in the warning notification module, the triggering process of the warning notification includes:

[0054] Receive the output of the illegal parking determination module, determine the illegally parked vehicle, and classify the vehicle as a licensed vehicle or a warning vehicle based on the result of the license plate recognition module;

[0055] For licensed vehicles, the corresponding owner contact information is retrieved through the vehicle database, and a parking violation warning SMS is sent to the owner of the licensed illegally parked vehicle using the SMS gateway. The SMS content includes the location of the illegally parked vehicle, the illegal parking time, the preset moving time, possible penalty information, and a guide for the owner to take action;

[0056] For warning vehicles (vehicles without license plates, temporary license plates, or severely damaged license plates), the alarm mechanism is triggered, and the illegal parking information and vehicle description (model, color, location) are sent to the traffic management department, and law enforcement officers are dispatched to the scene for verification and processing. The owner is contacted according to the vehicle's reserved number to move the vehicle. If the owner cannot be contacted, a penalty list is issued;

[0057] Track the responses of the vehicle owner and the traffic management department, and update the status of illegal parking. For licensed vehicles, if the owner does not respond to the SMS notification and move the vehicle within the preset time (15 minutes), the system will trigger the alarm mechanism and automatically send the detailed information of the illegally parked vehicle (including license plate number, illegal parking time, location, photo evidence, etc.) to the traffic management department. After receiving the information, the traffic management department will dispatch law enforcement officers to the scene for verification and processing. Law enforcement officers will impose fines, towing and other penalties on illegally parked vehicles according to the on-site situation;

[0058] The system automatically records key information of each illegal parking notification, the owner's response, and the handling by the traffic management department for subsequent analysis and tracking, and generates a parking violation handling report, including statistics on illegal parking incidents, handling efficiency, and the owner's response. Based on the feedback and statistics of illegal parking handling, the early warning notification process and strategy are continuously optimized.

[0059] Due to the adoption of the above technical solution, the present invention has the following technical advances compared with the prior art:

[0060] 1. The present invention provides a vehicle illegal parking detection system based on drone inspection and photography. The drone inspection and photography can quickly cover a large area and realize real-time monitoring and detection of vehicle illegal parking behaviors. Compared with the traditional manual inspection method, drone inspection not only greatly shortens the detection cycle, but also significantly improves the detection accuracy. Through the high-precision image recognition algorithm, the vehicle illegal parking detection system can accurately identify the license plate number, vehicle type and illegal parking location, effectively avoiding the subjectivity and errors of manual judgment.

[0061] 2. The present invention provides a vehicle illegal parking detection system based on drone inspection and photography. It identifies vehicles and makes illegal parking judgments through deep learning algorithms, thereby reducing errors caused by human factors and improving the accuracy of illegal parking judgments. It can also send illegal parking warning text messages to illegal parking owners in a timely manner to inform them of their illegal parking behavior and possible penalties. The instant feedback mechanism helps car owners to correct illegal parking behavior in a timely manner to avoid further penalties. It also enables traffic management departments to monitor and manage urban traffic more effectively and correct illegal parking behavior in a timely manner, thereby maintaining traffic order and improving road use efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0063] Figure 1 It is a module structure diagram of the present invention;

[0064] Figure 2 A flow chart of the present invention for determining whether a vehicle is illegally parked;

[0065] Figure 3 This is a flow chart of the recognition of the license plate of an illegally parked vehicle according to the present invention;

[0066] Figure 4 This is a triggering flow chart of the early warning notification of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Embodiment 1, as Figure 1 , Figure 2 As shown, the present invention provides a vehicle illegal parking detection system based on drone inspection and photography, including a vehicle illegal parking detection platform, the vehicle illegal parking detection platform is communicatively connected with a drone image acquisition module, an image preprocessing module, a vehicle detection and analysis module, an illegal parking determination module, a license plate recognition module and an early warning notification module, wherein the modules are connected by electrical signals;

[0069] The UAV image acquisition module is responsible for the take-off, flight trajectory planning, altitude adjustment, hovering and landing operations of the UAV, ensuring that the UAV can conduct inspections according to the predetermined route and altitude, and through the high-definition camera carried by the UAV, it can capture the image information of ground vehicles in real time to provide data support for subsequent vehicle status analysis, configure the task planning system, mark the inspection area on the map, clarify the inspection tasks of vehicle illegal parking detection, match the inspection area and inspection tasks, build a task planning system, and enter the geographic information of the inspection area in the task planning system to plan the flight trajectory of the UAV, including setting the take-off point, inspection route, hovering point and landing point. According to the requirements of the inspection task, adjust the height of the UAV to ensure that the UAV maintains a constant altitude during the flight to ensure that the captured ground vehicle image information has a certain The drone is started to inspect along the preset flight trajectory with consistent resolution and clarity. When the drone reaches the predetermined hovering point, the drone system's built-in hovering function is used to keep the drone in a stable hovering state. When hovering, the drone maintains a horizontal posture to avoid tilting or shaking to ensure the accuracy of image capture. In the hovering state, the drone's high-definition camera is used to capture image information of ground vehicles in real time to ensure that the captured image information has consistent resolution and clarity. Wireless communication technology (such as Wi-Fi, 4G / 5G, etc.) is configured to transmit the ground vehicle image information captured by the drone camera to the vehicle illegal parking detection platform in real time, and the received ground vehicle image information is stored in the cloud storage of the vehicle illegal parking detection platform to ensure that the stored image information is complete, clear and easy to retrieve.

[0070] The image preprocessing module preprocesses the collected vehicle image information, including denoising, contrast enhancement, and color correction to improve image quality, provide clear image input for subsequent vehicle detection and identification, and extract vehicle feature information from the preprocessed image. The vehicle image information captured by the drone is read from the cloud storage of the vehicle parking violation detection platform, and the vehicle image information is preprocessed by denoising, contrast enhancement, and color correction. Among them, the denoising algorithm is applied to reduce random noise in the image and improve the clarity of the image. The denoising process helps to reduce the noise introduced by sensor noise and environmental interference factors during the image acquisition process. The contrast of the image is enhanced by histogram equalization and contrast-limited adaptive histogram equalization methods. Contrast enhancement helps to highlight the details in the image and make the characteristics of the vehicle more obvious. The color balance of the image is adjusted according to the ambient light conditions and camera characteristics. Including brightness, saturation and hue. Color correction helps to improve the visual quality of the image, making the color more realistic and helpful for feature extraction. The pre-processed vehicle image information is normalized to scale the pixel values ​​to a uniform range so that the brightness and contrast of the image remain consistent under different conditions. Normalization helps to reduce the impact of lighting changes on image analysis and improve the robustness of vehicle detection and recognition. The processed vehicle image information is subjected to feature analysis. The edge detection, corner detection and contour extraction methods are combined to extract feature information for assisting vehicle recognition and license plate recognition. Among them, the Canny edge detection algorithm is used to extract edge information in the vehicle image, the Harris corner detection algorithm is used to detect corners in the image, the contour tracking algorithm is used to extract the contour information of the vehicle, and the color features and shape features are extracted from the vehicle image. The extracted features are combined to obtain a feature data set.

[0071] The vehicle detection and analysis module combines deep learning algorithms and vehicle feature information to detect the location and outline of the vehicle, identify the vehicle in the image information, and collect a large amount of vehicle image data from public data sets (KITTI, Pascal VOC, COCO, etc.), covering vehicle categories under different environmental conditions, ensuring that the data set contains vehicle images in different weather, different lighting conditions, different angles and different backgrounds. Vehicle categories include cars, trucks, and motorcycles. The collected images are annotated using the LabelImg annotation tool to determine the location of the vehicle and create a bounding box. The vehicle category is also marked, and the annotated data set is divided into a training set and a test set. A convolutional neural network is used as the infrastructure of the vehicle recognition model. The annotated training set data is combined with the convolutional neural network to train the vehicle recognition model. During the training process, the model is adjusted. The parameters of the model, such as learning rate, batch size, etc., are adjusted to achieve better detection results. The trained vehicle recognition model is tested using the data of the test set to ensure the recognition effect of the vehicle recognition model. The image feature information of the feature data set is input into the trained vehicle recognition model. The input image is detected in real time. The non-maximum suppression method is used to eliminate repeated detection frames, determine the position and outline of the vehicle, and output the bounding box coordinates. After the position and outline of the vehicle are detected, the vehicle recognition model further identifies the vehicle and determines the category of the vehicle (sedan, truck, motorcycle), and outputs the final detection and recognition results, including the position, outline, and category of the vehicle.

[0072] The illegal parking determination module automatically determines whether a vehicle is illegally parked based on traffic regulations and vehicles identified from drone images, reduces errors in human judgment, improves the fairness and accuracy of illegal parking determination, integrates relevant traffic regulations, uses geographic information systems to mark prohibited parking areas and time-limited parking spaces on the map of the inspection area, and sets parking times and prohibited parking signs as vehicle illegal parking standards. The vehicle illegal parking standards in traffic regulations are converted into a computer-recognizable format, and then the marked prohibited parking area information, parking time limits, and vehicle illegal parking standards are imported into the database of the vehicle illegal parking detection platform. Based on the set vehicle illegal parking standards, the benchmark stay time and benchmark distance are determined. For each detected vehicle, the vehicle's stay time in the prohibited parking area or time-limited parking space, and the relative distance between the vehicle and the parking line are extracted. Associated data, wherein the benchmark dwell time is a threshold value preset according to traffic regulations and actual conditions, and is used as a time benchmark to determine whether a vehicle is illegally parked; the benchmark distance is a preset distance threshold value, and is used to determine whether a vehicle has crossed a line or has not been parked in accordance with regulations; based on the vehicle recognition result obtained from the drone image, the identified vehicle position is matched with the prohibited parking area and parking time limit in the database to determine whether the vehicle is in the prohibited parking area or has exceeded the parking time limit; combined with the relative position of the vehicle and the parking line, an illegal parking evaluation index is calculated to quantify the degree of illegal parking of the vehicle; an illegal parking evaluation threshold value is preset; based on the calculated illegal parking evaluation index, the position of the vehicle is analyzed to determine whether the identified vehicle is illegally parked; and the evaluation result of the illegally parked vehicle is transmitted to the license plate recognition module; if the result of the illegal parking evaluation index exceeds the illegal parking evaluation threshold value, the vehicle is determined to be illegally parked;

[0073] Furthermore, the calculation expression of the illegal parking evaluation index is:

[0074]

[0075] Among them, VI is the illegal parking evaluation index, T is the time the vehicle stays in the prohibited parking area or the time-limited parking space, and T base is the benchmark stay time, which is a preset threshold used to determine whether a vehicle may park illegally. s is the time scale parameter used to adjust the sensitivity of the exponential function, D is the relative distance between the vehicle and the stop line, and D base The reference distance is a preset distance threshold used to determine whether the vehicle has crossed the line or parked in violation of regulations. The value range of VI is between 0 and 1. When the vehicle stay time T is less than the reference stay time T base When the exponential part tends to 1, the value of VI is mainly determined by the minimum function, that is, when the vehicle does not cross the line or exceed the time limit, VI is close to 1. When the vehicle stay time T exceeds T baseWhen the relative distance D between the vehicle and the parking line exceeds the reference distance D, the exponential part tends to 0, indicating that the possibility of illegal parking increases and the value of VI decreases. base When , the minimum function part tends to 0, indicating that the vehicle crosses the line or does not park according to regulations, and the value of VI will also decrease;

[0076] The license plate recognition module divides the illegally parked vehicles into licensed vehicles and warning vehicles, recognizes the license plates and extracts vehicle information. Warning vehicles include vehicles without license plates, temporary license plates, and vehicles with severely damaged license plates.

[0077] The early warning notification module can remotely send text messages to notify the owners of illegally parked vehicles with license plates, improve the timeliness of illegal parking processing, and provide owners with timely warnings and correction opportunities. For warning vehicles, the alarm mechanism is triggered and the illegal parking information is sent to the traffic management department for on-site processing.

[0078] Embodiment 2, as Figure 3 , Figure 4 As shown, based on Example 1, the present invention provides a technical solution: Preferably, in the license plate recognition module, the recognition process of the license plate of an illegally parked vehicle includes:

[0079] The image of the illegally parked vehicle is further processed to more accurately locate the license plate area. The edge detection algorithm (Canny edge detection) is used to process the image of the illegally parked vehicle to highlight the edge features in the image. The license plate area is located in the image after edge detection based on the shape, size and color features of the license plate. The license plate image is accurately cut out from the located license plate area using image cropping technology so that the cut license plate image contains the complete license plate characters and is ready for further recognition processing. The cut license plate image is geometrically corrected using affine transformation image transformation technology to correct the tilt or distortion of the license plate and ensure the clarity of the license plate characters. In the corrected license plate image, individual characters are identified and segmented, including Letters, numbers and special characters, ensure that each character is accurately segmented and the boundaries between characters are clear, use optical character recognition technology to identify each segmented character, convert the characters in the image into text information, and match the identified license plate characters with the registered license plates in the database, use the string matching algorithm to calculate the license plate matching coefficient, preset the matching threshold, and divide the illegally parked vehicles into licensed vehicles and warning vehicles according to the license plate matching coefficient. Vehicles with license plate matching coefficients higher than the matching threshold are regarded as licensed vehicles, and vehicles with license plate matching coefficients lower than the matching threshold are regarded as warning vehicles. Among them, licensed vehicles refer to vehicles with complete license plate information and matching with the database, and warning vehicles include vehicles without license plates, temporary license plates, and vehicles with severely damaged license plates;

[0080] Furthermore, the calculation process of the license plate matching coefficient is:

[0081] Determine the total number of characters in the license plate and the preset edit distance reference value, and extract the text information of each character in the license plate image after optical character recognition conversion. For each character i in the license plate, compare it with the standard character in the database to obtain a matching score. The matching score is determined by comparing the visual similarity and shape matching of the characters, and the average and standard deviation of all matching scores are calculated. For each letter, use an exponential function to calculate its matching score deviation to obtain a matching score deviation function F(m), and sum the matching score deviation function values ​​of all characters. Use the Levenshtein distance string matching algorithm to calculate the edit distance between the recognized license plate and the standard license plate in the database, and calculate the minimum function of the license plate character distance. Multiply the sum of the matching score deviation function values ​​of all characters by the result of the minimum function to obtain a final license plate matching coefficient, which is compared with the matching threshold to determine whether the vehicle is a licensed vehicle or a warning vehicle.

[0082] Furthermore, the calculation expression of the license plate matching coefficient is:

[0083]

[0084] Among them, PMI is the license plate matching coefficient, F(m) is the matching score deviation function, n is the total number of characters in the license plate, and m is i is the matching score of the ith character, obtained by comparing the recognized character with the standard character in the database, b is the average of all matching scores, used as a benchmark, s is the standard deviation of the matching scores, used to adjust the sensitivity of the exponential function, d is the edit distance (Levenshtein distance) between the recognized license plate and the standard license plate in the database, and d base It is a preset edit distance benchmark value used to determine the minimum acceptable difference between license plate characters. The PMI value range is between 0 and 1. When the matching score of each character is m i When the index is close to or equal to the reference value b, the index part tends to 1, indicating that the character matches well. i When far away from the reference value b, the exponential part tends to 0, indicating that the character matching is poor. The minimum function ensures that if the edit distance d exceeds the reference value d base , the license plate matching coefficient PMI will be limited, indicating that the overall matching degree is not high;

[0085] In the warning notification module, the triggering process of the warning notification includes:

[0086] Receive the output of the illegal parking determination module, determine the illegally parked vehicles, and classify the vehicles into licensed vehicles or warning vehicles based on the results of the license plate recognition module. For licensed vehicles, retrieve the corresponding owner's contact information through the vehicle database, and use the SMS gateway to send a parking warning SMS to the owner of the licensed illegally parked vehicle. The SMS content includes the location of the illegally parked vehicle, the illegal parking time, the preset moving time, possible penalty information and a guide for the owner to take action. For warning vehicles (vehicles without license plates, temporary license plates or severely damaged license plates), trigger the alarm mechanism, send the illegal parking information and the vehicle description (model, color, location) to the traffic management department, dispatch law enforcement personnel to the scene for verification and processing, contact the owner to move the vehicle according to the vehicle's reserved number, and if the owner cannot be contacted, issue a penalty list and track the owner and traffic management. The system will respond to the SMS notification and update the status of the illegal parking handling. For licensed vehicles, if the owner does not respond to the SMS notification and move the vehicle within the preset time (15 minutes), the system will trigger the alarm mechanism and automatically send the detailed information of the illegally parked vehicle (including license plate number, illegal parking time, location, photo evidence, etc.) to the traffic management department. After receiving the information, the traffic management department will dispatch law enforcement officers to the scene for verification and handling. Law enforcement officers will impose fines, towing and other penalties on illegally parked vehicles according to the on-site situation. The system automatically records each illegal parking notification, owner response, and key information handled by the traffic management department for subsequent analysis and tracking, and generates an illegal parking handling report, including statistical data of illegal parking incidents, handling efficiency and owner response. According to the feedback and statistical data of illegal parking handling, the early warning notification process and strategy are continuously optimized.

[0087] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A vehicle illegal parking detection system based on drone inspection and photography, including a vehicle illegal parking detection platform, characterized in that: The vehicle illegal parking detection platform is communicatively connected with a drone image acquisition module, an image preprocessing module, a vehicle detection and analysis module, an illegal parking determination module, a license plate recognition module, and an early warning notification module, wherein electrical signals are connected between the modules; The UAV image acquisition module captures image information of ground vehicles in real time through a high-definition camera carried by the UAV; The image preprocessing module preprocesses the collected vehicle image information and extracts vehicle feature information from the preprocessed image; The vehicle detection and analysis module detects the position and outline of the vehicle by combining the deep learning algorithm and the vehicle feature information, and identifies the vehicle in the image information; The illegal parking determination module automatically determines whether a vehicle is illegally parked based on the vehicle identified from the drone image; The license plate recognition module, for the identified illegally parked vehicles, divides the illegally parked vehicles into licensed vehicles and warning vehicles, recognizes the license plates and extracts vehicle information, wherein the warning vehicles include vehicles without license plates, temporary license plates and vehicles with severely damaged license plates; The early warning notification module triggers corresponding early warning notifications for vehicles with license plates illegally parked and warning vehicles.

2. According to claim 1, a vehicle parking violation detection system based on drone inspection and photography is characterized in that: In the UAV image acquisition module, the process of acquiring ground vehicle image information includes: Configure the task planning system, mark the inspection areas on the map, clarify the inspection tasks for illegal parking detection, and match the inspection areas and inspection tasks; Build a mission planning system, input the geographic information of the inspection area into the mission planning system, and plan the flight trajectory of the drone, including setting the take-off point, inspection route, hovering point, and landing point; Start the drone to conduct inspections along the preset flight path. In the hovering state, use the high-definition camera on the drone to capture image information of ground vehicles in real time; Wireless communication technology is configured to transmit ground vehicle image information captured by the drone camera to the vehicle illegal parking detection platform in real time, and the received ground vehicle image information is stored in the cloud storage of the vehicle illegal parking detection platform.

3. The vehicle parking violation detection system based on drone inspection and photography according to claim 2 is characterized by: In the image preprocessing module, the vehicle feature information extraction process includes: Read the vehicle image information captured by the drone from the cloud storage of the vehicle illegal parking detection platform, and perform pre-processing operations such as denoising, contrast enhancement, and color correction on the vehicle image information; Normalize the preprocessed vehicle image information and scale the pixel values ​​to a uniform range so that the brightness and contrast of the image remain consistent under different conditions; The processed vehicle image information is subjected to feature analysis, and the methods of edge detection, corner detection, and contour extraction are integrated to extract feature information for assisting vehicle identification and license plate recognition. The Canny edge detection algorithm is used to extract edge information in vehicle images, the Harris corner detection algorithm is used to detect corner points in the image, the contour tracking algorithm is used to extract the contour information of the vehicle, and the color features and shape features are extracted from the vehicle image. The extracted features are integrated to obtain a feature data set.

4. The vehicle parking violation detection system based on drone inspection and photography according to claim 3 is characterized by: In the vehicle detection and analysis module, the process of vehicle identification in image information includes: Collect vehicle image data from public dataset sources, covering vehicle categories in different environmental conditions, including cars, trucks, and motorcycles; Use the Label Img annotation tool to annotate the collected images, determine the location of the vehicle and create a bounding box, mark the vehicle category, and divide the annotated dataset into a training set and a test set; Use convolutional neural network as the basic architecture of the vehicle recognition model, use the labeled training set data combined with the convolutional neural network to train the vehicle recognition model, and use the test set data to test the trained vehicle recognition model; The image feature information of the feature data set is input into the trained vehicle recognition model, the input image is detected in real time, the non-maximum suppression method is used to eliminate duplicate detection frames, the position and outline of the vehicle are determined, the bounding box coordinates are output, and after the position and outline of the vehicle are detected, the vehicle recognition model further identifies the vehicle and determines the category of the vehicle; Output the final detection and recognition results, including the location, outline, and category of the vehicle.

5. The vehicle parking violation detection system based on drone inspection and photography according to claim 4 is characterized by: In the illegal parking determination module, the process of determining whether a vehicle is illegally parked includes: Use geographic information systems to mark prohibited parking areas and time-limited parking spaces on the map of the inspection area, set parking times and prohibited parking signs as vehicle illegal parking standards, convert vehicle illegal parking standards into a computer-recognizable format, and then import the marked prohibited parking area information, parking time limits, and vehicle illegal parking standards into the database of the vehicle illegal parking detection platform; Based on the set vehicle illegal parking standards, the benchmark stay time and benchmark distance are determined. For each detected vehicle, the associated data of the vehicle's stay time in the prohibited parking area or time-limited parking space and the relative distance between the vehicle and the parking line are extracted. Among them, the benchmark stay time is a threshold preset according to the actual situation, which is used to determine whether the vehicle is illegally parked. The benchmark distance is a preset distance threshold, which is used to determine whether the vehicle has crossed the line or is not parked according to regulations; Based on the vehicle recognition results obtained from the drone image, the identified vehicle location is matched with the prohibited parking areas and parking time limits in the database to determine whether the vehicle is in the prohibited parking area or exceeds the parking time limit; Combined with the relative position of the vehicle and the parking line, the illegal parking evaluation index is calculated to quantify the degree of illegal parking of the vehicle; A parking violation evaluation threshold is preset, and the position of the vehicle is analyzed based on the calculated parking violation evaluation index to determine whether the identified vehicle is illegally parked. The evaluation result of the illegally parked vehicle is transmitted to the license plate recognition module. If the result of the illegal parking evaluation index exceeds the parking violation evaluation threshold, the vehicle is determined to be illegally parked.

6. The vehicle parking violation detection system based on drone inspection and photography according to claim 5 is characterized by: The calculation expression of the illegal parking evaluation index is: Among them, VI is the illegal parking evaluation index, T is the time the vehicle stays in the prohibited parking area or the time-limited parking space, and T base is the reference residence time, T s is the time scale parameter used to adjust the sensitivity of the exponential function, D is the relative distance between the vehicle and the stop line, and D base is the reference distance, and the value of VI ranges from 0 to 1.

7. The vehicle parking violation detection system based on drone inspection and photography according to claim 6 is characterized by: In the license plate recognition module, the recognition process of the illegally parked vehicle license plate includes: Further processing is performed on the determined illegally parked vehicle images, using edge detection algorithms to process the illegally parked vehicle images and highlight edge features in the images; According to the shape, size and color features of the license plate, the license plate area is located in the image after edge detection, and the license plate image is cut out from the located license plate area using image cropping technology so that the cut license plate image contains the complete license plate characters; The cut license plate image is geometrically corrected using the image transformation technology of affine transformation, and individual characters, including letters, numbers and special characters, are recognized and segmented in the corrected license plate image; Use optical character recognition technology to recognize each segmented character, convert the characters in the image into text information, match the recognized license plate characters with the registered license plates in the database, and use the string matching algorithm to calculate the license plate matching coefficient; A matching threshold is preset, and illegally parked vehicles are divided into licensed vehicles and warning vehicles according to the license plate matching coefficient. Vehicles with license plate matching coefficients higher than the matching threshold are regarded as licensed vehicles, and vehicles with license plate matching coefficients lower than the matching threshold are regarded as warning vehicles. Licensed vehicles refer to vehicles whose license plate information is complete and matches the database, and warning vehicles include vehicles without license plates, temporary license plates, and vehicles with severely damaged license plates.

8. The vehicle parking violation detection system based on drone inspection and photography according to claim 7 is characterized by: The calculation process of the license plate matching coefficient is: Determine the total number of characters in the license plate and a preset edit distance reference value, and extract text information converted by optical character recognition for each character in the license plate image; For each character i in the license plate, compare it with the standard characters in the database to obtain a matching score, which is determined by comparing the visual similarity and shape matching of the characters, and calculating the average and standard deviation of all matching scores; For each letter, use the exponential function to calculate its matching score deviation to obtain the matching score deviation function F(m), and sum the matching score deviation function values ​​of all characters; Use the Levenshtein distance string matching algorithm to calculate the edit distance between the recognized license plate and the standard license plate in the database, and calculate the minimum function of the license plate character distance; The sum of the matching score deviation function values ​​of all characters is multiplied by the result of the minimum function to obtain the final license plate matching coefficient, which is compared with the matching threshold to determine whether the vehicle is a licensed vehicle or a warning vehicle.

9. The vehicle parking violation detection system based on drone inspection and photography according to claim 8 is characterized by: The calculation expression of the license plate matching coefficient is: Among them, PMI is the license plate matching coefficient, F(m) is the matching score deviation function, n is the total number of characters in the license plate, and m is i is the matching score of the ith character, b is the average of all matching scores, which is used as the benchmark value, s is the standard deviation of the matching score, which is used to adjust the sensitivity of the exponential function, and d is the edit distance between the recognized license plate and the standard license plate in the database, d base It is the preset edit distance benchmark value, and the value range of PMI is between 0 and 1.

10. The vehicle illegal parking detection system based on drone inspection and photography according to claim 9 is characterized by: In the warning notification module, the triggering process of the warning notification includes: Receive the output of the illegal parking determination module, determine the illegally parked vehicle, and classify the vehicle as a licensed vehicle or a warning vehicle based on the result of the license plate recognition module; For licensed vehicles, the corresponding owner contact information is retrieved through the vehicle database, and a parking violation warning SMS is sent to the owner of the licensed illegally parked vehicle using the SMS gateway. The SMS content includes the location of the illegally parked vehicle, the illegal parking time, the preset moving time, and the owner's guide to take action; For the warning vehicle, the alarm mechanism is triggered, and the illegal parking information and vehicle description are sent to the traffic management department, and law enforcement officers are dispatched to the scene for verification and processing. The owner is contacted according to the vehicle reserved number to move the vehicle. If the owner cannot be contacted, a penalty list is issued; Track the responses of the vehicle owner and the traffic management department, and update the status of illegal parking. For licensed vehicles, if the owner does not respond to the SMS notification and move the vehicle within the preset time, the system will trigger the alarm mechanism and automatically send the detailed information of the illegally parked vehicle to the traffic management department. After receiving the information, the traffic management department will dispatch law enforcement officers to the scene for verification and processing, and the law enforcement officers will punish the illegally parked vehicle according to the on-site situation; The system automatically records key information of each illegal parking notification, the owner's response, and the handling by the traffic management department, and generates a parking violation handling report, including statistics on illegal parking incidents, handling efficiency, and the owner's response. Based on the feedback and statistics of illegal parking handling, the early warning notification process and strategy are continuously optimized.

Images