Vehicle road illegal parking detection method and system based on automatic cruise unmanned aerial vehicle

Abstract

The invention discloses a vehicle road illegal parking detection method and system based on an automatic cruise unmanned aerial vehicle, and relates to the technical field of illegal parking detection, and the method comprises the following steps: firstly, controlling the unmanned aerial vehicle to fly above a target area, and enabling the unmanned aerial vehicle to carry out the automatic cruise according to a preset route, and to cover a target monitoring area; inputting road network basic information into the data platform, inputting point location longitudes and latitudes and point location names of roads every 500 meters, and marking the attribute of whether the roads can be parked or not; real-time latitude and longitude of the unmanned aerial vehicle are obtained and compared with a road network information base, parking attributes of the road are searched, and video data are obtained for continuous recognition. According to the invention, illegal parking vehicle information can be accurately identified and recorded in real time, the illegal parking detection efficiency is improved, a good warning effect can be provided for illegal parking vehicles with fluke mind, and smooth operation of a road can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of illegal parking detection, and in particular to a method and system for detecting illegal parking of vehicles on roads based on an automatic cruising drone. Background Art

[0002] Road refers to highways, urban roads and places under the jurisdiction of a unit but where social motor vehicles are allowed to pass, including squares, public parking lots and other places for public passage; roads are the arteries of urban transportation network infrastructure construction, supporting the city's economy and vitality, and carrying people's yearning for a better life. In order to ensure the smooth operation of roads, it is necessary to warn and punish vehicles that illegally park on the roads.

[0003] With the rapid development of urban traffic, illegal parking is becoming more and more serious. The existing illegal parking detection method is through manual inspection by traffic police. This illegal parking detection method is inefficient and cannot give a good warning to illegal parking vehicles that are lucky enough to park. Therefore, there is room for improvement. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and propose a method and system for detecting illegal parking on roads based on an automatic cruise drone. Its advantages are that it can accurately identify and record illegal parking vehicle information in real time, improve the efficiency of illegal parking detection, and give a good warning to illegal parking vehicles that are lucky enough to park, which is conducive to ensuring the smooth operation of the road.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for detecting illegal parking of vehicles on a road based on an automatic cruise drone comprises the following steps:

[0007] Step 1: First, control the drone to fly over the target area, and the drone will automatically cruise according to the predetermined route to cover the target monitoring area; enter the basic information of the road network in the data platform, enter the latitude and longitude and point name of the road every 500 meters, and mark the attributes of whether the road can be parked;

[0008] Step 2: Obtain the real-time latitude and longitude of the drone, compare it with the road network information database, find the parking attributes of the road, and obtain video data for further identification;

[0009] Step 3: Identify vehicle features and lane lines in the video data, and use image segmentation technology to identify grid lines; determine the speed of the vehicle, determine the location of the vehicle based on the lane lines and grid lines, filter out stationary vehicles, and mark vehicles on the grid lines or hard shoulders as possible illegally parked vehicles;

[0010] Step 4: For vehicles marked as potentially illegally parked, hover the drone to identify the license plate and extract the license plate information; save the license plate and vehicle image into the database; push the illegally parked vehicle data to other platforms for further processing.

[0011] The present invention is further configured that, in the process of the drone camera collecting images, there is generally interference noise, which will deteriorate the quality of the image, reduce the clarity of the image, make the image features unclear or even disappear, and make subsequent image processing difficult; Gaussian filtering is required to perform noise reduction processing on the aerial image, and the Gaussian function expression is: Where x and y are the coordinates of the image pixels.

[0012] The present invention is further configured such that the vehicle features are extracted using gradient histogram features, and the calculation steps include color space normalization, gamma correction, gradient calculation and overlapping block histogram normalization.

[0013] The present invention is further configured such that the color space normalization is used to convert the RGB components of a color image into a grayscale image, and the conversion formula is: Gray=0.3*R+0.59*G+0.11*B.

[0014] The present invention is further configured that the gamma correction is used to increase or decrease the overall brightness of the image when the image illumination intensity is uneven, and the calculation formula is: Y(x,y)=I(x,y) γ , where γ is equal to 0.5, I(x,y) is the original image pixel, and Y(x,y) is the corrected pixel.

[0015] The present invention is further configured that the gradient calculation is used to calculate the gradient magnitude and gradient direction angle for each pixel of the image, the gradient magnitude includes a horizontal gradient value and a vertical gradient value, and the gradient operator is: horizontal direction [-101], vertical direction [-101] T , horizontal gradient G x The calculation formula for (x,y) is: x (x,y)=I(x+1,y)-I(x-1,y), vertical gradient G y The calculation formula for (x,y) is: y (x,y)=I(x,y+1)-I(x,y-1), the gradient amplitude G(x,y) is calculated as: The calculation formula of gradient direction angle θ(x,y) is:

[0016]

[0017] The present invention is further configured that the lane line recognition adopts a K-means clustering algorithm to divide the points in space into K categories according to the size of the distance, and the general steps are as follows:

[0018] Step 1: Select K objects from the classified data as the initial clustering centers;

[0019] Step 2: Calculate the distance from each cluster object to the cluster center to divide:

[0020] Step 3: Calculate each cluster center again;

[0021] Step 4: Calculate the standard measure function until the maximum number of iterations is reached.

[0022] The present invention is further configured such that the stroke width of the connected domain where the lane line is located generally changes very little, and the coefficient of variation D is used to measure the magnitude of the change in the stroke width, and its expression is: Among them, N is the number of pixels in the category, is the stroke width value of the pixel, and is the average value of the pixel stroke width.

[0023] The present invention is further configured such that the vehicle speed calculation formula is: v represents the speed of the vehicle, s represents the distance the vehicle moves per unit time, and t represents time.

[0024] The present invention is further configured such that the camera of the drone should be located at the center of the field of view during license plate recognition. Therefore, the shooting angle of the camera needs to be calculated according to the actual situation. The calculation formula is: Among them, α represents the slope of the road; S represents the length of the license plate, d represents the distance between the license plate and the ground, h represents the flight altitude of the drone; l represents the horizontal distance between the drone and the vehicle.

[0025] The present invention also provides a vehicle road parking violation detection system based on an automatic cruising drone, comprising:

[0026] Module 1: Control the drone to fly over the target area, and the drone will automatically cruise according to the predetermined route to cover the target monitoring area; enter the basic information of the road network in the data platform, enter the latitude and longitude and point name of the road every 500 meters, and mark the attributes of whether the road can be parked;

[0027] Module 2: Obtain the real-time latitude and longitude of the drone, compare it with the road network information database, find the parking attributes of the road, and obtain video data for further identification;

[0028] Module 3: Identify vehicle features and lane lines in video data, and use image segmentation technology to identify grid lines; determine the speed of the vehicle, determine the location of the vehicle based on the lane lines and grid lines, filter out stationary vehicles, and mark vehicles on the grid lines or hard shoulders as possible illegally parked vehicles;

[0029] Module 4: For vehicles marked as potentially illegally parked, hover drones to identify license plates and extract license plate information; save license plates and vehicle images into a database; and push illegally parked vehicle data to other platforms for further processing.

[0030] The beneficial effects of the present invention are as follows: the vehicle road illegal parking detection method based on automatic cruising drone uses drones to replace traditional manual detection. The drone camera can identify vehicle features and lane lines in video data, and use image segmentation technology to identify grid lines; judge the speed of the vehicle, judge the vehicle's position according to the lane lines and grid lines, filter out stationary vehicles, and mark vehicles on grid lines or hard shoulders as possible illegally parked vehicles. For vehicles marked as possible illegally parked vehicles, the hovering drone identifies the license plate, extracts the license plate information for processing, and can identify and record the information of illegally parked vehicles in real time and accurately, thereby improving the efficiency of illegal parking detection, and can give a good warning to illegally parked vehicles that are lucky enough to park, which is conducive to ensuring smooth operation of roads. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the detection process of the vehicle road illegal parking detection method based on the automatic cruising drone proposed in the present invention. DETAILED DESCRIPTION

[0032] The technical solution of this patent is further described in detail below in conjunction with specific implementation methods.

[0033] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0034] With the development of society, the continuous innovation of science and technology, and the continuous improvement of people's material living standards, the automobile industry has developed rapidly, and cars have gradually entered thousands of households.

[0035] Although private cars can bring convenience to people's travel, with the continuous acceleration of urbanization, a large number of people have poured into cities, resulting in a rapid increase in the number of vehicles in cities. The shortage of urban parking spaces has led to serious illegal parking of vehicles, chaos and even paralysis of urban traffic, and ultimately caused huge economic losses. Therefore, it is necessary to effectively supervise vehicles to ensure the normal operation of urban traffic, and the most direct and effective way to supervise vehicles is to ticket illegally parked vehicles. Commonly used vehicle parking detection and ticketing still rely on traffic police patrols to complete. This method of manual vehicle supervision is time-consuming and labor-intensive, and has a narrow coverage and is prone to missed inspections.

[0036] In recent years, drone technology has developed rapidly, and drones have been widely used in all walks of life.

[0037] For example, in the power industry, some people equip drones with high-definition digital cameras and GPS positioning systems, so that drones can autonomously cruise along the power grid. Through wireless image transmission, the images of the power grid are transmitted to the computer in real time, and the staff can monitor the power grid and operate the drone in real time on the computer. This method of monitoring the power grid does not require climbing poles, which reduces the labor intensity of staff, improves safety, and is conducive to the rapid restoration of power supply after power grid failure. Another example is the spraying of pesticides by drones, which has achieved great commercial value as a typical application of drones. Traditional pesticide spraying relies on manual spraying of pesticide tanks in the fields. This spraying method is time-consuming, labor-intensive, and inefficient. Install the pesticide spraying mechanism on the drone and set the flight path for the drone, and you can efficiently spray pesticides on large tracts of farmland, thus saving farmers from the hard work of spraying pesticides under the scorching sun. Operations that cannot be done on the ground can be performed in the air, and the natural environment in the air is relatively simple. As an aerial flight platform, drones have great advantages in some applications, so the application of drones is becoming more and more extensive.

[0038] Based on the above analysis, it is of great research significance to combine the UAV flight platform with the vehicle illegal parking detection technology. The UAV can cruise in the air, thus solving the problem of the limited detection area of ​​fixed cameras. As long as a reasonable path is planned, a UAV can detect the illegal parking of vehicles in a large area. This detection method is much cheaper than deploying a large number of cameras.

[0039] Reference Figure 1 , a vehicle road illegal parking detection method based on an automatic cruise drone includes the following steps:

[0040] First, control the drone to fly over the target area, and the drone will automatically cruise according to the predetermined route to cover the target monitoring area; enter the basic information of the road network into the data platform, enter the latitude and longitude and point name of the road every 500 meters, and mark whether the road is suitable for parking;

[0041] Obtain the real-time latitude and longitude of the drone, compare it with the road network information database, find the parking attributes of the road, and obtain video data for further recognition; in the process of collecting images by the drone camera, there is generally interference noise, which will deteriorate the quality of the image, reduce the clarity of the image, make the image features less obvious or even disappear, and make subsequent image processing difficult; Gaussian filtering is needed to reduce the noise of the aerial image, and the Gaussian function expression is: Where x and y are the coordinates of the image pixels.

[0042] Identify vehicle features and lane lines in video data, and use image segmentation technology to identify grid lines; determine the speed of the vehicle, determine the location of the vehicle based on the lane lines and grid lines, filter out stationary vehicles, and mark vehicles on grid lines or hard shoulders as possible illegally parked vehicles;

[0043] In image recognition, the image pixel data is directly used as input for classifier training. Since the amount of image pixel data is generally large, the training of the classifier model is very slow, and the trained classifier model is also very complex. In addition, the image recognition process has a large amount of calculations and the recognition is slow. Therefore, feature extraction is generally performed on the image. Feature extraction can reduce the dimension of the training input, speed up the training speed, reduce the complexity of the training model, and improve the recognition efficiency.

[0044] Vehicle features are extracted using gradient histogram features, and the calculation steps include color space normalization, gamma correction, gradient calculation, and overlapping block histogram normalization.

[0045] Color space normalization: used to convert the RGB components of a color image into a grayscale image. The conversion formula is: Gray = 0.3*R + 0.59*G + 0.11*B.

[0046] Gamma correction: used to increase or decrease the overall brightness of the image when the image illumination intensity is uneven. The calculation formula is: Y(x,y)=I(x,y) γ , where γ is equal to 0.5, I(x,y) is the original image pixel, and Y(x,y) is the corrected pixel.

[0047] Gradient calculation: used to calculate the gradient magnitude and gradient direction angle for each pixel of the image. The gradient magnitude includes the horizontal gradient value and the vertical gradient value. The gradient operator is: horizontal direction [-101], vertical direction [-101] T , horizontal gradient G x The calculation formula for (x,y) is: x (x,y)=I(x+1,y)-I(x-1,y), vertical gradient G y The calculation formula for (x,y) is: y (x,y)=I(x,y+1)-I(x,y-1), the gradient amplitude G(x,y) is calculated as: The calculation formula of gradient direction angle θ(x,y) is:

[0048]

[0049] Overlapping block histogram normalization: Because the illumination intensity and background of the image are uncertain, the calculated image gradient values ​​will generally fluctuate greatly, so a better feature normalization method needs to be selected to improve the image detection performance. There are many commonly used normalization methods, basically putting image cells into image blocks and then standardizing all image blocks. An image block consists of 2x2 image cells, and there is overlap between adjacent image blocks. The overlap of image blocks allows adjacent blocks to share pixel information, which can significantly improve the image detection effect.

[0050] Lane line recognition uses the K-means clustering algorithm to divide the points in space into K categories according to the size of the distance. The general steps are as follows:

[0051] 1. Select K objects from the classified data as the initial cluster centers;

[0052] 2. Calculate the distance from each cluster object to the cluster center to divide:

[0053] 3. Calculate each cluster center again;

[0054] 4. Calculate the standard measure function until the maximum number of iterations is reached.

[0055] The stroke width of the connected domain where the lane line is located generally changes very little. The coefficient of variation D is used to measure the magnitude of the change in stroke width, and its expression is: Among them, N is the number of pixels in the category, is the stroke width value of the pixel, and is the average value of the pixel stroke width.

[0056] The vehicle speed calculation formula is: v represents the speed of the vehicle, s represents the distance the vehicle moves per unit time, and t represents time.

[0057] When recognizing license plates, the drone’s camera should be located at the center of the field of view. Therefore, the camera’s shooting angle needs to be calculated based on the actual situation. The calculation formula is: Among them, α represents the slope of the road; S represents the length of the license plate, d represents the distance between the license plate and the ground, h represents the flight altitude of the drone; l represents the horizontal distance between the drone and the vehicle.

[0058] For vehicles marked as potentially illegally parked, the drone will hover to identify the license plate and extract the license plate information; save the license plate and vehicle image into the database; and push the illegally parked vehicle data to other platforms for further processing.

[0059] The vehicle road illegal parking detection method based on automatic cruising drone uses drones to replace traditional manual detection. The drone camera can identify vehicle features and lane lines in video data, and use image segmentation technology to identify grid lines; judge the vehicle's speed, determine the vehicle's location based on lane lines and grid lines, filter out stationary vehicles, and mark vehicles on grid lines or hard shoulders as possible illegal parking vehicles. For vehicles marked as possible illegal parking, the drone is hovered to identify the license plate, and the license plate information is extracted for processing. It can identify and record the illegal parking vehicle information in real time and accurately, improve the illegal parking detection efficiency, and can give a good warning to illegal parking vehicles that are lucky enough to park, which is conducive to ensuring smooth road operation.

[0060] The present invention also provides a vehicle road illegal parking detection system based on an automatic cruise drone, which can be implemented by executing the process steps of the vehicle road illegal parking detection method based on an automatic cruise drone, that is, those skilled in the art can understand the vehicle road illegal parking detection method based on an automatic cruise drone as a preferred implementation of the vehicle road illegal parking detection system based on an automatic cruise drone. The system includes:

[0061] Module 1: Control the drone to fly over the target area. The drone will automatically cruise along the predetermined route to cover the target monitoring area. Enter the basic information of the road network into the data platform, enter the latitude and longitude of the road points and the name of the points every 500 meters, and mark the attributes of whether the road can be parked.

[0062] Module 2: Obtain the real-time latitude and longitude of the drone, compare it with the road network information database, find the parking attributes of the road, and obtain video data for further identification.

[0063] Module Three: Identify vehicle features and lane lines in video data, and use image segmentation technology to identify grid lines; determine the speed of the vehicle, determine the location of the vehicle based on the lane lines and grid lines, filter out stationary vehicles, and mark vehicles on grid lines or hard shoulders as possible illegally parked vehicles.

[0064] Module 4: For vehicles marked as potentially illegally parked, hover drones to identify license plates and extract license plate information; save license plates and vehicle images into a database; and push illegally parked vehicle data to other platforms for further processing.

[0065] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0066] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A vehicle road illegal parking detection method based on an automatic cruise drone, characterized in that: The following steps are involved: Step 1: First, control the drone to fly over the target area, and the drone will automatically cruise according to the predetermined route to cover the target monitoring area; enter the basic information of the road network in the data platform, enter the latitude and longitude and point name of the road every 500 meters, and mark the attributes of whether the road can be parked; Step 2: Obtain the real-time latitude and longitude of the drone, compare it with the road network information database, find the parking attributes of the road, and obtain video data for further identification; Step 3: Identify vehicle features and lane lines in the video data, and use image segmentation technology to identify grid lines; determine the speed of the vehicle, determine the location of the vehicle based on the lane lines and grid lines, filter out stationary vehicles, and mark vehicles on the grid lines or hard shoulders as possible illegally parked vehicles; Step 4: For vehicles marked as potentially illegally parked, hover the drone to identify the license plate and extract the license plate information; save the license plate and vehicle image into the database; Push illegally parked vehicle data to other platforms for further processing.

2. The vehicle road parking violation detection method based on automatic cruise drone according to claim 1 is characterized in that: In the process of collecting images by the drone camera, there is generally interference noise, which will deteriorate the quality of the image, reduce the clarity of the image, make the image features unclear or even disappear, and make subsequent image processing difficult; Gaussian filtering is needed to reduce the noise of the aerial image, and the Gaussian function expression is: Where x and y are the coordinates of the image pixels.

3. The vehicle road illegal parking detection method based on automatic cruising drone according to claim 1 is characterized in that: The vehicle features are extracted using gradient histogram features, and the calculation steps include color space normalization, gamma correction, gradient calculation and overlapping block histogram normalization.

4. The vehicle road parking violation detection method based on the automatic cruise drone according to claim 3 is characterized in that: The color space normalization is used to convert the RGB components of a color image into a grayscale image, and the conversion formula is: Gray=0.3*R+0.59*G+0.11*B.

5. The method for detecting illegal parking of vehicles on roads based on an automatic cruising drone according to claim 4 is characterized in that: The gamma correction is used to increase or decrease the overall brightness of the image when the image illumination intensity is uneven. The calculation formula is: Y(x,y)=I(x,y) γ , where γ is equal to 0.5, I(x,y) is the original image pixel, and Y(x,y) is the corrected pixel.

6. The vehicle road illegal parking detection method based on automatic cruising drone according to claim 5 is characterized in that: The gradient calculation is used to calculate the gradient magnitude and gradient direction angle for each pixel of the image. The gradient magnitude includes the horizontal gradient value and the vertical gradient value. The gradient operator is: horizontal direction [-101], vertical direction [-101] T , horizontal gradient G x The calculation formula for (x,y) is: x (x,y)=I(x+1,y)-I(x-1,y), vertical gradient G y The calculation formula for (x,y) is: y (x,y)=I(x,y+1)-I(x,y-1), the gradient amplitude G(x,y) is calculated as: The calculation formula of gradient direction angle θ(x,y) is:

7. The vehicle road illegal parking detection method based on automatic cruising drone according to claim 1 is characterized in that: The lane line recognition adopts the K-means clustering algorithm to divide the points in space into K categories according to the size of the distance. The general steps are as follows: Step 1: Select K objects from the classified data as the initial clustering centers; Step 2: Calculate the distance from each cluster object to the cluster center to divide: Step 3: Calculate each cluster center again; Step 4: Calculate the standard measure function until the maximum number of iterations is reached.

8. The vehicle road parking violation detection method based on automatic cruise drone according to claim 7 is characterized in that: The stroke width of the connected domain where the lane line is located generally changes very little. The coefficient of variation D is used to measure the magnitude of the change in stroke width, and its expression is: Among them, N is the number of pixels in the category, is the stroke width value of the pixel, and is the average value of the pixel stroke width.

9. The vehicle road illegal parking detection method based on automatic cruising drone according to claim 1 is characterized in that: The camera of the drone should be located at the center of the field of view during license plate recognition. Therefore, the camera shooting angle needs to be calculated according to the actual situation. The calculation formula is: Among them, α represents the slope of the road; S represents the length of the license plate, d represents the distance between the license plate and the ground, h represents the flight altitude of the drone; l represents the horizontal distance between the drone and the vehicle.

10. A vehicle road parking violation detection system based on an automatic cruise drone, characterized in that: include: Module 1: Control the drone to fly over the target area, and the drone will automatically cruise along the predetermined route to cover the target monitoring area; Enter the basic information of the road network into the data platform, enter the latitude and longitude of the road points and the name of the points every 500 meters, and mark the attributes of whether the road is suitable for parking; Module 2: Obtain the real-time latitude and longitude of the drone, compare it with the road network information database, find the parking attributes of the road, and obtain video data for further identification; Module 3: Identify vehicle features and lane lines in video data, and use image segmentation technology to identify grid lines; determine the speed of the vehicle, determine the location of the vehicle based on the lane lines and grid lines, filter out stationary vehicles, and mark vehicles on the grid lines or hard shoulders as possible illegally parked vehicles; Module 4: For vehicles marked as potentially illegally parked, hover the drone to identify the license plate and extract the license plate information; save the license plate and vehicle image into the database; Push illegally parked vehicle data to other platforms for further processing.

Images