A parking fee collection method based on drone recognition and infrared technology

Abstract

The invention discloses a parking fee collection method based on unmanned aerial vehicle identification and infrared technology, which belongs to the technical field of parking fee collection. The method comprises the following steps: performing cruise detection on a toll road section by a unmanned aerial vehicle and judging whether a vehicle is in a stationary state; if the vehicle is stationary, performing infrared temperature measurement on a hood and a tail cover of the vehicle, inputting temperature difference information, weather and outdoor temperature into a parking time detection model, and obtaining the parking time; using an OpenCV library to perform license plate positioning and character segmentation, and identifying the license plate number; the unmanned aerial vehicle updates the parking time at time intervals of t; if the parking time is greater than the free parking time of the toll road section, calculating the payment amount according to the parking time of the vehicle, and finally transmitting the payment information to a car owner corresponding to the vehicle license plate; the car owner pays the parking fee through a payment QR code in the payment information. The method can improve the processing efficiency of parking fees on toll roads, and the detection content is more accurate and comprehensive.

Description

Technical Field

[0001] The present invention belongs to the technical field of parking fee collection, and in particular relates to a parking fee collection method based on drone recognition and infrared technology. Background Art

[0002] As a means of transportation, more and more cars are owned in cities. As a result, there are fewer and fewer free spaces for parking cars in cities, and many cities have many paid parking areas.

[0003] As the process of urbanization gradually advances, urban traffic problems are becoming more and more numerous, and urban traffic congestion is becoming more and more serious. At present, the number of cars in various cities is also growing rapidly, and parking spaces in urban areas are becoming increasingly tight, so roadside parking fees have become one of the ways to solve the problem of urban congestion.

[0004] At present, the detection and charging methods for parking fee areas are mainly manual. The staff needs to record the relevant information of the vehicle, manually time and apply tickets, etc. This method requires the staff to manually monitor the parking fee places for a long time, which is inefficient. Therefore, the traditional method has great limitations and greatly wastes the manpower of relevant departments. Summary of the invention

[0005] In view of the above problems existing in the prior art, the present invention proposes a parking fee collection method based on drone recognition and infrared technology, which has a reasonable design, solves the shortcomings of the prior art and has good effects.

[0006] A parking fee collection method based on drone recognition and infrared technology comprises the following steps:

[0007] S1. Use drones to patrol the toll road section and use the optical flow method to determine whether the vehicle in the toll road section is stationary;

[0008] S2. If the vehicle is stationary, use the thermal imager on the drone to measure the temperature of the hood and tailgate of the vehicle by infrared to obtain temperature difference information;

[0009] S3: Build a parking time detection model, input temperature difference information, weather, and outdoor temperature into the parking time detection model to obtain the parking time of the vehicle ;

[0010] S4, photographing the toll vehicle to obtain a parking image of the toll vehicle in the toll road section and an image of the license plate number;

[0011] S5, use OpenCV library to locate the license plate and segment the characters to identify the license plate number;

[0012] S6. The drone continues to monitor the temperature difference information of the toll vehicle until the toll vehicle leaves the toll road section, and records the final parking time of the toll vehicle;

[0013] S7. If the parking time last updated Longer than the free parking time of the toll road section , the car owner’s mobile phone number corresponding to the license plate number is retrieved from the database, the payment amount is calculated according to the total parking time, and finally the payment information is transmitted to the car owner, who pays the parking fee through the payment QR code in the payment information.

[0014] Furthermore, in S1, the drone uses automatic cruise technology to perform cruise detection. The drone captures a continuous image sequence of the vehicle and the surrounding environment in the toll section through a camera, uses the Lucas-Kanade optical flow method to calculate the motion vector of each pixel in the image sequence, and analyzes the optical flow vector of the area where the vehicle is located. If the optical flow vector of the area is close to zero or has no obvious change over a period of time, it indicates that the vehicle is stationary.

[0015] Furthermore, in order to facilitate detection and reduce the number of times drones need to be charged, drone charging points are selected near the center points of multiple toll sections and in a reasonable area away from the airport. A group of drones are dispatched to conduct cyclic detection between various toll areas. When the drone battery is low, a second group of drones are dispatched for cyclic detection, and the first group of drones return to the charging point for charging.

[0016] Furthermore, in S2, if the vehicle is stationary, the thermal imager on the drone is used to perform infrared temperature measurement on the hood and tailgate of the vehicle; the maximum temperature value of the hood is subtracted from the maximum temperature value of the tailgate and the absolute value is taken to obtain the temperature difference information between the hood and tailgate.

[0017] Furthermore, in S3, after comparative experiments, the minimum initial temperature of the hood is set to N°C, 50≤N≤55, and the temperature of the hood and tailgate of the vehicle is measured multiple times using a thermal imager, the absolute value of the temperature difference is taken, and the time, weather, and outdoor temperature are recorded. After a large number of experiments, a polynomial regression model is established using weather, outdoor temperature, and temperature difference information, namely, a parking duration detection model, which is specifically:

[0018] The expression for parking time in cloudy weather is:

[0019] ;

[0020] The expression for parking time in sunny weather is;

[0021] ;

[0022] The expression for parking time in cloudy weather is:

[0023] ;

[0024] The expression for parking time in rainy weather is:

[0025] ;

[0026] in, is the absolute value of the temperature difference between the maximum temperature of the front cover and the maximum temperature of the rear cover, is the outdoor temperature;

[0027] The temperature difference information, weather, and outdoor temperature are input into the parking time detection model to obtain the vehicle parking time information.

[0028] Furthermore, in S4, the drone takes multiple sets of photos of the toll vehicle and the license plate number, filters the acquired sets of image information, and extracts the images containing the front and rear information of the vehicle and the evidence pictures of the vehicle parked on the toll road section.

[0029] Furthermore, in S5, the image containing the front or rear information of the vehicle is preprocessed, and the preprocessing operation includes graying of the color image, Gaussian filtering, Sobel edge detection, color positioning, image binarization and morphological processing;

[0030] Use the cv2.findContours() function in the OpenCV library to perform rectangle detection on the preprocessed image and extract the license plate area when it is identified;

[0031] After locating the license plate, calibrate the level of the license plate;

[0032] The corrected license plate image is processed by grayscale value and thresholding to obtain a binary image, and then the cv2.boundingRect() function is used to obtain the information in the minimum rectangular border of the image, sort the license plate information, and finally display the license plate character segmentation map;

[0033] The first character of the license plate selects the Chinese template list in the license plate character template for matching, the second character selects the English template list for matching, and the remaining characters select the English and digital template list for matching, so as to read the license plate number.

[0034] Furthermore, in S6, the drone monitors the temperature difference information of the toll vehicle at every t time, and updates the parking time through the parking time detection model. ;

[0035] If the monitored temperature difference is always greater than 0, when the parking time exceeds time t If the toll vehicle's license plate number is not updated and cannot be identified, it indicates that the toll vehicle has left the toll road section, and the parking time of the toll vehicle is the last updated time. ;

[0036] If the monitored temperature difference decreases to 0, the time from the temperature difference being 0 to the last recognition of the license plate number is recorded, and the parking time of the toll vehicle is the sum of the time output by the parking time detection model when the temperature difference is 0 and the recorded time.

[0037] Beneficial technical effects brought by the present invention:

[0038] The present invention reduces the workload of staff to a certain extent, improves the efficiency of parking fee processing, and makes the detection content more accurate and comprehensive, thereby improving the effect of urban traffic management, reducing road traffic pressure, meeting the needs of staff, and alleviating the problem of urban traffic congestion. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The present invention is a flow chart of a parking fee collection method based on drone recognition and infrared technology.

[0040] Figure 2 This is a line graph comparing the actual value and the predicted value in the parking duration detection model under cloudy weather in the present invention.

[0041] Figure 3 It is a scatter plot of the actual value and the predicted value in cloudy weather in the parking duration detection model of the present invention.

[0042] Figure 4 It is a predicted three-dimensional surface diagram under cloudy weather in the parking duration detection model of the present invention.

[0043] Figure 5 This is a line graph comparing the actual value and the predicted value in the parking duration detection model in the present invention under sunny weather conditions.

[0044] Figure 6 It is a scatter plot of the actual value and the predicted value in the parking duration detection model in sunny weather in the present invention.

[0045] Figure 7 It is a predicted three-dimensional surface diagram under sunny weather in the parking duration detection model of the present invention.

[0046] Figure 8 This is a line graph comparing the actual value and the predicted value in the parking duration detection model under cloudy weather conditions in the present invention.

[0047] Fig. 9 It is a scatter plot of the actual value and the predicted value in the parking duration detection model under cloudy weather conditions in the present invention.

[0048] Fig.10 It is a predicted three-dimensional surface diagram under cloudy weather in the parking duration detection model of the present invention.

[0049] Fig.11 This is a line graph comparing the actual value and the predicted value in the parking duration detection model in rainy weather in the present invention.

[0050] Fig.12 It is a scatter plot of the actual value and the predicted value in the parking duration detection model in rainy weather in the present invention.

[0051] Fig.13 It is a predicted three-dimensional surface diagram under rainy weather in the parking duration detection model of the present invention. DETAILED DESCRIPTION

[0052] The specific implementation of the present invention is further described below in conjunction with specific embodiments:

[0053] A parking fee collection method based on drone recognition and infrared technology, such as Figure 1 As shown, the following steps are included:

[0054] S1. Use drones to patrol the toll road section and use the optical flow method to determine whether the vehicle in the toll road section is stationary;

[0055] The staff used the automatic cruise technology to dispatch drones to patrol the toll road sections. The drones captured continuous image sequences of the vehicles and the surrounding environment in the toll road sections through cameras, and used the Lucas-Kanade optical flow method to determine whether the vehicle was stationary. First, the motion vector of each pixel in the image sequence was calculated, and the optical flow vector of the area where the vehicle was located was analyzed. If the optical flow vector of the area was close to zero or did not change significantly over a period of time, it indicated that the vehicle was stationary. A threshold could be set to distinguish between tiny jitters and actual movement. For example, only when the optical flow vector exceeded a certain value, the vehicle was considered to be moving; otherwise, it was considered to be stationary.

[0056] To facilitate detection, reduce detection time and reduce the number of times drones need to be charged, drone charging points are selected near the center points of multiple toll sections and in a reasonable area away from the airport. A batch of drones are dispatched to conduct cyclic detection between various toll areas. When the drone battery is low, a second batch of drones are dispatched for cyclic detection, and the first batch of drones return to the charging point for charging.

[0057] S2. If the vehicle is stationary, use the thermal imager on the drone to measure the temperature of the hood and tailgate of the vehicle by infrared to obtain temperature difference information;

[0058] A thermal imager is installed on the drone, and the infrared temperature of the hood and tailgate of the stationary vehicle is measured by the thermal imager on the drone. The maximum temperature of the hood and the maximum temperature of the tailgate are subtracted, and then the absolute value is taken to obtain the temperature difference information, which is then used to further determine the parking time.

[0059] S3: Build a parking time detection model, input temperature difference information, weather, and outdoor temperature into the parking time detection model to obtain the parking time of the vehicle ;

[0060] When the vehicle is in motion or temporarily stopped, the surface temperature of the vehicle engine area will be higher and the surface temperature of the non-engine area will be lower due to the high temperature of the vehicle engine or motor. Therefore, the temperature difference information is used to determine the parking time. Due to the different engine structures and driving times between vehicles, there is a certain difference between the initial temperatures of the hood on the engine side when the vehicle is just stopped. Therefore, a minimum initial temperature is selected as the experimental temperature. The initial temperature of the hood on the engine side of each type of vehicle is measured after driving for 10 minutes. After comparative experiments and a large amount of vehicle data analysis, the minimum initial temperature N (50≤N≤55) of the hood on the engine side is finally selected as the experimental temperature. A large number of experiments are carried out on this type of vehicle. The vehicle hood and tail cover are measured multiple times using a thermal imager to obtain the absolute value of the temperature difference, obtain the temperature difference information, and record the time, weather, and outdoor temperature.

[0061] After obtaining a large amount of data, the data is stored in an Excel file. Pandas is used to read the data in the Excel file. The data is grouped according to the "weather" column. Each weather condition is used as an independent regression analysis object. For each weather group, "temperature difference" and "outdoor temperature" are extracted as features x, and "time" is used as the target variable y. Then, the features are transformed into quadratic polynomials, and the data is fitted for regression analysis. A line chart comparing the true value and the predicted value under each weather condition is drawn to show the changing trend of the true value and the predicted value with the number; a scatter plot of the true value and the predicted value under each weather condition is drawn, where the predicted value is the value corresponding to the y-axis and the true value is the value corresponding to the x-axis. At the same time, a diagonal line from the minimum value to the maximum value (the straight line of y=x) is drawn to represent the ideal situation of perfect prediction. The comparison between the true value and the predicted value is shown through a scatter plot, showing the situation where the true value and the predicted value should coincide under ideal conditions. A three-dimensional surface map of the prediction is drawn to show the relationship between the temperature difference and the outdoor temperature and the predicted value of time under specific weather conditions, which shows the connection between multiple variables in an intuitive way.

[0062] The expression for parking time in cloudy weather is;

[0063] ;

[0064] in is the absolute value of the temperature difference between the maximum temperature of the front cover and the maximum temperature of the rear cover, is the outdoor temperature;

[0065] The comparison chart of the actual value and predicted value number under cloudy weather is as follows Figure 2 As shown;

[0066] The scatter plot of the actual value and the predicted value under cloudy weather is as follows Figure 3 As shown;

[0067] The predicted three-dimensional surface map under cloudy weather is as follows Figure 4 As shown;

[0068] The expression for parking time in sunny weather is:

[0069] ;

[0070] The comparison chart of the actual value and predicted value number under sunny weather is as follows Figure 5 As shown;

[0071] The scatter plot of the actual value and the predicted value under sunny weather is as follows Figure 6 As shown;

[0072] The predicted three-dimensional surface map under sunny weather is as follows Figure 7 As shown;

[0073] The expression for parking time in cloudy weather is:

[0074] ;

[0075] The comparison chart of the actual value and predicted value number under cloudy weather is as follows Figure 8 As shown;

[0076] The scatter plot of the actual value and the predicted value under cloudy weather is as follows Fig. 9 As shown;

[0077] The predicted three-dimensional surface map under cloudy weather is as follows Fig.10 As shown;

[0078] The expression for parking time in rainy weather is:

[0079] ;

[0080] The comparison chart of the actual value and predicted value number under rainy weather is as follows Fig.11 As shown;

[0081] The scatter plot of the actual value and the predicted value under rainy weather is as follows Fig.12 As shown;

[0082] The predicted three-dimensional surface diagram under rainy weather is as follows Fig.13 As shown;

[0083] Through polynomial regression analysis under different weather conditions, multiple models are generated, and the regression effect is displayed in a variety of visualization methods. Finally, the regression equation of each weather group is output, and finally a polynomial regression model is created and trained, that is, a parking duration detection model;

[0084] Normally, the temperature of a vehicle just after it stops is greater than N°C, so when the temperature difference of the vehicle is input into the parking time detection model, the actual parking time is greater than the time predicted by the parking time detection model. For example, if the weather is cloudy and the outdoor temperature is 25°C, the thermal imager measures a temperature difference of 20°C between the hood and the tailgate of the vehicle, and the parking time detection model predicts 50 minutes, which means that the actual parking time of the vehicle is at least 50 minutes. However, since the hood temperatures of various types of vehicles just stopped are not very different, the time consumed for the temperature of the hood on the engine side to drop to N°C when the vehicle just stopped is almost negligible.

[0085] S4, photographing the toll vehicle to obtain a parking image of the toll vehicle on the toll road section and an image of the license plate number;

[0086] The drone is used to take multiple sets of photos of toll vehicles and license plates, filter the acquired sets of image information, extract images containing the front and rear information of the vehicle, and evidence-collecting images of vehicles parked in toll sections.

[0087] S5, use OpenCV library to locate the license plate and segment the characters to identify the license plate number;

[0088] Preprocess the front side image of the car. The preprocessing operations include grayscale conversion of color images, Gaussian filtering, Sobel edge detection, color positioning, image binarization and morphological processing.

[0089] Use the cv2.findContours() function in the OpenCV library to perform rectangle detection on the preprocessed image. First, set the candidate area suitable for the license plate size. According to the national license plate size, set the aspect ratio of the candidate area between 3.0-3.5 to ensure that the framed area has license plate information and remove invalid areas. When the license plate area is identified, it will be extracted.

[0090] After locating the license plate, calibrate the level of the license plate;

[0091] The corrected license plate image is processed by grayscale value and thresholding to obtain a binary image, and then the cv2.boundingRect() function is used to obtain the information in the minimum rectangular border of the image, sort the license plate information, and finally display the license plate character segmentation map;

[0092] Since the first character of the license plate displays the province in Chinese, the second character is an English character, and the other characters are a mixture of English and numbers, the first character chooses to obtain the Chinese template list in the license plate character template for matching, the second character chooses to obtain the English template in the license plate character template for matching, and the remaining characters choose to obtain the English and number template list in the license plate character template for matching. The address of the template is read to match the picture, the template is formatted and converted, the template image is thresholded, the size of the picture to be detected is obtained, the template is adjusted to the same size as the image, and finally template matching is performed to read the license plate number.

[0093] The drone is equipped with a control system, which includes a parking duration detection module and a license plate recognition module, which are used to detect the parking duration of S4 and S5 and recognize the license plate number respectively;

[0094] S6. The drone continues to monitor the temperature difference information of the toll vehicle until the toll vehicle leaves the toll road section, and records the final parking time of the toll vehicle;

[0095] The drone monitors the temperature difference information of the toll vehicle every t seconds and updates the parking time through the parking time detection model. ;

[0096] If the monitored temperature difference is always greater than 0, when the parking time exceeds time t If the toll vehicle's license plate number is not updated and cannot be identified, it indicates that the toll vehicle has left the toll road section, and the parking time of the toll vehicle is the last updated time. ;

[0097] If the monitored temperature difference decreases to 0, the drone will monitor the toll vehicle every t seconds by identifying the license plate number, and record the time from the temperature difference being 0 to the last time the license plate number was identified. The parking time of the toll vehicle is the sum of the time output by the parking time detection model when the temperature difference is 0 and the recorded time.

[0098] S7. If the parking time last updated Longer than the free parking time of the toll road section , the car owner’s mobile phone number corresponding to the license plate number is retrieved from the database, the payment amount is calculated according to the total parking time, and finally the payment information is transmitted to the car owner, who pays the parking fee through the payment QR code in the payment information.

[0099] The payment information includes the payment amount, payment QR code, parking time information, and an image of the vehicle parked in the toll section.

[0100] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A parking fee collection method based on drone recognition and infrared technology, characterized in that: The following steps are involved: S1. Use drones to patrol the toll road section and use the optical flow method to determine whether the vehicle in the toll road section is stationary; S2. If the vehicle is stationary, use the thermal imager on the drone to measure the temperature of the hood and tailgate of the vehicle by infrared to obtain temperature difference information; S3, constructing a parking time detection model, inputting temperature difference information, weather, and outdoor temperature into the parking time detection model, thereby obtaining the parking time T of the vehicle; S4, photographing the toll vehicle to obtain a parking image of the toll vehicle in the toll section and an image of the license plate number; S5, use OpenCV library to locate the license plate and segment the characters to identify the license plate number; S6. The drone continues to monitor the temperature difference information of the toll vehicle until the toll vehicle leaves the toll road section, and records the final parking time of the toll vehicle; S7. If the final parking time is longer than the free parking time T′ of the toll road section, the mobile phone number of the car owner corresponding to the license plate number is retrieved from the database, the payment amount is calculated according to the total parking time, and the payment information is finally transmitted to the car owner, who pays the parking fee through the payment QR code in the payment information; In S1, the drone uses automatic cruise technology to perform cruise detection. The drone captures a continuous image sequence of the vehicle and the surrounding environment in the toll road section through a camera, calculates the motion vector of each pixel in the image sequence using the Lucas-Kanade optical flow method, and analyzes the optical flow vector of the area where the vehicle is located. If the optical flow vector of the area is close to zero or has no obvious change over a period of time, it indicates that the vehicle is stationary; In S2, if the vehicle is in a stationary state, the infrared temperature measurement of the hood and the tailgate of the vehicle is performed using the thermal imager on the drone; the maximum temperature value of the hood is subtracted from the maximum temperature value of the tailgate and the absolute value is taken to obtain the temperature difference information between the hood and the tailgate; In S3, after comparative experiments, the minimum initial temperature of the hood is set to N°C, 50≤N≤55, and the temperature of the hood and tailgate of the vehicle is measured multiple times using a thermal imager, the absolute value of the temperature difference is taken, and the time, weather, and outdoor temperature are recorded. After a large number of experiments, a polynomial regression model is established using weather, outdoor temperature, and temperature difference information, namely, a parking duration detection model, which is specifically: The expression for parking time in cloudy weather is: T1=392.946-6.521a-12.650b+0.022a^2-0.148ab+0.276b^2; The expression for parking time in sunny weather is; T2=144.320-8.024a+7.229b+0.013a^2-0.084ab-0.117b^2; The expression for parking time in cloudy weather is: T3=628.012-6.856a-32.776b+0.043a^2-0.156ab-0.698b^2; The expression for parking time in rainy weather is: T4=94.793-8.512a+11.691b+0.070a^2-0.111ab-0.218b^2; Among them, a is the absolute value of the temperature difference between the maximum temperature of the front cover and the maximum temperature of the rear cover, and b is the outdoor temperature; Inputting temperature difference information, weather, and outdoor temperature into a parking duration detection model to obtain vehicle parking duration information; In S6, the drone monitors the temperature difference information of the toll vehicle at every t time interval, and updates the parking time T through the parking time detection model; If the monitored temperature difference is always greater than 0, when the parking time T is not updated and the license plate number of the toll vehicle cannot be identified after time t, it indicates that the toll vehicle has left the toll section, and the parking time of the toll vehicle is the last updated T; If the monitored temperature difference decreases to 0, the time from the temperature difference being 0 to the last recognition of the license plate number is recorded, and the parking time of the toll vehicle is the sum of the time output by the parking time detection model when the temperature difference is 0 and the recorded time.

2. A parking fee collection method based on drone recognition and infrared technology according to claim 1, characterized in that: To facilitate detection and reduce the number of times drones need to be charged, drone charging points are selected near the center points of multiple toll sections and in a reasonable area away from the airport. A batch of drones are dispatched to conduct cyclic detection between the various toll areas. When the drone battery is low, a second batch of drones are dispatched for cyclic detection, and the first batch of drones return to the charging point for charging.

3. The parking fee collection method based on drone recognition and infrared technology according to claim 2 is characterized in that: In S4, the drone takes multiple sets of photos of the toll vehicle and the license plate number, filters the acquired sets of image information, and extracts the images containing the front and rear information of the vehicle and the evidence pictures of the vehicle parked in the toll section.

4. The parking fee collection method based on drone recognition and infrared technology according to claim 3 is characterized in that: The S5 specifically includes the following steps: Preprocess the license plate image. The preprocessing operations include grayscale conversion of color images, Gaussian filtering, Sobel edge detection, color location, image binarization and morphological processing. Use the cv2.findContours() function in the OpenCV library to perform rectangle detection on the preprocessed image and extract the license plate area when it is identified; After locating the license plate, calibrate the level of the license plate; The corrected license plate image is processed by grayscale value and thresholding to obtain a binary image, and then the cv2.boundingRect() function is used to obtain the information in the minimum rectangular border of the image, sort the license plate information, and finally display the license plate character segmentation map; The first character of the license plate selects the Chinese template list in the license plate character template for matching, the second character selects the English template list for matching, and the remaining characters select the English and digital template list for matching, so as to read the license plate number.

Images