Expressway license plate recognition and vehicle attribute recognition mode and system based on view angle of unmanned aerial vehicle

Abstract

The invention discloses an expressway license plate recognition and vehicle attribute recognition mode and system based on the view angle of an unmanned aerial vehicle. The system comprises an unmanned aerial vehicle module, a server module, a lane classification module, a vehicle attribute recognition and license plate recognition module and a control module. The unmanned aerial vehicle module snapshots an expressway video in real time and transmits the expressway video to the server through a wireless network; the server module calls a lane classification module, a vehicle attribute recognition module and a license plate recognition module to analyze and process the pictures; the control module remotely controls the unmanned aerial vehicle according to the analysis result. The system can accurately identify the road type of the expressway, and provides basic data for subsequent vehicle attribute identification and license plate identification. The vehicle attribute recognition module can recognize attributes such as vehicle types, colors and brands, the license plate recognition module accurately recognizes license plate numbers, powerful support is provided for traffic management and law enforcement, the inspection efficiency and accuracy are improved, the emergency response capability is enhanced, and the manual inspection cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic monitoring drones, and in particular to a method and system for recognizing license plates and vehicle attributes on highways based on the perspective of drones. Background Art

[0002] The highway drone inspection solution can realize the automated inspection of highways, improve the inspection accuracy, efficiency and safety, and reduce the cost of manual inspection. Drones can conduct regular inspections and real-time monitoring. Through airborne sensors and laser radar and other equipment, they can realize comprehensive monitoring of complex terrain and hidden points such as road conditions, signs and markings, and road facilities. The highway drone inspection solution can also assist traffic police in traffic diversion, ensure road safety, and reduce traffic congestion. Drones can quickly obtain on-site conditions through aerial photography, collect relevant information, and report and dispatch to relevant departments in a timely manner, providing important technical support for the handling of emergencies.

[0003] The current inspection plan is generally for pilots to manually fly drones to designated roads, hover the drones, and then manually adjust the focus to monitor vehicles on the road. This method requires a large number of pilots to operate during busy traffic periods such as holidays, and the pilots must have a certain amount of experience to control the position and angle of the drones.

[0004] During holidays, the traffic police department needs to check whether there are vehicles occupying the emergency lane. And it needs to check whether there are trucks occupying the fast lane for a long time. The definition of trucks occupying the fast lane for a long time is: 1. Trucks driving in the fast lane for more than 10 seconds. If it is an overtaking behavior, it is not a violation if it does not exceed 10 seconds. 2. If there is no car in the lane next to the truck, it is a violation regardless of how long it drives in the fast lane.

[0005] Traditional traffic monitoring technology, when used in practice, mainly relies on ground-based fixed cameras for monitoring. However, this approach has exposed some significant shortcomings in practical applications, as follows:

[0006] (1) The road conditions on highways are complex and changeable, including different road types (such as emergency lanes, express lanes, etc.), signs and markings, and road facilities. Fixed cameras are difficult to fully capture this complex information, resulting in limited monitoring effects.

[0007] (2) Traditional monitoring methods mainly rely on manual observation and judgment and lack intelligent recognition functions. This results in relatively low efficiency and accuracy when detecting traffic violations (such as occupying the emergency lane, trucks occupying the fast lane for a long time, etc.). Summary of the invention

[0008] The technical problem to be solved by the present invention is to overcome the above technical defects and provide a method and system for license plate recognition and vehicle attribute recognition on highways based on the perspective of a drone.

[0009] In order to solve the above problems, the technical solution of the present invention is a license plate recognition and vehicle attribute recognition system based on the highway from the perspective of a drone, comprising the following modules:

[0010] A drone module, which is used to inspect above the highway and capture the video of the current road section in real time;

[0011] A server module, which is wirelessly connected to the drone module, is used to receive pictures transmitted by the drone module, and call the lane classification module, the vehicle attribute recognition and the license plate recognition module to analyze and process the pictures;

[0012] A lane classification module, which is used to classify the current road surface image and return the types of all roads in the current image;

[0013] A vehicle attribute recognition and license plate recognition module, which is used to detect and recognize vehicle attributes and license plate numbers on the corresponding road according to the road type and corresponding road coordinates returned by the lane classification module;

[0014] The control module is used to remotely control the flight position, focal length adjustment and monitoring functions of the drone module according to the analysis results of the server module.

[0015] Furthermore, the server may be a cloud server or a local server to provide sufficient computing resources to process image analysis tasks. The server module includes an alarm module, which is used to record violation events and return alarm pictures and information when vehicle violations are detected.

[0016] Furthermore, the vehicle attribute recognition and license plate recognition module includes a vehicle detection unit, a vehicle attribute classification unit and a license plate recognition unit.

[0017] Furthermore, the drone is equipped with a camera that can automatically adjust the focal length to meet the needs of different monitoring distances. The drone module includes a wireless network transmission module, and the wireless network transmission module is used to transmit the captured video or picture to the server module via the wireless network.

[0018] Furthermore, the lane classification module uses a segmentation model to detect road marking lines in the image, and determines the road type and its coordinate information in combination with road rules.

[0019] Furthermore, the vehicle detection unit uses target detection models such as SSD and YOLO to perform vehicle detection, the vehicle attribute classification unit uses a CNN network to classify and identify the attributes of the vehicle, and the license plate recognition unit uses an OCR algorithm to recognize characters in the license plate area.

[0020] A method for recognizing license plates and vehicle attributes on a highway based on the perspective of an unmanned aerial vehicle comprises the following steps:

[0021] S1. The drone conducts inspections above the highway and captures the video of the current road section in real time;

[0022] S2, the drone transmits the captured video or picture to the server via the wireless network;

[0023] S3. The server receives the image transmitted by the drone, calls the lane classification module to classify the current road image, and returns the types of all roads in the current image;

[0024] S4. The server calls the vehicle attribute recognition and license plate recognition module to detect and recognize the vehicle attributes and license plate number on the corresponding road according to the road type and corresponding road coordinates returned by the lane classification module;

[0025] S5. The server remotely controls the flight position, focal length adjustment and monitoring functions of the drone through the control module according to the analysis results;

[0026] S6. When a vehicle violation is detected, the server records the violation and returns a warning image and information.

[0027] Furthermore, the specific steps of the vehicle attribute recognition and license plate recognition module detecting and recognizing the vehicle attributes and license plate number include:

[0028] S4.1. Obtain the road type and corresponding road coordinates obtained from the lane classification module analysis;

[0029] S4.2, detecting vehicles on the corresponding road and identifying the attributes of the vehicles;

[0030] S4.3, detecting whether the vehicle has a license plate and identifying the vehicle's license plate number;

[0031] S4.4. Output vehicle attributes and license plate number.

[0032] Furthermore, the specific steps of the lane classification module performing road classification on the current road surface image include:

[0033] S3.1, using the segmentation model to detect road marking lines in the image;

[0034] S3.2, determining the road type in the image by combining road rules and detected road markings;

[0035] S3.3, obtaining the coordinates of the road in the image;

[0036] S3.4. Return the types of all roads in the image and the corresponding polygon coordinates.

[0037] The advantages of the present invention compared with the prior art are:

[0038] 1. The present invention provides a method and system for license plate recognition and vehicle attribute recognition of highways based on the perspective of drones. The lane classification module uses a segmentation model to detect road marking lines in an image, and determines the road type and its coordinate information in combination with road rules. The system can accurately identify the road type of the highway and provide basic data for subsequent vehicle attribute recognition and license plate recognition.

[0039] 2. The present invention provides a method and system for license plate recognition and vehicle attribute recognition on a highway based on the perspective of a drone. The vehicle detection unit adopts target detection models such as SSD and YOLO to detect the vehicle. The vehicle attribute classification unit adopts a CNN network to classify and identify the attributes of the vehicle. The license plate recognition unit adopts an OCR algorithm to recognize the characters in the license plate area. The vehicle attribute recognition module can recognize the attributes of the vehicle, including vehicle type, color, brand, etc., and the license plate recognition module can accurately recognize the license plate number, providing strong support for traffic management and law enforcement. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The present invention is a flow chart of a system for recognizing license plates and vehicle attributes on a highway based on the perspective of an unmanned aerial vehicle.

[0041] Figure 2 The present invention is a flow chart of a method for recognizing license plates and vehicle attributes on a highway based on the perspective of an unmanned aerial vehicle.

[0042] Figure 3 The present invention is a flowchart of a road type recognition module in a flowchart of a license plate recognition and vehicle attribute recognition system for a highway based on the perspective of an unmanned aerial vehicle.

[0043] Figure 4 The present invention discloses a flow chart of vehicle attribute and license plate recognition in a license plate recognition and vehicle attribute recognition system for a highway based on a drone's perspective. DETAILED DESCRIPTION

[0044] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] A license plate recognition and vehicle attribute recognition system for highways based on the perspective of drones, including the following modules:

[0047] Drone module: The drone module is used to inspect above the highway and capture the video of the current road section in real time;

[0048] The server module is wirelessly connected to the drone module, and is used to receive the pictures transmitted by the drone module, and call the lane classification module, vehicle attribute recognition and license plate recognition module to analyze and process the pictures;

[0049] Lane classification module: The lane classification module is used to classify the current road image and return the types of all roads in the current image;

[0050] The vehicle attribute recognition and license plate recognition module is used to detect and recognize the vehicle attributes and license plate numbers on the corresponding road according to the road type and corresponding road coordinates returned by the lane classification module;

[0051] The control module is used to remotely control the flight position, focal length adjustment and monitoring functions of the drone module according to the analysis results of the server module.

[0052] Furthermore, the drone is equipped with a camera that can automatically adjust the focal length to meet the needs of different monitoring distances. The drone module includes a wireless network transmission module, and the wireless network transmission module is used to transmit the captured video or picture to the server module via the wireless network.

[0053] Furthermore, the server can be a cloud server or a local server to provide sufficient computing resources to process image analysis tasks. The server module includes an alarm module, which is used to record violation events and return alarm pictures and information when vehicle violations are detected.

[0054] Furthermore, the lane classification module uses a segmentation model to detect road marking lines in the image and determines the road type and its coordinate information in combination with road rules.

[0055] Furthermore, the vehicle attribute recognition and license plate recognition module includes a vehicle detection unit, a vehicle attribute classification unit and a license plate recognition unit.

[0056] Furthermore, the vehicle detection unit uses target detection models such as SSD and YOLO to perform vehicle detection, the vehicle attribute classification unit uses a CNN network to classify and identify the attributes of the vehicle, and the license plate recognition unit uses an OCR algorithm to recognize characters in the license plate area.

[0057] A method for recognizing license plates and vehicle attributes on a highway based on the perspective of an unmanned aerial vehicle comprises the following steps:

[0058] S1. The drone conducts inspections above the highway and captures the video of the current road section in real time;

[0059] S2, the drone transmits the captured video or picture to the server via the wireless network;

[0060] S3. The server receives the image transmitted by the drone, calls the lane classification module to classify the current road image, and returns the types of all roads in the current image;

[0061] S4. The server calls the vehicle attribute recognition and license plate recognition module to detect and recognize the vehicle attributes and license plate number on the corresponding road according to the road type and corresponding road coordinates returned by the lane classification module;

[0062] S5. The server remotely controls the flight position, focal length adjustment and monitoring functions of the drone through the control module according to the analysis results;

[0063] S6. When a vehicle violation is detected, the server records the violation and returns a warning image and information.

[0064] Furthermore, the specific steps of the lane classification module for classifying the current road surface image include:

[0065] S3.1, using the segmentation model to detect road marking lines in the image;

[0066] S3.2, determining the road type in the image by combining road rules and detected road markings;

[0067] S3.3, obtaining the coordinates of the road in the image;

[0068] S3.4. Return the types of all roads in the image and the corresponding polygon coordinates.

[0069] Furthermore, the specific steps of the vehicle attribute recognition and license plate recognition module detecting and recognizing the vehicle attributes and license plate number include:

[0070] S4.1. Obtain the road type and corresponding road coordinates obtained from the lane classification module analysis;

[0071] S4.2, detecting vehicles on the corresponding road and identifying the attributes of the vehicles;

[0072] S4.3, detecting whether the vehicle has a license plate and identifying the vehicle's license plate number;

[0073] S4.4. Output vehicle attributes and license plate number.

[0074] For specific use, refer to Figures 1 to 4 As shown, the pilot releases the drone on the highway section that needs to be monitored. After the drone takes off, it captures the video of the current section in real time and transmits the picture to the server through the wireless network. The server can be a cloud server or a local server. The server uses the wireless network module to receive the pictures sent back by the drone. The server calls the lane classification module to classify the current road image and returns the types of all roads in the current picture. Then, according to the set detection process, the section to be monitored is found, and the signal is returned to remotely control the drone to fly to the road to be monitored. For example, if the vehicle occupancy of the emergency lane needs to be monitored, the position of the current emergency vehicle is returned to control the drone to fly. The drone interacts with the server through the wireless network until it hovers and adjusts the focal length after flying to the appropriate position, and then monitors the section. A wide-angle lens is generally used when flying, and it is switched to a telephoto lens as needed when monitoring. The control of the drone flight and the electric adjustment of the focal length are not within the scope of the present invention. The drone transmits the picture of the section to be monitored to the server in real time through the wireless network. The server calls the vehicle attribute recognition and license plate recognition modules to analyze in real time whether there are any violations of the vehicles on the section. Once a violation is found, the violation event is recorded and the warning picture and information are returned at the same time. Staff can remotely issue commands to the drone, control it to fly to a new monitoring location, and adjust the monitoring functions.

[0075] First, the real-time highway road image acquired by the drone is input. Then, the lane classification module is called to classify the current road image and return the types of all roads in the current picture. Secondly, the drone vehicle attribute recognition and license plate recognition module obtains the coordinate information of the road position. Finally, the drone vehicle attribute recognition and license plate recognition module identifies the road conditions of the current road in real time and returns alarm information.

[0076] The process of the road type recognition module is to input an image, use the segmentation model to detect the road marking lines in the image, determine the type of road in the image by combining road rules and the detected road marking lines, obtain the coordinates of the road in the image (obtain the coordinates of the four vertices of the polygon), and return the types of all roads in the image and the corresponding polygon coordinates.

[0077] The process of vehicle attribute and license plate recognition is to input the image, obtain the road type and the coordinates of the corresponding road obtained from the road type classification module, and detect the vehicle on the corresponding road. If a vehicle is detected, the attributes of the vehicle are identified. Common target detection models such as SSD and YOLO can be used for vehicle detection. The classification model can use a CNN network to detect whether the vehicle has a license plate. If a license plate is detected, the license plate number of the vehicle is identified. Common target detection models such as SSD and YOLO can be used for license plate detection. After the license plate is detected, the characters in the license plate area are identified using a common OCR algorithm, and the vehicle attributes and license plate number are output.

[0078] Road marking recognition model training, the samples used by the model come from the field. They include: white dotted line, yellow dotted line, white solid line, yellow solid line, white special dotted line, yellow special dotted line, white special solid line, yellow special solid line, white grid line, yellow grid line, white guide line, yellow guide line, zebra crossing, guardrail, green belt, etc. Road markings, guardrails and green belts are annotated with polygons. The annotated samples are used to train the segmentation model.

[0079] Vehicle attribute recognition model training, the samples used by the model come from the scene. Use 2D boxes to mark the vehicle, and then mark the vehicle attributes, including but not limited to vehicle type, vehicle color, vehicle brand, etc. The labeled samples are used to train the vehicle detection model and vehicle attribute classification model.

[0080] License plate recognition model training, the model uses samples from the scene. Use 2D boxes to mark the license plate, and for the license plate area, mark the characters of the license plate. After the samples are marked, they are used to train the license plate detection model and the license plate character recognition model.

[0081] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0082] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0083] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A license plate recognition and vehicle attribute recognition system for highways based on the perspective of drones, characterized in that: Includes the following modules: A drone module, which is used to inspect above the highway and capture the video of the current road section in real time; A server module, which is wirelessly connected to the drone module, is used to receive pictures transmitted by the drone module, and call the lane classification module, the vehicle attribute recognition and the license plate recognition module to analyze and process the pictures; A lane classification module, which is used to classify the current road surface image and return the types of all roads in the current image; A vehicle attribute recognition and license plate recognition module, which is used to detect and recognize vehicle attributes and license plate numbers on the corresponding road according to the road type and corresponding road coordinates returned by the lane classification module; The control module is used to remotely control the flight position, focal length adjustment and monitoring functions of the drone module according to the analysis results of the server module.

2. The system for license plate recognition and vehicle attribute recognition based on a highway from the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: The drone is equipped with a camera that can automatically adjust the focal length to meet the needs of different monitoring distances. The drone module includes a wireless network transmission module, and the wireless network transmission module is used to transmit the captured video or picture to the server module via the wireless network.

3. The system for license plate recognition and vehicle attribute recognition based on a highway from the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: The server can be a cloud server or a local server to provide sufficient computing resources to process image analysis tasks. The server module includes an alarm module, which is used to record violation events and return alarm pictures and information when vehicle violations are detected.

4. The system for license plate recognition and vehicle attribute recognition based on a highway from the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: The lane classification module uses a segmentation model to detect road marking lines in an image, and determines the road type and its coordinate information in combination with road rules.

5. The system for license plate recognition and vehicle attribute recognition based on a highway from the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: The vehicle attribute recognition and license plate recognition module includes a vehicle detection unit, a vehicle attribute classification unit and a license plate recognition unit.

6. The system for license plate recognition and vehicle attribute recognition based on the highway from the perspective of an unmanned aerial vehicle according to claim 5, characterized in that: The vehicle detection unit uses target detection models such as SSD and YOLO to detect vehicles, the vehicle attribute classification unit uses a CNN network to classify and identify the attributes of the vehicle, and the license plate recognition unit uses an OCR algorithm to recognize characters in the license plate area.

7. A method for license plate recognition and vehicle attribute recognition on a highway based on the perspective of an unmanned aerial vehicle, characterized in that: The following steps are involved: S1. The drone conducts inspections above the highway and captures the video of the current road section in real time; S2, the drone transmits the captured video or picture to the server via the wireless network; S3. The server receives the image transmitted by the drone, calls the lane classification module to classify the current road image, and returns the types of all roads in the current image; S4. The server calls the vehicle attribute recognition and license plate recognition module to detect and recognize the vehicle attributes and license plate number on the corresponding road according to the road type and corresponding road coordinates returned by the lane classification module; S5. The server remotely controls the flight position, focal length adjustment and monitoring functions of the drone through the control module according to the analysis results; S6. When a vehicle violation is detected, the server records the violation and returns a warning image and information.

8. The method for license plate recognition and vehicle attribute recognition on a highway based on the perspective of an unmanned aerial vehicle according to claim 7, characterized in that: The specific steps of the lane classification module performing road classification on the current road surface image include: S3.1, using the segmentation model to detect road marking lines in the image; S3.2, determining the road type in the image by combining road rules and detected road markings; S3.3, obtaining the coordinates of the road in the image; S3.

4. Return the types of all roads in the image and the corresponding polygon coordinates.

9. The method for license plate recognition and vehicle attribute recognition on a highway based on the perspective of an unmanned aerial vehicle according to claim 7, characterized in that: The specific steps of the vehicle attribute recognition and license plate recognition module detecting and recognizing the vehicle attributes and license plate number include: S4.

1. Obtain the road type and corresponding road coordinates obtained from the lane classification module analysis; S4.2, detecting vehicles on the corresponding road and identifying the attributes of the vehicles; S4.3, detecting whether the vehicle has a license plate and identifying the vehicle's license plate number; S4.

4. Output vehicle attributes and license plate number.

Images