Intelligent inspection unmanned aerial vehicle image recognition system

Abstract

The invention relates to the technical field of unmanned aerial vehicle road inspection, and discloses an intelligent inspection unmanned aerial vehicle image recognition system, which comprises an image capture module which comprises a camera arranged on an unmanned aerial vehicle and is used for capturing video image information of a traffic road; the analysis end is used for identifying and analyzing the video image information data of the traffic road and determining a hovering control strategy and an information prompting strategy according to an identification and analysis result; the execution end comprises a control module and a display module, and the control module is used for executing the hovering control strategy and controlling the flight state of the unmanned aerial vehicle; the display module comprises an electronic induction screen arranged on a traffic road and is used for executing the information prompt strategy and displaying information prompt content. And the communication module is used for establishing a communication relationship between the analysis end and the execution end. According to the method, the road congestion problem can be relieved, the road dredging efficiency is improved, and in addition, manual participation in the road dredging process is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) road inspection, and particularly to an intelligent inspection UAV image recognition system. Background Art

[0002] With the improvement of people's living standards, the number of private cars has been increasing continuously, resulting in an increased load on roads and thus traffic congestion occurring almost every day in various cities. Traffic congestion is a complex and widespread phenomenon, which involves multiple factors, including the number of vehicles, traffic flow, driving behavior, and the relatively uncontrollable factor of road design. Traffic congestion is defined as a traffic phenomenon in which, within a certain period of time, due to the increase in traffic demand, the total traffic volume passing through a certain section or intersection of a road is greater than the traffic capacity of the road (i.e., the passing capacity of the section or intersection), resulting in the traffic flow on the road being unable to flow smoothly and the excess traffic flow remaining on the road.

[0003] There are many reasons for road congestion. From a macroscopic perspective, there are the following aspects: The mismatch between the number of vehicles and road resources, the growth rate of vehicle ownership is much higher than the road construction speed. With the development of the economy and the improvement of people's living standards, the number of private cars has increased rapidly, while the speed of road construction and expansion cannot match it, resulting in a shortage of road resources; Unreasonable road design and planning: The roads in some cities are designed too narrow, and there are no fork roads between roads, making it impossible to divert vehicles, resulting in traffic congestion; Driving behavior and traffic management problems, some citizens have weak awareness of traffic safety and legal concepts, and some motor vehicle drivers have illegal behaviors, such as running red lights and going in the wrong direction. These behaviors not only endanger the safety of themselves and others, but also exacerbate traffic congestion. However, it is not easy to alleviate the actual road congestion problem from the above perspectives. To solve the road congestion problem, road workers often conduct inspections and road dredging on congested sections during specific time periods. Although this solution can play a role in alleviating road congestion, the efficiency is low, and there are also personal safety problems of road workers in the specific implementation process. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent inspection UAV image recognition system to solve the above technical problems:

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] An intelligent inspection UAV image recognition system, comprising: an image capture module, including a camera installed on the UAV, for capturing video image information of traffic roads; an analysis terminal, for identifying and analyzing the video image information data of traffic roads, and establishing a hover control strategy and an information prompt strategy according to the identification and analysis results; an execution terminal, including a control module and a display module, the control module is used to execute the hover control strategy and control the flight state of the UAV; the display module includes an electronic induction screen installed on the traffic road, for executing the information prompt strategy and displaying information prompt content. A communication module, for establishing a communication relationship between the analysis terminal and the execution terminal. Through the above technical solution, the main content of an intelligent inspection UAV image recognition system is provided. The video image information of traffic roads is collected by the image capture module carried by the UAV. The specific collection time and location can be set at sections prone to congestion during peak periods. The analysis terminal identifies and analyzes the video image information data of traffic roads, and then controls the working conditions of the UAV and establishes an information prompt strategy according to the results of the identification and analysis. In the execution terminal, the control module can correspondingly control the flight and hover of the UAV according to the identification and analysis results, and the basket-induced screen can display the information prompt content and show the road conditions in front of the driver. The content of the road conditions can specifically indicate the reasons for the congestion of the road ahead, such as lane occupation for construction or a car accident in a certain lane. Therefore, it can remind vehicles to change lanes in advance to relieve the congestion of the road. It should be noted that the UAV shoots in a hover state, mainly for focusing on monitoring key positions of the road, such as traffic bottlenecks. The above technical solution uses the camera carried by the UAV to obtain the video image information of the congested road, then the analysis terminal identifies and analyzes the video image information, converts the results of the identification and analysis into prompt content indicating the road conditions, and displays it to the driver through the display module to guide the driver to drive correctly according to the road conditions ahead. Therefore, it can not only relieve the problem of road congestion, but also improve the efficiency of road dredging. In addition, it also reduces the manual participation in the process of road dredging.

[0007] As a further technical solution, the analysis terminal includes: an edge analysis module for performing real-time analysis on the video image information data of the traffic road and establishing a hovering control strategy according to the factual analysis results; a data processing center arranged on the ground for identifying and analyzing the video image information captured by the camera when the UAV is in a hovering state and establishing an information prompt strategy according to the identification and analysis. Through the above technical solution, the specific composition of the analysis terminal is provided. Among them, the edge analysis module can be arranged on the UAV to facilitate real-time analysis of the video image information data of the traffic road in a timely manner, and then establish a hovering control strategy according to the factual analysis results. The data processing center, arranged on the ground, can identify and analyze the video image information captured by the camera when the UAV is in a hovering state, and then establish an information prompt strategy according to the identification and analysis. The relevant data of the information prompt content is transmitted to the electronic induction screen through the communication module and displayed to the driver.

[0008] As a further technical solution, the process of capturing the video image information of the traffic road includes:

[0009] The UAV flies over the section to be detected at a preset flight speed, and the flight direction of the UAV is the same as the driving direction of the vehicles on the road, and the video image of the top view of the section to be detected is collected through the camera.

[0010] The camera captures the video image of the road directly below the UAV in the hovering state.

[0011] As a further technical solution, the edge analysis module includes:

[0012] An image analysis unit for analyzing the video image of the traffic road captured by the camera, obtaining road condition-related data according to the analysis process, and the road condition-related data includes: the number of vehicles on the traffic road, the average vehicle speed, and the driving state of the vehicles.

[0013] A data analysis unit for comprehensively analyzing the road condition-related data and controlling the flight state of the UAV according to the comprehensive analysis results.

[0014] As a further technical solution, the process of analyzing the video image of the traffic road captured by the camera includes:

[0015] Preprocessing the video image, including adjusting the size and format conversion of the video image according to the number of lanes to be detected;

[0016] Identifying the vehicles in the video image based on the Haar feature algorithm;

[0017] Obtaining the total number of vehicles in the field of view captured by the camera by counting the number of vehicles in the video frames.

[0018] As a further technical solution, the process of analyzing the video images of the traffic road captured by the camera further includes:

[0019] Tracking the vehicles in the camera's field of view based on the target tracking algorithm, and obtaining the relative position information and time information of the vehicles with respect to the drone;

[0020] Calculating the displacement Δd of the vehicle between adjacent frame rates;

[0021] Through the formula Calculating and obtaining the driving speed v of the vehicle;

[0022] Where, v0 is the flight speed of the drone, Z is the distance from the camera to the vehicle; f is the focal length of the camera when shooting the vehicle; F is the frame rate of the video.

[0023] As a further technical solution, the process of comprehensively analyzing the road condition-related data includes:

[0024] Through the formula Calculating and obtaining the road condition evaluation coefficient K;

[0025] Where, C1 is the total number of vehicles in the camera's field of view; n is the number of lanes of the road section to be detected; C2 is the number of lane-changing vehicles in the camera's field of view; is the average vehicle speed of each vehicle in the camera's field of view; v s is the standard passing speed of the road section to be measured; α1, α2 are preset weight coefficients;

[0026] Comparing the road condition evaluation coefficient K with the preset threshold K t for comparison:

[0027] If K > K t , then control the drone to hover, and continuously collect image of the road under the drone through the camera;

[0028] If K ≤ K t , the drone continues to fly at the preset flight speed.

[0029] As a further technical solution, the process of establishing the information prompt strategy includes:

[0030] The camera captures the static aerial video image of the road under the drone when the drone is in the hovering state;

[0031] Extracting the feature parameters in the static aerial video image based on the convolutional neural network;

[0032] Importing the feature parameter data into the trained recognition model, and the output result indicates the road condition;

[0033] The prompt content corresponding to the road conditions of the section to be measured is displayed through an electronic induction screen.

[0034] Advantages of the present invention:

[0035] In the present invention, video image information of a congested road is obtained by a camera carried by a drone, and then the video image information is identified and analyzed by an analysis terminal. The result of the identification and analysis is converted into a prompt content indicating the road condition information, and is displayed to the driver through a display module to guide the driver to drive correctly according to the road condition ahead. Therefore, it can not only relieve the road congestion problem, but also improve the efficiency of road dredging. In addition, it reduces the manual participation in the process of road dredging. Description of the drawings

[0036] The present invention will be further described below with reference to the drawings.

[0037] Figure 1 It is a content summary block diagram of an intelligent inspection drone image recognition system in the present invention. Specific embodiments

[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] Please refer to Figure 1 As shown, an intelligent inspection drone image recognition system includes: an image capture module, including a camera disposed on the drone for capturing video image information of a traffic road; an analysis terminal for identifying and analyzing the video image information data of the traffic road and establishing a hover control strategy and an information prompt strategy according to the identification and analysis results; an execution terminal, including a control module and a display module, the control module is used to execute the hover control strategy and control the flight state of the drone; the display module includes an electronic induction screen disposed on the traffic road for executing the information prompt strategy and displaying the information prompt content. A communication module is used to establish a communication relationship between the analysis terminal and the execution terminal.

[0040] Through the above technical solutions, this embodiment provides the main content of an intelligent inspection UAV image recognition system. The video image information of the traffic road is collected by an image capture module carried by the UAV. The specific collection time and location can be set on the sections prone to congestion during peak hours. The analysis terminal identifies and analyzes the video image information data of the traffic road, and then controls the working conditions of the UAV and determines the information prompt strategy according to the results of the identification and analysis. In the execution terminal, the control module can control the flight and hovering of the UAV according to the results of the identification and analysis. The basket-shaped induction screen can display the information prompt content and show the road conditions in front of the driver. The content of the road conditions can specifically indicate the reasons for the congestion ahead, such as lane occupation for construction or a car accident in a certain lane. Therefore, it can remind vehicles to change lanes in advance to relieve the congestion of the road. It should be noted that the UAV takes pictures in the hovering state, mainly for focusing on monitoring key positions of the road, such as traffic bottlenecks. To sum up, this embodiment obtains the video image information of the congested road through the camera carried by the UAV, then identifies and analyzes the video image information through the analysis terminal, converts the results of the identification and analysis into prompt content indicating the road conditions, and shows it to the driver through the display module, guiding the driver to drive correctly according to the road conditions ahead. Therefore, it can not only relieve the problem of road congestion, but also improve the efficiency of road dredging. In addition, it reduces the manual participation in the process of road dredging.

[0041] The analysis terminal includes:

[0042] An edge analysis module, which is used for real-time analysis of the video image information data of the traffic road and determines a hovering control strategy according to the results of the factual analysis;

[0043] A data processing center, which is set on the ground and is used for identifying and analyzing the video image information captured by the camera when the UAV is in the hovering state, and determines an information prompt strategy according to the identification and analysis.

[0044] Through the above technical solutions, this embodiment provides the specific composition of the analysis terminal. Among them, the edge analysis module can be set on the UAV to facilitate real-time analysis of the video image information data of the traffic road on the spot, and then determine a hovering control strategy according to the results of the factual analysis. The data processing center, which is set on the ground, can identify and analyze the video image information captured by the camera when the UAV is in the hovering state, and then determine an information prompt strategy according to the identification and analysis. The relevant data of the information prompt content is transmitted to the electronic induction screen through the communication module and shown to the driver.

[0045] The process of capturing the video image information of the traffic road includes:

[0046] The drone flies over the section to be detected at a preset flight speed, and the flight direction of the drone is the same as the driving direction of the vehicles on the road. The drone captures video images of the top-down view of the section to be detected through a camera;

[0047] The camera captures video images of the road directly below the drone in a hovering state.

[0048] Through the above technical solution, this embodiment provides a process for capturing video image information of a traffic road. Specifically, the drone flies over the section to be detected at a preset flight speed, where the section to be detected is preferably set as a section prone to traffic congestion. It should be noted that the flight direction of the drone is the same as the driving direction of the vehicles on the road. The drone captures video images of the top-down view of the section through a camera.

[0049] The edge analysis module includes:

[0050] An image parsing unit, which is used to parse the video images of the traffic road captured by the camera, and obtain road condition related data according to the parsing process. The road condition related data includes: the number of vehicles on the traffic road, the average vehicle speed, and the driving state of the vehicles.

[0051] A data analysis unit, which is used to comprehensively analyze the road condition related data and control the flight state of the drone according to the comprehensive analysis result.

[0052] Through the above technical solution, this embodiment provides the specific content of the edge analysis module. The edge analysis module includes an image parsing unit and a data analysis unit. Among them, the image parsing unit can parse the video images of the traffic road captured by the camera, and obtain road condition related data according to the parsing process. The road condition related data includes: the number of vehicles on the traffic road, the average vehicle speed, and the driving state of the vehicles, where the driving state mainly includes lane change and straight driving of the vehicles. The data analysis unit can comprehensively analyze the road condition related data and control the flight state of the drone according to the comprehensive analysis result.

[0053] The process of parsing the video images of the traffic road captured by the camera includes:

[0054] Preprocess the video images, including adjusting the size and format conversion of the video images according to the number of lanes to be detected;

[0055] Identify the vehicles in the video images based on the Haar feature algorithm;

[0056] Obtain the total number of vehicles in the camera shooting field of view by counting the number of vehicles in the video frames.

[0057] The process of parsing the video images of the traffic road captured by the camera also includes:

[0058] Track the vehicles in the camera's field of view based on the target tracking algorithm, and obtain the relative position information and time information of the vehicles with respect to the drone;

[0059] Calculate the displacement Δd of the vehicle between adjacent frame rates;

[0060] Through the formula Calculate and obtain the driving speed v of the vehicle;

[0061] where, v0 is the flight speed of the drone, Z is the distance from the camera to the vehicle; f is the focal length of the camera when shooting the vehicle; F is the frame rate of the video.

[0062] Through the above technical solution, this embodiment provides a process for parsing the video image of the traffic road captured by the camera. The purpose of parsing is to obtain the number of vehicles, vehicle speeds, and driving states of vehicles on the traffic road according to the video image. Specifically, first preprocess the video image, including adjusting the size and format conversion of the video image according to the number of lanes to be detected, to ensure that the video image can display the states of all lanes in a certain direction of the section to be detected. Then, based on the Haar feature algorithm, identify the vehicles in the video image, and obtain the total number of vehicles in the camera's shooting field of view by counting the number of vehicles in the video frames. On the basis of identifying the vehicles in the video image, use a target tracking algorithm such as the Kalman filter algorithm to track the vehicles in the camera's field of view, and obtain the relative position information and time information of the vehicles with respect to the drone, calculate the displacement Δd of the vehicle between adjacent frame rates, and finally through the formula Calculate and obtain the driving speed v of the vehicle. It should be noted that in the above formula, v0 is the preset flight speed of the drone, Z is the distance from the camera to the vehicle; f is the focal length of the camera when shooting the vehicle; F is the frame rate of the video. The driving speed v of the vehicle and the flight speed v0 of the drone are in the same direction.

[0063] The process of comprehensively analyzing the road condition-related data includes:

[0064] Through the formula Calculate and obtain the road condition evaluation coefficient K;

[0065] where, C1 is the total number of vehicles in the camera's field of view; n is the number of lanes of the section to be detected; C2 is the number of lane-changing vehicles in the camera's field of view; is the average vehicle speed of each vehicle in the camera's field of view; v s is the standard passing speed of the section to be measured; α1, α2 are preset weight coefficients;

[0066] Compare the road condition evaluation coefficient K with the preset threshold K t as follows:

[0067] If K > Kt Then, the drone is controlled to hover, and continuous image acquisition of the road below the drone is performed through the camera.

[0068] If K ≤ K t The drone continues to fly at the preset flight speed.

[0069] Through the above technical solution, this embodiment provides a process for comprehensively analyzing road condition-related data. Specifically, first, according to the formula the road condition evaluation coefficient K is calculated and obtained, and then the road condition evaluation coefficient K is compared with the preset threshold K t When K > K t , it indicates that there are many vehicles, slow vehicle speeds, and multiple lane-changing vehicles in the camera shooting field of view. Therefore, there is a high risk that the corresponding position in the field of view is the traffic bottleneck of the congested section. So, the drone is controlled to hover, and continuous image acquisition of the road below the drone is performed through the camera. When K ≤ K t , it indicates that the vehicles on the road are driving normally, so the drone continues to fly at the preset flight speed. It should be noted that in the formula, C1 is the total number of vehicles in the camera field of view; n is the number of lanes of the section to be detected; C2 is the number of lane-changing vehicles in the camera field of view; is the average vehicle speed of each vehicle in the camera field of view; α1, α2 are preset weight coefficients; v s is the standard passing speed of the section to be measured, which is specifically obtained according to the empirical data of the normal driving speed of vehicles on this section of the road.

[0070] The process of establishing the information prompt strategy includes:

[0071] The camera acquires static aerial video images of the road below the drone when the drone is in a hovering state.

[0072] Based on the convolutional neural network, the feature parameters in the static aerial video images are extracted.

[0073] The feature parameter data is imported into the trained recognition model, and the output result indicates the road condition.

[0074] The prompt content corresponding to the road condition of the section to be measured is displayed through the electronic induction screen.

[0075] Through the above technical solution, this embodiment provides a process for establishing an information prompting strategy. Specifically, the camera collects static aerial video images of the road below the drone when the drone is in a hovering state, and extracts feature parameters from the static aerial video images based on a convolutional neural network. The feature parameters include: vehicle speed, number of people getting off the vehicle, number of vehicles expressing lane changes, and the corresponding lane-changing positions. The reason for using these as feature parameters is that in the event of a traffic accident, the speed of the accident vehicle is zero, and there is usually someone getting off the vehicle to negotiate. In addition, if multiple vehicles change lanes in the same lane, it is highly likely that the lane is occupied. Therefore, the feature parameter data is imported into the trained recognition model, and the output result indicates the road condition. The prompt content corresponding to the road condition of the section to be measured is displayed through an electronic induction screen. The prompt content specifically includes: lane occupation and construction in a certain lane, traffic accident in a certain lane, etc. It reminds other drivers to decelerate or change lanes in advance after obtaining these prompt messages, reducing the accident rate and alleviating the road congestion situation.

[0076] The above has described a specific embodiment of the present invention in detail, but the described content is only the preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. An intelligent inspection UAV image recognition system, characterized in that, Including: An image capture module, including a camera installed on the drone, for capturing video image information of the traffic road; An analysis terminal, for identifying and analyzing the video image information data of the traffic road, and establishing a hovering control strategy and an information prompt strategy according to the identification and analysis results; An execution terminal, including a control module and a display module, the control module is used to execute the hovering control strategy and control the flight state of the drone; the display module includes an electronic induction screen installed on the traffic road, for executing the information prompt strategy and displaying the information prompt content; A communication module, for establishing a communication relationship between the analysis terminal and the execution terminal.

2. The intelligent inspection UAV image recognition system according to claim 1, wherein, The analysis terminal includes: An edge analysis module, for performing real-time analysis on the video image information data of the traffic road, and establishing a hovering control strategy according to the factual analysis results; A data processing center, installed on the ground, for identifying and analyzing the video image information captured by the camera when the drone is in a hovering state, and establishing an information prompt strategy according to the identification and analysis.

3. The intelligent inspection UAV image recognition system according to claim 2, characterized in that, The process of capturing video image information of the traffic road includes: The drone flies over the section to be detected at a preset flight speed, and the flight direction of the drone is the same as the driving direction of the vehicles on the road, and the video image of the top view of the section to be detected is collected through the camera; The camera captures the video image of the road directly below the drone in a hovering state.

4. An intelligent inspection UAV image recognition system according to claim 3, characterized in that The edge analysis module includes: An image parsing unit, for parsing the video image of the traffic road captured by the camera, and obtaining road condition-related data according to the parsing process, the road condition-related data includes: the number of vehicles on the traffic road, the average vehicle speed, and the driving state of the vehicles; A data analysis unit, for comprehensively analyzing the road condition-related data, and controlling the flight state of the drone according to the comprehensive analysis results.

5. The intelligent inspection UAV image recognition system according to claim 4, wherein, The process of parsing the video image of the traffic road captured by the camera includes: Preprocessing the video image, including adjusting the size and format conversion of the video image according to the number of lanes to be detected; Identifying the vehicles in the video image based on the Haar feature algorithm; Obtaining the total number of vehicles in the camera shooting field of view by counting the number of vehicles in the video frames.

6. The intelligent inspection drone image recognition system according to claim 5, wherein, The process of parsing the video image of the traffic road captured by the camera further includes: Tracking the vehicles in the camera field of view based on the target tracking algorithm, and obtaining the relative drone position information and time information of the vehicles; Calculating the displacement Δd of the vehicle between adjacent frame rates; Obtain the driving speed v of the vehicle through the formula Calculate and obtain the driving speed v of the vehicle; Wherein, v0 is the flight speed of the drone, Z is the distance from the camera to the vehicle; f is the focal length of the camera when shooting the vehicle; F is the frame rate of the video.

7. An intelligent inspection UAV image recognition system according to claim 6, characterized in that, The process of comprehensively analyzing the road condition-related data includes: Obtain the road condition evaluation coefficient K through the formula Calculate and obtain the road condition evaluation coefficient K; Among them, C1 is the total number of vehicles in the camera's field of view; n is the number of lanes on the section to be detected; C2 is the number of lane-changing vehicles in the camera's field of view; is the average vehicle speed of each vehicle in the camera's field of view; v s is the standard passing speed of the section to be measured; α1 and α2 are preset weight coefficients; Compare the road condition evaluation coefficient K with the preset threshold K t for comparison: If K > K t , the drone is controlled to hover, and continuous image acquisition of the road under the drone is performed through the camera; If K ≤ K t , the drone continues to fly at the preset flight speed.

8. An intelligent inspection UAV image recognition system according to claim 7, characterized in that, The process of establishing the information prompt strategy includes: The camera captures the static aerial video image of the road below the drone when the drone is in a hovering state; Extracting the feature parameters in the static aerial video image based on the convolutional neural network; Importing the feature parameter data into the trained recognition model, and the output result indicates the road condition; Displaying the prompt content corresponding to the road condition of the section to be detected through the electronic induction screen.

Images