Fixed-wing unmanned aerial vehicle real-time route planning method based on intelligent identification technology

Abstract

The invention discloses a fixed-wing unmanned aerial vehicle real-time route planning method based on an intelligent identification technology. The method comprises the following steps: acquiring a flight instruction; collecting an unmanned aerial vehicle flight environment value, and detecting whether the unmanned aerial vehicle take-off environment value accords with the unmanned aerial vehicle patrol flight instruction; based on the collected patrol target point and patrol range information value of the unmanned aerial vehicle, obtaining a flight task target parameter of the unmanned aerial vehicle; acquiring initial unmanned aerial vehicle flight route information, and activating the unmanned aerial vehicle to start a flight task of the unmanned aerial vehicle along the initial unmanned aerial vehicle flight route; detecting the flight environment value of the unmanned aerial vehicle in real time based on the flight process; and completing the current round of unmanned aerial vehicle patrol based on the collected patrol target point and patrol range information value of the unmanned aerial vehicle. The invention designs and develops a highly intelligent self-inspection and self-inspection unmanned aerial vehicle inspection technology capable of adjusting an optimal line in real time based on a flight environment, and the technology greatly reduces the cost of manual inspection and avoids the negative influence of a complex environment on inspection.

Description

Technical Field

[0001] The present invention belongs to the technical field of UAV navigation, and particularly relates to a real-time route planning method for a fixed-wing UAV based on intelligent recognition technology. Background Art

[0002] In current outdoor monitoring and inspection tasks, despite continuous technological progress, many industries still rely on traditional manual inspection methods. Although this method is relatively reliable and flexible in personnel arrangement, it has significant deficiencies and difficulties in many aspects. First of all, the low efficiency of manual inspection is a prominent problem. Inspection personnel need to move between different locations to check equipment and facilities one by one, which is not only time-consuming and laborious but also easily affected by weather and terrain. Under complex terrain conditions, such as mountains, forests or densely built urban areas, the movement of inspection personnel is restricted, resulting in a significant extension of the task completion time. Secondly, the cost of manual inspection is too high, and these costs accumulate in long-term monitoring and inspection tasks, bringing a heavy economic burden to the operation of enterprises. In addition, manual inspection often requires the cooperation of multiple people, increasing the difficulty of human resource allocation and management and further driving up the overall cost.

[0003] To address the above technical problems, an intelligent inspection solution that relies on the flexibility of UAVs has now been developed. People only need to remotely control the UAV to complete the inspection work according to the set service plan. However, in the existing technology, UAV technology is not yet mature. For example, it requires full cooperation from personnel to issue instructions. At the same time, since the UAV operates remotely, it highly depends on the intelligent algorithm of the UAV to be able to adaptively adjust according to the actual situation. However, in the existing scheme design, it is difficult for the UAV to intelligently plan the route, and its response ability is poor, and the actual experience is not ideal.

[0004] In view of the deficiencies of the existing technology, it is urgent to design a real-time route planning method for a fixed-wing UAV based on intelligent recognition technology to solve the above technical problems. Summary of the Invention

[0005] To solve the above deficiencies in the existing technology, the purpose of the present invention is to overcome the existing deficiencies and provide a real-time route planning method for a fixed-wing UAV based on intelligent recognition technology. The method includes the following:

[0006] Obtain a flight instruction, and based on the received flight instruction, collect the inspection target points and inspection scope of the UAV;

[0007] Collect the UAV flight environment value, and detect whether the UAV takeoff environment value conforms to the UAV inspection flight instruction; if it conforms, activate the UAV inspection flight;

[0008] Based on the acquired inspection target points and inspection range information values of the drone, obtain the flight mission target parameters of the drone, and starting from the current position, construct the flight route of the drone for this time;

[0009] Obtain the initial drone flight route information, activate the drone to start the flight mission of the drone along the initial drone flight route; detect the drone flight environment value in real time during the flight process; if during the flight process, the drone flight environment value meets the flight requirements of the drone, maintain the corresponding state, otherwise make corresponding adjustments;

[0010] Based on the acquired inspection target points and inspection range information values of the drone, complete the inspection of the drone for this round.

[0011] As a further optimization of the above solution, the method for constructing the flight route of the drone for this time includes the following:

[0012] Mark all the inspection target points and inspection ranges of the drone to form an inspection area; set other areas as non-inspection areas; mark the number of inspection areas as M, and based on the M-scale drone flight solution space dimension as L, the initialization of the drone inspection area is as follows:

[0013] A i =(a i,1 ,a i,2 ,…,a i,L )

[0014] In the above formula, i = 1, 2, …, M, a i represents the initial state position of the drone during the inspection process;

[0015] Starting from the current position, based on each marked inspection area, respectively form the corresponding optimal path, and construct the optimal path trend:

[0016]

[0017] In the above formula, j = 1, 2, …, L; C max is the maximum number of iterations for the drone to find the optimal point in the inspection area, c is the current number of c rounds of iterations; ε, μ are adjustment parameters of the optimal path trend; Δl is the displacement; f i,j (a i ) represents the gradient value corresponding to the j-th dimension of the drone solution space at the i-th place;

[0018] Based on the constructed optimal path trend, respectively perform the optimal position positioning of the drone for each marked inspection area, and monitor all positioning records:

[0019] If the optimal position of the drone in any inspection area is obtained, form a preferred domain based on this position Substitute all boundary points into the best path trend calculation formula within the said preferred domain to determine whether the current position is the optimal position of the stable drone. If so, update this point as the latest optimal position of the drone; otherwise, keep it unchanged.

[0020] Repeat the above steps until the optimal positions of the stable drones in all inspection areas are found, and record the shortest flight route as the flight route of the drone.

[0021] As a further optimization of the above solution, the method for constructing the flight route of the drone in this time further includes the following:

[0022] For the preferred domain Conduct an anti-premature convergence design and construct an anti-convergence relational expression:

[0023] a i,j =(1 - γ)a i,j +γa n,j

[0024] where a i,j is the optimal position of the drone, a n,j is any position in any inspection area, n is not equal to i; γ is the learning weight of the best path trend of the drone.

[0025] As a further optimization of the above solution, the specific process of real-time detecting the flight environment value of the drone during the flight includes the following:

[0026] Based on the drone positioning to obtain the longitude and latitude coordinates, determine the world coordinate system of the drone. Based on the collected images of the drone, apply the camera coordinate system of the collected images to be perspective-transformed into the image coordinate system, and calculate the relationship between the camera imaging position points of the drone in the three-dimensional world coordinate system.

[0027] Continuously conduct omnidirectional image acquisition of the drone, mark the identified objects based on the image algorithm, and detect the relative displacement between the identified object and the drone based on the continuously captured images; if the relative displacement between the identified object and the drone remains unchanged or decreases, mark it as a moving object; if the relative displacement increases, respectively obtain the image coordinates of the identified object detected for the first and second times, obtain the displacement of the drone at the corresponding time based on the coordinate transformation. Let the angle between the drone's travel route and the distance from the drone to the identified object be θ, and the drone speed be V 机 , and the occurrence time be t; then calculate the relationship between the distance l from the image coordinates of the identified object detected for the second time to the drone and V 机 ·t / cosθ; if they are equal, mark the identified object as stationary, otherwise mark it as a moving object.

[0028] Continuously conduct all-round image acquisition of the drone, and record the image coordinates of any detected moving object based on multiple time periods; if the relative displacement between the moving object and the drone remains unchanged or increases within consecutive time periods, mark it as a non-obstacle moving object; if the relative displacement becomes smaller, continue the monitoring;

[0029] If the drone and the moving object are on the same plane, based on the current position, select the perpendicular to the traveling direction as the reference line, and mark the traveling direction of the drone as the front; based on the all-round image acquisition of the drone, determine the front-back relationship between the moving object and the drone;

[0030] If the moving object is behind the drone at this time, record the image coordinates of different positions of the moving object based on multiple time periods; then, based on coordinate system conversion, obtain the actual displacement of the moving object in multiple time periods, and calculate the average speed V of the moving object corresponding to multiple time periods based on the corresponding occurrence times recorded by the drone 物 ; if V 机 ≥V 物 , mark it as a non-obstacle moving object; otherwise, increase the speed or altitude of the drone;

[0031] If the moving object is on the reference line of the drone at this time, respectively detect the speed relationship between the drone and the moving object. If V 机 ≥V 物 , mark it as a non-obstacle moving object; otherwise, increase the speed or altitude of the drone;

[0032] If the moving object is in front of the drone at this time, respectively detect the speed and moving direction of the drone and the moving object.

[0033] As a further optimization of the above solution, the real-time detection of the drone flight environment value during the flight process specifically includes the following:

[0034] If the moving directions of the drone and the moving object are perpendicular, further obtain the angle formed by the line connecting the drone to the moving object and the moving direction of the drone. If the angle does not exceed 45°, based on the monitored speeds of the drone and the moving object, if V 机 <V 物 , mark it as a non-obstacle moving object; otherwise, mark it as an obstacle object; if the angle exceeds 45°, based on the monitored speeds of the drone and the moving object, if V 机 ≥V 物 , mark it as a non-obstacle moving object; otherwise, mark it as an obstacle object;

[0035] If the moving directions of the drone and the moving object form an acute or obtuse angle, further obtain the angle formed by the line connecting the drone to the moving object and the moving direction of the drone and the second angle formed by the line connecting the drone to the moving object and the moving direction; if the first angle is greater than the second angle, mark it as a non-obstacle moving object; otherwise, mark it as an obstacle object.

[0036] As a further optimization of the above solution, the trigger for the corresponding adjustment includes the following:

[0037] Based on real-time detection of the UAV flight environment value during the flight process;

[0038] If the humidity value of the detected UAV flight environment value is greater than the UAV flight humidity adaptation threshold, or the wind force of the detected UAV flight environment value is greater than the UAV flight wind force adaptation threshold, end the current flight instruction response;

[0039] If the UAV detects an obstacle object during flight along the planned flight route information route, activate the UAV to fly around the obstacle object.

[0040] As a further optimization of the above solution, the UAV flight environment value includes humidity value and wind force:

[0041] If the humidity value of the detected UAV flight environment value is greater than the UAV flight humidity adaptation threshold, or the wind force of the detected UAV flight environment value is greater than the UAV flight wind force adaptation threshold, cancel the current flight instruction response obtained.

[0042] As a further optimization of the above solution, the shortest path of the UAV during flight is based on two-dimensional Dubins path planning.

[0043] A computer-readable storage medium storing one or more programs, characterized in that the one or more programs include instructions that, when executed by a computing device, cause the computing device to execute any of the methods according to claims 1 to 8.

[0044] A computing device, characterized in that it includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods according to the claims.

[0045] The present invention adopts the above technical solution, and compared with the prior art, has the following beneficial effects:

[0046] 1. The present invention manipulates a fixed-wing UAV device to send inspection instructions and remotely control it to perform task inspections. After the UAV receives the work instructions, it quickly completes a flight self-check to ensure that it is currently in a flight condition. Then, according to the inspection requirements of the UAV, key information is extracted to form parameters that are easy for machines to recognize. Finally, the UAV quickly and real-time plans the route of this voyage to formulate an inspection route with high efficiency, short time consumption, and high safety. The present invention makes good use of the development of existing intelligent technologies, integrates them with the UAV, and designs and develops a highly intelligent self-checking and self-adjusting UAV inspection technology that optimizes the route based on the flight environment in real time. This technology greatly reduces the cost of manual inspection and avoids the negative impact of complex environments on inspection.

[0047] 2. The method for planning the flight route of the UAV designed by the present invention, in actual processing, adds a non-linear change adaptation factor to ensure that when the UAV plans the inspection area, it considers the timing of iteration, gives a higher control means at the beginning, so that when retrieving multiple areas, it has sufficient algorithm planning ability, and the algorithm coverage range covers all areas to be inspected; when the retrieval and matching of all work areas are completed, due to the designed non-linear change adaptation factor, it can accurately process the local computing ability and improve the planning accuracy of the optimal path.

[0048] 3. The present invention analyzes the flight trajectory of the UAV, converts the coordinate system of the captured image of the UAV into the world coordinate system, and accurately maps the image capture information to the real world; then, by capturing images in real time, it obtains the effective flight trajectory of the UAV and the detection of unidentified flying objects, and based on the detection results and the UAV status, it makes a pre-judgment of the UAV's air obstacle avoidance and guiding actions in advance, so that the UAV can safely and effectively plan the flight route. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0050] Figure 1 is a flow schematic diagram of the present invention;

[0051] Figure 2 is another flow schematic diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] As Figure 1-2 shown, the embodiments of the present invention disclose a real-time route planning method for a fixed-wing UAV based on intelligent recognition technology, and the method includes the following:

[0054] Obtain a flight instruction. Based on the received flight instruction, collect the inspection target points and inspection scope of the drone.

[0055] Collect the flight environment values of the drone, and detect whether the takeoff environment value of the drone meets the inspection flight instruction of the drone; if it meets the requirements, activate the inspection flight of the drone.

[0056] Based on the collected information values of the inspection target points and inspection scope of the drone, obtain the flight mission target parameters of the drone, and construct the flight route of the drone for this time with the current position as the starting point.

[0057] Obtain the initial drone flight route information, activate the drone to start the flight mission along the initial drone flight route; detect the drone flight environment values in real time during the flight process; if during the flight process, the drone flight environment values meet the flight requirements of the drone, maintain the corresponding state, otherwise make corresponding adjustments.

[0058] Based on the collected information values of the inspection target points and inspection scope of the drone, complete the inspection of the drone for this round.

[0059] Specifically, grid drone cruising plays a crucial role in modern power systems. With the continuous growth of power demand and the increasing complexity of grid facilities, traditional manual inspection methods are no longer able to meet the requirements of efficient and safe management. Drone cruising can quickly and comprehensively monitor the power grid, promptly detect equipment failures, line damages, and potential safety hazards, greatly improving the efficiency and accuracy of inspections. In addition, the high-definition cameras and infrared imaging technologies equipped on drones can operate in adverse weather or complex terrains to ensure the stable operation of the power grid. Through real-time data transmission, drones can also provide immediate monitoring information to the dispatching center, helping decision-makers respond quickly, thereby ensuring the safety and reliability of power supply. Therefore, grid drone cruising is not only an important means to improve power management levels but also one of the key technologies for realizing the construction of smart grids.

[0060] Based on existing requirements, the present invention designs and develops a real-time route planning method for a fixed-wing unmanned aerial vehicle (UAV) based on intelligent recognition technology. Specifically, by operating the fixed-wing UAV device, an inspection instruction is sent to remotely control it to perform task inspections; after the UAV receives the work instruction, it quickly completes a flight self-check to ensure that it currently has flight conditions; then, according to the inspection requirements of the UAV, key information is extracted to form parameters that are easy for machines to recognize; finally, the UAV quickly and real-time plans the route of this voyage, formulating an inspection route with high efficiency, short time consumption, and high safety. The present invention makes good use of the development of existing intelligent technologies, integrates them with the UAV, designs and develops a highly intelligent self-checking and self-adjusting UAV inspection technology that optimizes the route in real time based on the flight environment. This technology greatly reduces the cost of manual inspections and avoids the negative impacts of complex environments on inspections.

[0061] Specifically, the method for constructing the flight route of the UAV this time includes the following:

[0062] Mark all the inspection target points and inspection ranges of the UAV to form an inspection area; set other areas as non-inspection areas; mark the number of inspection areas as M, and based on the M-scale UAV flight solution space dimension as L, the initialization of the UAV inspection area is as follows:

[0063] A i =(a i,1 ,a i,2 ,…,a i,L )

[0064] In the above formula, i = 1, 2, …, M, and a i represents the initial state position of the UAV during the inspection process;

[0065] More specifically, since there are multiple areas that the UAV may inspect, it is required that each area be inspected and photographed or checked; the present invention specifically constructs a route planning method for the flight route, marks each area separately, and then calculates the most suitable points for the UAV inspection efficiency based on the marked areas, finally forming an efficient and stable inspection route;

[0066] Starting from the current position, based on each marked inspection area, the corresponding best path is formed respectively, and the best path trend is constructed:

[0067]

[0068] In the above formula, j = 1, 2, …, L; C max is the maximum number of iterations for the UAV to find the best point in the inspection area, c is the current number of c rounds of iterations; ε, μ are adjustment parameters for the best path trend; Δl is the displacement; f i,j (a i)Characterized as the gradient value corresponding to the j-th dimension of the UAV line solution space at the i-th position;

[0069] More specifically, in the process of the UAV planning route in the present invention, the early intelligent planning self-regulation and planning require high-precision algorithm execution and calculation, so as to obtain a UAV inspection result with high accuracy and suitable for actual flight;

[0070] Based on the above, the method for planning the flight route of the UAV designed by the present invention, by adding a non-linear change adaptation factor in actual processing, ensures that when the UAV conducts inspection area planning, it takes into account the timing of iteration, gives a higher control means at the beginning, so that when conducting multiple area retrievals, it has sufficient algorithm planning ability, and the algorithm action range covers all the areas to be inspected; when the retrieval and matching of all working areas are completed, due to the designed non-linear change adaptation factor, it can process the local calculation ability with high precision and improve the planning accuracy of the optimal path.

[0071] The present invention provides an important embodiment, that is, after testing, ε and μ of the present invention are respectively taken as 2.45 and 0.04, and Δl is set to 0.55; the algorithm states corresponding to these several parameters are the best convergence states; based on the characteristics of the sinh function, it can perfectly feedback the matching of the non-linear change requirements of this algorithm.

[0072] Based on the constructed best path trend, the best position of the UAV is located for each marked inspection area respectively, and all positioning records are monitored:

[0073] If the best position of the UAV in any inspection area is obtained, a preferred domain is formed based on this position All boundary points in the preferred domain are substituted into the best path trend calculation formula to determine whether the current is the stable best position of the UAV. If so, update this point as the latest best position of the UAV, otherwise, keep it unchanged;

[0074] Repeat the above steps until the stable best positions of the UAV in all inspection areas are found, and record the shortest flight route as the flight route of the UAV.

[0075] Specifically, the method for constructing the flight route of the UAV this time further includes the following:

[0076] For the preferred domain Conduct an anti-premature convergence design and construct an anti-convergence relational expression:

[0077] a i,j =(1 - γ)a i,j +γa n,j

[0078] Among them, a i,j is the best position of the UAV, an,j is any position in any inspection area, where n is not equal to i; γ is the learning weight of the optimal path trend of the UAV.

[0079] More specifically, in order to further prevent the algorithm from being unable to obtain a highly accurate process optimum prematurely due to the convergence of the stable optimal position of the UAV, and to prevent the emergence of a local optimal position of the UAV with greater restrictions, the present invention innovatively designs the learning weight of the optimal path trend. This parameter is mainly based on the selection quantity of any position in any inspection area, that is, the number of optimal positions of the UAV corresponding to all inspection areas to be inspected, based on a n,j range of sample information, so that when this algorithm processes local computing capabilities, the learning weight based on γ brings data fluctuations, avoiding being unable to get rid of local influences when performing the final calculation of the local optimal position of the UAV, which improves the convergence of the optimization solution process and further improves the planning accuracy of the optimal path.

[0080] As another embodiment of the present invention, the method for real-time detecting the UAV flight environment value during the flight process specifically includes the following:

[0081] Based on the UAV positioning to obtain the longitude and latitude coordinates, determine the world coordinate system of the UAV. Based on the collected images of the UAV, apply the camera coordinate system of the collected images to be perspective-transformed into the image coordinate system, and calculate the relationship between the camera imaging position points of the UAV in the three-dimensional world coordinate system;

[0082] Continuously perform all-round image acquisition of the UAV, mark the image recognition objects based on the image algorithm, and detect the relative displacement between the identified object and the UAV based on the continuously captured images; if the relative displacement between the identified object and the UAV remains unchanged or decreases, mark it as a moving object; if the relative displacement increases, respectively obtain the image coordinates of the identified object detected for the first and second times, and obtain the displacement of the UAV at the corresponding time based on the coordinate transformation. Let the angle between the UAV's traveling route and the UAV's distance to the identified object be θ, and the UAV speed be V 机 , and the occurrence time be t; then calculate the relationship between the distance l from the image coordinates of the identified object detected for the second time to the UAV and V 机 ·t / cos; if they are equal, mark the identified object as stationary, otherwise mark it as a moving object;

[0083] Continuously perform all-round image acquisition of the UAV, and record the image coordinates of any moving object detected based on multiple time periods; if the relative displacement between the moving object and the UAV remains unchanged or increases within consecutive time periods, mark it as a moving non-obstacle object; if the relative displacement becomes smaller, continue to monitor;

[0084] If the UAV and the moving object are in the same plane, based on the current position, select the perpendicular to the traveling direction as the directrix, and mark the traveling direction of the UAV as the front; based on the all-round image acquisition of the UAV, determine the front-back relationship between the moving object and the UAV.

[0085] If the moving object is behind the UAV at this time, record the image coordinates of different positions of the moving object in multiple time periods; then, based on coordinate system conversion, obtain the actual displacement of the moving object in multiple time periods, and calculate the average speed V of the moving object corresponding to multiple time periods based on the occurrence time recorded by the UAV 物 ; If V 机 ≥V 物 , mark it as a non-obstacle moving object; otherwise, increase the speed or altitude of the UAV;

[0086] If the moving object is on the directrix of the UAV at this time, detect the speed relationship between the UAV and the moving object respectively. If V 机 ≥V 物 , mark it as a non-obstacle moving object; otherwise, increase the speed or altitude of the UAV;

[0087] If the moving object is in front of the UAV at this time, detect the speed and moving direction of the UAV and the moving object respectively; if the moving directions of the UAV and the moving object are perpendicular, further obtain the angle formed by the line connecting the UAV to the moving object and the moving direction of the UAV. If the angle does not exceed 45°, based on the monitored speeds of the UAV and the moving object, if V 机 <V 物 , mark it as a non-obstacle moving object; otherwise, mark it as an obstacle object; if the angle exceeds 45°, based on the monitored speeds of the UAV and the moving object, if V 机 ≥V 物 , mark it as a non-obstacle moving object; otherwise, mark it as an obstacle object;

[0088] If the angle formed by the moving directions of the UAV and the moving object is acute or obtuse, further obtain the angle formed by the line connecting the UAV to the moving object and the moving direction of the UAV and the second angle formed by the line connecting the UAV to the moving object and the moving direction; if the first angle is greater than the second angle, mark it as a non-obstacle moving object; otherwise, mark it as an obstacle object;

[0089] Through the analysis of the operation trajectory of the UAV, the present invention converts the image coordinate system of the UAV's captured images and the world coordinate system, accurately maps the image capture information to the real world; then, through real-time captured images, obtains effective UAV flight trajectories and unidentified flying object detections, and based on the detection results and the UAV status, makes advance judgments on UAV aerial obstacle avoidance and guiding actions in advance, so as to facilitate the safe and effective route planning of the UAV.

[0090] The present invention also discloses an obstacle detection method, which specifically includes the following:

[0091] If it is detected that the traveling speed of the UAV is 0, mark it as a hovering state; continuously collect omnidirectional images based on the UAV's shooting device; perform detection on the collected omnidirectional images based on image algorithms:

[0092] If no moving object is detected in the omnidirectional image captured within a unit time, maintain the current state;

[0093] If a moving object is detected in the omnidirectional image captured within a unit time, determine the traveling speed and direction of the moving object;

[0094] If it is detected that the traveling direction of the moving object is parallel to the plane where the drone is located, continuously monitor whether the moving direction of the moving object and the drone are on the same straight line; if not on the same straight line, mark it as a non-obstacle moving object; It should be noted that when it is mentioned in the present invention that it is parallel to the plane where the drone is located, the relative distance between the plane where the moving object is located and the plane where the drone is located needs to be monitored. If the relative distance is outside the safe vertical distance range of the drone, it is marked as not in the same plane as the drone; if the relative distance is within the safe vertical distance range of the drone, it is marked as in the same plane as the drone; For subsequent mentions of being parallel to the plane where the drone is located, this situation needs to be considered, and it will not be elaborated here;

[0095] If on the same straight line, determine the moving direction of the moving object based on the omnidirectional image captured within a unit time. If the distance between the moving object and the drone becomes shorter within consecutive unit times, mark it as an obstacle object; otherwise, mark it as a non-obstacle moving object;

[0096] If it is detected that the traveling direction of the moving object intersects with the plane where the drone is located, obtain the moving direction of the moving object based on the continuous images of the moving object captured within a unit time; if not on the same straight line, mark it as a non-obstacle moving object;

[0097] If on the same straight line, determine the moving direction of the moving object based on the omnidirectional image captured within a unit time. If the distance between the moving object and the drone becomes shorter within consecutive unit times, mark it as an obstacle object; otherwise, mark it as a non-obstacle moving object; More specifically, the traveling speed of the drone in the present invention refers to the traveling speed of the plane straight line along the main direction of the drone at the current moment;

[0098] If it is detected that the traveling speed of the drone is within the first threshold range, continuously collect omnidirectional images based on the shooting device of the drone; perform detection on the collected omnidirectional images based on the image algorithm:

[0099] If no moving object is detected in the omnidirectional image captured within a unit time, maintain the current state;

[0100] If a moving object is detected in the omnidirectional image captured within a unit time, determine the traveling speed and direction of the moving object;

[0101] If it is detected that the moving direction of the moving object is parallel to the plane where the UAV is located, continuously monitor whether there is an intersection point between the moving direction and speed of the moving object and the moving direction and speed of the UAV; if there is no intersection point, mark it as a non-obstacle moving object;

[0102] If there is an intersection point, judge the moving speed and direction of the moving object based on the omnidirectional images captured within a unit time. If the time difference between the time calculated based on the monitored speed of the moving object and the distance to the intersection point and the time for the UAV to reach the intersection point at the current speed is outside the first-time monitoring threshold range, mark it as a non-obstacle moving object; if it is within the first-time monitoring threshold range, mark it as an obstacle object;

[0103] If it is detected that the moving direction of the moving object intersects with the plane where the UAV is located, based on the continuous images of the moving object captured within a unit time, obtain the moving direction of the moving object from the continuous images; if there is no intersection point between the moving direction of the moving object and the straight-line direction of the moving direction of the UAV, mark it as a non-obstacle moving object;

[0104] If there is an intersection point, judge the moving speed and direction of the moving object based on the omnidirectional images captured within a unit time. If the time difference between the time calculated based on the monitored speed of the moving object and the distance to the intersection point and the time for the UAV to reach the intersection point at the current speed is outside the first-time monitoring threshold range, mark it as a non-obstacle moving object;; if it is within the first-time monitoring threshold range, mark it as an obstacle object;

[0105] If it is detected that the moving speed of the UAV is within the second threshold range, based on the UAV's shooting device, continuously capture omnidirectional images; perform detection on the captured omnidirectional images based on the image algorithm:

[0106] If no moving object is detected in the omnidirectional images captured within a unit time, maintain the current state;

[0107] If a moving object is detected in the omnidirectional images captured within a unit time, judge the moving speed and direction of the moving object;

[0108] If it is detected that the moving direction of the moving object is parallel to the plane where the UAV is located, set the second safety distance threshold, and judge whether the plane where the moving object is located falls within the second safety distance threshold. If it does not fall within the height range of the second safety distance threshold, mark it as a non-obstacle moving object; if it falls within the second safety distance threshold, mark it as an obstacle object; It should be noted that setting the second safety distance threshold means adding effective heights above and below the UAV's flight plane, which further expands its safety perception space within the UAV's safe flight distance space. And for subsequent judgments of the relative distance between the UAV and the moving object, the second safety distance threshold needs to be considered, which will not be elaborated here;

[0109] Continuously monitor whether there is an intersection point between the moving direction of the moving object and the moving direction of the UAV; if there is no intersection point, mark it as a non-obstacle moving object;

[0110] If there is an intersection point, judge the moving speed and direction of the moving object based on the omnidirectional images captured per unit time. If the time difference between the time calculated based on the speed of the monitored moving object and the distance to the intersection point and the time for the UAV to reach the intersection point at the current speed is outside the second time monitoring threshold range, mark it as a non-obstacle moving object; if it is within the second time monitoring threshold range, mark it as an obstacle object;

[0111] If it is detected that the moving direction of the moving object intersects with the plane where the UAV is located, set a second safety distance threshold, and judge whether the plane where the moving object is located falls within the second safety distance threshold. If it does not fall within the height range of the second safety distance threshold, mark it as a non-obstacle moving object; if it falls within the second safety distance threshold, mark it as an obstacle object;

[0112] Continuously monitor whether there is an intersection point between the moving direction of the moving object and the moving direction of the UAV; if there is no intersection point between the straight-line direction of the moving direction of the moving object and the moving direction of the UAV, mark it as a non-obstacle moving object;

[0113] If there is an intersection point, judge the moving speed and direction of the moving object based on the omnidirectional images captured per unit time. If the time difference between the time calculated based on the speed of the monitored moving object and the distance to the intersection point and the time for the UAV to reach the intersection point at the current speed is outside the second time monitoring threshold range, mark it as a non-obstacle moving object;; if it is within the second time monitoring threshold range, mark it as an obstacle object;

[0114] Furthermore, if it is detected that the UAV's moving speed is within the second threshold range and an obstacle object is marked, calculate the closest point between the UAV and the moving object based on image acquisition and the UAV's current speed;

[0115] If the closest point is within the second safety distance threshold of the UAV, set a buffer point within the distance for the UAV to reach the closest point, reduce the UAV's current speed to the maximum value of the first threshold, and make it re-detect the closest point between the UAV and the moving object after reaching the buffer point; if the closest point is still within the second safety distance threshold of the UAV, adjust the UAV's current spatial height so that it is at least higher than the second safety distance threshold.

[0116] Specifically, the triggers for corresponding adjustments include the following:

[0117] Based on the real-time detection of the UAV flight environment value during the flight process;

[0118] If the humidity value of the detected UAV flight environment value is greater than the UAV flight humidity adaptation threshold, or the wind force of the detected UAV flight environment value is greater than the UAV flight wind force adaptation threshold, end the current flight instruction response;

[0119] If the UAV detects an obstacle during flight along the planned flight route information route, activate the UAV to fly around the obstacle.

[0120] More specifically, the corresponding adjustment disclosed in the present invention refers to that during the route planning operation of the UAV, it is necessary to continuously detect the flight environment information, including state parameter values directly affecting the UAV flight such as air humidity value and wind force; and, during normal path planning, due to real-time monitoring, there may be obstacles in the planned path, and correspondingly, the UAV needs to make a real-time avoidance response, that is, respond to the flight instruction of the UAV to fly around the obstacle. The end of the current flight instruction response of the present invention can be any interruptive response scheme different from the current UAV working state, such as lifting and lowering in a different location, returning to the departure place, taking shelter nearby, etc.

[0121] It should be particularly noted that since the UAV flight environment value can be monitored through technical means such as weather prediction, the present invention also supports real-time reception of information such as warnings of unstable weather systems. Based on this method, the time and technical costs of the UAV's real-time flight environment self-check are saved, and the effect is good.

[0122] Specifically, the UAV flight environment value includes humidity value and wind force:

[0123] If the humidity value of the detected UAV flight environment value is greater than the UAV flight humidity adaptation threshold, or the wind force of the detected UAV flight environment value is greater than the UAV flight wind force adaptation threshold, cancel the current obtained flight instruction response.

[0124] More specifically, based on the triggering of the UAV's inspection task, considering that the UAV is for remote operation and requires a stable flight environment and no large differences for its operation; specifically, the present invention designs a flight humidity adaptation threshold and a flight wind force adaptation threshold, which respectively limit the UAV's flight task from two aspects of precipitation and wind force, avoiding its attendance in extremely harsh environments, ensuring a good and stable operation environment for the UAV, and avoiding economic losses.

[0125] Specifically, the shortest path of the UAV during flight is based on two-dimensional Dubins path planning.

[0126] It should be noted that in this text, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising such element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0127] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present application.

[0128] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Those of ordinary skill in the art, under the inspiration of the present application and without departing from the spirit of the present application and the scope protected by the claims, can also make many forms, all of which fall within the protection scope of the present application.

[0129] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0130] Although embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present application. The scope of the present application is defined by the claims and their equivalents.

Claims

1. A real-time route planning method for fixed-wing UAV based on intelligent recognition technology, characterized in that: The method comprises the following: Obtain flight instructions, and based on the received flight instructions, collect the patrol target points and patrol ranges of the UAV; Collect the flight environment values ​​of the drone and detect whether the drone take-off environment values ​​meet the patrol flight instructions of the drone; if they meet, activate the drone patrol flight; Based on the collected patrol target point and patrol range information values ​​of the drone, the flight mission target parameters of the drone are obtained, and the flight route of the drone is constructed with the current position as the starting point; Acquire initial UAV flight route information, activate the UAV and start the UAV flight mission along the initial UAV flight route; Real-time detection of the drone flight environment value based on the flight process; if during the flight process, the drone flight environment value meets the drone flight requirements, maintain the corresponding state, otherwise adjust accordingly; Based on the collected information values ​​of the drone's patrol target points and patrol range, this round of drone patrol is completed.

2. According to claim 1, a fixed-wing UAV real-time route planning method based on intelligent recognition technology is characterized in that: The method for constructing the flight route of the drone includes the following steps: All patrol target points and patrol ranges of the drone are marked to form patrol areas; other areas are set as non-patrol areas; the number of marked patrol areas is M, and the dimension of the drone flight solution space based on the M scale is L. The drone patrol area is initialized as follows: A i =(a i,1 ,a i,2 ,…,a i,L ) In the above formula, i = 1, 2, ..., M, a i It is represented by the initial position of the UAV during the patrol process; Taking the current position as the starting point, based on each marked patrol area, the corresponding optimal path is formed respectively, and the optimal path trend is constructed: In the above formula, j = 1, 2, ..., L; C max is the maximum iteration value for the UAV to find the best point in the patrol area, and c is the current c-round iteration; ε, μ are the adjustment parameters of the optimal path trend; Δl is the displacement; f i,j (a i ) is represented as the gradient value corresponding to the jth dimension of the solution space of the UAV at the i-th location; Based on the constructed optimal path trend, the best position of the drone is located for each marked patrol area, and all positioning records are monitored: If the best position of the drone in any patrol area is obtained, a preferred area is formed based on the position Substitute all boundary points in the preferred domain into the optimal path trend calculation formula to determine whether the current point is the optimal position of the stable drone. If so, update the point to the latest optimal position of the drone, otherwise, keep it unchanged; Repeat the above steps until the best stable drone position in all patrol areas is found, and record the shortest flight route as the drone flight route.

3. According to claim 2, a fixed-wing UAV real-time route planning method based on intelligent recognition technology is characterized in that: The method for constructing the flight route of the drone also includes the following: Preferred Domain Conduct anti-early convergence design and construct an anti-convergence relationship: a i,j =(1-γ)a i,j +ga n,j Among them, a i,j is the best position for the drone, a n,j is any position in any inspection area, n and i are not equal; γ is the optimal path trend learning weight of the UAV.

4. According to claim 1, a fixed-wing UAV real-time route planning method based on intelligent recognition technology is characterized in that: The real-time detection of the UAV flight environment value based on the flight process specifically includes the following: Based on the positioning of the drone, the longitude and latitude coordinates are obtained to determine the world coordinate system of the drone. Based on the collected images of the drone, the camera coordinate system of the collected images is converted into the image coordinate system through perspective, and the relationship between the camera mapping positions of the drone in the three-dimensional world coordinate system is calculated; Continuously collect all-round images of the drone, mark the image recognition object based on the image algorithm, and detect the relative displacement between the marker and the drone based on the continuously captured images; if the relative displacement between the marker and the drone remains unchanged or decreases, mark it as a moving object; The relative displacement increases, and the image coordinates of the first and second detections of the marker are obtained respectively. Based on the coordinate transformation, the displacement of the drone at the corresponding time is obtained. Suppose the angle between the drone route and the drone to the marker is θ, and the drone speed V 机 , the occurrence time t; then calculate the distance l and V from the image coordinates of the second detection of the marker to the drone 机 t / cosθ relationship; If they are equal, the object is marked as stationary, otherwise it is marked as a moving object; Continuously collect all-round images from drones, and record the image coordinates of any moving objects detected in multiple time periods; If the relative displacement between the moving object and the UAV remains unchanged or increases during a continuous period, it is marked as a moving non-obstructive object; If the relative displacement becomes smaller, continue monitoring; If the drone and the moving object are in the same plane, select the line perpendicular to the direction of travel based on the current position, and mark the direction of the drone's travel as the front; Based on the omnidirectional image acquisition of the UAV, the front and back relationship between the moving object and the UAV is determined; If the moving object is behind the drone at this time, the image coordinates of different positions of the moving object are recorded based on multiple time periods; Then, based on the coordinate system conversion, the actual displacement of the moving object in multiple time periods is obtained, and the average speed V corresponding to the moving object in multiple time periods is calculated based on the corresponding multi-time period occurrence time recorded by the drone. 物 ; If V 机 ≥V 物 , marked as a moving non-obstacle object; otherwise, increase the speed or altitude of the drone; If the moving object is in the line of sight of the drone at this time, the speed relationship between the drone and the moving object is detected respectively. If V 机 ≥V 物 , marked as a moving non-obstacle object; otherwise, increase the speed or altitude of the drone; If the moving object is in front of the drone at this time, the speed and moving direction of the drone and the moving object are detected respectively.

5. The method for real-time route planning of a fixed-wing UAV based on intelligent recognition technology according to claim 4 is characterized in that: The real-time detection of the UAV flight environment value based on the flight process specifically includes the following: If the UAV is perpendicular to the moving object, further obtain the angle between the UAV and the moving object and the UAV's moving direction. If the angle does not exceed 45°, based on the speed of the UAV and the moving object, if V 机 <V 物 , marked as a moving non-obstructive object; Otherwise, it is marked as an obstacle; if the angle exceeds 45°, based on the speed monitoring of the drone and the moving object, if V 机 ≥V 物 , marked as a moving non-obstruction object; otherwise, marked as an obstacle object; If the UAV forms an acute angle or an obtuse angle with the moving direction of the moving object, further obtain the angle formed by the line connecting the UAV to the moving object and the moving direction of the UAV and the second angle formed by the line connecting the UAV to the moving object and the moving direction; if the angle is greater than the second angle, it is marked as a moving non-obstacle object; otherwise, it is marked as an obstacle object.

6. The method for real-time route planning of a fixed-wing UAV based on intelligent recognition technology according to claim 5 is characterized in that: The triggering of the corresponding adjustment includes the following: Real-time detection of drone flight environment values ​​based on the flight process; If the humidity value of the detected drone flight environment value is greater than the drone flight humidity adaptation threshold, or the wind force of the detected drone flight environment value is greater than the drone flight wind force adaptation threshold, the current flight command response is terminated; If the drone detects an obstruction while flying based on the planned flight route information, the drone will be activated to fly around the obstruction.

7. The method for real-time route planning of a fixed-wing UAV based on intelligent recognition technology according to claim 6 is characterized in that: The UAV flight environment values ​​also include humidity and wind speed: If the humidity value of the detected drone flight environment value is greater than the drone flight humidity adaptation threshold, or the wind force of the detected drone flight environment value is greater than the drone flight wind force adaptation threshold, the currently acquired flight command response is canceled.

8. The method for real-time route planning of a fixed-wing UAV based on intelligent recognition technology according to claim 7 is characterized in that: The shortest path of the UAV during flight is based on two-dimensional Dubins path planning.

9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions which, when executed by a computing device, cause the computing device to perform any one of the methods according to claims 1 to 8.

10. A computing device, characterized in that: The method comprises one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods according to claims 1 to 8.

Images