An unmanned aerial vehicle autonomous flight path planning system and method based on machine vision

Abstract

The application discloses a kind of unmanned aerial vehicle autonomous flight path planning system and method based on machine vision, belong to unmanned aerial vehicle technical field, including the following steps: obtaining and processing environmental perception data, constructs the three-dimensional confidence map for quantifying environmental cognitive degree;Establish the dynamics constraint model of fixed-wing unmanned aerial vehicle and the associated model of attitude and perception range;Based on the three-dimensional confidence map and associated model, sampling within the range defined by the dynamics constraint model, generate multiple candidate flight paths that satisfy the dynamics constraint;By constructing the mapping relationship between attitude and field of view, the body tilt action that must be generated when fixed-wing unmanned aerial vehicle turns into active perception extension means, in the path planning stage, the system according to the turning radius on the planned path predicts the body tilt angle, and determines the lateral observation area additionally covered due to tilt accordingly, so that unmanned aerial vehicle when executing steering maneuver.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a machine vision-based autonomous flight path planning system and method for UAVs. Background Technology

[0002] In the field of unmanned aerial vehicle (UAV) technology, fixed-wing UAVs, with their advantages of long endurance, high speed, and wide coverage, play an increasingly important role in large-area search, reconnaissance, mapping, and emergency rescue missions in complex and unknown environments. Compared with rotary-wing UAVs, fixed-wing UAVs can perform detection tasks over a larger area per unit time. Their aerodynamic layout determines their higher energy efficiency and longer operating radius, giving them irreplaceable advantages in applications requiring rapid coverage of vast areas, such as military reconnaissance, border patrol, disaster area mapping, and forest fire monitoring. However, when performing these tasks, fixed-wing UAVs typically need to perform autonomous flight and path planning in completely unknown and structurally complex environments, which places extremely high demands on the system's real-time perception and decision-making capabilities. Unknown environments often contain a large number of unexplored areas, potential obstacles, and uncertain terrain structures. How to efficiently plan flight paths to achieve a comprehensive understanding of the environment while ensuring flight safety and energy efficiency, without prior information, has become a core technical problem that urgently needs to be solved in this field.

[0003] To address path planning in complex and unknown environments, existing technologies have proposed a series of exploration methods driven by reducing global cognitive uncertainty. The core idea of ​​these methods is to construct an uncertainty model of the environment, quantify the risks of unknown areas into calculable indicators, and take the information gains obtained by reducing this uncertainty as an important optimization objective for path planning. Typically, a probabilistic occupancy grid map is used to model the environment, and metrics such as information entropy are used to characterize the cognitive level of each area, thereby guiding UAVs to prioritize flying to areas with higher uncertainty, thus achieving systematic exploration. These methods can effectively avoid the problem of falling into local optima due to limited perception range or simple decision logic in traditional path planning, and show relatively ideal performance improvements in terms of total task completion time, total energy consumption, and final map integrity.

[0004] However, the aforementioned uncertainty-driven path planning methods are mainly applicable to flight platforms such as rotary-wing UAVs that have hovering and arbitrary turning capabilities. For fixed-wing UAVs, due to their inherent nonholonomic constraints, including the inability to hover, the existence of a minimum turning radius, and the need to change course by tilting the fuselage when turning, directly applying existing solutions will face the problem of insufficient adaptability. Existing methods do not fully consider the kinematic boundaries of fixed-wing UAVs when generating paths, resulting in planned trajectories that are physically difficult to execute. At the same time, traditional methods treat fuselage tilting as a purely dynamic burden and fail to effectively utilize the lateral field of view expansion opportunities brought about by tilting maneuvers, thus restricting further improvements in perception efficiency. Summary of the Invention

[0005] This invention overcomes the shortcomings of the prior art and provides a machine vision-based autonomous flight path planning system and method for unmanned aerial vehicles.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a method for autonomous flight path planning of unmanned aerial vehicles based on machine vision, comprising the following steps:

[0007] S1. Acquire and process environmental perception data to construct a three-dimensional confidence map for quantifying the degree of environmental awareness;

[0008] S2. Establish a dynamic constraint model for fixed-wing UAVs and a correlation model between attitude and perception range;

[0009] S3. Based on the three-dimensional confidence map and correlation model, sample within the range defined by the dynamic constraint model to generate multiple candidate flight paths that satisfy the dynamic constraints;

[0010] S4. Select the optimal path from the candidate paths that balances information acquisition efficiency and flight cost.

[0011] S5. The flight trajectory of the optimal path is parsed into control commands and executed, while the path is rolled and replanned based on real-time perceived environmental changes.

[0012] In a preferred embodiment of the present invention, step S1, the three-dimensional confidence map includes: acquiring sequential images, fusing real-time pose information to update the probability of occupying the grid map, calculating the information entropy of each grid cell, and defining the global information entropy distribution as an environmental uncertainty field.

[0013] In a preferred embodiment of the present invention, in step S2, the dynamic constraint model includes at least the differential equation relationship between the heading angle, roll angle and flight speed, and the minimum turning radius constraint.

[0014] In a preferred embodiment of the present invention, in step S2, the correlation model between attitude and perception range is a mapping relationship model between the fuselage roll angle and the sensor's lateral field of view. The mapping relationship model is used to calculate the additional lateral observation area covered by the fuselage tilt based on the turning radius on the planned path.

[0015] In a preferred embodiment of the present invention, step S3 includes: obtaining the current pose, sampling within the constraints of the maximum yaw rate and the minimum turning radius, generating multiple candidate flight paths composed of arcs, and generating a detour path for the high uncertainty area on the side and rear of the fuselage by splicing multiple arcs.

[0016] In a preferred embodiment of the present invention, step S4, the optimal path includes: calculating a comprehensive evaluation index for each candidate path, the index including an information gain term, a path length cost term, and an attitude smoothing term composed of forward observation and lateral observation caused by fuselage tilt, and selecting the path with the optimal index as the desired flight trajectory.

[0017] In a preferred embodiment of the present invention, the information gain term is obtained by calculating the sum of entropy reduction of the grid cells traversed by the probe rays emitted along the candidate path direction in the environmental uncertainty field, and an attenuation factor that decreases with increasing propagation distance is introduced during the calculation.

[0018] In a preferred embodiment of the present invention, step S5, the rolling replanning includes: continuously collecting new data and updating the three-dimensional confidence map in real time during the execution of the current flight trajectory; when the updated map shows that the current path has a collision risk or an area with higher information urgency, interrupting the current trajectory and returning to step S3 to regenerate the candidate flight path.

[0019] Secondly, the present invention provides a system for autonomous flight path planning of unmanned aerial vehicles based on machine vision, comprising:

[0020] The environment modeling module is used to build and maintain 3D confidence maps;

[0021] The constraint establishment module is used to store kinematic constraint models and establish the relationship between posture and perception range;

[0022] The path generation module is used to generate candidate flight paths that satisfy dynamic constraints;

[0023] The evaluation and optimization module is used to select the optimal flight trajectory from the candidate paths;

[0024] The control execution module is used to parse the flight trajectory into control commands and execute them, and to trigger replanning based on environmental updates.

[0025] Thirdly, the present invention provides an electronic device, comprising:

[0026] At least one processor; and

[0027] A memory that is communicatively connected to the at least one processor;

[0028] The memory stores a computer program that can be executed by at least one processor, which is then executed by the at least one processor to enable the at least one processor to execute the machine vision-based unmanned aerial vehicle autonomous flight path planning system and method described in any of the preceding claims.

[0029] This invention addresses the shortcomings of the prior art and has the following beneficial effects:

[0030] (1) By constructing the mapping relationship between attitude and field of view, the body tilting action that must occur when a fixed-wing UAV turns is transformed into an extended means of active perception. In the path planning stage, the system predicts the body tilt angle based on the turning radius on the planned path and determines the additional lateral observation area covered by the tilt. This allows the UAV's sensor field of view to no longer be limited to the forward direction when performing turning maneuvers, but to actively cover the information blind spot that was originally located to the side and rear. This enables the UAV to complete the heading adjustment and lateral area detection simultaneously through one maneuver during flight, solving the physical limitation that fixed-wing UAVs cannot hover and have a minimum turning radius, making it difficult to fly directly to the high-value area to the side and rear. Compared with the traditional approach of treating body tilting as a dynamic burden, this inevitable physical action is transformed into a perception gain, enabling the UAV to overcome the structural contradiction of being able to see but not fly in a complex and unknown environment.

[0031] (2) By incorporating the lateral observation gains brought about by fuselage tilt as an independent attitude coordination term into the multi-objective optimization function, the synchronous optimization of perception behavior and flight path is realized. During the path evaluation process, the system not only calculates the information gain of the forward coverage area along the path, but also accumulates the uncertainty value in the lateral coverage area due to fuselage tilt. This enables the path planning algorithm to quantitatively evaluate the additional information value brought about by tilt maneuvering, allowing the UAV to actively identify and select flight trajectories that can obtain more lateral information through tilt attitude during the path search phase, rather than passively accepting the trajectory deviation caused by tilt. This enables the UAV to obtain richer environmental information per unit flight distance, significantly improving the detection efficiency of a single flight. Compared with the planning mode where perception and maneuvering are separated, this realizes the deep coupling of attitude and perception, making each maneuver have the dual functions of flight control and information acquisition.

[0032] (3) By adopting a rolling optimization closed-loop control mechanism, new data is continuously collected and the three-dimensional confidence map is updated in real time during the execution of the current optimal trajectory. When a collision risk is detected in the current path or an area with higher information urgency appears, replanning is triggered immediately. This enables the UAV to dynamically adapt to environmental changes and correct path deviations caused by insufficient initial information or sudden environmental changes in a timely manner. This enables the system to maintain global exploration efficiency while having the ability to respond quickly to sudden obstacles or newly discovered high-value targets, ensuring the continuity and safety of mission execution. Compared with the traditional open-loop planning method, which can only rely on preset paths or frequently interrupt the mission when the environment changes, the rolling replanning mechanism realizes the adaptive adjustment of the exploration process, enabling the UAV to continuously and stably complete autonomous exploration tasks in a completely unknown and dynamically changing complex environment. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of a preferred embodiment of the present invention;

[0035] Figure 2 This is a flowchart of the construction of a three-dimensional confidence map according to a preferred embodiment of the present invention;

[0036] Figure 3 This is a flowchart of the optimal path evaluation in a preferred embodiment of the present invention;

[0037] Figure 4 This is a flowchart of a preferred embodiment of the rolling replanning of the present invention;

[0038] Figure 5 This is a flowchart of the control system of a preferred embodiment of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Application Overview:

[0041] There is a structural contradiction between the physical mobility capabilities of fixed-wing UAVs due to nonholonomic constraints and their information acquisition needs. When high-value information areas in the environment are located to the side or rear of the fuselage or require sharp turns to reach, the UAV, limited by its minimum turning radius and non-hovering dynamic characteristics, cannot directly turn or hover to adjust its attitude to cover the area like a rotorcraft, creating a dilemma of being able to see but not fly. At the same time, traditional methods regard fuselage tilting as a burden caused by dynamic constraints, focusing only on how to limit or compensate for the impact of tilting maneuvers on the trajectory, without recognizing that the tilting action itself can be used as an extension of active perception. This results in the waste of lateral visual information that could have been acquired simultaneously, and perception efficiency cannot be further improved.

[0042] The applicant discovered that when a fixed-wing UAV performs a turning maneuver, the fuselage inevitably rolls and tilts. During this process, the optical axis of the onboard sensors deflects with the fuselage, allowing areas that were originally located to the side and rear to enter the sensor's field of view at a specific tilt angle. Based on this observation of the physical phenomenon, the applicant realized that the tilting action is not simply a dynamic burden, but a natural coupling opportunity between attitude and perception. If the fuselage tilt angle at different turning radii can be predicted during the path planning stage, and the additional lateral observation area gained due to the tilt can be calculated accordingly, the originally passive dynamic constraint can be transformed into an active perception gain tool.

[0043] The applicant proposes, firstly, to construct an information gain map, quantifying the risk of unknown areas in the environment into a calculable uncertainty index, and taking the information gain from reducing this uncertainty as the core optimization objective. Based on this, a mapping relationship between attitude and field of view is established for fixed-wing UAVs. The expected fuselage tilt angle is calculated based on the turning radius along the planned path, and the additional lateral observation area covered due to tilt is determined. At the path planning level, the information gain term is decomposed into forward observation gain and attitude coordination gain. The attitude coordination gain is obtained by accumulating the uncertainty values ​​in the laterally covered areas at each point along the path due to tilt, and this gain is incorporated into a multi-objective optimization function. Through this method, the UAV can predict the additional perception value brought by tilting maneuvers during the planning stage, thereby achieving simultaneous coverage of multiple unknown areas in a single maneuver, balancing physical constraints and perception efficiency in a coordinated optimization.

[0044] like Figure 1 As shown, a method for autonomous flight path planning of unmanned aerial vehicles based on machine vision includes the following steps:

[0045] S1. Acquire and process environmental perception data to construct a three-dimensional confidence map for quantifying the degree of environmental awareness;

[0046] S2. Establish a dynamic constraint model for fixed-wing UAVs and a correlation model between attitude and perception range;

[0047] S3. Based on the three-dimensional confidence map and correlation model, sample within the range defined by the dynamic constraint model to generate multiple candidate flight paths that satisfy the dynamic constraints;

[0048] S4. Select the optimal path from the candidate paths that balances information acquisition efficiency and flight cost.

[0049] S5. The flight trajectory of the optimal path is parsed into control commands and executed, while the path is rolled and replanned based on real-time perceived environmental changes.

[0050] The core innovation of this invention lies in transforming the fuselage tilt that a fixed-wing UAV must generate when turning from a dynamic burden into an active sensing tool. By establishing a mapping relationship between attitude and field of view and embedding it into a multi-objective optimization function for path planning, the sensing behavior and flight path are synchronously coupled, enabling the UAV to simultaneously complete heading adjustment and lateral area detection in a single maneuver.

[0051] Each step will be explained in detail below.

[0052] like Figure 2 As shown, in step S1, the UAV uses its onboard visual sensors to continuously collect sequential images of the surrounding environment during flight. The system simultaneously acquires the UAV's real-time pose information, including its three-dimensional spatial position coordinates, heading angle, roll angle, etc.

[0053] Specifically, visual synchronous positioning and mapping technology is used to register newly acquired image data with existing maps and update a three-dimensional probabilistic occupancy grid map.

[0054] Specifically, the map discretizes the space into individual cubic units, or grid units. Each grid unit stores a probability value, representing the likelihood that the unit is occupied by an obstacle. Based on the occupancy probability of each grid unit, its information entropy is calculated.

[0055] The formula for calculating information entropy is:

[0056] ;

[0057] in, This indicates the probability that the grid cell is occupied. This indicates the probability that the grid cell is idle. This is the information entropy of the unit. When When the time is completely uncertain, the entropy is at its maximum; when... When the entropy value approaches 0 or 1, i.e. when it is determined to be idle or occupied, the entropy value approaches 0. The system combines the information entropy of all grid cells in the entire domain to form a three-dimensional information entropy distribution map, i.e., the environmental uncertainty field, which is the three-dimensional confidence map of this step.

[0058] Understandably, in an unknown forest area, initially all raster cells... All information entropies are at their maximum values, indicating complete unknown. When a drone flies and observes an open area, the corresponding grid cell's... When updated to 0.1, its information entropy drops to near 0, indicating that the region has been recognized.

[0059] This step constructs a three-dimensional confidence map, which is not a simple obstacle map, but rather quantifies the degree of environmental awareness into the information entropy of each spatial unit, forming a global environmental uncertainty field.

[0060] Traditional path planning often relies solely on known obstacle information, simply treating unknown areas as passable or high-risk zones, lacking a precise measurement of the level of awareness. This invention introduces the concept of information entropy, transforming environmental uncertainty into a continuous, calculable numerical indicator.

[0061] This map provides direct navigation guidance for route planning. High-entropy areas represent areas with scarce information but worth exploring, while low-entropy areas represent areas that are already well-understood. Drones can use entropy levels to determine the value and risk of flight.

[0062] Compared to traditional grid maps that can only distinguish between known and unknown, the three-dimensional confidence map of this invention can finely depict the gradient changes in cognition, enabling the path planning algorithm to prioritize the areas that need to be explored, just like a human explorer, rather than blindly wandering randomly in unknown areas.

[0063] This map forms the basis for all subsequent decisions. It provides goal guidance for generating candidate paths in step S3 and provides a quantitative basis for evaluating paths in step S4. Without this quantitative map, subsequent attitude coordination and path optimization will lose their objective, and the UAV will be unable to determine which path is more valuable to explore.

[0064] In step S2, a dynamic constraint model of the fixed-wing UAV is established, which describes the physical laws that the UAV must follow to move in space.

[0065] Specifically, the dynamic constraint model includes at least the differential equation relationship between the heading angle, roll angle and flight speed, and the minimum turning radius constraint.

[0066] Specifically, the dynamic constraint model includes at least the heading angle. , With flight speed The differential equation relationship between them. For example, the relationship between the rate of change of heading angle and roll angle can be expressed as:

[0067] ;

[0068] in, It is the acceleration due to gravity. This represents the rate of change of heading angle.

[0069] This formula shows that, in order to change its course, the drone must generate a roll angle. Furthermore, the larger the roll angle, the sharper the turn; the model also includes a minimum turning radius constraint. From the maximum roll angle And speed determines:

[0070] ;

[0071] This means that drones cannot achieve smaller than The turning radius.

[0072] Specifically, the relationship model between attitude and perception range is a mapping model between the fuselage roll angle and the sensor's lateral field of view. This mapping model is used to calculate the additional lateral observation area covered by the fuselage tilt based on the turning radius on the planned path.

[0073] Secondly, a correlation model between attitude and perception range is established, which describes the fuselage roll angle. The mapping relationship between the sensor's field of view and the actual view. Taking an airborne forward-looking camera as an example, its optical axis is usually parallel to the longitudinal axis of the fuselage. When the drone is flying horizontally, At this time, the camera's field of view mainly covers the area directly in front of the drone. When the drone turns, it generates a roll angle. At this time, the camera's optical axis tilts along with the camera body, causing a shift in its lateral field of view. This model can be expressed as a mapping function. Input is the roll angle. The output is the angle range of the additional lateral observation area covered by the tilt relative to the fuselage coordinate system. For example, when When tilted to the right, the camera's field of view will additionally cover the lower right side of the camera body, bringing the original right-side blind spot into view.

[0074] Specifically, the core of the mapping model lies in establishing a quantitative mapping relationship between the roll angle of a fixed-wing UAV and the sensor's field of view coverage. It uses the turning radius provided by the path planning module and the UAV's current flight speed as its main inputs. Based on coordinated turning dynamics, the formula is:

[0075] ;

[0076] Calculate the fuselage roll angle required to execute the turning path. .

[0077] in, It is the acceleration due to gravity. The turning radius is This refers to flight speed.

[0078] Building upon this, the model incorporates the sensor's inherent parameters, including the horizontal field of view, vertical field of view, and mounting angle on the fuselage. The sensor's original field of view in the fuselage coordinate system is described as a defined spatial cone or quadrangular pyramid. The model establishes a rotation matrix from the fuselage coordinate system to the ground inertial coordinate system. This matrix represents the original field of view as a function of the roll angle. A rotational transformation is performed to calculate the actual spatial area covered by the sensor in the ground coordinate system.

[0079] The model calculates the difference between the theoretical field of view during level flight and the actual field of view at the current roll angle, thereby quantitatively identifying the additional lateral observation area covered by the aircraft's tilt. This area typically appears as a set of grid cells located on the side of the fuselage, which were originally in a blind spot but have entered the field of view after tilting.

[0080] In the software architecture, the model is encapsulated as an independent computational unit, providing a standardized interface. When the path evaluation module is called, only the turning radius and current pose information of the candidate path need to be input, and the model can return the lateral coverage area obtained due to tilt maneuvers during the execution of that path and its corresponding estimated information entropy gain.

[0081] Using the analytical model described above, the system can accurately predict the perception benefits of each turning maneuver during the path planning phase without relying on experimental data or approximate fitting. This mechanism transforms the originally passive dynamic constraints of fixed-wing UAVs into a quantifiable and calculable active perception extension method, providing a clear mathematical foundation and engineering implementation path for incorporating attitude coordination benefits into path evaluation.

[0082] This step does not treat the dynamic constraints of fixed-wing UAVs merely as planning limitations, but also establishes a model that correlates fuselage tilt with perception range, deeply coupling physical maneuvering with perception behavior.

[0083] Due to their nonholonomic constraints, fixed-wing UAVs must generate a roll angle when turning, which is an inevitable physical action. Traditional methods consider this action as a burden to be overcome when planning the path. This invention, however, recognizes that this inevitable roll angle changes the field of view of the onboard sensors.

[0084] This correlation model enables the system to anticipate and utilize the additional perception opportunities that arise from each turn. During the planning phase, the system can calculate which lateral areas the drone will gain observation capabilities by choosing a path with a specific turning radius.

[0085] Existing planning methods for fixed-wing aircraft often only address dynamic constraints to ensure path feasibility, separating attitude changes from perception. This invention, for the first time, transforms attitude changes from a planning cost into a source of perception benefits, laying the foundation for achieving dual benefits from a single maneuver.

[0086] This model introduces a new dimension to path generation in step S3. The path is no longer a simple geometric arc, but a sequence of maneuvers associated with a specific roll angle profile. More importantly, it provides crucial input for calculating attitude coordination benefits in step S4, enabling the algorithm to quantitatively evaluate the additional information value brought by different maneuvers, thereby selecting the optimal path that satisfies both dynamic requirements and maximizes perception efficiency.

[0087] In step S3, candidate flight paths that satisfy dynamic constraints are generated. After obtaining a 3D confidence map and establishing a dynamic constraint model, the system enters the candidate path generation stage. The core of this stage is to perform effective sampling within the feasible domain defined by the dynamic constraint model and generate physically flyable paths based on the sampling results.

[0088] The system first acquires the current pose of the UAV, including its spatial position coordinates and heading angle. and flight speed Subsequently, the system performs discrete sampling within the feasible range of yaw rate. Yaw rate The upper limit of sampling is determined by the maximum roll angle of the fixed-wing UAV. The decision is made using the following formula:

[0089] ;

[0090] in, This is the acceleration due to gravity.

[0091] Lower limit of yaw rate The formula corresponding to the minimum turning radius is:

[0092] ;

[0093] ;

[0094] The system in the interval Inner uniform sampling Yaw rate value For each sample obtained The system calculates the corresponding turning radius based on the coordinated turning dynamics. and automatically satisfy The constraints ensure that the generated path meets the platform's physical constraints.

[0095] At the path generation level, the system employs a path construction method based on motion primitives. Each sampled yaw angular velocity corresponds to a basic motion primitive, which is represented as a segment with a fixed curvature. The system uses the current pose as the starting point and extends the arc forward along the current heading direction, forming an arc segment. The parametric equation of this arc can be expressed as: Taking the current UAV position as the starting point, in the local coordinate system, the arc trajectory originates from the center coordinates of:

[0096] ;

[0097] and central angle Confirmed, among which For flight time.

[0098] Building upon this, the system further splices arc segments with straight line segments: completing a segment with a central angle of... After the arc, the drone resumes level flight and flies in a straight line along the tangent direction at the end of the arc, thus forming a standard path structure composed of "arc-straight line".

[0099] To address situations involving highly uncertain areas to the side and rear of the drone, the system employs a multi-segment arc splicing method to generate a specialized detour path. The specific method is as follows: The system first calculates the azimuth and distance of the target area relative to the drone's current position. If the target area is located to the side and rear of the drone and exceeds the reach of a single arc segment, the system generates a composite path composed of two or three interconnected arc segments. Let the curvature of the first arc segment be... The curvature of the second arc is The two circular arcs must satisfy the curvature continuity condition at the connection point, that is... And the turning direction is the same, or And the steering direction is opposite. The absolute value of the curvature of each arc segment does not exceed the upper limit of curvature corresponding to the maximum roll angle:

[0100] ;

[0101] By adjusting the radius and direction of each arc segment, the system can construct a continuous trajectory that bypasses blind spots to the sides and covers the target area. For example, to explore a high-entropy region to the left rear, the system can first generate a trajectory with a curvature of... The left-turning arc caused the drone to deviate from its original course, generating another segment with the same curvature. The left-turning arc guides the drone back to face the target area. The two arcs have the same tangential direction at the connection point, and the total turning angle is... This ensures that the flight trajectory is smooth and meets dynamic constraints.

[0102] Through the aforementioned sampling and path generation mechanism, the system can output a set of physically feasible candidate flight paths. This set includes both conventional circular-straight-line paths and multi-segment arc-connected detour paths designed for lateral and rearward exploration, providing ample decision-making space for subsequent path evaluation and optimization.

[0103] This step, under the premise of strictly meeting the dynamic constraints such as the minimum turning radius of the fixed-wing UAV, generates a large number of structured candidate paths through sampling, and can generate detour paths spliced ​​with multiple arcs for targets to the side and rear.

[0104] The core challenge of path planning for fixed-wing UAVs lies in their nonholonomic constraints. Generating a seemingly geometrically correct path is not difficult; the challenge lies in ensuring that every point and every curvature change along that path conforms to the UAV's physical flight capabilities. This step fundamentally guarantees that all candidate paths are physically flyable by decomposing the path into motion primitives that satisfy dynamic constraints and limiting the sampling range.

[0105] The output of this step is a candidate set of multiple physically feasible paths. This provides the basis for subsequent optimal path selection. Without this step, subsequent steps would face an infinite space of possibilities, making effective optimization impossible, or the optimized paths might be unworkable.

[0106] Many planning algorithms applicable to rotary-wing UAVs can generate polyline paths at arbitrary angles. However, directly applying these paths to fixed-wing aircraft results in UAVs being unable to track them. This invention solves the fundamental problem of path executability by employing a specialized motion primitive-based sampling strategy, ensuring consistency between the planning results and the platform's physical characteristics.

[0107] The diversity and completeness of the candidate paths generated in this step directly impact the quality of the final planning results. By transforming the exploration needs of high-uncertainty areas to the lateral rear into a multi-segment arc-connected detour path, this step initially integrates the exploration intent of step S1 with the dynamic capabilities of step S2. It provides the optimization process of step S4 with paths that include both forward exploration paths and special paths capable of covering high-value areas to the lateral rear, ensuring that the final optimal path truly balances information acquisition efficiency and flight safety.

[0108] like Figure 3As shown, the core of step S4 is to design a multi-objective evaluation function to calculate a comprehensive score for each candidate path generated in step S3, and select the path with the highest score as the optimal trajectory.

[0109] The formula for calculating the comprehensive evaluation index can be expressed as:

[0110] ;

[0111] in, As a comprehensive evaluation index, a larger value indicates a better path. These are the weighting coefficients for each item, used to balance the importance of different objectives.

[0112] Information gain term This item consists of two parts: forward observation benefit and attitude coordination benefit.

[0113] ;

[0114] Forward observation benefits A forward-facing sensor is simulated by emitting probe rays along the candidate path. The information entropy of each grid cell traversed by the ray in the 3D confidence map of step S1 is calculated, and these entropy values ​​are summed to obtain a total. The formula is:

[0115] ;

[0116] in, It is the information entropy of the grid cell. It is the distance from the current drone position to the grid cell. This is a decay factor, between 0 and 1, indicating that the greater the distance, the greater the uncertainty of the observation, and the less valuable the information. Attitude synergy benefits. Using the association model of step S2, calculate the sum of information entropy in the laterally covered region due to the execution of this path (whose roll angle profile is known), and accumulate it in the same way as... similar.

[0117] Path length cost term This item represents the geometric length of the path, used to penalize excessively long paths and encourage drones to complete exploration tasks over shorter distances.

[0118] attitude smoothing term This option penalizes drastic changes in roll angle during the path, encouraging the generation of trajectories with gentle attitude changes to conserve energy and ensure flight stability.

[0119] Ultimately, the system selects to make The candidate path with the largest value is taken as the expected flight trajectory.

[0120] This step decomposes the evaluation metrics into an information gain term, a path length cost term, and an attitude smoothing term. The information gain term is further subdivided into forward observation gains and attitude coordination gains caused by fuselage tilt, and a distance attenuation factor is introduced to simulate sensor uncertainties.

[0121] This is the core decision-making step in this invention to achieve the coupling of perception and maneuverability. It transforms the correlation between attitude and perception discovered in step S2 into a quantifiable benefit term that can participate in path competition. Attitude Coordination Benefit Term The existence of this means that a path that needs to turn will not be penalized for changes in its length or orientation, but will instead receive bonuses because it can cover additional lateral high-entropy regions.

[0122] Through multi-objective optimization, the system can find the path with the best overall performance among many candidate paths. This path may not be the shortest or the one with the greatest information gain, but it will certainly be the path that achieves the best balance between information acquisition efficiency, flight cost, and flight safety.

[0123] Traditional information path planning only considers information ahead, causing UAVs to explore unknown areas in a straight line. This invention, however, incorporates attitude coordination benefits into the optimization function, enabling the algorithm to proactively identify and select paths that allow for exploration of lateral blind spots through tilting maneuvers, achieving coordinated optimization of both path and visibility.

[0124] The optimal path output in this step reflects a deep integration of the target exploration in step S1 and the dynamics and perception capabilities in step S2. It ensures that the UAV can acquire environmental information with maximum efficiency, provided it is physically feasible. More importantly, the introduction of attitude coordination benefits means that each turn maneuver of the UAV during mission execution is no longer a pure path adjustment, but simultaneously undertakes the task of information collection, thereby acquiring richer environmental information per unit flight distance and improving the detection efficiency of a single flight.

[0125] like Figure 4 As shown, step S5 is responsible for implementing the planning results and achieving closed-loop control.

[0126] Specifically, after selecting the optimal path, the system first describes the path in a parameterized form, that is, a series of desired state points discretized by time or distance. Each desired state point contains the desired location coordinates. Desired heading angle Desired flight speed and expected roll angle .

[0127] The system employs a model predictive control algorithm to achieve accurate tracking of the optimal trajectory. The algorithm's construction process is as follows: First, a discretized kinematic model of the fixed-wing UAV is established as the prediction model. The current pose and velocity are used as state variables, and the aileron deflection angle, elevator deflection angle, and throttle opening are used as control inputs. The state equations are used to predict the future trajectory evolution within a finite time domain. Let the state vector be: ;

[0128] in, Spatial location coordinates, For heading angle, This is the roll angle. For flight speed; control vector is ,in For the aileron deflection angle, For elevator deflection angle, Let be the throttle opening. The discretized prediction model of the system is expressed as:

[0129] ;

[0130] in, A continuous function describing the kinematic relationships of an unmanned aerial vehicle (UAV). is the discrete time step.

[0131] Within each control cycle, the system is based on the current state. Solve an optimization problem in a finite-time domain, where the objective is to minimize the weighted sum of the tracking error and control energy of the desired trajectory. The mathematical form of this optimization problem is:

[0132] ;

[0133] in, To predict the length of the time domain, For the first One desired state point, These are the state error weight matrix, control quantity weight matrix, and terminal cost weight matrix, respectively. Solving the optimization problem requires satisfying the physical constraints of the UAV, including the aileron deflection angle. Upper and lower limit constraints, elevator deflection angle Upper and lower limit constraints, throttle opening Upper and lower limits of constraints, and roll angle change rate The constraints ensure that the solved control commands are executable in actual flight. The system solves this optimization problem and extracts the first control variable from the optimization sequence. The control commands for the current cycle are sent to the flight control computer via serial port for execution. In the next control cycle, the system reconstructs the optimization problem based on the updated real state and solves it again, forming a rolling optimization closed-loop control structure, thereby achieving precise tracking of the optimal trajectory.

[0134] During the execution of the current flight path, the system continuously collects new data and updates the 3D confidence map in real time. When the updated map shows a newly discovered obstacle ahead of the current path, and the distance is less than the safe braking distance, or when a high-value unknown area with an information entropy value exceeding that of the current target area is detected on the map, the system triggers a replanning process. After replanning is initiated, the system uses the UAV's current true pose, speed, and roll angle as the new initial state.

[0135] To ensure flight safety, the system does not immediately terminate the current trajectory, but instead generates a smooth transition path from the current state to the newly planned trajectory. This transition path uses a curve with minimum jerk to ensure the continuity of roll angle, heading angle, and their rate of change, avoiding flight instability caused by sudden changes in commands.

[0136] After the transition path is generated, the system merges the transition segment with the new path generated by replanning to form a complete remaining flight trajectory, and then re-executes the above trajectory analysis and control command output process.

[0137] To avoid frequent replanning due to drastic changes in environmental information, the system introduces a replanning cooling mechanism. After each replanning, the system sets a minimum time interval threshold. Within this threshold, even if the triggering conditions are met again, the system will not execute a new replanning; instead, it will re-evaluate after the current trajectory is completed. This mechanism ensures system responsiveness while avoiding the waste of computational resources and flight path oscillations caused by excessive replanning. Through this closed-loop rolling optimization mechanism, the UAV can continuously and stably complete autonomous exploration tasks in dynamically changing and unknown environments, achieving a synergistic optimization of safety and exploration efficiency.

[0138] In a completely unknown environment, the initial map information is bound to be incomplete. During flight, the drone will continuously discover new obstacles or new areas of interest. If it insists on following the initially planned path, it may collide with unexpected obstacles or miss newly discovered high-value exploration targets.

[0139] This mechanism endows drones with real-time adaptability in dynamic environments. Instead of blindly executing a static plan, it becomes an autonomous intelligent agent capable of sensing environmental changes and continuously adjusting its behavior. This fundamentally guarantees flight safety and ensures that exploration missions remain focused on the most valuable targets in the current environment.

[0140] Traditional open-loop programming does not adjust once the task starts, making it extremely unresponsive to changes. At the other extreme, frequent replanning is used, but this incurs high computational costs and can lead to path jitter due to overreaction. This invention employs an event-triggered replanning strategy, which avoids unnecessary waste of computational resources while ensuring safety and task efficiency.

[0141] The rolling replanning mechanism links the entire autonomous flight mission into an organic whole. It elevates the planning capabilities of steps S3 and S4 from one-off mission deployments to continuous, real-time decision-making capabilities. When step S1 discovers a new high-entropy region, step S5 treats it as a new target, rapidly guiding steps S3 and S4 to generate a new path that can explore the region while taking into account attitude coordination benefits. This enables the UAV to continuously and safely complete the complete cognitive loop of exploration, perception, and re-exploration in complex and unknown environments.

[0142] like Figure 5 As shown, a machine vision-based system for autonomous flight path planning of unmanned aerial vehicles includes:

[0143] Specifically, the environmental modeling module is responsible for processing sensor data, dynamically building and maintaining a 3D confidence map that serves as the basis for all decisions, and quantifying the physical world into a computable information entropy field.

[0144] Specifically, the constraint establishment module not only stores the dynamic constraint model of the fixed-wing UAV, but more importantly, it establishes the correlation model between attitude and perception range. This model is the key mathematical bridge to transform dynamic burden into perception benefit, enabling the system to predict the lateral observation gain brought about by the tilt of the fuselage when turning.

[0145] Specifically, guided by dynamic constraints and correlation models, the path generation module samples in the physically feasible solution space to generate a series of candidate flight paths composed of arcs and straight lines. In particular, it can generate specialized multi-segment arc splicing detour paths for high uncertainty areas to the side and rear, solving the core problem that fixed-wing UAVs can see but cannot fly.

[0146] Specifically, the evaluation and optimization module is the decision-making center for realizing the coupling optimization of perception and maneuver. It calculates a comprehensive evaluation index for each candidate path, decomposes the information gain into forward observation gain and attitude coordination gain, and optimizes them together with path length cost and attitude smoothing term. This enables the system to actively select the optimal trajectory that can obtain more lateral information by tilting maneuver, rather than passively accepting turns.

[0147] Specifically, the control execution and replanning module is the execution and feedback link in the interaction between the system and the physical world. It is responsible for parsing the optimal trajectory into control commands and implementing rolling replanning based on the real-time updated map. This makes the system no longer an open-loop plan executor, but an autonomous intelligent agent with dynamic adaptability, capable of dealing with sudden obstacles or newly discovered high-value targets.

[0148] An electronic device, comprising:

[0149] Specifically, the processor, as the core computing unit of the device, is responsible for executing computer programs stored in memory. In this context, the processor's task is to run the algorithm code that implements the path planning method.

[0150] It should be noted that this specifically includes processing sensor data to build and update 3D confidence maps, calculating dynamics and perception association models, performing path sampling and generation, running multi-objective optimization algorithms to evaluate and select optimal paths, and generating control commands and managing the replanning process.

[0151] Specifically, the memory includes an internal memory and an external memory interface for storing two types of key information: one is the executable computer program; the other is the data generated during program execution, such as real-time constructed 3D confidence maps, candidate path sets, intermediate calculation results of optimization functions, and current flight status and control commands.

[0152] Furthermore, the multi-source perception unit includes an airborne visual sensing module, a wireless data transmission & GNSS positioning module, and an attitude sensor interface. The airborne visual sensing module uses an airborne forward-looking camera to continuously acquire sequential environmental images during flight, providing raw perception data for the environment modeling module. The wireless data transmission & GNSS positioning module acquires real-time global pose information such as the UAV's 3D spatial position and heading angle. The attitude sensor interface interfaces with the airborne inertial measurement unit to acquire real-time attitude data such as the UAV's roll angle and pitch angle. The pose and attitude data are synchronously input to the environment modeling module and the constraint establishment module to ensure real-time updates of the 3D confidence map and matching of dynamic constraints for path planning.

[0153] Furthermore, the flight control interaction unit includes a flight control communication interface and an actuator drive interface. The flight control communication interface uses a serial communication link to realize bidirectional data interaction between the processor and the UAV flight control computer, sending the discrete control commands parsed by the control execution module to the flight control system, while simultaneously transmitting the UAV's real-time flight status back. The actuator drive interface is used to interface with the UAV's ailerons, elevators, throttle, and other actuators to achieve precise control of the UAV's flight attitude and speed, ensuring stable tracking of the optimal flight trajectory.

[0154] Specifically, the power management unit includes a power management module, a charging management module, and an onboard battery. The onboard battery is externally charged and managed through the charging management module. The power management module provides graded and regulated power supply to the processor and all peripheral units to ensure the continuous and stable operation of the system during the drone's flight.

[0155] It should be noted that the peripheral interface unit includes a USB debugging interface, which is used for ground-side program burning, system debugging, offline data export and parameter configuration, improving the convenience of system engineering debugging.

[0156] Based on the preferred embodiments of the present invention described above, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for autonomous flight path planning of unmanned aerial vehicles (UAVs) based on machine vision, characterized in that, Includes the following steps: S1. Acquire and process environmental perception data to construct a three-dimensional confidence map for quantifying the degree of environmental awareness; S2. Establish a dynamic constraint model for fixed-wing UAVs and a correlation model between attitude and perception range; S3. Based on the three-dimensional confidence map and correlation model, sample within the range defined by the dynamic constraint model to generate multiple candidate flight paths that satisfy the dynamic constraints; S4. Select the optimal path from the candidate paths that balances information acquisition efficiency and flight cost. S5. The flight trajectory of the optimal path is parsed into control commands and executed, while the path is rolled and replanned based on real-time perceived environmental changes.

2. The method for autonomous flight path planning of unmanned aerial vehicles based on machine vision according to claim 1, characterized in that: In step S1, the three-dimensional confidence map includes: acquiring sequential images, fusing real-time pose information to update the probability of occupying the grid map, calculating the information entropy of each grid cell, and defining the global information entropy distribution as an environmental uncertainty field.

3. The method for autonomous flight path planning of unmanned aerial vehicles based on machine vision according to claim 1, characterized in that: In step S2, the dynamic constraint model includes at least the differential equation relationship between the heading angle, roll angle and flight speed, and the minimum turning radius constraint.

4. The method for autonomous flight path planning of unmanned aerial vehicles based on machine vision according to claim 1, characterized in that: In step S2, the relationship model between attitude and perception range is a mapping model between the fuselage roll angle and the sensor's lateral field of view. The mapping model is used to calculate the additional lateral observation area covered by the fuselage tilt based on the turning radius on the planned path.

5. The method for autonomous flight path planning of unmanned aerial vehicles based on machine vision according to claim 1, characterized in that: In step S3, the candidate flight path includes: obtaining the current pose, sampling within the constraints of the maximum yaw rate and the minimum turning radius, generating multiple candidate flight paths composed of arcs, and generating a detour path for the high uncertainty area on the side and rear of the fuselage by splicing multiple arcs.

6. The method for autonomous flight path planning of unmanned aerial vehicles based on machine vision according to claim 1, characterized in that: In step S4, the optimal path includes: calculating a comprehensive evaluation index for each candidate path, the index including an information gain term, a path length cost term, and an attitude smoothing term composed of forward observation and lateral observation caused by fuselage tilt, and selecting the path with the optimal index as the desired flight trajectory.

7. The method for autonomous flight path planning of unmanned aerial vehicles based on machine vision according to claim 6, characterized in that: The information gain term is obtained by calculating the sum of entropy reduction of the grid cells traversed by the probe rays in the uncertain field of the environment by emitting probe rays along the candidate path direction, and an attenuation factor that decreases with increasing propagation distance is introduced during the calculation.

8. The method for autonomous flight path planning of unmanned aerial vehicles based on machine vision according to claim 1, characterized in that: In step S5, the rolling replanning includes: continuously collecting new data and updating the three-dimensional confidence map in real time during the execution of the current flight path; when the updated map shows that the current path has a collision risk or an area with higher information urgency, interrupting the current path and returning to step S3 to regenerate the candidate flight path.

9. A system for autonomous flight path planning of unmanned aerial vehicles (UAVs) based on machine vision, comprising a method for autonomous flight path planning of UAVs based on machine vision as described in any one of claims 1-8, characterized in that, include: The environment modeling module is used to build and maintain 3D confidence maps; The constraint establishment module is used to store kinematic constraint models and establish the relationship between posture and perception range; The path generation module is used to generate candidate flight paths that satisfy dynamic constraints; The evaluation and optimization module is used to select the optimal flight trajectory from the candidate paths; The control execution module is used to parse the flight trajectory into control commands and execute them, and to trigger replanning based on environmental updates.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores a computer program executable by at least one processor, which is executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images