An aircraft return path construction method based on visual intelligence

Abstract

The application relates to the technical field of aircraft navigation and discloses a flight path construction method based on visual intelligence. The method collects a multi-view image sequence of a flight environment through an airborne visual sensor, generates visual observation data with unified time stamps and spatial coordinates through space-time alignment processing, calculates a motion vector field of an aircraft relative to a ground reference based on the light flow features between continuous frames in the data, builds a three-dimensional environmental kinematic model in combination with attitude data of an inertial measurement unit, inversely deduces a current flight path, extracts a key flight path point set of a historical path, constructs an initial topological network of a return flight path according to the spatial distribution density of the key flight path point set, dynamically allocates the connection relationship of network nodes, generates a wind disturbance-resistant candidate return flight path set in combination with real-time meteorological wind field intensity parameters, evaluates the candidate set by using a multi-objective optimization algorithm, and outputs an optimal return flight path meeting a fuel consumption threshold and a flight time constraint.

Description

Technical Field

[0001] This invention relates to the field of aircraft navigation technology, specifically to a method for constructing an aircraft return path based on visual intelligence. Background Technology

[0002] In the field of aviation, the rational construction of return routes is a crucial component of ensuring flight safety. Especially when an aircraft encounters sudden malfunctions, mission changes, or abnormal external environments, the ability to quickly generate a reliable return route directly impacts the safety of the crew and the aircraft. Currently, aircraft return route construction primarily relies on the Global Navigation Satellite System (GNSS) and pre-set fixed route data. However, this method is prone to path planning interruptions or deviations in scenarios where satellite signals are blocked, interfered with, or malfunction, such as in complex mountainous areas, densely populated high-rise buildings, or environments with electronic countermeasures.

[0003] With the continuous development of aviation technology, path planning technology based on multi-sensor fusion is gradually being applied to the field of aircraft navigation. Inertial Measurement Units (INS), due to their independence from external signals, are often used in conjunction with other sensors. However, during long-term operation, INS can accumulate errors due to device drift, leading to a decrease in the accuracy of the calculated aircraft trajectory and consequently affecting the accuracy of the return path. Furthermore, traditional path planning methods often neglect the impact of real-time weather conditions on flight, especially changes in wind intensity, which alter the aircraft's actual flight attitude and energy consumption. Failure to consider these factors in path planning may result in extended flight time and unexpected fuel consumption during the return journey.

[0004] Existing return path construction methods lack efficient topology network construction when fusing historical flight data with real-time environmental data, making it difficult to quickly extract key information from historical flight paths and transform it into usable return path foundation data. Most methods employ fixed path generation patterns, failing to adjust path weights according to dynamic changes in the flight environment. This results in poor adaptability of the generated return paths, making it difficult to meet the return requirements of different scenarios in complex and ever-changing aerial environments. Furthermore, in the path optimization stage, traditional single-objective optimization algorithms can only focus on one metric, failing to consider multiple constraints. This makes it difficult for the final output return path to achieve an ideal balance across multiple dimensions such as fuel consumption and flight time, further reducing the safety and economy of the return process. Summary of the Invention

[0005] The purpose of this invention is to provide a visual intelligence-based method for constructing aircraft return paths, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides a method for constructing an aircraft return path based on visual intelligence, the method comprising:

[0007] The flight environment is acquired by an airborne visual sensor through a multi-view image sequence. The multi-view image sequence is then spatiotemporally aligned to generate visual observation data with a unified timestamp and spatial coordinates.

[0008] Based on the continuous inter-frame optical flow characteristics in the visual observation data, the motion vector field of the aircraft relative to the ground reference is calculated, and combined with the attitude data output by the inertial measurement unit, a three-dimensional environmental kinematic model is established.

[0009] The current flight path of the aircraft is reverse-engineered using the three-dimensional environmental kinematic model. The set of key waypoints in the historical flight path is extracted, and the initial topology network of the return path is constructed based on the spatial distribution density of the set of key waypoints.

[0010] Dynamic weight allocation is performed on the node connection relationships in the initial topology network, and combined with the wind field intensity parameters in real-time meteorological data, a set of candidate return paths with wind disturbance resistance is generated.

[0011] A multi-objective optimization algorithm is used to evaluate the feasibility of the candidate return route set and output the optimal return route that meets the fuel consumption threshold and flight time constraints.

[0012] Preferably, the spatiotemporal alignment processing of the multi-view image sequence includes:

[0013] Significant edge feature points are detected in each frame of the multi-view image sequence, and cross-view feature correspondence is established through feature descriptor matching;

[0014] Based on the aforementioned feature correspondence, the homography transformation matrix between adjacent viewpoints is calculated, and images from different viewpoints are projected onto the same virtual observation plane.

[0015] A sliding window mechanism is used to perform inter-frame consistency verification in the time dimension of the projected image sequence, eliminating the spatiotemporal misalignment caused by sensor delay.

[0016] Preferably, establishing the three-dimensional environmental kinematic model includes:

[0017] Based on the visual observation data of the unified timestamp and spatial coordinates, a spherical coordinate system centered on the aircraft is constructed;

[0018] By integrating the optical flow characteristics between consecutive frames in the spherical coordinate system, the displacement increments of the aircraft in three degrees of freedom are obtained.

[0019] The displacement increment and the angular velocity data output by the inertial measurement unit are fused using Kalman filtering to calculate the six-degree-of-freedom motion trajectory of the aircraft in the global coordinate system.

[0020] Preferably, the initial topology network for constructing the return path includes:

[0021] Extract the heading change points from the six-degree-of-freedom motion trajectory as key waypoints, and calculate the curvature continuity index between adjacent key waypoints;

[0022] Using the curvature continuity index as a constraint, a smooth curve segment connecting key waypoints is generated using piecewise cubic spline interpolation.

[0023] The intersection points of the smooth curve segments are used as nodes of the topological network, and the curve segments themselves are used as edges, forming an initial topological network with a weighted directed graph structure.

[0024] Preferably, the dynamic weight allocation of node connection relationships in the initial topology network includes:

[0025] The ratio of the actual length of each edge to the theoretical shortest path length in the initial topology network is obtained and used as the basic weight coefficient.

[0026] Based on the gradient distribution of wind field intensity in three-dimensional space from real-time meteorological data, the additional drag influence factor of the wind field on each side is calculated.

[0027] The basic weight coefficients and the additional resistance influence factor are weighted and superimposed to generate a dynamic edge weight matrix that takes into account environmental disturbances.

[0028] Preferably, generating a set of candidate return paths with wind disturbance resistance includes:

[0029] An improved Dijkstra algorithm is run on the dynamic edge weight matrix to search for the first K shortest paths from the current position to the preset landing point;

[0030] For each shortest path, wind speed field simulation tests are performed. Paths with continuous headwinds exceeding a preset distance are eliminated, and the remaining paths are retained to form a set of candidate return paths.

[0031] Preferably, the feasibility assessment of the candidate return path set using a multi-objective optimization algorithm includes:

[0032] Establish an objective function space with fuel efficiency, route length, and risk coefficient as variables;

[0033] Pareto front analysis is performed on the candidate return path set within the objective function space to filter out the non-dominated solution set;

[0034] The non-dominated solution set is subjected to secondary constraint filtering based on the aircraft's current remaining fuel to obtain an optimized path subset that meets the conditions for safe return.

[0035] Preferably, the optimal return path that satisfies the fuel consumption threshold and flight time constraints includes:

[0036] The path with the highest matching degree between the fuel consumption rate and the current engine operating condition is selected from the optimized path subset as the first preferred solution;

[0037] When the flight time of the first preferred option exceeds a preset threshold, the alternative mechanism is activated, and the path with the shortest path length and the risk coefficient lower than the warning value is selected as the alternative.

[0038] The final selected return path is discretized into a sequence of waypoint commands and sent to the flight control system for execution.

[0039] Preferably, the method further includes:

[0040] During path execution, the deviation between the visual observation data and the kinematic model prediction is continuously monitored. When the cumulative deviation exceeds the fault tolerance threshold, path replanning is triggered.

[0041] The route replanning adopts an incremental update strategy, retaining the segments of the original route that are not affected by environmental changes, and only reconstructing the local topology network in areas where deviations exceed limits.

[0042] Preferably, the local topology network reconstruction includes:

[0043] Using the current deviation position as the center, visual data is re-acquired within a preset radius to establish a temporary environment model;

[0044] The temporary environment model is seamlessly spliced ​​with the unaffected part of the original topology network to generate a hybrid topology structure;

[0045] A real-time path optimization algorithm is run on the hybrid topology to output a corrected continuous return trajectory.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] This visual intelligence-based method for constructing aircraft return paths uses an airborne visual sensor to collect multi-view image sequences and perform spatiotemporal alignment processing to generate visual observation data with unified timestamps and spatial coordinates. This method can make full use of the rich environmental information acquired by the visual sensor, reduce excessive reliance on global navigation satellite system signals, and even in scenarios where satellite signals are interfered with or fail, visual data can still provide stable basic data support for the construction of return paths, thus broadening the applicable scenarios of the return path construction method.

[0048] The motion vector field is calculated based on the continuous inter-frame optical flow characteristics in the visual observation data, and a three-dimensional environmental kinematic model is established by combining the attitude data output by the inertial measurement unit. This model integrates the advantages of visual data and inertial data. Visual data can provide high-precision relative motion information, while inertial data can supplement the attitude changes of the aircraft. The combination of the two can effectively reduce the errors that may occur when a single sensor is working, improve the accuracy of the description of the aircraft's motion state, and provide a more reliable model foundation for the reverse inference and construction of the subsequent return path.

[0049] By using a three-dimensional environmental kinematic model to reverse-engineer the current flight path, extract the set of key waypoints, and construct an initial topology network, this process can accurately capture important position nodes during flight by analyzing key information in historical flight paths. At the same time, the topology network is constructed based on the spatial distribution density of key waypoints, making the initial network structure more consistent with the actual flight trajectory characteristics. This avoids the problem of path decoupling from the actual flight environment that may occur in traditional fixed route planning, and provides a more reasonable initial framework for subsequent path optimization.

[0050] Dynamic weights are assigned to the node connections in the initial topology network, and a set of candidate return paths is generated by combining wind field intensity parameters from real-time meteorological data. The dynamic weight assignment can be adjusted according to the actual connection between network nodes, while the introduction of wind field intensity parameters can fully consider the impact of meteorological conditions on flight, enabling the generated candidate paths to have wind disturbance resistance, reducing problems such as increased flight energy consumption and extended flight time caused by wind field changes, and improving the adaptability of return paths in complex meteorological environments.

[0051] A multi-objective optimization algorithm is used to evaluate the feasibility of the candidate return route set and output the optimal return route that meets the fuel consumption threshold and flight time constraints. The multi-objective optimization algorithm can take into account both fuel consumption and flight time constraints at the same time, and select the solution with better overall performance from multiple candidate routes. This avoids the limitation of traditional single-objective optimization, which only focuses on a single indicator. The final output return route can control fuel consumption within a reasonable range and meet flight time requirements, thereby improving the economy and efficiency of the return process while ensuring flight safety.

[0052] The entire approach, from data acquisition, model building, network construction, path generation to optimization and evaluation, forms a complete return path construction process. The various stages are closely connected, and the full integration of data and models improves the accuracy and reliability of path construction. The combination of dynamic adjustment and multi-objective optimization enhances the adaptability of the path to environmental changes and its overall performance, providing a more efficient and reliable path solution for aircraft return. Attached Figure Description

[0053] Figure 1This is a schematic diagram illustrating the working principle of the visual intelligence-based aircraft return path construction method described in this invention.

[0054] Figure 2 A flowchart for spatiotemporal alignment processing of multi-view image sequences;

[0055] Figure 3 A three-dimensional environmental kinematic model and a six-degree-of-freedom trajectory diagram;

[0056] Figure 4 A flowchart for constructing the initial topology network for the return path. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please see Figure 1This invention provides a visual intelligence-based method for constructing an aircraft return path. The method integrates data from airborne visual sensors and inertial measurement units (IMUs) to enable autonomous return-to-home capability for the aircraft in environments without satellite navigation signals. The method acquires multi-view image sequences of the flight environment using airborne visual sensors. These sequences are typically captured simultaneously by multiple cameras positioned on the aircraft's nose, belly, and sides, covering forward-looking, downward-looking, and side-looking perspectives. The acquired multi-view image sequences undergo spatiotemporal alignment processing using feature point matching and projection transformation techniques to unify images from different perspectives to the same timestamp and spatial coordinate system, generating visual observation data. This visual observation data serves as the basis for subsequent processing and contains optical flow information between consecutive frames. Based on the optical flow characteristics between consecutive frames in the visual observation data, the motion vector field of the aircraft relative to a ground reference is calculated. The motion vector field is estimated pixel-by-pixel using the optical flow method to represent two-dimensional motion. This is then combined with attitude data output in real-time by the IMU, which includes three-axis angular velocity and acceleration. By fusing visual and inertial data, a three-dimensional environmental kinematic model describing the aircraft's six degrees of freedom motion is established. A three-dimensional environmental kinematic model is used to back-engineer the aircraft's current trajectory, extracting a set of critical waypoints from historical flight paths. These waypoints consist of locations with significant heading changes. Based on the spatial distribution density of the critical waypoints, a graph theory approach is used to construct an initial topology network for the return path, with waypoints as nodes and path segments as edges. Dynamic weights are assigned to the node connections in the initial topology network, incorporating wind intensity parameters from real-time meteorological data obtained via airborne weather radar or data link. Considering the wind field's influence, multiple candidate return paths with wind resistance capabilities are generated. Finally, a multi-objective optimization algorithm is used to evaluate the feasibility of the candidate return paths, simultaneously considering fuel consumption thresholds and flight time constraints, outputting an optimal return path for the flight control system to execute.

[0059] Example 1: See Figure 2The spatiotemporal alignment of multi-view image sequences begins with the detection of significant edge feature points in each frame. The feature point detection process employs an adaptive thresholding algorithm to adapt to environmental changes under varying lighting conditions. Detected feature points must possess rotation invariance and scale invariance to ensure the stability of subsequent matching. When establishing cross-view feature correspondences through feature descriptor matching, the feature descriptor calculation process generates high-dimensional vectors to represent the image patch information surrounding each feature point. The matching algorithm uses bidirectional consistency checks to eliminate false matches, thereby establishing accurate feature point correspondences between images from different viewpoints. When calculating the homography transformation matrix between adjacent viewpoints based on the feature correspondences, the matrix calculation process uses the least squares method to fit the projection relationship between feature points, while employing a random sampling consistency algorithm to eliminate interference from abnormal match points in the transformation matrix estimation. The homography transformation matrix projects images from different viewpoints onto the same virtual observation plane. The projection transformation process corrects perspective distortion caused by different camera viewpoints. The selection of the virtual observation plane must ensure that the image content from all viewpoints can be fully projected while minimizing image distortion. When using a sliding window mechanism to perform inter-frame consistency verification in the time dimension of the projected image sequence, the size of the sliding window is dynamically adjusted according to the image acquisition frequency and the speed of the aircraft. The images of consecutive frames within the window will undergo pixel-level optical flow consistency analysis. By comparing the motion trajectories of feature points between adjacent frames, spatiotemporal misalignment caused by sensor transmission delay can be identified and corrected.

[0060] When establishing a 3D environmental kinematic model, a spherical coordinate system centered on the aircraft is constructed. The origin of the spherical coordinate system is the aircraft's center of mass, and the coordinate axes are aligned with the body coordinate system to facilitate subsequent data fusion. When integrating the optical flow features between consecutive frames in the spherical coordinate system, the integration process considers the influence of aircraft attitude changes on the direction of the optical flow vector. The displacement increment of the aircraft in three degrees of freedom is obtained by accumulating the optical flow vectors over time. During the displacement increment calculation, the optical flow component caused by camera rotation is compensated to ensure that the displacement increment is purely caused by the aircraft's translational motion. When fusing the displacement increment with the angular velocity data output by the inertial measurement unit (IMU) using Kalman filtering, the Kalman filter design includes two main stages: state prediction and measurement update. The state vector includes position, velocity, attitude angle, and their rate of change. During the filtering process, the visual displacement increment serves as the observation input, and the angular velocity data from the IMU is used to correct the state prediction model. Optimal fusion of visual and inertial data is achieved through dynamic adjustment of the covariance matrix. When calculating the six-degree-of-freedom motion trajectory of the aircraft in the global coordinate system, the trajectory calculation process uses quaternions to represent attitude to avoid gimbal lock problems. The position information is obtained by integrating the velocity vector and then accumulating it in the global coordinate system. The generation of the motion trajectory updates the spatial relationship of the aircraft relative to its initial position in real time, providing accurate attitude estimation for subsequent path deduction.

[0061] Throughout the processing, the accuracy of spatiotemporal alignment directly impacts the accuracy of subsequent kinematic models. Therefore, the robustness of feature point matching and the accuracy of homography transformation require continuous monitoring. When a decrease in feature point matching success rate or an increase in transformation matrix reprojection error is detected, the system automatically adjusts feature detection parameters or switches to a backup feature descriptor algorithm. Similarly, during data fusion, the innovative covariance and observation covariance of the Kalman filter are adaptively adjusted based on the confidence level of the sensor data. When the quality of visual data deteriorates due to weather conditions, the system appropriately increases the weight of inertial data to ensure the continuity of trajectory estimation. The establishment of the 3D environmental kinematic model not only relies on the data at the current moment but also references historical motion states to smooth the trajectory estimation results. The model output includes uncertainties in position and attitude, which serve as a risk assessment basis for subsequent path planning. The update frequency of the kinematic model is synchronized with the image acquisition frequency to ensure the capture of rapid maneuvers of the aircraft.

[0062] See Figure 3 This image showcases the core technology of visual and inertial data fusion. In the upper half, the aircraft trajectory curve clearly displays the motion vector field calculated based on continuous inter-frame optical flow features. Arrows indicate the aircraft's direction of motion and velocity changes at different time points. The distribution of ground reference points provides stable benchmarks for visual positioning. These reference points establish correspondences between images from different viewpoints through feature point matching technology, providing accurate relative position information for the kinematic model. The lower half of the image shows the aircraft's six-degree-of-freedom motion in the global coordinate system, including three translational and three rotational degrees of freedom. Key points on the trajectory mark important pose sampling locations, reflecting the results of Kalman filtering fusion of visual displacement increments and inertial measurement unit angular velocity data. This data fusion effectively overcomes the limitations of a single sensor; visual data provides high-precision relative motion information, while inertial data compensates for the aircraft's attitude changes, jointly ensuring the accuracy of trajectory estimation. Through optical flow feature integration in spherical coordinates and the Kalman filtering algorithm, the system can output the aircraft's spatial position and attitude information in real time, laying a solid technical foundation for autonomous return in environments without satellite navigation signals.

[0063] Example 2: See Figure 4The construction process begins with an in-depth analysis of the aircraft's historical trajectory. Points where the heading changes significantly are identified by calculating the instantaneous curvature at each point on the trajectory. These abrupt changes reflect critical maneuvering decisions made by the aircraft during flight, such as obstacle avoidance or alteration of the cruise path. Curvature calculation relies on higher-order derivatives of the position data, estimating the curvature of the current path through the geometric relationships between adjacent trajectory points. When the curvature value exceeds a threshold dynamically calculated based on the aircraft's maximum roll rate and airspeed, the point is marked as a critical waypoint. The extraction of critical waypoints is not isolated; their spatial distribution density is regulated by a minimum distance constraint determined by the aircraft's turning radius. This prevents overly dense nodes in areas of small maneuvers, ensuring the simplicity and manageability of the topology network.

[0064] After obtaining the set of critical waypoints, it is necessary to evaluate the path smoothness between these points. Calculating the curvature continuity index between adjacent critical waypoints becomes the basis for generating connecting segments. This index quantifies the smoothness of the path direction change when transitioning from one waypoint to the next, and its calculation involves analyzing the rate of curvature change of the trajectory segments before and after the waypoint. Using the curvature continuity index as the core constraint, the system employs piecewise cubic spline interpolation to generate smooth curve segments connecting each critical waypoint. The interpolation process defines an independent cubic polynomial curve for every two adjacent critical waypoints. The coefficients of the curve are determined by solving the boundary conditions, which force the curve to maintain continuity in its position, tangent direction (first derivative), and curvature (second derivative) at the waypoints. This mathematical continuity ensures that the generated path segments are physically flyable, and the aircraft does not need to make abrupt directional or attitude adjustments when flying along such paths.

[0065] The generation of smooth curve segments is an iterative optimization process. When the curvature of a curve segment exceeds the maximum curvature allowed by the aircraft's performance envelope, the system automatically inserts auxiliary control points into that segment, reducing the local curvature by increasing the number of segments until all curve segments satisfy flight dynamics constraints. Each smooth curve segment is mathematically parameterized, and its arc length can be precisely calculated through numerical integration. This arc length serves as the initial weight of the edges in the topology network. The endpoints of all these smooth curve segments (i.e., critical waypoints) are defined as nodes in the topology network, while the curve segments connecting the nodes are defined as edges, thus forming an initial topology network with a weighted directed graph structure. In this directed graph, the direction of the edges is consistent with the aircraft's historical flight direction, and the weights are initialized to the actual lengths of the curve segments. The network structure captures the flightable airspace that the aircraft has already verified. The construction of the initial topology network is not merely a simple replication of historical paths; it is more of an abstraction and generalization process. The nodes and edges in the network constitute a search space that can be used for future path planning. Because this network is derived from actual flight trajectories, it implicitly includes information on avoiding environmental factors (such as no-fly zones and terrain), thus possessing high reliability. This network forms the foundational framework for subsequent dynamic weight allocation and optimized path search, and its quality directly impacts the feasibility and optimality of the final return path.

[0066] Example 3: Dynamic weight allocation and generation of a set of candidate paths to withstand wind disturbances are performed on the initial topology network. Each edge in the initial topology network is associated with the basic cost information of the aircraft moving from one node to another. The dynamic weight allocation process aims to transform the static geometric path length into a comprehensive flight cost affected by real-time environmental factors. The allocation process calculates the ratio of the actual length of each edge to the theoretical shortest path length as the basic weight coefficient. The actual length is obtained by integrating the arc length of the smooth curve segment connecting the nodes, while the theoretical shortest path length is the Euclidean straight-line distance between the nodes. This ratio reflects the spatial efficiency of the path relative to straight-line flight; the more curved the path, the larger the ratio.

[0067] Wind field intensity parameters from real-time meteorological data are incorporated to assess the impact of environmental disturbances. Wind field data typically originates from airborne weather radar scans or ground-based weather forecasts received via data links. Its gradient distribution in three-dimensional space is constructed into a continuous wind vector field using spatial interpolation algorithms. For each edge in the topology network, the system samples multiple points along its path, calculating the dot product between the wind vector and the unit vector of the aircraft's expected flight direction (path tangent direction) at each sampling point. The sign and magnitude of this dot product are crucial: negative values ​​indicate headwind (drag), positive values ​​indicate tailwind (assistance), and absolute values ​​indicate the degree of influence. The additional drag impact factor is calculated based on a weighted average of these dot product values, and its mathematical expression is:

[0068]

[0069] Where: symbol This represents the calculated additional drag influence factor. (Symbol) This is a scaling factor, a normal number, used to adapt the degree of wind influence to the order of magnitude of the weighting system. (Symbol) This represents the total arc length of the edge in the currently calculated topological network. (Integral variable) This represents the arc length parameter along the path of this edge. (Symbol) It is a vector function, representing the path arc length as... The three-dimensional wind vector at the location is obtained from meteorological data. (Symbol) Also a vector function, it represents the path arc length as... The unit vector in the tangential direction of the path at the given location. Operator This represents the dot product operation between two vectors. The dot product result. It directly reflects the component of wind in the direction of flight. When its value is negative (headwind), after the negative sign before the formula, it ultimately affects... The contribution is positive, thus increasing the weight; when its value is positive (tailwind), it increases the weight. The contribution is negative, thus reducing the weight. The entire integral operation calculates the average influence of the tangential component of the wind field along the entire path and scales it using a coefficient.

[0070] The basic weight coefficients and the additional drag influence factor are fused through a weighted superposition model to generate the final dynamic edge weights. The weight coefficients in the weighted superposition model are configured according to the specific aerodynamic characteristics of the aircraft and mission priorities. The dynamic edge weight matrix ultimately constructs a graph network model that reflects the flight cost in real time in the current environment. This model enables path search to actively avoid high-drag regions and favor assist regions. On the topology network with updated weights, path search uses an improved Dijkstra algorithm to find the top K shortest paths from the aircraft's current position to the preset landing point. The value of K is determined based on available computing resources and path diversity requirements. The standard Dijkstra algorithm typically only finds a single shortest path. The improvement lies in that the algorithm maintains a priority queue of size K, recording the top K optimal paths to the target point found so far. When a new path is discovered, it is compared and selected with the existing paths in the queue to maintain path diversity.

[0071] After generating the path set, the system performs wind speed field simulation tests to further evaluate its wind resistance capability. The simulation process mimics the aircraft flying along each candidate path at a typical cruise speed, densely sampling points along the path to analyze the continuous impact of the wind field at each point on the aircraft. The system presets a continuous headwind distance threshold, which is related to the aircraft's engine performance margin and fuel endurance. Any path containing a continuous headwind segment exceeding this threshold is eliminated, and the remaining paths constitute the candidate return path set. The paths in this set may differ in spatial distribution, but all meet basic flight safety and feasibility requirements, providing input for subsequent multi-objective optimization. The entire dynamic weight allocation and candidate path generation process is a closed loop; as the aircraft moves and weather conditions update, the weights of the topology network and the candidate path set are periodically refreshed.

[0072] Example 4: A multi-objective optimization algorithm is used to evaluate the feasibility of the candidate return path set and finally output the optimal path that meets the constraints. Assume that after the above screening, the system obtains four candidate return paths with wind resistance capabilities, forming a path set to be evaluated. The evaluation process requires establishing a multi-dimensional objective function space, whose core variables include fuel efficiency, path length, and risk coefficient. Fuel efficiency estimation relies on a fuel consumption rate integral model based on the aircraft's aerodynamic model. This model considers the altitude, airspeed, and local wind field conditions at each point on the path to calculate the total fuel consumption required to complete each path. Obtaining the path length is relatively straightforward; it is the sum of the dynamic weights of all constituent edges of the candidate path in the topology network. This length is the actual flight distance after wind field correction. The risk coefficient assessment is more comprehensive. It analyzes multiple risk sources, such as the minimum distance between the path and known obstacles (e.g., terrain, buildings), the intensity and probability of meteorological activities around the path (e.g., turbulent areas, thunderstorms), and the duration of the path segment in the communication blind spot, and finally integrates them into a normalized scalar value. The higher the risk coefficient, the greater the uncertainty of flight safety.

[0073] In multi-objective optimization problems, a path that performs well in one objective may have weaknesses in another. Therefore, a method is needed to identify solutions that achieve a good balance among multiple objectives. Pareto front analysis is used for this purpose, and its core is non-dominated ranking. Specifically, in a three-dimensional objective space consisting of fuel consumption, path length, and risk coefficient, if path A is no worse than path B in both fuel consumption and path length, and is strictly superior to path B in at least one of these indicators, while having a risk coefficient no higher than B, then path A is said to dominate path B. Through this pairwise comparison, paths not dominated by any other path can be selected from the candidate set. The set of these paths is called the non-dominated solution set or Pareto front, representing the optimal trade-off set achievable under current technological conditions. Referring to Table 1, assume the evaluation parameters of the current candidate paths are as follows.

[0074] Table 1: Evaluation Parameters for Candidate Return Paths

[0075]

[0076] Based on the data in the table above, a non-dominated ranking analysis is performed. Path A is superior to Path C in both fuel consumption and risk coefficient, and their path lengths are similar; therefore, Path A dominates Path C. Path B is inferior to Path A in fuel consumption but shorter in path length, and its risk coefficient is similar to that of Path A; there is no dominance relationship between Path A and Path B. Although Path D has the lowest fuel consumption, its path length and flight time are the longest; its comparison with other paths depends on specific constraints. After initial screening, Path A and Path B are usually included in the non-dominated solution set, while Path C is likely to be excluded due to its significantly higher risk coefficient. The fate of Path D depends on subsequent hard constraints. Next, the system introduces the hard constraint of the current remaining fuel amount for secondary filtering; assuming the aircraft currently has 165 kg of remaining fuel and a safe fuel reserve of 10 kg, the fuel budget available for the return flight is 155 kg. Based on this condition, route D is retained in the optimized route subset because its estimated fuel consumption (140 kg) is lower than the budget; routes A (145 kg) and B (158 kg) also meet the requirements; while route C's fuel consumption (162 kg) exceeds the budget and is therefore ultimately eliminated. The optimized route subset includes routes A, B, and D.

[0077] The decision to output the final optimal return path from the optimized path subset is hierarchical. The system attempts to select the path with the highest fuel consumption rate matching the current engine operating conditions as the first preferred option. The matching degree evaluation examines whether the engine's thrust requirements in each flight phase of the path fall within its highest efficiency range, avoiding long-term inefficient operation. Assuming that the flight profile (altitude and speed changes) of path A has the highest degree of matching with the engine's optimal operating envelope, it is initially selected as the first option. However, the system immediately checks whether the estimated flight time (48 minutes) of this option exceeds a preset threshold. Assuming that the mission commander sets the maximum allowable return flight time threshold to be 50 minutes, then the path A option is feasible, and the system will directly discretize it into a waypoint command sequence and issue it to the flight control system.

[0078] In another scenario, where the preset flight time threshold is set to 45 minutes (e.g., due to rapidly deteriorating weather requiring immediate landing), both Path A (48 minutes) and Path D (52 minutes) exceed the time limit. In this case, the system activates an alternative mechanism. This mechanism selects the shortest path from those that do not exceed the time limit (in this example, only Path B, 46 minutes) with a risk factor below the warning value (assuming a warning value of 0.45). Path B meets all these conditions and is therefore selected as the alternative output. The path discretization process transforms the selected three-dimensional continuous path into a series of waypoint instructions containing latitude, longitude, altitude, target airspeed, and heading angle. The interval between each waypoint is set according to the accuracy requirements of the flight management system. The final instruction sequence is sent to the flight control system via the aviation data bus, guiding the aircraft back along the optimized path. The entire evaluation and decision-making process is highly automated, capable of comprehensively weighing the three key factors of fuel, time, and safety within seconds to generate the most reliable return route under the current conditions.

[0079] Example 5: Continuous monitoring of system status and path replanning mechanism in abnormal situations during the return path execution. While the aircraft flies along the optimal path, the onboard visual sensors and inertial measurement unit (IMU) do not cease operation but continuously generate new observation data. The system monitors the difference between the actual flight state and the expected model through a high-frequency comparison process. This process compares real-time visual observation data with the scene predicted for the next moment based on an established 3D environmental kinematic model. For example, the visual system calculates the real-time optical flow field to predict the expected position of specific ground feature points in the image within a short future time window. When the aircraft actually arrives at that moment, the system recaptures the image and detects the actual positions of these feature points. The pixel deviation between the two is transformed into the position-level prediction deviation. Data from the IMU is also used to construct a short-term trajectory simulation for cross-validation with the visual prediction results.

[0080] These deviations are continuously accumulated, forming a cumulative deviation value. The system sets a fault tolerance threshold for this cumulative deviation. The fault tolerance threshold is not fixed; it is a dynamically adjusted value, and its magnitude is positively correlated with the aircraft's current speed, altitude, and the complexity of the terrain. For example, when flying at low altitudes and high speeds over undulating mountainous areas, the uncertainty of model predictions is naturally higher, and the fault tolerance threshold will be relaxed accordingly to avoid overly sensitive false alarms. Conversely, during high-altitude stable cruise, the threshold will be tightened. When the system determines, through a sliding window averaging algorithm, that the cumulative deviation exceeds the fault tolerance threshold for the current stage, it triggers a path replanning process. This triggering means that there is a significant difference between the original path assumptions and the actual situation, which may stem from unforeseen strong wind shear, local terrain changes, or the slow drift of the sensors themselves.

[0081] The path replanning employs an incremental update strategy. Its core idea is to maximize the use of previously planned effective portions, recalculating only areas where problems have occurred or where the environment has changed. This ensures computational efficiency in the planning process and meets the real-time requirements of flight. The system analyzes the current flight trajectory, locating the spatial starting point where the cumulative deviation begins to increase significantly, and marks this point as the deviation source. Subsequently, the system divides the original path into affected and unaffected segments. The path from the deviation source to the current flight position is marked as unreliable and discarded because the environmental model it relies on no longer matches reality. However, among the remaining paths from the current flight position to the preset landing point, those path segments that are sufficiently far from the deviation source and are naturally isolated from it (such as by mountains) are considered less affected by current local environmental changes and are retained.

[0082] This partitioning method avoids the complete rejection of the problem, saving significant computational resources. Local topology network reconstruction unfolds within a predetermined radius around the deviation source point. Determining this radius requires consideration of the aircraft's perception range, maneuverability, and computational latency; typically, it's a fan-shaped area extending forward and laterally from the current flight position. The system immediately mobilizes the onboard visual sensor array to perform a high-priority scan of this area, acquiring the latest multi-view image sequences. These image data undergo rapid but necessary spatiotemporal alignment and feature extraction processing to generate a high-precision temporary environment model focused on this local region. This temporary model may exist in the form of a 3D point cloud or an occupancy raster map, depicting the most realistic terrain and obstacle distribution in the area at the current moment.

[0083] The newly developed temporary environment model is seamlessly stitched together with the preserved original global topology network. This stitching process requires finding common feature anchor points at the boundary between the two models. These anchor points might be prominent features such as mountain peaks, river confluences, or man-made structures, all of which have clear correspondences in both the original global model and the temporary local model. By matching these anchor points, the system calculates a coordinate transformation matrix, precisely aligning the coordinate system of the temporary local model to the global model. Like patching, the new temporary local model covers the outdated, corresponding areas in the original global model, while ensuring that the network connectivity at the stitching boundary is smooth and natural, without abrupt path breaks or illogical jumps.

[0084] On this generated hybrid topology, the system runs an optimized real-time path planning algorithm. This algorithm begins working upon receiving the hybrid topology, which is a concatenation of the temporary environment model and the original topology network. Its core task is to find a safe and feasible path from the aircraft's instantaneous position to a preset intermediate target point within a very short time. During the algorithm's initialization phase, the hybrid topology is simplified based on an important spatial assumption: that the aircraft, within a local replanning region, requires a path that can quickly revert to the main flight path, rather than a globally optimal path. Therefore, the algorithm creates a search subgraph containing only nodes and edges within a certain radius of the current flight position. The radius of this radius matches the aircraft's speed and the system's maximum allowable decision delay, thereby reducing the search space and improving computational efficiency. On the simplified search subgraph, the algorithm employs a variant of the heuristic search strategy for path exploration. The heuristic function of this strategy is specially designed to consider not only the geometric straight-line distance between nodes and the target point but also incorporate a lightweight risk estimation term. This risk estimation term is quickly calculated by querying the surrounding environmental attributes of nodes stored in the hybrid topology (such as average terrain height and known obstacle density), guiding the search process to prioritize exploring safer areas. The algorithm maintains a priority queue to manage nodes to be expanded. The queue is ordered based on the sum of the known path cost and the heuristic function estimate for each node. At each iteration, the node with the lowest cost is selected from the queue for expansion.

[0085] The node expansion process involves querying the hybrid topology to obtain all adjacent edges and corresponding neighboring nodes of the current node. For each adjacent edge, the algorithm obtains its latest dynamic weight in real time. The calculation of the dynamic weight integrates the latest local wind field information and the terrain complexity factor of the region where the edge is located. The path cost of a neighboring node is obtained by adding the dynamic weight of the connecting edge to the cumulative cost of the current node. If this cost is lower than the known best cost of the neighboring node, the cost value of the neighboring node is updated and it is added back to the priority queue. At the same time, the optimal parent node information for reaching the neighboring node is recorded for backtracking and reconstruction of the final path. To meet real-time requirements, the algorithm introduces an early termination mechanism. This mechanism does not necessarily end the search only after finding the exact intermediate target point. When the distance between the expanded node and the intermediate target point is less than a dynamic threshold, and the currently constructed path segment satisfies the basic maneuver constraints of the aircraft (such as curvature continuity), the algorithm can terminate the search early and connect the current path with the remaining original route to form a complete solution. The final output path undergoes smoothness verification to ensure that its curvature change is within the aircraft's maneuverability range, and is converted into a waypoint command sequence executable by the flight control system for output. The entire algorithm process balances path quality and planning speed with limited computing resources, achieving rapid response capability in the face of sudden environmental changes.

[0086] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0087] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for constructing an aircraft return path based on visual intelligence, characterized in that, Includes the following steps: The flight environment is acquired by an airborne visual sensor through a multi-view image sequence. The multi-view image sequence is then spatiotemporally aligned to generate visual observation data with a unified timestamp and spatial coordinates. Based on the continuous inter-frame optical flow characteristics in the visual observation data, the motion vector field of the aircraft relative to the ground reference is calculated, and combined with the attitude data output by the inertial measurement unit, a three-dimensional environmental kinematic model is established. The current flight path of the aircraft is reverse-engineered using the three-dimensional environmental kinematic model. The set of key waypoints in the historical flight path is extracted, and the initial topology network of the return path is constructed based on the spatial distribution density of the set of key waypoints. Dynamic weight allocation is performed on the node connection relationships in the initial topology network, and combined with the wind field intensity parameters in real-time meteorological data, a set of candidate return paths with wind disturbance resistance is generated. A multi-objective optimization algorithm is used to evaluate the feasibility of the candidate return route set and output the optimal return route that meets the fuel consumption threshold and flight time constraints.

2. The method for constructing an aircraft return path based on visual intelligence according to claim 1, characterized in that, Spatiotemporal alignment processing of the multi-view image sequence includes: Significant edge feature points are detected in each frame of the multi-view image sequence, and cross-view feature correspondence is established through feature descriptor matching; Based on the aforementioned feature correspondence, the homography transformation matrix between adjacent viewpoints is calculated, and images from different viewpoints are projected onto the same virtual observation plane. A sliding window mechanism is used to perform inter-frame consistency verification in the time dimension of the projected image sequence, eliminating the spatiotemporal misalignment caused by sensor delay.

3. The method for constructing an aircraft return path based on visual intelligence according to claim 2, characterized in that, The establishment of the three-dimensional environmental kinematic model includes: Based on the visual observation data of the unified timestamp and spatial coordinates, a spherical coordinate system centered on the aircraft is constructed; By integrating the optical flow characteristics between consecutive frames in the spherical coordinate system, the displacement increments of the aircraft in three degrees of freedom are obtained. The displacement increment and the angular velocity data output by the inertial measurement unit are fused using Kalman filtering to calculate the six-degree-of-freedom motion trajectory of the aircraft in the global coordinate system.

4. The method for constructing an aircraft return path based on visual intelligence according to claim 3, characterized in that, The initial topology network for constructing the return path includes: Extract the heading change points from the six-degree-of-freedom motion trajectory as key waypoints, and calculate the curvature continuity index between adjacent key waypoints; Using the curvature continuity index as a constraint, a smooth curve segment connecting key waypoints is generated using piecewise cubic spline interpolation. The intersection points of the smooth curve segments are used as nodes of the topological network, and the curve segments themselves are used as edges, forming an initial topological network with a weighted directed graph structure.

5. The method for constructing an aircraft return path based on visual intelligence according to claim 4, characterized in that, The dynamic weight allocation of node connection relationships in the initial topology network includes: The ratio of the actual length of each edge to the theoretical shortest path length in the initial topology network is obtained and used as the basic weight coefficient. Based on the gradient distribution of wind field intensity in three-dimensional space from real-time meteorological data, the additional drag influence factor of the wind field on each side is calculated. The basic weight coefficients and the additional resistance influence factor are weighted and superimposed to generate a dynamic edge weight matrix that takes into account environmental disturbances.

6. The method for constructing an aircraft return path based on visual intelligence according to claim 5, characterized in that, The generation of a set of candidate return paths with wind disturbance resistance includes: An improved Dijkstra algorithm is run on the dynamic edge weight matrix to search for the first K shortest paths from the current position to the preset landing point; For each shortest path, wind speed field simulation tests are performed. Paths with continuous headwinds exceeding a preset distance are eliminated, and the remaining paths are retained to form a set of candidate return paths.

7. The method for constructing an aircraft return path based on visual intelligence according to claim 6, characterized in that, The feasibility assessment of the candidate return path set using a multi-objective optimization algorithm includes: Establish an objective function space with fuel efficiency, route length, and risk coefficient as variables; Pareto front analysis is performed on the candidate return path set within the objective function space to filter out the non-dominated solution set; The non-dominated solution set is subjected to secondary constraint filtering based on the aircraft's current remaining fuel to obtain an optimized path subset that meets the conditions for safe return.

8. The method for constructing an aircraft return path based on visual intelligence according to claim 7, characterized in that, The optimal return path that satisfies the fuel consumption threshold and flight time constraints includes: The path with the highest matching degree between the fuel consumption rate and the current engine operating condition is selected from the optimized path subset as the first preferred solution; When the flight time of the first preferred option exceeds a preset threshold, the alternative mechanism is activated, and the path with the shortest path length and the risk coefficient lower than the warning value is selected as the alternative. The final selected return path is discretized into a sequence of waypoint commands and sent to the flight control system for execution.

9. The method for constructing an aircraft return path based on visual intelligence according to claim 8, characterized in that, Also includes: During path execution, the deviation between the visual observation data and the kinematic model prediction is continuously monitored. When the cumulative deviation exceeds the fault tolerance threshold, path replanning is triggered. The route replanning adopts an incremental update strategy, retaining the segments of the original route that are not affected by environmental changes, and only reconstructing the local topology network in areas where deviations exceed limits.

10. The method for constructing an aircraft return path based on visual intelligence according to claim 9, characterized in that, The local topology network reconstruction includes: Using the current deviation position as the center, visual data is re-acquired within a preset radius to establish a temporary environment model; The temporary environment model is seamlessly spliced ​​with the unaffected part of the original topology network to generate a hybrid topology structure; A real-time path optimization algorithm is run on the hybrid topology to output a corrected continuous return trajectory.

Images