Unmanned aerial vehicle path planning method and system using adaptive dynamic programming

Abstract

The application provides a kind of unmanned aerial vehicle path planning method and system based on adaptive dynamic programming. Wherein, the method comprises: calculating the sun azimuth according to the atmospheric polarization mode, and using the sun azimuth to compensate the heading drift of visual odometry, to generate the corrected unmanned aerial vehicle position information; according to the illumination invariance edge information, combined with convolutional neural network, the key landmarks in texture enhanced image are segmented, bidirectional rapid random tree is used to generate semantic navigation landmark point sequence in the constraint space of key landmarks; a model predictive controller is constructed according to the semantic navigation landmark point sequence; through the corrected position information, the flight path of the unmanned aerial vehicle is optimized by adaptive dynamic programming under the constraint of dynamic window, and the field of view angle and exposure parameter of the dual-band imaging unit are adjusted. The application improves the autonomous navigation accuracy and semantic understanding ability of the unmanned aerial vehicle in complex lighting environment, and realizes the anti-interference, high-precision autonomous path planning and flight control.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV path planning method and system based on adaptive dynamic programming. Background Technology

[0002] Autonomous navigation systems for unmanned aerial vehicles (UAVs) operating under complex lighting conditions face severe challenges. Specifically, this environment requires the UAV autonomous navigation system to possess the following capabilities: strong resistance to lighting interference to ensure the stability and reliability of visual perception, unaffected by drastic changes in lighting or low contrast; high-precision heading estimation and position correction capabilities to overcome heading drift errors accumulated by visual odometry or inertial measurement units in long-distance, complex environments; and intelligent dynamic path planning capabilities that incorporate semantic understanding, enabling real-time generation of safe, reachable flight paths that comply with high-level semantic constraints in urban or near-ground environments containing structured landmarks, and adaptively adapting to environmental changes.

[0003] Existing solutions provide an autonomous navigation system based on simultaneous localization and mapping (SLAM) of feature points. This system utilizes an airborne visible light camera for multi-sensor fusion to achieve real-time estimation of position and attitude, while simultaneously constructing a grid map by extracting and tracking visual feature points in the environment. Path planning typically employs sampling-based algorithms or optimization methods to select feasible paths that avoid obstacles within the grid map. The system may include basic illumination compensation algorithms to improve image quality.

[0004] Existing solutions suffer from the following main drawbacks: First, the visual feature points relied upon are prone to failure or mismatch under extreme lighting changes or homogeneous texture areas, leading to lost feature tracking, map degradation, or errors, severely impacting the stability and accuracy of perception and positioning. Second, in the absence of a global absolute reference or during long-term operation, the drift error of the heading angle accumulates continuously and is difficult to suppress effectively through internal sensors, ultimately resulting in inaccurate position estimation. Path planning based on grid maps struggles to effectively incorporate and understand structured semantic information in the environment, and the generated paths may not conform to actual navigation semantics or ignore available guidance structures. Furthermore, the system has limited adaptive adjustment capabilities to dynamic environmental changes. Summary of the Invention

[0005] This application provides a method and system for UAV path planning based on adaptive dynamic programming, which solves the problems of low UAV path planning accuracy caused by perception instability in extreme lighting and homogeneous environments, cumulative heading drift during long-term operation, lack of semantic guidance and environmental adaptation in path planning.

[0006] Firstly, this application provides a UAV path planning method based on adaptive dynamic programming, including:

[0007] Visible light image data and infrared image data of the target area are collected and fused by histogram matching to generate a texture-enhanced image. Illumination-invariant edge information is extracted from the texture-enhanced image.

[0008] The atmospheric polarization pattern is obtained by a polarized light navigation sensor, the solar azimuth angle is calculated based on the atmospheric polarization pattern, and the solar azimuth angle is used to compensate for the heading drift of the visual odometry to generate corrected UAV position information.

[0009] Based on the illumination-invariant edge information, combined with a convolutional neural network, key landmarks in the texture enhancement image are segmented. A bidirectional fast random tree is used to generate a semantic navigation landmark sequence within the constraint space of the key landmarks. The key landmarks include roads and buildings.

[0010] A model prediction controller is constructed based on the semantic navigation landmark sequence;

[0011] Using the model predictive controller and the corrected position information, the UAV's flight path is optimized through adaptive dynamic programming under dynamic window constraints, while the field of view and exposure parameters of the dual-band imaging unit are adjusted according to the ambient light intensity.

[0012] Optionally, the step of segmenting key landmarks in the texture enhancement image based on the illumination-invariant edge information and combining it with a convolutional neural network, and generating a semantic navigation landmark sequence within the constraint space of the key landmarks using a bidirectional fast random tree, includes:

[0013] Based on the texture-enhanced image and the illumination-invariant edge information, a pixel-level probability distribution is generated by fusing feature maps of the same scale during the encoding and decoding processes using a convolutional neural network.

[0014] Determine the regions belonging to the key landmarks in the pixel-level probability distribution, output the key landmark regions, and construct a constraint space based on the key landmark regions;

[0015] Two search trees are alternately expanded in the constrained space using a bidirectional fast random tree. The connectivity between the nearest nodes of the two trees is detected by sampling random spatial points until the two search trees are successfully connected.

[0016] Backtrack the path nodes where the two search trees are successfully connected, and extract the spatial coordinates of the path nodes;

[0017] The spatial coordinates are sorted according to the path order from the starting point of the two search trees to the target point of the two search trees, and the semantic navigation landmark sequence is output.

[0018] Optionally, constructing the constraint space based on the key landmark area includes:

[0019] Extract the road centerline and building outline from the key landmark areas;

[0020] Generate the geometric boundaries of key landmarks based on the road centerline and building outlines;

[0021] The geometric boundary is transformed into Euclidean space constraints to construct the constraint space.

[0022] Optionally, the step of constructing a model prediction controller based on the semantic navigation landmark sequence includes:

[0023] Map the semantic navigation landmark sequence to a reference trajectory;

[0024] Based on the current state vector of the UAV and a predefined motion model, a state prediction sequence in the prediction time domain is generated.

[0025] A composite cost function is defined based on the deviation between the state prediction sequence and the reference trajectory;

[0026] Within the feasible region of control input defined by the dynamic window, the optimal control input sequence is obtained by solving the composite cost function through rolling optimization.

[0027] Extract the first term of the optimal control input sequence and output it to the actuator of the UAV;

[0028] The current state vector is iteratively updated to form a closed-loop execution mechanism to construct a model predictive controller.

[0029] Optionally, the step of obtaining the optimal control input sequence by solving the composite cost function through rolling optimization within the control input feasible region defined by the dynamic window includes:

[0030] The boundary constraints of the dynamic window are calculated based on the real-time flight status of the UAV, and the feasible domain of the control input is generated.

[0031] Starting from the current control input sequence and aiming to minimize the composite cost function, an iterative search is performed within the feasible region of the control input to generate candidate cost values. The composite cost function includes a trajectory tracking accuracy term and a control input smoothing term, with different terms assigned corresponding weight coefficients.

[0032] The control input sequence with the smallest candidate cost value is selected as the optimal control input sequence.

[0033] Optionally, the step of performing an iterative search within the feasible region of the control inputs, starting from the current control input sequence and aiming to minimize the composite cost function, to generate candidate cost values ​​includes:

[0034] Based on a predefined kinematic model, an iterative search is performed within the feasible region of the control input, using the current control input sequence as the initial value, to generate a candidate state sequence.

[0035] Based on the candidate state sequence and the reference trajectory, the cumulative deviation is calculated;

[0036] Evaluate the change in control input relative to the current control input sequence of the candidate state sequence;

[0037] The cumulative deviation and the change in control input are fused to generate candidate cost values.

[0038] Secondly, this application provides a UAV path planning system based on adaptive dynamic programming, comprising:

[0039] The acquisition module is used to acquire visible light image data and infrared image data of the target area, and generate a texture-enhanced image by histogram matching and fusion, and extract illumination-invariant edge information from the texture-enhanced image;

[0040] The compensation module is used to obtain the atmospheric polarization pattern through the polarized light navigation sensor, calculate the solar azimuth angle based on the atmospheric polarization pattern, and use the solar azimuth angle to compensate for the heading drift of the visual odometry, thereby generating the corrected UAV position information.

[0041] The segmentation module is used to segment key landmarks in the texture enhancement image based on the illumination invariant edge information and in combination with a convolutional neural network, and to generate a semantic navigation landmark sequence within the constraint space of the key landmarks using a bidirectional fast random tree. The key landmarks include roads and buildings.

[0042] The construction module is used to construct a model prediction controller based on the semantic navigation landmark sequence;

[0043] The adjustment module is used to optimize the flight path of the UAV through adaptive dynamic programming under dynamic window constraints by using the model prediction controller and the corrected position information, while adjusting the field of view and exposure parameters of the dual-band imaging unit according to the ambient light intensity.

[0044] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a UAV path planning method based on adaptive dynamic programming as described in any of the first aspects.

[0045] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a UAV path planning method based on adaptive dynamic programming as described in any of the first aspects.

[0046] This application provides a method for UAV path planning based on adaptive dynamic programming. The method includes: acquiring visible light and infrared image data of a target area, fusing them through histogram matching to generate a texture-enhanced image, and extracting illumination-invariant edge information from the texture-enhanced image; acquiring atmospheric polarization patterns using a polarized light navigation sensor, calculating the solar azimuth angle based on the atmospheric polarization pattern, and using the solar azimuth angle to compensate for the heading drift of the visual odometry, generating corrected UAV position information; segmenting key landmarks in the texture-enhanced image based on the illumination-invariant edge information and combining it with a convolutional neural network, and generating a semantic navigation landmark sequence within the constraint space of the key landmarks using a bidirectional fast random tree, where key landmarks include roads and buildings; constructing a model predictive controller based on the semantic navigation landmark sequence; and optimizing the UAV's flight path using adaptive dynamic programming under dynamic window constraints using the model predictive controller and the corrected position information, while simultaneously adjusting the field of view and exposure parameters of the dual-band imaging unit according to the ambient light intensity.

[0047] This application generates texture-enhanced images by fusing visible light and infrared image histograms, effectively overcoming illumination interference and extracting robust edge features with illumination invariance. It utilizes polarized light navigation to calculate the solar azimuth angle to compensate for visual odometry heading drift, suppressing long-term accumulated positioning errors and improving position accuracy. Based on illumination-invariant edge information, it drives a convolutional neural network to segment key landmarks and combines this with a bidirectional fast random tree to generate navigation landmark sequences within a semantically constrained space, achieving a structured understanding of the environment and accurate modeling of passable areas. A model prediction controller is constructed based on the semantic landmark sequence, fusing and correcting position information. Under dynamic window constraints, it uses adaptive dynamic programming to continuously optimize the flight path in real time, ensuring trajectory safety and global optimality. Simultaneously, it dynamically adjusts dual-band imaging parameters according to ambient illumination, ensuring perception quality under all operating conditions. Ultimately, it achieves high-precision, robust, and semantically-based autonomous navigation capabilities for UAVs in complex lighting environments.

[0048] Furthermore, based on texture-enhanced images and illumination-invariant edge information, a pixel-level probability distribution is generated by fusing feature maps of the same scale from the encoding-decoding process using a convolutional neural network. Key landmark areas in the probability distribution are identified, and geometric boundaries are constructed by extracting the road centerline and building outlines, which are then converted into Euclidean space constraints. Within the constraint space, two search trees are alternately expanded using bidirectional fast random trees. Random sampling and connectivity detection of the nearest nodes in both trees are used until a connection is successfully established and the path node coordinates are traced back. A semantic navigation landmark sequence is output in the order from the starting point to the target point.

[0049] Furthermore, the road sign sequence is mapped to a reference trajectory, and a state prediction sequence is generated based on the UAV state vector and motion model. A composite cost function is defined by combining the reference trajectory deviation. The optimal control input is solved through rolling optimization within the control input feasible region limited by a dynamic window. The first term is extracted and output to the actuator, and a closed-loop iterative model predictive controller is constructed. High-precision pixel-level segmentation of key landmarks is achieved through multi-scale feature fusion of convolutional neural networks, improving the robustness of road and building recognition in complex environments. Euclidean space constraints based on geometric boundaries effectively narrow the path search range. Combined with the alternating expansion and connection detection mechanism of bidirectional fast random trees, the efficiency of semantic road sign generation and path connectivity are significantly improved. Through reference trajectory mapping and rolling optimization mechanisms, the model predictive controller generates optimal control commands that conform to semantic constraints in real time under dynamic window constraints, ensuring the smoothness, safety, and adaptability of the UAV trajectory to dynamic environments, ultimately achieving synergistic optimization of navigation accuracy and intelligent decision-making.

[0050] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating an UAV path planning method based on adaptive dynamic programming, provided for an embodiment of this application;

[0053] Figure 2 A schematic diagram of the structure of an unmanned aerial vehicle (UAV) path planning system based on adaptive dynamic programming, provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0055] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0056] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 11, 12, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] To address the problems of low path planning accuracy in existing technologies, such as perception instability under extreme lighting and homogeneous environments, cumulative heading drift during long-term operation, and lack of semantic guidance and environmental adaptation in path planning, this application provides a UAV path planning method based on adaptive dynamic programming. This method employs the following concept: To address the perception drift, accumulated positioning errors, and poor path planning adaptability of UAV navigation under complex lighting conditions, the complementary characteristics of visible light and infrared spectra are fused at the perception layer. Histogram matching is used to generate texture-enhanced images and extract illumination-invariant edge features, overcoming the limitations of single-spectrum environmental adaptability. At the positioning layer, polarized light navigation is introduced as an independent absolute heading reference. The system calculates the solar azimuth angle in real time to compensate for visual odometry heading drift, effectively suppressing the accumulation of long-term positioning errors. At the decision level, it utilizes illumination-invariant edges to enhance the robustness of segmentation of key landmarks such as roads and buildings, transforming geometric boundaries into Euclidean constraint spaces. Based on bidirectional fast random trees, it efficiently generates a sequence of passable road signs within the semantic constraint domain. At the control level, it maps the semantic road sign sequence to a reference trajectory, constructs a model predictive controller, and performs rolling optimization under dynamic window constraints to solve for the optimal control command that minimizes the state-trajectory deviation. Simultaneously, it dynamically and adaptively adjusts dual-band imaging parameters according to real-time illumination intensity, ultimately forming a fully closed-loop autonomous navigation optimization system from perception enhancement, positioning correction, semantic planning to control execution.

[0059] Figure 1 A flowchart of an UAV path planning method based on adaptive dynamic programming is provided for embodiments of this application, as shown below. Figure 1 As shown, the method includes:

[0060] S11. Collect visible light image data and infrared image data of the target area, and generate a texture-enhanced image by histogram matching and fusion. Extract illumination-invariant edge information from the texture-enhanced image.

[0061] The target area refers to the specific geographical range that the UAV flight mission needs to cover, including urban blocks, wilderness terrain, and other navigational scenarios. Visible light image data is three-channel image data captured by visible light imaging equipment, reflecting the color and surface texture details of objects. Infrared image data is thermal radiation data collected by infrared sensors, used to characterize the temperature distribution and thermal properties of objects. Histogram matching is a technique that maps the grayscale histogram of the source image to the distribution of the target image, used to unify the contrast characteristics of images from different modalities. Texture-enhanced images are output images generated by fusing visible light and infrared images, possessing both detailed texture and thermal radiation information, enhancing the expressiveness of environmental features. Illumination-invariant edge information is edge feature extracted based on gradient operators, exhibiting stability to changes in illumination intensity and avoiding interference from shadows or strong light.

[0062] In this embodiment, firstly, visible light image data and infrared image data of the target area are simultaneously acquired by a dual-band imaging unit; secondly, histogram matching is performed on the two types of images to map the grayscale distribution of the infrared image to the histogram space of the visible light image, thereby achieving spectral feature fusion; then, a texture-enhanced image is generated based on the fusion result; finally, illumination-invariant edge information is extracted from the texture-enhanced image using a gradient operator, which is robust to interference such as shadows and strong light.

[0063] S12. Obtain the atmospheric polarization pattern through the polarized light navigation sensor, calculate the solar azimuth angle based on the atmospheric polarization pattern, and use the solar azimuth angle to compensate for the heading drift of the visual odometry, thereby generating the corrected UAV position information.

[0064] The polarization navigation sensor is an optical device that measures the atmospheric polarization state to interpret solar azimuth information. Atmospheric polarization pattern refers to the spatial distribution pattern of polarization angles formed by Rayleigh scattering from atmospheric particles, which is geometrically related to the solar azimuth angle. The solar azimuth angle is the horizontal direction angle of the sun measured clockwise with true north as the reference, providing a reference for absolute heading. Visual odometry is an algorithm that estimates the UAV's pose change through feature point matching between consecutive image frames. Heading drift refers to the heading angle error caused by accumulated feature mismatches in visual odometry. The corrected UAV position information is the UAV's three-dimensional coordinate data after compensating for heading drift with the solar azimuth angle. Optionally, the visual odometry and polarization navigation sensor are associated through timestamp alignment. When the polarization navigation sensor acquires the atmospheric polarization pattern, it synchronously records the timestamp and calculates the solar azimuth angle at that moment. Simultaneously, the visual odometry outputs a heading angle sequence with the same timestamp. Using the solar azimuth angle as an absolute reference, the heading drift at the corresponding moment is compensated, ultimately generating time-synchronized corrected position information.

[0065] In this embodiment, the atmospheric polarization pattern is first obtained by a polarized light navigation sensor; then, the polarization angle distribution of the atmospheric polarization pattern is calculated based on the Rayleigh scattering model to derive the solar azimuth angle; subsequently, the solar azimuth angle is used as the absolute heading reference to compensate for the heading drift caused by the accumulation of image matching errors in the visual odometry; finally, the corrected UAV position information is output to suppress long-term positioning deviation.

[0066] S13. Based on the illumination-invariant edge information and combined with a convolutional neural network, segment the key landmarks in the texture enhancement image. Use a bidirectional fast random tree to generate a semantic navigation landmark sequence within the constraint space of the key landmarks. The key landmarks include roads and buildings.

[0067] Here, convolutional neural networks refer to deep learning models that automatically extract image features using convolutional layers. Key landmarks refer to objects with structural recognizability in navigation, exemplified by roads and buildings. Bidirectional fast random tree refers to a path search algorithm that expands a random tree bidirectionally from the starting point and the target point until a connection is reached. Constraint space refers to the Euclidean geometric feasible region formed by the road centerline and the outer contour of buildings. Semantic navigation landmark sequence refers to the set of coordinates of key landmark pathpoints arranged in the order of passage. Roads refer to ground or air corridor areas accessible to drones. Buildings are obstacles or navigation references with vertical facade structures.

[0068] In this embodiment, illumination-invariant edge information is first input into a convolutional neural network. Multi-scale features are extracted by the encoder, and the same-scale feature maps are fused in the decoder to generate a pixel-level probability distribution. Next, key landmark areas belonging to roads and buildings in the probability distribution are determined. Then, the road centerline and building outline are extracted to construct a Euclidean geometric boundary constraint space. Finally, a bidirectional fast random tree is used to alternately expand two search trees in the constraint space. The nearest nodes of the two trees are connected by random sampling points until they are connected. The coordinates of the path nodes are traced back and the semantic navigation landmark sequence is output in sequence.

[0069] S14. Construct a model prediction controller based on the semantic navigation landmark sequence.

[0070] Among them, the model predictive controller refers to a feedback controller that solves for the optimal control quantity in the future time domain based on rolling optimization.

[0071] In this embodiment, the semantic navigation landmark sequence is first mapped to a reference trajectory in spatial order; then, the state sequence in the future time domain is predicted based on the UAV kinematic model; subsequently, a composite cost function consisting of the position deviation and velocity error between the state sequence and the reference trajectory is defined; finally, the minimum value of the cost function is solved by rolling within the control input range limited by the dynamic window to generate the optimal control command sequence.

[0072] S15. Using the model predictor controller and the corrected position information, the flight path of the UAV is optimized by adaptive dynamic programming under dynamic window constraints. At the same time, the field of view and exposure parameters of the dual-band imaging unit are adjusted according to the ambient light intensity.

[0073] The dynamic window is a feasible range of speed and acceleration defined in real time based on the dynamic constraints of the UAV. Rolling optimization is an iterative strategy that re-solves the finite-time optimal problem in each control cycle; the dynamic window constraint and the constraint space of key landmarks act on the planning and control layers, respectively. The flight path refers to the continuous motion trajectory of the UAV in three-dimensional space. Ambient light intensity refers to the visible light radiation energy intensity acting on the imaging unit. The dual-band imaging unit refers to an integrated camera module that simultaneously acquires visible light and infrared images. The field of view refers to the spatial angular range observable by the imaging device. Exposure parameters refer to the time and gain coefficient that control the amount of light entering the image sensor.

[0074] In this embodiment, the current state of the model prediction controller is first initialized based on the corrected UAV position information; then, under dynamic window constraints, the flight path in the future time domain is continuously optimized through an adaptive dynamic programming algorithm; subsequently, the first instruction of the optimal control sequence is extracted and sent to the actuator; finally, the field of view and exposure parameters of the dual-band imaging unit are adaptively adjusted according to the ambient light intensity to ensure imaging quality.

[0075] Here is a specific example: A drone performs an inspection mission on a city street with strong and changing lighting conditions, flying from a starting point (0,0) to a destination (100,0), executing the following process:

[0076] Dual cameras simultaneously capture visible light and infrared images. Histogram matching technology is used for fusion: first, the grayscale distribution of the infrared image is adjusted to match that of the visible light image, generating a fused image with clear texture and rich detail. Edge features unaffected by illumination are extracted from the fused image: the brightness variation intensity of each pixel in the horizontal and vertical directions is calculated, and stable contour lines are identified by combining the intensity values ​​in both directions, filtering out weak illumination interference. A polarization sensor captures the light distribution pattern in the atmosphere, calculating the precise direction of the sun in the UAV's coordinate system. After projecting this direction onto the horizontal plane, a fixed angle is calculated by comparing it with the geographic North Pole direction to establish a true north reference direction. Simultaneously, historical heading data from the visual odometry is analyzed to separate the heading deviation component accumulated over time. Reverse rotation compensation is performed based on the true north direction to eliminate heading errors. Finally, combined with the distance traveled measured by the visual odometry, precise 3D position coordinates (5, 0, 10) are output. The network fuses image features at different scales to generate a probability map of each pixel belonging to a road or building. A probability threshold is set to identify reliable landmark areas, extracting the center trajectory of road areas and the edge lines connecting buildings. A safe passage corridor is generated based on a road width of 10 meters, and a no-fly zone is set up 2 meters outside the building, jointly constructing a navigation constraint space. Within this space, two search trees are grown simultaneously from the start and end points. Spatial points are randomly selected to guide the expansion of branches, and when the distance between the nearest nodes of two trees is less than 1 meter, they are directly connected. The complete path nodes are traced back, and the sequence of critical waypoints arranged in an ordered manner from the start to the end point is output.

[0077] The waypoint sequence is converted into a smooth flight trajectory. Based on the UAV's current position (5,0,10), speed of 1 m / s, and heading of 0 degrees, the flight path for the next 5 seconds is predicted. A control objective function is set: 70% weight ensures accurate tracking of the planned path, and 30% weight ensures smooth and continuous control commands. Within physical limits, such as acceleration not exceeding ±2 m / s², the control scheme is repeatedly adjusted to select the optimal command sequence. The first acceleration command of 0.5 m / s² is extracted and sent to the propulsion system. The UAV is repositioned and control commands are updated every 0.1 seconds, forming a closed-loop control system. Simultaneously, imaging parameters are adjusted in real time according to light intensity: exposure time is shortened proportionally when light intensity increases (e.g., 1000 lux corresponds to 10 milliseconds), and the field of view is reduced to 70 degrees according to a preset relationship to ensure clear and usable images under various lighting conditions. The UAV flies along the planned path at the precise position (5,0,10) with an acceleration of 0.5 m / s², adaptively completing the inspection task in the shaded area of ​​the building.

[0078] By executing S11~S15, this embodiment of the application enhances the environmental texture representation by fusing visible light and infrared images, and effectively overcomes perception distortion and positioning drift caused by complex lighting interference by combining a polarized light heading correction mechanism; it generates a sequence of passable road signs that conform to the landmark topology by using semantic constraint path search, thereby improving the rationality of planning; and it achieves dynamic smooth optimization of flight trajectory and perception stability under all working conditions based on model predictive control and adaptive imaging adjustment, ultimately achieving a highly robust and low-drift closed-loop control for UAV autonomous navigation.

[0079] In one possible embodiment, S13, based on illumination-invariant edge information and combined with a convolutional neural network, key landmarks in the texture enhancement image are segmented, and a semantic navigation landmark sequence is generated within the constraint space of the key landmarks using a bidirectional fast random tree, including:

[0080] Step 131: Based on the texture enhancement image and the illumination invariant edge information, a pixel-level probability distribution is generated by fusing feature maps of the same scale from the encoding and decoding processes using a convolutional neural network.

[0081] In this context, the encoding process refers to the downsampling stage in a convolutional neural network, where convolutional and pooling layers progressively compress the image size and extract abstract features. The decoding process refers to the upsampling stage, where transposed convolutional layers progressively restore the feature map resolution and fuse it with the encoded features. A feature map is a multidimensional data tensor output by each layer of a convolutional neural network, reflecting the local feature responses of the input image. Pixel-level probability distribution refers to the probability matrix of each pixel belonging to different categories.

[0082] In this embodiment, the texture enhancement image and illumination-invariant edge information are first input into a convolutional neural network; then, during the encoding process, multi-scale features are extracted by downsampling through convolutional layers; subsequently, during the decoding process, the feature map is upsampled to the same scale as the encoding stage and fused with the corresponding layer's encoded feature map through channel overlay; finally, a classifier is used to generate a probability distribution map of each pixel belonging to a key landmark.

[0083] Step 132: Determine the regions belonging to key landmarks in the pixel-level probability distribution, output the key landmark regions, and construct a constraint space based on the key landmark regions.

[0084] Among them, the key landmark area refers to the set of connected pixels in the probability distribution that exceed a set threshold, corresponding to the actual image area of ​​the road or building.

[0085] In this embodiment, a probability threshold is first set based on the pixel-level probability distribution; then, pixel regions exceeding the threshold are determined to belong to key landmarks of roads or buildings; subsequently, the geometric center line and outer contour of the key landmark region are extracted; finally, the contour boundary is transformed into a feasible region constraint in Euclidean space to construct a constraint space for navigation path search.

[0086] Step 133: Use a bidirectional fast random tree to alternately expand the two search trees in the constrained space. Sampling and detecting the connectivity of the nearest nodes in the two trees through random space points until the two search trees are successfully connected.

[0087] In this context, a search tree refers to the tree-like data structure used in the bidirectional fast random tree algorithm to store path nodes and their connections. Random space point sampling refers to the probabilistic selection operation of randomly generating candidate path points within a constrained space. The nearest node connectivity in two trees refers to detecting whether the geometrically closest nodes in two search trees can be connected without obstacles by a straight line.

[0088] In this embodiment, two search trees growing from the starting point and the target point are first initialized in the constraint space. Then, the two trees are expanded alternately: spatial points are randomly sampled and the nearest node in each tree is selected to extend a new path segment to that point. Subsequently, it is detected in real time whether there is an unobstructed straight path between the nodes with the closest geometric distance in the two trees. Finally, the expansion is terminated when the nearest nodes of the two trees are successfully connected.

[0089] Step 134: Backtrack the path nodes where the two search trees are successfully connected and extract the spatial coordinates of the path nodes.

[0090] In this context, a node refers to a data unit in the search tree that stores the coordinates of path points and pointers to their parent nodes. Connectivity refers to the Boolean condition for determining whether a straight path between nodes lies within the constraint space and does not intersect with obstacles. Path nodes refer to the coordinates of key turning points that constitute a complete navigation path. Spatial coordinates refer to three-dimensional position data based on the UAV navigation coordinate system.

[0091] In this embodiment, the process begins with the nearest node in the successfully connected twin trees; then, it recursively backtracks along the parent node of the search tree to the starting point and the target point; subsequently, it extracts the three-dimensional spatial coordinates of all nodes in the backtracking path; and finally, it records the coordinate set in the backtracking order.

[0092] Step 135: Sort the spatial coordinates according to the path order from the starting point of the two search trees to the target point of the two search trees, and output the semantic navigation landmark sequence.

[0093] The starting point refers to the initial position of the path search, corresponding to the current position of the UAV. The target point refers to the ending position of the path search, corresponding to the UAV's mission objective point.

[0094] In this embodiment, the coordinates of the path nodes from the starting point to the connection point are first sorted in the forward direction; then the coordinates of the path nodes from the target point to the connection point are arranged in reverse order; then the coordinate sequences of the two ordered paths are merged; finally, the semantic navigation landmark sequence extending continuously from the starting point to the target point is output.

[0095] Here is a specific example: First, texture-enhanced images of urban blocks and illumination-invariant edges are input into a convolutional neural network. Feature maps of the same scale from the encoding and decoding layers are fused to generate a pixel-level landmark probability distribution. Second, road and building areas with probabilities exceeding a threshold are extracted, and a constraint space is constructed based on the road centerline and building outline. Then, within the constraint space, two search trees are simultaneously expanded from the current location of the drone and the inspection target point. Random sampling points guide the extension of branches and detect the linear connectivity of the nearest nodes in the two trees until the two trees are connected. Next, the coordinates of all path nodes are extracted by traversing back from the connecting nodes to the starting point and the target point. Finally, the coordinates are sorted according to the passage order from the starting point to the target point, and a semantic navigation sign sequence for safe passage along the building edge is output.

[0096] By executing steps 131 to 135, this embodiment of the application improves the segmentation accuracy of key landmarks through multi-scale feature fusion and narrows the path search range by utilizing geometric constraint space; bidirectional random trees efficiently generate connected paths within the semantically feasible domain, ensuring that the landmark sequence conforms to the actual travel logic, and ultimately achieving the synergistic optimization of the robustness and rationality of navigation paths in complex environments.

[0097] In one possible embodiment, step 132, constructing a constraint space based on key landmark areas, includes:

[0098] Step a1: Extract the road centerline and building outline from the key landmark areas.

[0099] The road centerline refers to the central trajectory line of the road area along the direction of travel, extracted using a skeletonization algorithm, reflecting the guide axis of the passable area. The building outline is a closed polygon formed by the vertices of the building's edges, obtained through edge detection and fitting, representing the geometric boundary of the obstacle's occupancy.

[0100] In this embodiment, firstly, morphological skeletonization is performed on the road area in the key landmark area to extract the road centerline; secondly, edge tracking and polygon fitting are performed on the building area to extract the building outline; finally, the coordinate set of the road centerline and the vertex sequence of the building outline polygon are output.

[0101] Step a2: Generate the geometric boundaries of key landmarks based on the road centerline and the building outline.

[0102] Among them, the geometric boundary can refer to the spatial topology that integrates the road centerline guidance area and the no-fly zone of the building's outer contour, and is used to define the mathematical expression of the navigation feasible domain and the obstacle zone.

[0103] In this embodiment, the road centerline is first defined as a set of continuous points at predefined positions in the width direction; then the building outline is processed into a closed polygon boundary; subsequently, the road centerline and the building outline polygon are merged; finally, the spatial geometric boundary describing the passable area and no-fly zone of key landmarks is generated.

[0104] Step a3: Transform the geometric boundary into Euclidean space constraints to construct the constraint space.

[0105] Euclidean space constraints transform geometric boundaries into a set of linear inequalities in a three-dimensional Cartesian coordinate system, which are used to limit the range of coordinate values ​​of path points.

[0106] In this embodiment, the specific process of transforming the geometric boundary into Euclidean space constraints is as follows: First, the road width is expanded by a set ratio along the normal direction of the road centerline to generate a strip-shaped traffic corridor, which is transformed into a linear inequality constraint. Second, the outer contour of the building is expanded outward by a safe distance to form a no-fly expansion zone, which is transformed into a nonlinear obstacle constraint. Finally, the mathematical expressions of the traffic corridor and the no-fly zone are integrated to construct a hybrid constraint set containing position and distance restrictions, and to define the feasible region of path search.

[0107] In this embodiment, the road centerline in the geometric boundary is first transformed into a width-constrained corridor; the outer contour of the building is transformed into an obstacle expansion zone; a buffer zone is then set based on the safe distance of the UAV; and finally, the corridor and expansion zone are mapped into inequality constraints in a three-dimensional Euclidean coordinate system to construct a constraint space for path search.

[0108] Here is a specific example: First, the centerline of the main road and the outer contours of the buildings on both sides are extracted from the key landmark areas of the city block; second, the centerline of the road is processed into a five-meter-wide passageway, and the outer contours of the buildings are extended outward by two meters to generate a safety buffer zone; then, the passageway is defined as a linear constraint of the lower and upper limits of the coordinates, and the safety buffer zone is defined as an obstacle inequality; finally, all constraints are integrated to generate a Euclidean space constraint model for UAV path search, ensuring that the subsequent planned path flies strictly along the center of the road and stays away from building obstacles.

[0109] By executing steps a1 to a3, this embodiment of the application constructs a structured constraint space by accurately extracting the geometric features of roads and buildings, transforming semantic information into a mathematically feasible domain, providing physical rationality assurance for path search, and avoiding planned paths from intruding into obstacle areas or deviating from traffic corridors.

[0110] In one possible embodiment, S14, constructing a model prediction controller based on the semantic navigation landmark sequence, includes:

[0111] Step 141: Map the semantic navigation landmark sequence to a reference trajectory.

[0112] The reference trajectory is a continuous path curve generated from a sequence of semantic navigation landmarks, which serves as the target space trajectory tracked by the controller.

[0113] In this embodiment, the semantic navigation landmark sequence is first connected into a continuous polyline in spatial order; then, a smooth reference trajectory is generated by cubic spline interpolation; finally, the discrete-time position function of the trajectory is output.

[0114] Step 142: Based on the current state vector of the UAV and the predefined motion model, generate a state prediction sequence in the prediction time domain.

[0115] The current state vector is a set of parameters describing the instantaneous motion state of the UAV, including its three-dimensional position, linear velocity, heading angle, and angular velocity. The predefined motion model refers to a set of differential equations established based on the UAV's dynamic characteristics, used to predict the state evolution process. The prediction time domain refers to the time range within which the model predicts the controller's forward simulation of the state evolution. The state prediction sequence refers to the set of future state vectors discretized step-by-step within the prediction time domain.

[0116] In this embodiment, the current state vector of the UAV, including position, velocity, and heading angle, is first obtained; second, based on predefined kinematic differential equations, the state transition is iteratively calculated in the prediction time domain; and finally, a state prediction sequence for multiple future time steps is generated. It should be noted that this embodiment does not specifically limit the specific expression of the above equations.

[0117] Step 143: Define the composite cost function based on the deviation between the state prediction sequence and the reference trajectory.

[0118] Here, deviation refers to the geometric or kinematic difference between each point in the state prediction sequence and the corresponding point in the reference trajectory. The composite cost function is a multi-objective optimization function that integrates positional deviation, angular deviation, and control smoothness. It should be noted that the embodiments of this application do not specifically limit the specific expression of the above functions.

[0119] In this embodiment, the Euclidean distance deviation between each position point in the state prediction sequence and the reference trajectory is first calculated; then the angular deviation between the velocity direction and the tangent vector of the reference trajectory is superimposed; finally, a composite cost function including position deviation, angular deviation and control variable change rate is defined.

[0120] Step 144: Within the feasible region of the control input defined by the dynamic window, solve the composite cost function through rolling optimization to obtain the optimal control input sequence.

[0121] The feasible region of control inputs refers to the range of acceleration and angular acceleration values ​​determined by the physical limits of the UAV. The optimal control input sequence refers to the set of control commands that minimizes the composite cost function within the prediction time domain.

[0122] In this embodiment, the dynamic window range is first defined based on the maximum acceleration of the UAV; then, candidate control sequences are sampled within the feasible region of the control input; subsequently, the minimum value of the composite cost function is solved using a numerical optimization algorithm; and finally, the optimal control input sequence is output.

[0123] Step 145: Extract the first term of the optimal control input sequence and output it to the actuator of the UAV.

[0124] The first term refers to the immediate control command corresponding to the current moment in the optimal control input sequence. The actuator refers to the drive device such as a motor or servo motor that translates the control command into physical actions.

[0125] In this embodiment, the control command for the first time step is first extracted from the optimal control input sequence; then, the command containing the speed and heading angle increments is sent to the motor and servo actuators; finally, the UAV is driven to perform instantaneous actions.

[0126] Step 146: Iteratively update the current state vector to form a closed-loop execution mechanism to build a model predictive controller.

[0127] Among them, the closed-loop execution mechanism refers to the iterative optimization process of updating the control strategy in real time based on state feedback.

[0128] In this embodiment, the new state of the UAV after execution is first obtained through sensors; then the new state is used as the current state vector for the next control cycle; subsequently, the prediction optimization and command output process is repeated; finally, a closed-loop execution mechanism is formed to continuously update the model prediction controller.

[0129] Here is a specific example: First, the semantic landmark sequence is interpolated as the reference trajectory for urban inspection; second, based on the current position and velocity of the UAV, the flight state sequence for the next five seconds is predicted using a motion model; then, the deviation between the predicted position and the reference trajectory is calculated, and a composite cost function including position error and turning cost is defined; next, within the dynamic acceleration constraint range, the optimal speed control sequence is solved using sequential quadratic programming; the first instruction is extracted and sent to the power system for execution; finally, the state is updated through the pose sensor and the next control cycle is entered, continuously adjusting the UAV to fly stably along the edge of the building.

[0130] By executing steps 141 to 146, this embodiment of the application achieves accurate tracking of semantic paths by the UAV through trajectory mapping and rolling optimization, combines dynamic window constraints to ensure the physical feasibility of control commands, and continuously corrects trajectory deviations caused by environmental disturbances through a closed-loop mechanism, thereby improving the stability and adaptability of autonomous flight.

[0131] In one possible embodiment, step 144, within the feasible region of the control input defined by the dynamic window, obtains the optimal control input sequence by solving the composite cost function through rolling optimization, including:

[0132] Step b1: Calculate the boundary constraints of the dynamic window based on the real-time flight status of the UAV, and generate the feasible domain of control input.

[0133] Real-time flight status refers to the set of motion parameters of the UAV at the current moment, including three-dimensional position, linear velocity, angular velocity, and attitude angle, which can be updated in real time through inertial measurement units and vision sensors. Boundary constraints are the range of values ​​of control variables determined by the physical limits of the UAV; for example, they can be hard constraints including maximum linear acceleration and maximum angular acceleration.

[0134] In this embodiment, the real-time flight status of the UAV is first obtained through sensors; then, the velocity boundary constraints and steering boundary constraints of the dynamic window are calculated based on the maximum acceleration and angular velocity limits; finally, the mathematical expression range of the control input feasible domain is generated.

[0135] Step b2: Starting from the current control input sequence and aiming to minimize the composite cost function, perform an iterative search within the feasible region of the control input to generate candidate cost values. The composite cost function includes a trajectory tracking accuracy term and a control input smoothing term, with different terms assigned corresponding weight coefficients.

[0136] Iterative search refers to a loop optimization process that repeatedly adjusts the control input value and evaluates the cost function within the feasible region to approximate the global optimum. The candidate cost value is the composite cost function value calculated in each iteration, reflecting the tracking performance and smoothness level of the current control sequence. The trajectory tracking accuracy term is the cost function term describing the positional deviation between the predicted trajectory and the reference trajectory, typically calculated using the sum of squared Euclidean distances. The control input smoothness term is the cost function term measuring the magnitude of changes in adjacent control commands, used to suppress actuator jitter, and calculated using the sum of squared differences in the control quantities. Weighting coefficients are adjustment parameters assigned to different cost terms, used to balance the priority relationship between trajectory tracking accuracy and control smoothness.

[0137] In this embodiment, the current control input sequence is first used as the initial solution; then, gradient descent is used to perform an iterative search within the feasible region of the control input; subsequently, the candidate cost value for each iteration is calculated, which is obtained by multiplying the trajectory tracking accuracy term and the control input smoothing term by their respective weight coefficients and then adding them together; finally, the candidate cost values ​​for all iteration steps are recorded.

[0138] Step b3: Select the control input sequence with the smallest candidate cost value as the optimal control input sequence.

[0139] In this embodiment, firstly, all candidate cost values ​​generated in the iterations are compared; secondly, the control input sequence corresponding to the candidate cost value with the smallest value is selected; and finally, this sequence is determined as the optimal control input sequence.

[0140] Here is a specific example: First, calculate the acceleration boundary constraints of the dynamic window based on the real-time speed and attitude of the UAV; second, starting from the current speed control sequence, perform gradient descent iterations within the feasible region, and calculate the weighted composite cost including the sum of squares of position deviation and the sum of squares of acceleration change in each iteration; finally, select the control sequence corresponding to the minimum cost to drive the UAV to fly smoothly along the edge of the building under strong wind disturbance.

[0141] By executing steps b1 to b3, the embodiments of this application ensure the physical feasibility of control commands through dynamic windows, achieve efficient optimization of composite cost functions by combining iterative search mechanisms, and generate optimal control commands that take into account both tracking accuracy and flight stability under strict constraints, thereby improving the trajectory tracking quality in complex environments.

[0142] In one possible embodiment, step b2, starting with the current control input sequence and aiming to minimize the composite cost function, performs an iterative search within the feasible region of the control input to generate candidate cost values, including:

[0143] Step b21: Based on the predefined kinematic model, within the feasible region of the control input, perform an iterative search with the current control input sequence as the initial value to generate a candidate state sequence.

[0144] The predefined kinematic model refers to a set of differential equations describing the evolution of the UAV's position, velocity, and heading angle over time. This model can be established based on Newtonian mechanics and aerodynamics and is used to predict state changes under control inputs. It should be noted that this embodiment does not specifically limit the exact expression of this set of equations. The initial value refers to the starting control sequence for the iterative search, typically initialized using the optimal solution output from the previous control cycle or a zero-input sequence. The candidate state sequence is a set of future state vectors derived from the candidate control sequences through the kinematic model, containing predicted values ​​for parameters such as position, velocity, and heading angle.

[0145] In this embodiment, the current control input sequence is first used as the initial value for iterative search; then, candidate control sequences are generated in the feasible domain of the control input based on a predefined kinematic model; subsequently, the candidate control sequences are input into the kinematic model for state forward propagation; finally, the candidate state sequence in the prediction time domain is output.

[0146] Step b22: Calculate the cumulative deviation based on the candidate state sequence and the reference trajectory.

[0147] The cumulative bias refers to the sum of the position tracking biases at all time steps in the prediction time domain, reflecting the overall trajectory deviation of the candidate control sequence.

[0148] In this embodiment, firstly, each position point in the candidate state sequence is matched with the corresponding point of the reference trajectory; secondly, the Euclidean distance deviation of each position point at each time step is calculated; then, the deviation values ​​of all time steps are summed; finally, the cumulative deviation value in the prediction time domain is generated.

[0149] Step b23: Evaluate the change in control input relative to the current control input sequence of the candidate state sequence.

[0150] Among them, the change in control input refers to the average change in acceleration and steering angular velocity of the candidate control sequence relative to the initial control sequence, which characterizes the smoothness level of the control command.

[0151] In this embodiment, the acceleration and steering angular velocity values ​​of the current control input sequence are first extracted; then the control quantities at the same time step in the candidate control sequence are obtained; then the absolute value of the change amplitude between adjacent control quantities is calculated; finally, the change amplitude of the control quantities at all time steps is averaged.

[0152] Step b24: Combine the cumulative deviation and the change in control input to generate candidate cost values.

[0153] In this embodiment, firstly, a trajectory tracking accuracy weighting coefficient is assigned to the cumulative deviation; secondly, a smoothness weighting coefficient is assigned to the control input change; then, the weighted cumulative deviation is added to the weighted control input change; finally, a candidate cost value reflecting the overall performance is output.

[0154] Here is a specific example: First, the semantic landmark sequence is interpolated as the reference trajectory for urban inspection; second, based on the current position and velocity of the UAV, the flight state sequence for the next five seconds is predicted using a motion model; then, the deviation between the predicted position and the reference trajectory is calculated, and a composite cost function including position error and turning cost is defined; next, within the dynamic acceleration constraint range, the optimal speed control sequence is solved using sequential quadratic programming; the first instruction is extracted and sent to the power system for execution; finally, the state is updated through the pose sensor and the next control cycle is entered, continuously adjusting the UAV to fly stably along the edge of the building.

[0155] By executing steps b21 to b24, this embodiment first generates a candidate acceleration command set based on the UAV kinematic model and the current control sequence, within the dynamic window constraints, and derives a candidate state sequence; secondly, it aligns each position point of the candidate sequence with the inspection reference trajectory and calculates the cumulative position deviation throughout the process; then, it compares the average acceleration change amplitude of the candidate sequence with that of the current control sequence; finally, it adds the weighted cumulative deviation and the weighted change amount to generate a candidate cost value, which is used to screen the control command with the best anti-disturbance performance under strong crosswind conditions.

[0156] In one possible embodiment, in S12, the corrected UAV position information is generated by compensating for the heading drift of the visual odometry using the solar azimuth angle, including:

[0157] Step 121: Based on the atmospheric polarization mode, calculate the three-dimensional projection of the solar vector in the carrier coordinate system.

[0158] Here, the solar vector refers to the geometric direction vector pointing from the observation point to the sun, representing the sun's absolute position in space. The carrier coordinate system refers to the local coordinate system of the moving body, with its origin located at the UAV's center of mass and its X-axis pointing towards the nose. The three-dimensional projection refers to the set of direction components after transforming the solar vector from the inertial coordinate system to the carrier coordinate system.

[0159] In this embodiment, the atmospheric polarization mode is first obtained by a polarized light navigation sensor; then the direction of the solar vector is analyzed based on the Rayleigh scattering model; subsequently, the solar vector is transformed from the inertial coordinate system to the UAV carrier coordinate system; finally, the three-dimensional projection component of the solar vector in the carrier coordinate system is calculated.

[0160] Step 122: Convert the 3D projection to a horizontal plane to generate a reference direction, calculate the fixed angle between the reference direction and the geographic North Pole direction, and determine the true north reference based on the fixed angle.

[0161] In this context, the horizontal plane refers to a two-dimensional plane perpendicular to the local direction of gravity, used to define the direction reference. The reference direction is the projection of the solar vector onto the horizontal plane, used to indirectly characterize the solar azimuth. The geographic North Pole direction is the projection of the Earth's rotational axis North Pole onto the horizontal plane, serving as the absolute geographic reference. The fixed angle refers to the constant azimuth difference between the reference direction and the geographic North Pole direction. The true north azimuth reference is an absolute heading reference system established based on the geographic North Pole direction.

[0162] In this embodiment, the three-dimensional projection components are first projected onto the horizontal plane to generate a reference direction vector; then, the geographic North Pole direction is obtained by querying the geographic information system; subsequently, the fixed angle between the reference direction and the geographic North Pole direction is calculated; finally, the true north reference is determined based on this angle.

[0163] Step 123: Separate the heading drift component from the heading angle sequence of the visual odometry.

[0164] Here, the heading angle sequence refers to the historical dataset of the UAV's nose heading angle continuously output by the visual odometry. The heading drift component refers to the low-frequency angular offset in the heading angle sequence caused by accumulated errors.

[0165] In this embodiment, the historical heading angle sequence output by the visual odometry is first obtained; then, a high-pass filter is used to separate the low-frequency drift component in the heading angle sequence; finally, the heading drift component is extracted as the quantity to be corrected.

[0166] Step 124: Using the true north reference as the rotation reference, perform a reverse rotation operation on the heading drift component and output the corrected heading angle.

[0167] Here, the rotation reference refers to the true north reference frame used for coordinate system transformation. The reverse rotation operation refers to a coordinate transformation that rotates the drift component in the opposite direction relative to the true north reference to compensate for the error. The corrected heading angle refers to the accurate nose heading angle relative to true north after drift has been eliminated.

[0168] In the embodiments of this application, a rotation matrix is ​​first established using the true north reference as the rotation reference; then, the heading drift component is rotated in reverse; subsequently, the rotated angle is superimposed on the original heading angle; finally, the corrected heading angle to eliminate the drift is output.

[0169] Step 125: Merge the corrected heading angle and the translational displacement of the visual odometry to generate the corrected UAV position information.

[0170] Among them, translational displacement refers to the change in three-dimensional spatial position generated by visual odometry.

[0171] In this embodiment, the translational displacement of the UAV is first obtained from the visual odometry; then the corrected heading angle is converted into a rotation matrix; subsequently, the translational displacement is converted into a navigation coordinate system based on true north; finally, the corrected heading and displacement are fused to generate corrected position information in three-dimensional space.

[0172] Here is a specific example: First, the solar vector projection in the carrier coordinate system is calculated based on polarization sensor data; second, the projection is projected onto the horizontal plane to generate a reference direction, and the fixed angle between it and the geographic North Pole is calculated to determine the true north reference; then, the low-frequency drift component is separated from the visual odometry heading sequence; next, the drift component is rotated inversely with the true north reference as the rotation axis to output the corrected heading angle; finally, the corrected heading and the translational displacement of the visual odometry are fused to generate the real-time UAV position with centimeter-level accuracy in urban environments rejected by the Global Positioning System.

[0173] By executing steps 121 to 125, this embodiment of the application obtains the true north reference through polarized light analysis, accurately separates and reverse-compensates the heading drift of the visual odometry, and combines displacement data fusion to generate position information that resists cumulative error, thereby improving the heading accuracy and position reliability of long-distance navigation.

[0174] Figure 2 A schematic diagram of a UAV path planning system based on adaptive dynamic programming is provided in an embodiment of this application, as shown below. Figure 2 As shown, the system includes:

[0175] The acquisition module 21 is used to acquire visible light image data and infrared image data of the target area, and generate a texture-enhanced image by histogram matching and fusion, and extract illumination-invariant edge information from the texture-enhanced image.

[0176] The compensation module 22 is used to obtain the atmospheric polarization mode through the polarized light navigation sensor, calculate the solar azimuth angle based on the atmospheric polarization mode, and use the solar azimuth angle to compensate for the heading drift of the visual odometry, thereby generating the corrected UAV position information.

[0177] The segmentation module 23 is used to segment key landmarks in the texture enhancement image based on illumination invariant edge information and combined with a convolutional neural network. It uses a bidirectional fast random tree to generate a semantic navigation landmark sequence within the constraint space of the key landmarks. The key landmarks include roads and buildings.

[0178] Module 24 is used to build a model prediction controller based on the semantic navigation landmark sequence.

[0179] The adjustment module 24 is used to predict the controller and corrected position information through the model, and to optimize the flight path of the UAV through adaptive dynamic programming under dynamic window constraints. At the same time, it adjusts the field of view and exposure parameters of the dual-band imaging unit according to the ambient light intensity.

[0180] Figure 2 The aforementioned UAV path planning system based on adaptive dynamic programming can execute... Figure 1 The implementation principle and technical effects of the UAV path planning method based on adaptive dynamic programming described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit performs operations in the UAV path planning system based on adaptive dynamic programming in the above embodiments have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0181] In one possible design, Figure 2 The UAV path planning system based on adaptive dynamic programming in the illustrated embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32.

[0182] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0183] The processing component 32 is used to: acquire visible light image data and infrared image data of the target area, and generate a texture-enhanced image through histogram matching and fusion; extract illumination-invariant edge information from the texture-enhanced image; acquire atmospheric polarization mode through a polarized light navigation sensor, calculate the solar azimuth angle based on the atmospheric polarization mode, and use the solar azimuth angle to compensate for the heading drift of the visual odometry, generating corrected UAV position information; based on the illumination-invariant edge information, combine a convolutional neural network to segment key landmarks in the texture-enhanced image, and use a bidirectional fast random tree to generate a semantic navigation landmark sequence within the constraint space of the key landmarks, including roads and buildings; construct a model prediction controller based on the semantic navigation landmark sequence; and, using the model prediction controller and the corrected position information, optimize the UAV's flight path through adaptive dynamic programming under dynamic window constraints, while adjusting the field of view and exposure parameters of the dual-band imaging unit according to the ambient light intensity.

[0184] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0185] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Random Access Memory (RAM), Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0186] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0187] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0188] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0189] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0190] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a UAV path planning method based on adaptive dynamic programming.

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0192] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for UAV path planning based on adaptive dynamic programming, characterized in that, include: Visible light image data and infrared image data of the target area are collected and fused by histogram matching to generate a texture-enhanced image. Illumination-invariant edge information is extracted from the texture-enhanced image. The atmospheric polarization pattern is obtained by a polarized light navigation sensor, the solar azimuth angle is calculated based on the atmospheric polarization pattern, and the solar azimuth angle is used to compensate for the heading drift of the visual odometry to generate corrected UAV position information. Based on the illumination-invariant edge information, combined with a convolutional neural network, key landmarks in the texture enhancement image are segmented. A bidirectional fast random tree is used to generate a semantic navigation landmark sequence within the constraint space of the key landmarks. The key landmarks include roads and buildings. A model prediction controller is constructed based on the semantic navigation landmark sequence; Using the model predictive controller and the corrected position information, the UAV's flight path is optimized through adaptive dynamic programming under dynamic window constraints, while the field of view and exposure parameters of the dual-band imaging unit are adjusted according to the ambient light intensity.

2. The method according to claim 1, characterized in that, The step of segmenting key landmarks in the texture enhancement image based on the illumination-invariant edge information and combining it with a convolutional neural network, and generating a semantic navigation landmark sequence within the constraint space of the key landmarks using a bidirectional fast random tree, includes: Based on the texture-enhanced image and the illumination-invariant edge information, a pixel-level probability distribution is generated by fusing feature maps of the same scale during the encoding and decoding processes using a convolutional neural network. Determine the regions belonging to the key landmarks in the pixel-level probability distribution, output the key landmark regions, and construct a constraint space based on the key landmark regions; Two search trees are alternately expanded in the constrained space using a bidirectional fast random tree. The connectivity between the nearest nodes of the two trees is detected by sampling random spatial points until the two search trees are successfully connected. Backtrack the path nodes where the two search trees are successfully connected, and extract the spatial coordinates of the path nodes; The spatial coordinates are sorted according to the path order from the starting point of the two search trees to the target point of the two search trees, and the semantic navigation landmark sequence is output.

3. The method according to claim 2, characterized in that, The construction of the constrained space based on the key landmark area includes: Extract the road centerline and building outline from the key landmark areas; Generate the geometric boundaries of key landmarks based on the road centerline and building outlines; The geometric boundary is transformed into Euclidean space constraints to construct the constraint space.

4. The method according to claim 1, characterized in that, The step of constructing a model prediction controller based on the semantic navigation landmark sequence includes: Map the semantic navigation landmark sequence to a reference trajectory; Based on the current state vector of the UAV and a predefined motion model, a state prediction sequence in the prediction time domain is generated. A composite cost function is defined based on the deviation between the state prediction sequence and the reference trajectory; Within the feasible region of control input defined by the dynamic window, the optimal control input sequence is obtained by solving the composite cost function through rolling optimization. Extract the first term of the optimal control input sequence and output it to the actuator of the UAV; The current state vector is iteratively updated to form a closed-loop execution mechanism to construct a model predictive controller.

5. The method according to claim 4, characterized in that, The step of obtaining the optimal control input sequence by solving the composite cost function through rolling optimization within the control input feasible region defined by the dynamic window includes: The boundary constraints of the dynamic window are calculated based on the real-time flight status of the UAV, and the feasible domain of the control input is generated. Starting from the current control input sequence and aiming to minimize the composite cost function, an iterative search is performed within the feasible region of the control input to generate candidate cost values. The composite cost function includes a trajectory tracking accuracy term and a control input smoothing term, with different terms assigned corresponding weight coefficients. The control input sequence with the smallest candidate cost value is selected as the optimal control input sequence.

6. The method according to claim 5, characterized in that, The step of generating candidate cost values ​​by performing an iterative search within the feasible region of the control inputs, starting from the current control input sequence and aiming to minimize the composite cost function, includes: Based on a predefined kinematic model, an iterative search is performed within the feasible region of the control input, using the current control input sequence as the initial value, to generate a candidate state sequence. Based on the candidate state sequence and the reference trajectory, the cumulative deviation is calculated; Evaluate the change in control input relative to the current control input sequence of the candidate state sequence; The cumulative deviation and the change in control input are fused to generate candidate cost values.

7. The method according to claim 1, characterized in that, The process of using the solar azimuth angle to compensate for the heading drift of the visual odometry and generating corrected UAV position information includes: Based on the atmospheric polarization mode, calculate the three-dimensional projection of the solar vector in the carrier coordinate system; The three-dimensional projection is converted to a horizontal plane to generate a reference direction. The fixed angle between the reference direction and the geographic North Pole direction is calculated. The true north reference is determined based on the fixed angle. Separate the heading drift component from the heading angle sequence of the visual odometry; Using the true north reference as the rotation reference, the heading drift component is rotated in reverse to output a corrected heading angle; The corrected UAV position information is generated by combining the corrected heading angle with the translational displacement of the visual odometry.

8. A UAV path planning system based on adaptive dynamic programming, characterized in that, include: The acquisition module is used to acquire visible light image data and infrared image data of the target area, and generate a texture-enhanced image by histogram matching and fusion, and extract illumination-invariant edge information from the texture-enhanced image; The compensation module is used to obtain the atmospheric polarization pattern through the polarized light navigation sensor, calculate the solar azimuth angle based on the atmospheric polarization pattern, and use the solar azimuth angle to compensate for the heading drift of the visual odometry, thereby generating the corrected UAV position information. The segmentation module is used to segment key landmarks in the texture enhancement image based on the illumination invariant edge information and in combination with a convolutional neural network, and to generate a semantic navigation landmark sequence within the constraint space of the key landmarks using a bidirectional fast random tree. The key landmarks include roads and buildings. The construction module is used to construct a model prediction controller based on the semantic navigation landmark sequence; The adjustment module is used to optimize the flight path of the UAV through adaptive dynamic programming under dynamic window constraints by using the model prediction controller and the corrected position information, while adjusting the field of view and exposure parameters of the dual-band imaging unit according to the ambient light intensity.

9. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the UAV path planning method based on adaptive dynamic programming as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a UAV path planning method based on adaptive dynamic programming as described in any one of claims 1 to 7.

Images