UAV visual navigation method and device based on deep learning

Abstract

The present invention relates to a UAV visual navigation method and device based on deep learning, including: obtaining a visual image collected by a UAV visual system, extracting features from the visual image, and obtaining an image feature representation; performing type analysis on the image feature representation to obtain scene data, depth information, and continuous image frames; performing scene recognition on the scene data to obtain recognition results of obstacles and passable areas; performing optical flow estimation on the continuous image frames and the depth information through a preset optical flow algorithm to obtain a pose estimation result; obtaining the mission requirements of the UAV to obtain a navigation control strategy with environmental adaptability; optimizing the adaptive hybrid learning model based on the feedback information combined with a preset attention mechanism and reward shaping conditions to obtain the optimized adaptive hybrid learning model. The present invention can improve the adaptability of UAVs in complex and dynamic environments.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle visual navigation, and in particular to a method and device for unmanned aerial vehicle visual navigation based on deep learning. Background Art

[0002] As an advanced autonomous control method, UAV visual navigation technology is increasingly used in various complex environments. With the rapid development of artificial intelligence and computer vision technology, how to improve the navigation ability and adaptability of UAVs in dynamic and unknown environments has become one of the research focuses. Existing UAV visual navigation methods usually rely only on a single perception or decision-making model, such as path planning based only on traditional algorithms. This single method is difficult to cope with complex and changeable actual environments, and it is easy to lead to limitations and instability in navigation decisions. There are the following problems in current research: First, there is a lack of comprehensive utilization of multi-source visual information, which makes it difficult to fully perceive environmental information; second, existing methods often ignore the impact of task requirements on navigation strategies and cannot be flexibly adjusted according to specific tasks; third, there is a lack of effective adaptive learning mechanisms, which makes it difficult to continuously optimize navigation strategies according to environmental changes and feedback information. These problems limit the navigation performance and task execution capabilities of UAVs in complex and dynamic environments. Summary of the invention

[0003] The main purpose of the present invention is to provide a UAV visual navigation method and device based on deep learning, which can improve the adaptability of UAV in complex and dynamic environments.

[0004] To achieve the above object, the present invention provides a UAV visual navigation method based on deep learning, comprising:

[0005] Acquire a visual image collected by a visual system of the unmanned aerial vehicle, perform feature extraction on the visual image, and obtain an image feature representation;

[0006] Performing type analysis on the image feature representation to obtain scene data, depth information and continuous image frames;

[0007] Perform scene recognition on the scene data to obtain the recognition results of obstacles and passable areas;

[0008] Performing optical flow estimation on the continuous image frames and the depth information by using a preset optical flow algorithm to obtain a pose estimation result;

[0009] Obtaining the mission requirements of the UAV, inputting the recognition result, the pose estimation result and the mission requirements into a preset adaptive hybrid learning model for decision generation, and obtaining a navigation control strategy with environmental adaptability;

[0010] Acquire feedback information after applying the navigation control strategy, optimize the adaptive hybrid learning model according to the feedback information in combination with preset attention mechanism and reward shaping conditions, and obtain the optimized adaptive hybrid learning model.

[0011] Furthermore, the step of acquiring a visual image collected by a drone visual system and extracting features from the visual image to obtain an image feature representation includes:

[0012] Performing multi-scale feature extraction on the visual image to obtain multiple feature maps;

[0013] Performing visual space matching on the plurality of feature maps to obtain corresponding visual feature vectors;

[0014] Performing nonlinear feature transformation on the visual feature vector to obtain a visual feature representation;

[0015] Calculating feature weights on the visual feature representation to obtain a weighted feature representation;

[0016] The image feature representation is obtained by performing image conversion on the weighted feature representation using a preset principal component analysis method.

[0017] Furthermore, the performing type analysis on the image feature representation to obtain scene data, depth information and continuous image frames includes:

[0018] Decomposing the image feature representation in multiple levels to obtain a multi-level feature description;

[0019] Performing semantic segmentation on the image features to obtain a semantic segmentation result;

[0020] Performing scene extraction and fusion on the multi-level feature description according to the semantic segmentation result to obtain the scene data;

[0021] Performing depth calculation on the multi-level feature description to obtain initial depth data;

[0022] Extracting image frames from the image feature representation and performing time sequence sorting on the image frames to obtain the continuous image frames;

[0023] Performing motion estimation on the multi-level feature description according to the continuous image frames to obtain a motion vector;

[0024] The initial depth data is corrected according to the motion vector to obtain the depth information.

[0025] Furthermore, the performing scene recognition on the scene data to obtain recognition results of obstacles and passable areas includes:

[0026] Performing semantic segmentation on the scene data to obtain semantic labels for each area in the scene;

[0027] Preliminarily classifying the areas in the scene according to the semantic labels to obtain initial obstacles and traversable areas;

[0028] Performing edge detection and contour extraction on the initial obstacle and the traversable area to obtain comprehensive boundary information;

[0029] Estimating the three-dimensional spatial position of the initial obstacle according to the comprehensive boundary information to obtain spatial distribution information of the obstacle;

[0030] A three-dimensional occupancy grid map analysis is performed on the spatial distribution information and the range of the passable area to obtain the identification results of the obstacles and the passable area.

[0031] Furthermore, performing optical flow estimation on the continuous image frames and the depth information by using a preset optical flow algorithm to obtain a pose estimation result includes:

[0032] Performing image pyramid construction on the continuous image frames to obtain a multi-level image representation;

[0033] Performing feature point matching detection on each level image in the multi-level image representation to obtain corresponding level feature points;

[0034] Performing optical flow calculation on the hierarchical feature points according to the optical flow algorithm to obtain corresponding hierarchical optical flow vectors;

[0035] Performing dense optical flow interpolation on the multi-level image representation according to the hierarchical optical flow vectors to obtain a corresponding hierarchical optical flow field;

[0036] Performing a refinement optical flow fusion process on the hierarchical optical flow field to obtain an initial optical flow field;

[0037] Performing depth weighting on the initial optical flow field according to the depth information to obtain a weighted optical flow field;

[0038] Calculating the motion matrix between adjacent frames according to the weighted optical flow field to obtain the inter-frame pose transformation;

[0039] The inter-frame pose transformation is cumulatively calculated to obtain the pose estimation result.

[0040] Furthermore, the recognition result, the pose estimation result and the task requirement are input into a preset adaptive hybrid learning model for decision generation to obtain a navigation control strategy with environmental adaptability, including:

[0041] Inputting the recognition result, the pose estimation result and the task requirement into the adaptive hybrid learning model;

[0042] Performing feature convolution extraction on the recognition result through a convolutional network layer to obtain visual features;

[0043] Performing state transformation on the pose estimation result through a spatial transformation layer to generate a state vector;

[0044] Performing vector analysis on the task requirements through an embedding layer to obtain a corresponding task vector;

[0045] Integrate the visual features, the state vector and the task vector through a concatenation layer to obtain an input state vector;

[0046] Inputting the input state vector into the Deep Q-Network layer to estimate the action value and obtain an action value set;

[0047] Inputting the action value set into the argmax layer to perform navigation action decision analysis to obtain a preliminary decision result;

[0048] Inputting the preliminary decision result into the LSTM layer for decision optimization to obtain an optimized decision result;

[0049] Inputting the optimization decision result into the mapping layer for control command mapping to obtain the flight control command of the UAV;

[0050] The flight control instructions are input into the strategy generation layer for strategy generation, and the corresponding navigation control strategy is output.

[0051] Furthermore, the obtaining of feedback information after applying the navigation control strategy, and optimizing the adaptive hybrid learning model according to the feedback information in combination with a preset attention mechanism and reward shaping conditions, include:

[0052] The flight status of the UAV after applying the navigation control strategy is collected to obtain status feedback information including position, speed, acceleration and attitude angle;

[0053] Reconstructing the flight trajectory of the UAV according to the state feedback information to obtain an actual flight trajectory;

[0054] Calculating the deviation between the actual flight trajectory and the preset ideal flight trajectory to obtain a trajectory deviation;

[0055] Adjusting the attention weight of the adaptive hybrid learning model according to a preset adaptive multi-head attention mechanism combined with the trajectory deviation to obtain an updated attention distribution;

[0056] Performing reward evaluation calculation on the trajectory deviation according to the preset reward shaping condition to obtain a corresponding shaping reward value;

[0057] Optimizing the loss function of the adaptive hybrid learning model according to the shaping reward value to obtain an optimized loss function;

[0058] The parameters of the adaptive hybrid learning model are gradient updated based on the optimized loss function and the attention distribution to obtain the optimized adaptive hybrid learning model.

[0059] The present invention also provides a UAV visual navigation device based on deep learning, which is applied to any one of the above-mentioned UAV visual navigation methods based on deep learning, comprising:

[0060] An acquisition module, the acquisition module is used to acquire a visual image acquired by a UAV visual system, perform feature extraction on the visual image, and obtain an image feature representation;

[0061] An analysis module, the analysis module is used to perform type analysis on the image feature representation to obtain scene data, depth information and continuous image frames;

[0062] An association module, the association module is used to perform scene recognition on the scene data to obtain recognition results of obstacles and passable areas;

[0063] A processing module, the processing module is used to perform optical flow estimation on the continuous image frames and the depth information by using a preset optical flow algorithm to obtain a pose estimation result;

[0064] A control module, wherein the control module is used to obtain the mission requirements of the UAV, input the recognition result, the pose estimation result and the mission requirements into a preset adaptive hybrid learning model for decision generation, and obtain a navigation control strategy with environmental adaptability,

[0065] An execution module is used to obtain feedback information after applying the navigation control strategy, and optimize the adaptive hybrid learning model based on the feedback information in combination with a preset attention mechanism and reward shaping conditions.

[0066] The present invention provides a deep learning-based unmanned aerial vehicle visual navigation method and device, which has the following beneficial effects:

[0067] The deep learning-based UAV visual navigation method can perceive the environment more comprehensively by comprehensively utilizing multi-source visual information, thereby improving the accuracy and reliability of navigation decisions. This method can effectively extract scene data, depth information and continuous image frames by performing type analysis on image features, providing a rich information basis for subsequent scene recognition and pose estimation. By combining the scene recognition results and pose estimation results with task requirements and inputting them into an adaptive hybrid learning model for decision generation, the navigation strategy can be flexibly adapted to specific tasks. This method not only considers environmental factors, but also incorporates task requirements into the decision-making process, thereby improving the pertinence and effectiveness of the navigation strategy. By introducing an adaptive optimization mechanism based on attention mechanism and reward shaping conditions, the learning model can be continuously adjusted and optimized according to the feedback information during the navigation process. This adaptive learning mechanism significantly improves the adaptability of UAVs in complex and dynamic environments, enabling navigation strategies to be continuously optimized as the environment changes. By processing continuous image frames and depth information through the optical flow algorithm, more accurate pose estimation is achieved, providing reliable spatial positioning information for navigation decisions. This method effectively improves the positioning accuracy and navigation stability of UAVs in complex environments. In summary, the present invention effectively solves the problems of perception limitations, insufficient task adaptability and weak environmental adaptability in existing UAV visual navigation through multi-source information fusion, task adaptability decision-making and adaptive learning optimization, and significantly improves the navigation performance and task execution capability of UAVs in complex and dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a flow chart of a UAV visual navigation method based on deep learning provided by the present invention;

[0069] Figure 2 This is a structural diagram of a UAV visual navigation device based on deep learning provided by the present invention;

[0070] Figure 3 This is a structural diagram of a UAV visual navigation system based on deep learning provided by the present invention.

[0071] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0073] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0074] Reference Figure 1 As shown, the present invention provides a UAV visual navigation method based on deep learning, comprising:

[0075] Step S1: Obtain a visual image collected by a UAV visual system, extract features from the visual image, and obtain an image feature representation;

[0076] Step S2: performing type analysis on the image feature representation to obtain scene data, depth information and continuous image frames;

[0077] Step S3: Perform scene recognition on the scene data to obtain recognition results of obstacles and passable areas;

[0078] Step S4: performing optical flow estimation on continuous image frames and depth information using a preset optical flow algorithm to obtain a pose estimation result;

[0079] Step S5: Obtain the mission requirements of the UAV, input the recognition results, the pose estimation results and the mission requirements into a preset adaptive hybrid learning model for decision generation, and obtain a navigation control strategy with environmental adaptability;

[0080] Step S6: Obtain feedback information after applying the navigation control strategy, and optimize the adaptive hybrid learning model based on the feedback information combined with the preset attention mechanism and reward shaping conditions to obtain an optimized adaptive hybrid learning model.

[0081] Based on the above steps, the detailed steps are as follows:

[0082] Step S1: The visual system of the drone (usually including a camera or a multispectral camera) continuously collects visual images of the environment during flight. These visual images provide detailed information about the surrounding environment and are the basis for subsequent processing.

[0083] The collected original image contains interference factors such as noise and lighting changes, so the image is preprocessed. Common preprocessing steps include denoising, white balance, histogram equalization and other operations.

[0084] The preprocessed image will be subjected to feature extraction. The purpose of feature extraction is to extract important information from the image and convert it into a high-dimensional feature representation. These feature representations contain information such as edges, corners, textures, etc., which can effectively describe the image content.

[0085] After feature extraction, the image is converted into a feature vector or feature map.

[0086] Step S2: Perform type analysis on the obtained image feature representation to distinguish different types of visual information. Analyze the image feature representation and extract scene data. Scene data includes information such as the overall layout of the environment, the location and shape of objects, etc., which is usually achieved through semantic segmentation or target detection.

[0087] Depth information is extracted from the image feature representation through depth estimation algorithms (such as monocular depth estimation or stereo vision). Depth information describes the distance from each object in the scene to the drone and is an important basis for navigation and obstacle avoidance.

[0088] The continuously acquired image sequences are processed to obtain time series information. This usually involves feature point tracking or optical flow estimation to describe the motion of objects and scene changes.

[0089] Step S3: Identify the scene data and determine the various objects and their locations in the environment. Identify obstacles in the scene, including static and dynamic obstacles. Static obstacles include buildings and trees, and dynamic obstacles include pedestrians and vehicles.

[0090] Identify the traversable areas in the scene, such as roads, sidewalks, open areas, etc. This is usually achieved through semantic segmentation or path planning algorithms. The identified obstacle information and traversable area information are processed together to form a complete scene recognition result. The output of this step will serve as an important input to the navigation decision model.

[0091] Step S4: The optical flow algorithm is used to estimate the pixel motion between consecutive image frames. Combined with the depth information, the pixel changes in the image sequence of consecutive image frames are analyzed to obtain the motion information of the objects in the scene.

[0092] The obtained depth information is fused with the optical flow estimation results to improve the accuracy of pose estimation. The depth information provides the spatial distance of the object, while the optical flow provides the direction and speed of the object's movement.

[0093] By fusing the optical flow and depth information, the relative position change of the drone, including translation and rotation, is calculated. This is achieved by solving the camera motion equation or using an optimization algorithm such as the least squares method.

[0094] Finally, the pose estimation result of the UAV is obtained, including position and attitude information, which will be used for navigation and obstacle avoidance.

[0095] Step S5: Obtain the current mission requirements of the UAV, such as target location, path planning, obstacle avoidance requirements, etc. These mission requirements are usually provided by the ground control system or a preset mission plan.

[0096] The scene recognition results, pose estimation results, and task requirements are taken as input data.

[0097] Use a preset adaptive hybrid learning model for decision generation. This model usually combines reinforcement learning and supervised learning, and can adjust strategies in different environments through adaptive mechanisms.

[0098] The model generates environmentally adaptable navigation control strategies based on input data, including path planning, speed control, obstacle avoidance strategies, etc. These strategies will guide the actual flight of the drone.

[0099] Step S6: During the process of the UAV executing the navigation control strategy, real-time feedback of flight status and environmental information is obtained, including position, speed, obstacle approach, etc.

[0100] Analyze the feedback information and evaluate the effectiveness and adaptability of the navigation control strategy. This step is usually achieved by calculating indicators such as navigation error and obstacle avoidance success rate.

[0101] When analyzing feedback information, an attention mechanism is introduced to focus on feedback in key areas or time periods. For example, when approaching an obstacle, the attention weight of the obstacle information is increased.

[0102] Based on the feedback information and attention mechanism, a reward function is defined to optimize the adaptive hybrid learning model in a reinforcement learning manner. The reward function is usually designed based on indicators such as navigation accuracy and obstacle avoidance success rate.

[0103] By combining feedback information, attention mechanism and reward function, the adaptive hybrid learning model is optimized to perform better in similar environments.

[0104] The present invention provides a deep learning-based unmanned aerial vehicle visual navigation method. The deep learning-based unmanned aerial vehicle visual navigation method can perceive the environment more comprehensively by comprehensively utilizing multi-source visual information, thereby improving the accuracy and reliability of navigation decisions. The method realizes the effective extraction of scene data, depth information and continuous image frames by performing type analysis on image features, and provides a rich information basis for subsequent scene recognition and pose estimation. By combining the scene recognition results and pose estimation results with task requirements and inputting them into an adaptive hybrid learning model for decision generation, the flexible adaptation of the navigation strategy to specific tasks is realized. This method not only takes environmental factors into consideration, but also incorporates task requirements into the decision-making process, thereby improving the pertinence and effectiveness of the navigation strategy. By introducing an adaptive optimization mechanism based on an attention mechanism and a reward shaping condition, the learning model can be continuously adjusted and optimized according to the feedback information in the navigation process. This adaptive learning mechanism significantly improves the adaptability of the unmanned aerial vehicle in complex and dynamic environments, so that the navigation strategy can be continuously optimized as the environment changes. By processing continuous image frames and depth information through an optical flow algorithm, more accurate pose estimation is realized, providing reliable spatial positioning information for navigation decisions. This method effectively improves the positioning accuracy and navigation stability of the unmanned aerial vehicle in a complex environment. In summary, the present invention effectively solves the problems of perception limitations, insufficient task adaptability and weak environmental adaptability in existing UAV visual navigation through multi-source information fusion, task adaptability decision-making and adaptive learning optimization, and significantly improves the navigation performance and task execution capability of UAVs in complex and dynamic environments.

[0105] In one embodiment, a visual image collected by a drone visual system is obtained, and features are extracted from the visual image to obtain an image feature representation, including:

[0106] The visual system carried by the drone collects images of the surrounding environment. The visual system includes multiple cameras that can collect images from different angles and directions.

[0107] Perform multi-scale feature extraction on the collected visual images to obtain multiple feature maps. Multi-scale feature extraction refers to using convolution kernels of different sizes to perform convolution operations on images to extract features of different scales. For example, convolution kernels of sizes such as 3x3, 5x5, and 7x7 are used to convolve images to obtain multiple feature maps. These feature maps contain feature information at different levels such as edges and textures of the image.

[0108] Perform visual spatial matching on multiple feature maps to obtain corresponding visual feature vectors. Visual spatial matching refers to matching feature maps of different scales in spatial positions and merging them into a unified feature representation. In specific implementation, methods such as spatial pyramid pooling can be used to pool feature maps of different scales to the same size, and then splice them to obtain a visual feature vector of fixed dimension.

[0109] Perform nonlinear feature transformation on the visual feature vector to obtain visual feature representation. Nonlinear feature transformation is to enhance the expressiveness of features. Commonly used methods include ReLU activation function. Through nonlinear transformation, nonlinear factors can be introduced to improve the expressiveness and distinguishability of features.

[0110] The feature weights are calculated for the visual feature representation to obtain the weighted feature representation. The feature weight calculation is to highlight the important features and suppress the unimportant features. The importance weight of each feature is calculated by using methods such as the attention mechanism to weight the features.

[0111] The weighted feature representation is converted to an image using a preset principal component analysis method to obtain a final image feature representation. Principal component analysis (PCA) is a commonly used dimensionality reduction method that can extract the main features of the data and remove redundant information. In this embodiment, PCA is used to reduce the dimensionality of the weighted features to obtain a more compact feature representation.

[0112] This embodiment can capture feature information of different scales and levels in the image through multi-scale feature extraction and visual space matching, thereby improving the comprehensiveness and robustness of the features. Nonlinear feature transformation enhances the expressiveness of features, which is beneficial to subsequent navigation decisions. Feature weight calculation highlights important features, suppresses irrelevant features, and improves the pertinence of features. Principal component analysis dimensionality reduction processing removes redundant information to obtain a more compact and efficient feature representation. The entire feature extraction and representation process fully combines the advantages of deep learning, and can adaptively extract key features to adapt to complex and changing flight environments. Compared with traditional methods, the present invention does not require artificially designed features, has stronger environmental adaptability and generalization capabilities, and provides reliable feature input for autonomous visual navigation of drones.

[0113] In one embodiment, performing type analysis on the image feature representation to obtain scene data, depth information, and continuous image frames includes:

[0114] The image feature representation is decomposed into multiple levels to obtain a multi-level feature description. The multi-level feature description includes low-level features (such as edges, textures, etc.), mid-level features (such as shapes, structures, etc.) and high-level features (such as semantic information, etc.). These multi-level features can be extracted through multiple convolutional layers and pooling layers of the convolutional neural network.

[0115] The image features are semantically segmented to obtain the semantic segmentation result. Semantic segmentation uses a fully convolutional network to classify each pixel in the image into predefined semantic categories, such as roads, buildings, vegetation, etc. The semantic segmentation result is a segmentation map of the same size as the original image, in which each pixel is assigned a semantic label.

[0116] The scene data is obtained by performing scene extraction fusion on the multi-level feature description based on the semantic segmentation results. The scene extraction fusion process combines the semantic segmentation results with the multi-level feature description to extract a structured representation of the scene. This includes identifying the main objects in the scene, their spatial relationships, and the overall scene layout. The fusion uses an attention mechanism to weight the features at different levels according to the semantic information.

[0117] Perform depth calculation on the multi-level feature description to obtain initial depth data. Depth calculation can use a monocular depth estimation network, such as DepthNet, which takes the multi-level feature description as input and outputs the depth value of each pixel. The initial depth data is a depth map of the same size as the original image, which represents the distance from each point in the scene to the camera.

[0118] Image frame extraction is performed on the image feature representation, and the image frames are time-sequentially sorted to obtain continuous image frames. Image frame extraction selects key frames from a video stream or a sequence of continuously shot images. Time-sequential sorting ensures that the frames are arranged in the order of the shooting time to form a continuous sequence of image frames. This provides a basis for subsequent motion estimation. Motion estimation is performed on the multi-level feature description based on the continuous image frames to obtain motion vectors. Motion estimation uses optical flow or feature matching to calculate the displacement of pixels or feature points between continuous frames. Motion vectors describe the movement of the camera between continuous frames, including translation and rotation information.

[0119] The initial depth data is corrected according to the motion vector to obtain depth information. The correction process uses the camera motion information provided by the motion vector and combines the multi-view geometry principle to optimize the initial depth estimation result.

[0120] This embodiment achieves a comprehensive understanding of the scene and improves the accuracy and robustness of navigation by performing multi-level decomposition and semantic segmentation of image feature representation. The fusion of multi-level feature description and semantic segmentation results enables the drone to better identify and understand various elements in complex environments, thereby making more intelligent navigation decisions. The acquisition and correction process of depth information combines monocular depth estimation and multi-view geometry principles to significantly improve the accuracy of depth perception. This enables the drone to more accurately perceive the three-dimensional structure of the surrounding environment, effectively avoid obstacles, and achieve safe navigation. The extraction and motion estimation of continuous image frames provide the drone with accurate self-motion information, which helps to update the position and attitude in real time. This not only improves the accuracy of navigation, but also provides a basis for advanced functions such as SLAM (simultaneous localization and mapping).

[0121] In one embodiment, scene recognition is performed on scene data to obtain recognition results of obstacles and traversable areas, including:

[0122] The scene image is semantically annotated at the pixel level by using a deep learning semantic segmentation network. The semantic segmentation network can be a common network structure such as FCN, SegNet or DeepLab. Through training with a large amount of labeled data, it can divide the pixels in the scene into different semantic categories such as roads, buildings, vegetation, etc. The result of semantic segmentation is a label map of the same size as the original image, in which each pixel is assigned a semantic label.

[0123] Based on the results of semantic segmentation, the areas in the scene are initially divided into two categories: obstacles and passable areas. Specifically, semantic categories such as roads and flat land are classified as passable areas, and semantic categories such as buildings, trees, and vehicles are classified as obstacles. This step obtains a preliminary classification result based on semantic information, and further processing is required to obtain accurate obstacle information.

[0124] The obstacles and passable areas obtained by preliminary classification are subjected to edge detection and contour extraction. Edge detection uses classic algorithms such as the Canny operator to obtain a binary image of the object edge. Contour extraction uses a contour tracking algorithm to extract the complete object contour based on edge detection. The purpose of this step is to obtain accurate boundary information of obstacles in preparation for subsequent 3D position estimation.

[0125] Using the depth information obtained by the binocular camera or depth camera, combined with the obstacle contour obtained in the previous step, the three-dimensional spatial position of the obstacle is estimated. The specific method is to map the contour points into three-dimensional space to obtain a three-dimensional point cloud representation of the obstacle. Through cluster analysis, the spatial distribution range of each obstacle can be obtained, including information such as position and size.

[0126] The spatial distribution information is voxelized to obtain three-dimensional point cloud data, and an octree structure is constructed based on the three-dimensional point cloud data to obtain a hierarchical representation of the scene.

[0127] The occupancy probability of each node in the octree structure is calculated to obtain an initial occupancy grid map, and the initial occupancy grid map is corrected according to the range of the passable area to obtain a corrected occupancy grid map.

[0128] The spatial distribution information of obstacles and the range of the traversable area are converted into a modified occupancy grid map. The occupancy grid map divides the three-dimensional space into small cubic grids, and the state of each grid can be occupied, free or unknown. By projecting the obstacle point cloud and the traversable area into the grid map, a three-dimensional representation of the environment is obtained.

[0129] Perform cluster analysis on the three-dimensional representation of the environment to obtain the spatial distribution of obstacles and traversable areas, and perform three-dimensional reconstruction of obstacles based on the spatial distribution to obtain the geometric model of the obstacles;

[0130] The geometric model is subjected to collision detection analysis to obtain a safe path for the UAV to pass. The obstacles and passable areas are finally classified according to the safe path to obtain the identification results of obstacles and passable areas.

[0131] This embodiment achieves efficient representation and rapid retrieval of scenes by voxelizing spatial distribution information and constructing an octree structure. The accuracy of the occupancy grid map is improved by adopting occupancy probability calculation and correction steps. The application of cluster analysis and three-dimensional reconstruction technology enables a more accurate description of the spatial distribution and geometric characteristics of obstacles. The introduction of collision detection analysis ensures the safety of drone path planning. The combination of this series of steps not only improves the accuracy of obstacle and passable area identification, but also optimizes computational efficiency. It can adapt to complex and changing flight environments and provide reliable spatial perception capabilities for real-time navigation decisions of drones, thereby enhancing the autonomous flight capabilities and safety of drones in various scenarios.

[0132] In one embodiment, optical flow estimation is performed on continuous image frames and depth information using a preset optical flow algorithm to obtain a pose estimation result, including:

[0133] Image pyramids are constructed for consecutive image frames to obtain multi-level image representations. Image pyramids are a multi-scale representation method that generates image sequences of different resolutions by downsampling and smoothing the original image. This multi-level representation helps capture different scale features in the image and improve the robustness and efficiency of subsequent processing.

[0134] Perform feature point matching detection on each level of the multi-level image representation to obtain the corresponding level feature points. Feature points are local areas with significance and stability in the image, such as corners, edges, etc. Through feature point matching, a correspondence can be established between different image frames, providing a basis for subsequent optical flow calculation. Common feature point detection algorithms include SIFT, SURF, ORB, etc.

[0135] According to the optical flow algorithm, the optical flow of the hierarchical feature points is calculated to obtain the corresponding hierarchical optical flow vector. Optical flow is a vector field that describes the motion of pixels in an image and reflects the motion information between adjacent frames. Commonly used optical flow algorithms include the Lucas-Kanade method and the Horn-Schunck method. By calculating the optical flow of feature points, the motion vectors of these points between adjacent frames can be obtained.

[0136] Based on the hierarchical optical flow vectors, dense optical flow interpolation is performed on the multi-level image representation to obtain the corresponding hierarchical optical flow field. Dense optical flow interpolation is the process of expanding the sparse feature point optical flow vectors to the entire image area. Through interpolation algorithms such as bilinear interpolation or Gaussian interpolation, the optical flow estimation value of each pixel can be generated to form a complete optical flow field.

[0137] The hierarchical optical flow field is refined and fused to obtain the initial optical flow field. Refined optical flow fusion is to integrate the optical flow field information of different levels to obtain a more accurate optical flow estimate. This usually involves an iterative optimization process from a coarse level to a fine level, and the results of each level are used to guide and improve the estimation of the next level.

[0138] The initial optical flow field is depth-weighted according to the depth information to obtain a weighted optical flow field. Depth information can be obtained from stereo vision, structured light, or depth cameras. Depth weighting is the process of adjusting the optical flow according to the depth value of the pixel point. Generally, the optical flow of distant objects is smaller, and the optical flow of nearby objects is larger. This weighting can improve the accuracy of optical flow estimation, especially when there are objects of different depths in the scene.

[0139] Harris corner detection is performed on the weighted optical flow field to obtain a set of feature points. Local area sampling is performed on the weighted optical flow field according to the set of feature points to obtain a sparse optical flow field.

[0140] The RANSAC algorithm is applied to the sparse optical flow field to remove outliers and obtain the optimized sparse optical flow field. According to the optimized sparse optical flow field, the least squares optimization problem based on the reprojection error is constructed to obtain the optimization objective function. The Levenberg-Marquardt algorithm is applied to solve the optimization objective function to obtain the initial motion matrix estimate. The reprojection error of the sparse optical flow field is calculated based on the initial motion matrix estimate to obtain the reprojection error of each feature point.

[0141] An adaptive threshold screening method is applied to the reprojection error to obtain an inlier set. Based on the inlier set, the initial motion matrix estimate is optimized using an iterative closest point algorithm to obtain a refined motion matrix.

[0142] The fine motion matrix is ​​decomposed by singular value decomposition to obtain the rotation matrix and translation vector. A 4x4 homogeneous transformation matrix is ​​constructed based on the rotation matrix and translation vector to obtain the inter-frame pose transformation.

[0143] The pose transformation between frames is accumulated to obtain the pose estimation result. The cumulative calculation is to combine the pose transformation between consecutive frames to obtain the absolute pose relative to the initial position. This process involves matrix multiplication and coordinate transformation. It is necessary to pay attention to the processing of cumulative errors. It is usually combined with other sensor data or closed-loop detection technology to reduce error accumulation.

[0144] This embodiment adopts a multi-level image pyramid representation. The method can effectively capture image features of different scales and improve the accuracy and robustness of feature point matching and optical flow estimation. The optical flow field is weighted by using depth information to overcome the limitations of traditional optical flow algorithms in complex scenes, making pose estimation more accurate, especially in environments with near and far objects. By refining and fusing the hierarchical optical flow field, the method realizes iterative optimization from coarse to fine, significantly improving the quality and resolution of the optical flow field. Combined with the cumulative calculation of inter-frame pose transformations, this method can achieve long-term and large-scale pose tracking, providing reliable basic data for autonomous navigation of drones.

[0145] In one embodiment, the recognition results, the pose estimation results and the task requirements are input into a preset adaptive hybrid learning model for decision generation to obtain a navigation control strategy with environmental adaptability, including:

[0146] The adaptive hybrid learning model is used for decision generation, which receives recognition results, pose estimation results, and task requirements as input. The recognition results contain the object category and location information in the environment, the pose estimation results contain the current position and attitude of the drone, and the task requirements include the target location and obstacle avoidance requirements.

[0147] The structure of the adaptive hybrid learning model includes multiple functional layers. The convolutional network layer extracts features from the recognition results, using multi-layer convolution and pooling operations to extract key visual features in the environment, such as edges, textures, etc. The spatial transformation layer converts the pose estimation results into a state vector, and generates a vector representing the current state of the drone through coordinate transformation and normalization. The embedding layer vectorizes the task requirements and converts the task description into a vector representation of a fixed dimension. The concatenation layer concatenates the visual features, state vector, and task vector in the channel dimension to form a complete input state vector.

[0148] The Deep Q-Network layer uses a deep neural network structure to estimate the Q values ​​of different actions based on the input state vector to form an action value set. The network structure contains multiple fully connected layers and uses the ReLU activation function. The argmax layer selects the action with the largest Q value from the action value set as the preliminary decision result.

[0149] The LSTM layer introduces a loop structure, considers historical decision information, optimizes the initial decision results, and improves the temporal consistency of the decision. The mapping layer uses a fully connected network to map the optimized decision results to specific flight control instructions, such as speed and angle. The strategy generation layer generates a complete navigation control strategy based on the flight control instructions, including route planning, obstacle avoidance strategy, etc.

[0150] The model can adaptively generate navigation strategies based on the current environmental conditions and mission requirements. For example, in a complex obstacle environment, the model will generate a more conservative obstacle avoidance strategy, reduce flight speed, and increase safety distance; in an open environment, it will generate a more efficient straight route and increase flight speed. Through deep reinforcement learning, the model can continuously optimize decision-making capabilities and improve navigation performance in various scenarios.

[0151] This embodiment integrates multiple deep learning technologies such as convolutional networks, spatial transformation, Deep Q-Network, etc., so that the adaptive hybrid learning model can adaptively generate navigation strategies according to the current environmental conditions and task requirements, greatly improving the environmental adaptability and task adaptability of the system. The multi-layer structure design of the model enables it to effectively process complex visual input and state information and generate accurate decision results. The introduction of the LSTM layer considers historical decision information and improves the temporal consistency of decisions. Through deep reinforcement learning, the model can continuously optimize decision-making capabilities and maintain good navigation performance in various scenarios.

[0152] In one embodiment, feedback information after applying the navigation control strategy is obtained, and the adaptive hybrid learning model is optimized according to the feedback information combined with a preset attention mechanism and reward shaping conditions, including:

[0153] The flight status of the UAV after applying the navigation control strategy is collected to obtain status feedback information including position, speed, acceleration and attitude angle. Position information includes the coordinates of the UAV in three-dimensional space, speed information includes linear speed and angular speed, acceleration information includes linear acceleration and angular acceleration, and attitude angle information includes pitch angle, yaw angle and roll angle. These status feedback information fully describe the current flight status of the UAV.

[0154] Based on the collected state feedback information, the flight trajectory of the UAV is reconstructed to obtain the actual flight trajectory. The reconstruction process uses state estimation algorithms, such as Kalman filters or particle filters, to make discrete state information continuous and form a complete flight trajectory.

[0155] The deviation between the reconstructed actual flight trajectory and the preset ideal flight trajectory is calculated to obtain the trajectory deviation. The ideal flight trajectory is the optimal path pre-planned according to the navigation task, and the trajectory deviation reflects the gap between the actual flight and the expected target. The deviation calculation uses measurement methods such as Euclidean distance or Hausdorff distance.

[0156] According to the preset adaptive multi-head attention mechanism, the attention weight of the adaptive hybrid learning model is adjusted in combination with the trajectory deviation to obtain an updated attention distribution. The adaptive multi-head attention mechanism is a technology that dynamically adjusts the attention weight, and can adaptively allocate attention resources according to the importance of different input features. In this embodiment, the attention mechanism dynamically adjusts the degree of attention to different visual features and sensor data according to the size and direction of the trajectory deviation.

[0157] The reward evaluation calculation of the trajectory deviation is performed according to the preset reward shaping conditions to obtain the corresponding shaping reward value. The reward shaping conditions are a set of predefined rules used to convert trajectory deviations into specific reward values. These conditions take into account factors such as the size, duration, and change trend of the deviation, and quantify complex deviation situations into numerical signals that can be used for model optimization.

[0158] Based on the calculated shaping reward value, the loss function of the adaptive hybrid learning model is optimized to obtain the optimized loss function. The optimization process involves integrating the reward signal into the original loss function, and by adjusting the weights of each loss or introducing a new regularization term, the model pays more attention to reducing trajectory deviation during the optimization process.

[0159] Based on the optimized loss function and the updated attention distribution, the parameters of the adaptive hybrid learning model are gradient updated to obtain the optimized adaptive hybrid learning model. The gradient update uses the back propagation algorithm to calculate the partial derivatives of the loss function with respect to the model parameters and use an optimizer (such as Adam or RMSprop) to adjust the parameter values ​​so that the model can generate a more accurate control strategy in the next navigation task.

[0160] This embodiment can accurately evaluate the actual effect of the navigation control strategy by comprehensively collecting the flight status of the UAV and reconstructing the actual trajectory, providing a reliable basis for subsequent optimization. Secondly, the adaptive multi-head attention mechanism is used to dynamically adjust the model's attention to different features, thereby improving the model's efficiency in utilizing key information. Furthermore, the introduction of reward shaping conditions quantifies complex trajectory deviations into specific reward values, making model optimization more accurate and targeted. Finally, by integrating the optimized loss function and the updated attention distribution to adjust the model parameters, the continuous optimization of the adaptive hybrid learning model is achieved, which improves the accuracy and environmental adaptability of the UAV visual navigation.

[0161] The present invention also provides a UAV visual navigation device based on deep learning, which is applied to any one of the above-mentioned UAV visual navigation methods based on deep learning, comprising:

[0162] The acquisition module is used to obtain the visual images collected by the UAV visual system, extract features from the visual images, and obtain image feature representation;

[0163] An analysis module, the analysis module is used to perform type analysis on image feature representation to obtain scene data, depth information and continuous image frames;

[0164] The association module is used to perform scene recognition on the scene data to obtain recognition results of obstacles and passable areas;

[0165] A processing module, which is used to perform optical flow estimation on continuous image frames and depth information through a preset optical flow algorithm to obtain a pose estimation result;

[0166] The control module is used to obtain the mission requirements of the UAV, input the recognition results, pose estimation results and mission requirements into the preset adaptive hybrid learning model for decision generation, and obtain a navigation control strategy with environmental adaptability.

[0167] The execution module is used to obtain feedback information after applying the navigation control strategy, and optimize the adaptive hybrid learning model based on the feedback information combined with the preset attention mechanism and reward shaping conditions.

[0168] The present invention provides a deep learning-based unmanned aerial vehicle visual navigation device. The deep learning-based unmanned aerial vehicle visual navigation method can perceive the environment more comprehensively by comprehensively utilizing multi-source visual information, thereby improving the accuracy and reliability of navigation decisions. The method realizes the effective extraction of scene data, depth information and continuous image frames by performing type analysis on image features, and provides a rich information basis for subsequent scene recognition and pose estimation. By combining the scene recognition results and pose estimation results with task requirements and inputting them into an adaptive hybrid learning model for decision generation, the navigation strategy is flexibly adapted to specific tasks. This method not only takes environmental factors into consideration, but also incorporates task requirements into the decision-making process, thereby improving the pertinence and effectiveness of the navigation strategy. By introducing an adaptive optimization mechanism based on an attention mechanism and a reward shaping condition, the learning model can be continuously adjusted and optimized according to feedback information during the navigation process. This adaptive learning mechanism significantly improves the adaptability of the unmanned aerial vehicle in complex and dynamic environments, so that the navigation strategy can be continuously optimized as the environment changes. By processing continuous image frames and depth information through an optical flow algorithm, more accurate pose estimation is realized, providing reliable spatial positioning information for navigation decisions. This method effectively improves the positioning accuracy and navigation stability of the unmanned aerial vehicle in a complex environment. In summary, the present invention effectively solves the problems of perception limitations, insufficient task adaptability and weak environmental adaptability in existing UAV visual navigation through multi-source information fusion, task adaptability decision-making and adaptive learning optimization, and significantly improves the navigation performance and task execution capability of UAVs in complex and dynamic environments.

[0169] The present invention also provides a UAV visual navigation system based on deep learning, which is characterized by comprising:

[0170] Memory, used to store programs;

[0171] A processor is used to execute a program to implement each step of any of the above-mentioned deep learning-based unmanned aerial vehicle visual navigation methods.

[0172] In this embodiment, the processor and the memory may be connected via a bus or other means. The memory may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk, or a solid-state drive. The processor may be a general-purpose processor, such as a central processing unit, a digital signal processor, an application-specific integrated circuit, or one or more integrated circuits configured to implement the embodiments of the present invention.

[0173] The present invention also provides a storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute any of the above methods.

[0174] It should be noted that technicians in the relevant technical field can clearly understand that for the convenience and conciseness of description, the specific working process of the system and each module described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0175] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A UAV visual navigation method based on deep learning, characterized in that: include: Acquire a visual image collected by a visual system of the unmanned aerial vehicle, perform feature extraction on the visual image, and obtain an image feature representation; Performing type analysis on the image feature representation to obtain scene data, depth information and continuous image frames; Perform scene recognition on the scene data to obtain the recognition results of obstacles and passable areas; Performing optical flow estimation on the continuous image frames and the depth information by using a preset optical flow algorithm to obtain a pose estimation result; Obtaining the mission requirements of the UAV, inputting the recognition result, the pose estimation result and the mission requirements into a preset adaptive hybrid learning model for decision generation, and obtaining a navigation control strategy with environmental adaptability; Obtaining feedback information after applying the navigation control strategy, optimizing the adaptive hybrid learning model according to the feedback information combined with a preset attention mechanism and reward shaping conditions, and obtaining the optimized adaptive hybrid learning model; The step of inputting the recognition result, the pose estimation result and the task requirement into a preset adaptive hybrid learning model for decision generation to obtain a navigation control strategy with environmental adaptability includes: Inputting the recognition result, the pose estimation result and the task requirement into the adaptive hybrid learning model; Performing feature convolution extraction on the recognition result through a convolutional network layer to obtain visual features; Performing state transformation on the pose estimation result through a spatial transformation layer to generate a state vector; Performing vector analysis on the task requirements through an embedding layer to obtain a corresponding task vector; Integrate the visual features, the state vector and the task vector through a concatenation layer to obtain an input state vector; Inputting the input state vector into the Deep Q-Network layer to estimate the action value and obtain an action value set; Inputting the action value set into the argmax layer to perform navigation action decision analysis to obtain a preliminary decision result; Inputting the preliminary decision result into the LSTM layer for decision optimization to obtain an optimized decision result; Inputting the optimization decision result into the mapping layer for control command mapping to obtain the flight control command of the UAV; The flight control instructions are input into the strategy generation layer for strategy generation, and the corresponding navigation control strategy is output.

2. The deep learning-based UAV visual navigation method according to claim 1, characterized in that: The step of acquiring a visual image collected by a visual system of a drone and performing feature extraction on the visual image to obtain an image feature representation includes: Performing multi-scale feature extraction on the visual image to obtain multiple feature maps; Performing visual space matching on the plurality of feature maps to obtain corresponding visual feature vectors; Performing nonlinear feature transformation on the visual feature vector to obtain a visual feature representation; Calculating feature weights on the visual feature representation to obtain a weighted feature representation; The image feature representation is obtained by performing image conversion on the weighted feature representation using a preset principal component analysis method.

3. The deep learning-based UAV visual navigation method according to claim 1, characterized in that: The performing type analysis on the image feature representation to obtain scene data, depth information and continuous image frames includes: Decomposing the image feature representation in multiple levels to obtain a multi-level feature description; Performing semantic segmentation on the image features to obtain a semantic segmentation result; Performing scene extraction and fusion on the multi-level feature description according to the semantic segmentation result to obtain the scene data; Performing depth calculation on the multi-level feature description to obtain initial depth data; Extracting image frames from the image feature representation and performing time sequence sorting on the image frames to obtain the continuous image frames; Performing motion estimation on the multi-level feature description according to the continuous image frames to obtain a motion vector; The initial depth data is corrected according to the motion vector to obtain the depth information.

4. The deep learning-based UAV visual navigation method according to claim 1, characterized in that: The performing scene recognition on the scene data to obtain recognition results of obstacles and passable areas includes: Performing semantic segmentation on the scene data to obtain semantic labels for each area in the scene; Preliminarily classifying the areas in the scene according to the semantic labels to obtain initial obstacles and traversable areas; Performing edge detection and contour extraction on the initial obstacle and the traversable area to obtain comprehensive boundary information; Estimating the three-dimensional spatial position of the initial obstacle according to the comprehensive boundary information to obtain spatial distribution information of the obstacle; A three-dimensional occupancy grid map analysis is performed on the spatial distribution information and the range of the passable area to obtain the identification results of the obstacles and the passable area.

5. The deep learning-based UAV visual navigation method according to claim 1, characterized in that: The performing optical flow estimation on the continuous image frames and the depth information by using a preset optical flow algorithm to obtain a pose estimation result includes: Performing image pyramid construction on the continuous image frames to obtain a multi-level image representation; Performing feature point matching detection on each level image in the multi-level image representation to obtain corresponding level feature points; Performing optical flow calculation on the hierarchical feature points according to the optical flow algorithm to obtain corresponding hierarchical optical flow vectors; Performing dense optical flow interpolation on the multi-level image representation according to the hierarchical optical flow vectors to obtain a corresponding hierarchical optical flow field; Performing a refinement optical flow fusion process on the hierarchical optical flow field to obtain an initial optical flow field; Performing depth weighting on the initial optical flow field according to the depth information to obtain a weighted optical flow field; Calculating the motion matrix between adjacent frames according to the weighted optical flow field to obtain the inter-frame pose transformation; The inter-frame pose transformation is cumulatively calculated to obtain the pose estimation result.

6. The deep learning-based UAV visual navigation method according to claim 1, characterized in that: The obtaining of feedback information after applying the navigation control strategy, and optimizing the adaptive hybrid learning model according to the feedback information in combination with a preset attention mechanism and reward shaping conditions, comprises: The flight status of the UAV after applying the navigation control strategy is collected to obtain status feedback information including position, speed, acceleration and attitude angle; Reconstructing the flight trajectory of the UAV according to the state feedback information to obtain an actual flight trajectory; Calculating the deviation between the actual flight trajectory and the preset ideal flight trajectory to obtain a trajectory deviation; Adjusting the attention weight of the adaptive hybrid learning model according to a preset adaptive multi-head attention mechanism combined with the trajectory deviation to obtain an updated attention distribution; Performing reward evaluation calculation on the trajectory deviation according to the preset reward shaping condition to obtain a corresponding shaping reward value; Optimizing the loss function of the adaptive hybrid learning model according to the shaping reward value to obtain an optimized loss function; The parameters of the adaptive hybrid learning model are gradient updated based on the optimized loss function and the attention distribution to obtain the optimized adaptive hybrid learning model.

7. A UAV visual navigation device based on deep learning, characterized in that: The deep learning-based unmanned aerial vehicle visual navigation method applied to any one of claims 1 to 6 above comprises: An acquisition module, the acquisition module is used to acquire a visual image acquired by a UAV visual system, perform feature extraction on the visual image, and obtain an image feature representation; An analysis module, the analysis module is used to perform type analysis on the image feature representation to obtain scene data, depth information and continuous image frames; An association module, the association module is used to perform scene recognition on the scene data to obtain recognition results of obstacles and passable areas; A processing module, the processing module is used to perform optical flow estimation on the continuous image frames and the depth information by using a preset optical flow algorithm to obtain a pose estimation result; A control module, wherein the control module is used to obtain the mission requirements of the UAV, input the recognition result, the pose estimation result and the mission requirements into a preset adaptive hybrid learning model for decision generation, and obtain a navigation control strategy with environmental adaptability, An execution module, the execution module is used to obtain feedback information after applying the navigation control strategy, and optimize the adaptive hybrid learning model according to the feedback information in combination with a preset attention mechanism and reward shaping conditions; The step of inputting the recognition result, the pose estimation result and the task requirement into a preset adaptive hybrid learning model for decision generation to obtain a navigation control strategy with environmental adaptability includes: Inputting the recognition result, the pose estimation result and the task requirement into the adaptive hybrid learning model; Performing feature convolution extraction on the recognition result through a convolutional network layer to obtain visual features; Performing state transformation on the pose estimation result through a spatial transformation layer to generate a state vector; Performing vector analysis on the task requirements through an embedding layer to obtain a corresponding task vector; Integrate the visual features, the state vector and the task vector through a concatenation layer to obtain an input state vector; Inputting the input state vector into the Deep Q-Network layer to estimate the action value and obtain an action value set; Inputting the action value set into the argmax layer to perform navigation action decision analysis to obtain a preliminary decision result; Inputting the preliminary decision result into the LSTM layer for decision optimization to obtain an optimized decision result; Inputting the optimization decision result into the mapping layer for control command mapping to obtain the flight control command of the UAV; The flight control instructions are input into the strategy generation layer for strategy generation, and the corresponding navigation control strategy is output.

Images