Unmanned helicopter path planning method and system based on time sequence space prediction
By combining a temporal spatial prediction-based method with binocular image processing and a pyramid TCN temporal network model, the problem of local optima in path planning for unmanned helicopters is solved, achieving high-precision and efficient path planning that is suitable for obstacle avoidance in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-15
- Publication Date
- 2026-03-20
AI Technical Summary
Traditional unmanned helicopter path planning algorithms ignore the relationship between time and space, and are prone to getting trapped in local optima.
A temporal spatial prediction-based approach is adopted, which uses binocular image feature extraction, disparity map acquisition, and obstacle 3D position information, combined with the pyramid TCN temporal prediction network model for path planning, and integrates multi-source information for trajectory prediction.
It significantly improves the accuracy and efficiency of path planning, effectively copes with obstacle avoidance in complex dynamic environments, has strong robustness and adaptability, and provides reliable protection for the safe flight of unmanned helicopters.
Smart Images
Figure CN121702375A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of airborne system design technology, and in particular to a method and system for unmanned helicopter path planning based on temporal spatial prediction. Background Technology
[0002] This year, the unmanned helicopter market has flourished, with application scenarios continuously expanding and the industry scale gradually increasing. It is widely used in fields such as national defense and security, agricultural and forestry protection, aerial photography, logistics and transportation, urban planning, environmental monitoring, and emergency inspection. Traditional path planning algorithms use heuristic methods to plan the flight paths of unmanned helicopters, ignoring the temporal and spatial relationships between different characteristic parameters, and are prone to getting trapped in local optima. Summary of the Invention
[0003] In view of this, embodiments of this application provide a method and system for unmanned helicopter path planning based on temporal spatial prediction, which at least partially solves the problem that the path planning of unmanned helicopters in the prior art ignores the interrelationship of different characteristic parameters in time and space, and is prone to getting trapped in local optima.
[0004] In a first aspect, embodiments of this application provide a path planning method for unmanned helicopters based on temporal spatial prediction, the method comprising: Acquire binocular images of the target scene of the unmanned helicopter, extract features from the binocular images, and obtain feature information and two-dimensional position information of obstacles; Based on the binocular images, a disparity map is obtained. Based on the disparity map, the feature information of the obstacle, and the two-dimensional position information, the outline information and three-dimensional position information of the obstacle are obtained. Based on the outline information of obstacles, the three-dimensional position information of obstacles, and the path of unmanned helicopters at different times, the obstacle trajectory information and the distance information between the unmanned helicopter path and obstacles that change over time are obtained. For obstacle trajectory information that changes over time and distance information between the unmanned helicopter path and obstacles that changes over time, a pyramid-based TCN time-series prediction network model is used for calculation, and the calculation results are fused to perform trajectory prediction and obtain the final unmanned helicopter path planning result.
[0005] According to a specific implementation of an embodiment of this application, the feature extraction of the binocular image includes: The stereo images are used to extract features using either a VGG-16 network structure or a ResNet network structure.
[0006] According to a specific implementation of an embodiment of this application, the pyramid-based TCN time series prediction network model includes a first neural network layer, a second neural network layer, and a first TCN time series prediction network layer connected in sequence.
[0007] According to a specific implementation of an embodiment of this application, both the first neural network layer and the second neural network layer include a 1x1 convolutional layer, a 3x3 convolutional layer, and a ReLU residual layer connected in sequence.
[0008] According to a specific implementation of an embodiment of this application, the pyramid-based TCN time series prediction network model further includes a normalization layer, which is located between the second neural network layer and the first TCN time series prediction network layer.
[0009] According to a specific implementation of an embodiment of this application, the step of fusing the calculation results to perform trajectory prediction and obtain the final unmanned helicopter path planning result includes: The calculation results are then fused using a matrix expansion algorithm to obtain fused data. The fused data is input into the second TCN temporal prediction network layer for temporal feature extraction to capture the spatiotemporal correlation characteristics in path planning for trajectory prediction, thereby obtaining the final path planning result for the unmanned helicopter.
[0010] According to a specific implementation of an embodiment of this application, the method further includes: Set a dropout model after the second TCN time series prediction network layer.
[0011] According to a specific implementation of an embodiment of this application, the method further includes: Calculate the root mean square error between the final unmanned helicopter path planning result and the actual unmanned helicopter path; Loss convergence calculations are performed based on root mean square error and label data to optimize the path planning method for unmanned helicopters.
[0012] According to a specific implementation of an embodiment of this application, the step of obtaining a disparity map based on the binocular image includes: The disparity map is obtained using the SGBM algorithm based on the stereo image.
[0013] Secondly, embodiments of this application also provide an unmanned helicopter path planning system based on temporal spatial prediction, used to implement the unmanned helicopter path planning method based on temporal spatial prediction as described in any of the embodiments of the first aspect above, the system comprising: The feature extraction module is used to acquire binocular images of the target scene of the unmanned helicopter, extract features from the binocular images, and obtain feature information and two-dimensional position information of obstacles. The binocular ranging module is used to acquire a disparity map based on the binocular image, and to obtain the outline information and three-dimensional position information of the obstacle based on the disparity map, the feature information of the obstacle, and the two-dimensional position information. The dataset construction module is used to obtain time-varying obstacle trajectory information and time-varying distance information between the UAV path and obstacles based on the outline information of obstacles, the three-dimensional position information of obstacles, and the path of the UAV at different times. The path planning module is used to calculate the obstacle trajectory information and the distance information between the unmanned helicopter path and the obstacle information that change over time using a pyramid-based TCN time-series prediction network model. The calculation results are then fused to perform trajectory prediction and obtain the final unmanned helicopter path planning result.
[0014] Beneficial effects: The unmanned helicopter path planning method and system based on temporal spatial prediction in this application significantly improves the accuracy and efficiency of path planning by integrating multi-source information and deep learning technology. This method effectively addresses obstacle avoidance problems in complex dynamic environments while overcoming the tendency of traditional algorithms to get trapped in local optima. The system design fully considers the needs of practical application scenarios, exhibiting strong robustness and adaptability, and can be widely applied to various unmanned helicopter operations. By introducing an advanced temporal prediction network model, accurate prediction of obstacle trajectories is achieved, providing a reliable guarantee for the safe flight of unmanned helicopters. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart of an unmanned helicopter path planning method based on temporal spatial prediction according to an embodiment of the present invention. Detailed Implementation
[0017] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.
[0020] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The illustrations only show the components related to this application and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0021] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.
[0022] Firstly, referring to Figure 1 This application provides a method for unmanned helicopter path planning based on temporal spatial prediction, the method comprising: Acquire binocular images of the target scene of the unmanned helicopter, extract features from the binocular images, and obtain feature information and two-dimensional position information of obstacles; Based on the binocular images, a disparity map is obtained. Based on the disparity map, the feature information of the obstacle, and the two-dimensional position information, the outline information and three-dimensional position information of the obstacle are obtained. Based on the outline information of obstacles, the three-dimensional position information of obstacles, and the path of unmanned helicopters at different times, the obstacle trajectory information and the distance information between the unmanned helicopter path and obstacles that change over time are obtained. For obstacle trajectory information that changes over time and distance information between the unmanned helicopter path and obstacles that changes over time, a pyramid-based TCN time-series prediction network model is used for calculation, and the calculation results are fused to perform trajectory prediction and obtain the final unmanned helicopter path planning result.
[0023] In this embodiment, by fusing multi-source information and deep learning technology, the accuracy and efficiency of path planning are significantly improved. This method effectively addresses obstacle avoidance in complex dynamic environments while overcoming the tendency of traditional algorithms to get trapped in local optima. The system design fully considers the needs of practical application scenarios, exhibiting strong robustness and adaptability, and can be widely applied to various unmanned helicopter operations. By introducing an advanced temporal prediction network model, accurate prediction of obstacle trajectories is achieved, providing reliable assurance for the safe flight of unmanned helicopters.
[0024] In one embodiment, feature extraction of the binocular image includes: The stereo images are used to extract features using either a VGG-16 network structure or a ResNet network structure.
[0025] In practice, when using the VGG-16 network structure to extract features from binocular images, features from low to high levels are extracted progressively through the alternating stacking of 13 convolutional layers and 5 pooling layers. The first 5 sets of convolutional blocks (each containing 2-3 3×3 convolutional kernels) are responsible for capturing local details such as edges and textures, and are combined with 2×2 max pooling to achieve feature dimensionality reduction and receptive field expansion. The last 3 sets of fully connected layers map the convolutional features into fixed-dimensional vectors, effectively preserving the geometric contours and grayscale distribution features of obstacles.
[0026] The ResNet network architecture addresses the vanishing gradient problem in deep network training through residual skip connections, enabling the construction of deeper network architectures (such as 50 or 101 layers). When processing stereo images, the input image undergoes initial 7×7 convolutional layers and 3×3 pooling layers, followed by feature enhancement through multiple residual units (including 1×1 convolutional dimensionality reduction, 3×3 convolutional feature extraction, and 1×1 convolutional dimensionality increase). Finally, a high-dimensional feature vector is output through global average pooling. Both networks support end-to-end training. VGG-16, with its fixed receptive field, is suitable for extracting features from regular obstacles, while ResNet, with its depth advantage, more accurately captures the details of minute obstacles in complex scenes, providing multi-level feature support for subsequent stereo disparity calculation and 3D localization.
[0027] Depending on the specific requirements, either a VGG-16 network structure or a ResNet network structure can be selected to extract features from the binocular images, thereby obtaining obstacle feature information and two-dimensional position information. Obstacle feature information includes shape features, texture features, and structural features.
[0028] In one embodiment, the pyramid-based TCN time-series prediction network model includes a first neural network layer, a second neural network layer, and a first TCN time-series prediction network layer connected in sequence.
[0029] Furthermore, refer to Figure 1 Both the first neural network layer and the second neural network layer include a 1x1 convolutional layer, a 3x3 convolutional layer, and a ReLU residual layer connected in sequence.
[0030] Specifically, the pyramid-based TCN temporal prediction network model adopts a three-layer cascaded architecture, including a first neural network layer, a second neural network layer, and a first TCN temporal prediction network layer connected in sequence. The first neural network layer is responsible for preliminary feature extraction and dimensionality compression of the input obstacle trajectory temporal data. The second neural network layer enhances feature representation capabilities through multi-scale convolution operations, forming a pyramid-shaped feature hierarchy structure. Finally, the first TCN temporal prediction network layer performs long-sequence dependency modeling on the fused spatiotemporal features to achieve accurate prediction of dynamic obstacle trajectories. Both the first and second neural network layers adopt a "convolution-activation-residual" three-segment structure, specifically including a 1x1 convolutional layer, a 3x3 convolutional layer, and a ReLU residual layer connected in sequence. The 1x1 convolutional layer achieves feature dimensionality reduction (the number of channels is compressed to 1 / 2 of the original dimension) through cross-channel information interaction. The 3x3 convolutional layer uses depthwise separable convolution to extract local spatial features. The ReLU residual layer adds the input features and the convolution result element by element through skip connections, effectively alleviating the gradient vanishing problem in deep network training and enabling the network to stably learn obstacle motion patterns in complex dynamic scenes.
[0031] In this embodiment, two trajectory routes (obstacle trajectory information that changes over time and distance information between the unmanned helicopter path and the obstacle that changes over time) are used as inputs to two neural network layers. The neighboring trajectory information of the current point is comprehensively considered through two 3x3 convolutional layers. This network structure adopts a simple neural network structure, reducing the computational load by reducing the number of parameters.
[0032] Furthermore, the pyramid-based TCN time series prediction network model also includes a normalization layer, which is located between the second neural network layer and the first TCN time series prediction network layer.
[0033] In practice, to accelerate computation, the entire network is normalized, meaning all parameters are within the range of 0-1, thereby improving the model's convergence speed and stability. The normalization layer standardizes the data, ensuring that the input to each layer remains within the same scale, avoiding gradient update problems caused by uneven data distribution. This design not only optimizes the training process but also enhances the model's adaptability to complex dynamic environments. Furthermore, the introduction of the normalization layer effectively reduces the network's sensitivity to hyperparameters, further improving the model's robustness.
[0034] In one embodiment, fusing the calculation results to perform trajectory prediction and obtain the final unmanned helicopter path planning result includes: The calculation results are then fused using a matrix expansion algorithm to obtain fused data. The fused data is input into the second TCN temporal prediction network layer for temporal feature extraction to capture the spatiotemporal correlation characteristics in path planning for trajectory prediction, thereby obtaining the final path planning result for the unmanned helicopter.
[0035] In this embodiment, the method deeply fuses two different path trajectory information using matrix operation techniques. Specifically, the process first extracts features and spatially encodes the two initial path routes, then uses a matrix expansion algorithm to cross-fuse the multidimensional feature information of these two trajectories, forming a comprehensive path representation. Based on this, the system employs a one-layer Temporal Convolutional Network (TCN) model to process the fused path data. This model has powerful temporal feature extraction capabilities, effectively capturing the spatiotemporal correlation characteristics in path planning. Through deep learning and predictive analysis of the TCN network, the optimal path planning result is finally output. This result not only considers the spatial distribution of obstacles but also incorporates the dynamic changes in the time series, thus achieving more accurate and reliable path prediction. Through this multi-layered information fusion and modeling method, unmanned helicopters can quickly respond and generate optimal flight paths in complex environments, while effectively avoiding potential safety hazards.
[0036] Furthermore, this method demonstrates strong adaptability and scalability in practical applications. For example, performance can be further improved by adjusting network parameters, optimizing dataset distribution, or introducing new feature extraction algorithms to address different types of unmanned helicopters or specific mission requirements. Particularly for real-time path planning problems in highly dynamic scenarios, this method exhibits excellent computational efficiency and robustness, meeting the stringent requirements of real-world deployments.
[0037] To verify the effectiveness of this method, multiple simulation experiments and field tests were conducted. Experimental results show that, compared to traditional path planning algorithms, this method significantly improves path smoothness, obstacle avoidance accuracy, and computational efficiency. Especially when facing complex terrain and sudden obstacles, it demonstrates stronger environmental adaptability and decision-making flexibility, providing strong support for the intelligent development of unmanned helicopters.
[0038] Furthermore, the method also includes: Set a dropout model after the second TCN time series prediction network layer.
[0039] By randomly discarding some neurons during training, the dropout model effectively prevents overfitting and improves the generalization ability of path planning. In this way, the model can maintain high prediction accuracy and stability when facing unknown scenarios.
[0040] In practical deployments, to achieve good compatibility and scalability, environmental awareness can be enhanced by integrating more sensor data (such as LiDAR or infrared imaging), or the intelligence of path planning can be further improved by combining reinforcement learning algorithms. Simultaneously, users can flexibly adjust the hyperparameters in the network structure to achieve the optimal balance between performance and resource consumption for different task requirements.
[0041] In one embodiment, the method further includes: Calculate the root mean square error between the final unmanned helicopter path planning result and the actual unmanned helicopter path; Loss convergence calculations are performed based on root mean square error and label data to optimize the path planning method for unmanned helicopters.
[0042] In this embodiment, the root mean square error (RMSE) between the predicted path and the actual path is calculated to evaluate the model's performance. By continuously adjusting network parameters and optimizing the algorithm, the loss function gradually converges, thereby improving the accuracy and reliability of path planning. During training, a mini-batch gradient descent method is used to update the model weights, while a learning rate decay strategy is introduced to ensure that the model can quickly and stably reach its optimal state. Finally, after multiple rounds of iterative training, the model can generate high-quality path planning results in different scenarios, providing a solid technical guarantee for the practical application of unmanned helicopters.
[0043] In one embodiment, obtaining the disparity map based on the binocular images includes: The disparity map is obtained using the SGBM algorithm based on the stereo image.
[0044] This embodiment employs the Semi-Global Block Matching (SGBM) algorithm to process binocular images to obtain a high-precision disparity map. This algorithm effectively reduces noise and mismatch problems common in traditional local matching algorithms by optimizing paths in multiple directions. In practice, the binocular images are first preprocessed, including grayscale conversion, histogram equalization, and image correction, to ensure consistency and clarity between the left and right views. Subsequently, a cost calculation function is used to evaluate the matching cost of each pixel, and path costs are aggregated in multiple directions using dynamic programming to ultimately generate a globally optimal disparity map. This method not only significantly improves the accuracy of the disparity map but also possesses strong robustness, adapting to application needs under different lighting conditions and complex scenarios.
[0045] Secondly, embodiments of this application also provide an unmanned helicopter path planning system based on temporal spatial prediction, used to implement the unmanned helicopter path planning method based on temporal spatial prediction as described in any of the embodiments of the first aspect above, the system comprising: The feature extraction module is used to acquire binocular images of the target scene of the unmanned helicopter, extract features from the binocular images, and obtain feature information and two-dimensional position information of obstacles. The binocular ranging module is used to acquire a disparity map based on the binocular image, and to obtain the outline information and three-dimensional position information of the obstacle based on the disparity map, the feature information of the obstacle, and the two-dimensional position information. The dataset construction module is used to obtain time-varying obstacle trajectory information and time-varying distance information between the UAV path and obstacles based on the outline information of obstacles, the three-dimensional position information of obstacles, and the path of the UAV at different times. The path planning module is used to calculate the obstacle trajectory information and the distance information between the unmanned helicopter path and the obstacle information that change over time using a pyramid-based TCN time-series prediction network model. The calculation results are then fused to perform trajectory prediction and obtain the final unmanned helicopter path planning result.
[0046] The embodiments provided by this invention significantly improve the accuracy and efficiency of path planning by integrating multi-source information and deep learning technology. This method effectively addresses obstacle avoidance problems in complex dynamic environments while overcoming the tendency of traditional algorithms to get trapped in local optima. The system design fully considers the needs of practical application scenarios, exhibiting strong robustness and adaptability, and can be widely applied to various unmanned helicopter operations. By introducing an advanced temporal prediction network model, accurate prediction of obstacle trajectories is achieved, providing reliable assurance for the safe flight of unmanned helicopters.
[0047] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A path planning method for unmanned helicopters based on temporal spatial prediction, characterized in that, The method includes: Acquire binocular images of the target scene of the unmanned helicopter, extract features from the binocular images, and obtain feature information and two-dimensional position information of obstacles; Based on the binocular images, a disparity map is obtained. Based on the disparity map, the feature information of the obstacle, and the two-dimensional position information, the outline information and three-dimensional position information of the obstacle are obtained. Based on the outline information of obstacles, the three-dimensional position information of obstacles, and the path of unmanned helicopters at different times, the obstacle trajectory information and the distance information between the unmanned helicopter path and obstacles that change over time are obtained. For obstacle trajectory information that changes over time and distance information between the unmanned helicopter path and obstacles that changes over time, a pyramid-based TCN time-series prediction network model is used for calculation, and the calculation results are fused to perform trajectory prediction and obtain the final unmanned helicopter path planning result.
2. The unmanned helicopter path planning method based on temporal spatial prediction according to claim 1, characterized in that, The feature extraction of the binocular image includes: The stereo images are used to extract features using either a VGG-16 network structure or a ResNet network structure.
3. The unmanned helicopter path planning method based on temporal spatial prediction according to claim 1, characterized in that, The pyramid-based TCN time-series prediction network model includes a first neural network layer, a second neural network layer, and a first TCN time-series prediction network layer connected in sequence.
4. The unmanned helicopter path planning method based on temporal spatial prediction according to claim 3, characterized in that, Both the first neural network layer and the second neural network layer include a 1x1 convolutional layer, a 3x3 convolutional layer, and a ReLU residual layer connected in sequence.
5. The unmanned helicopter path planning method based on temporal spatial prediction according to claim 3, characterized in that, The pyramid-based TCN time-series prediction network model also includes a normalization layer, which is located between the second neural network layer and the first TCN time-series prediction network layer.
6. The unmanned helicopter path planning method based on temporal spatial prediction according to claim 1, characterized in that, The process of fusing the calculation results to predict the trajectory and obtain the final path planning result for the unmanned helicopter includes: The calculation results are then fused using a matrix expansion algorithm to obtain fused data. The fused data is input into the second TCN temporal prediction network layer for temporal feature extraction to capture the spatiotemporal correlation characteristics in path planning for trajectory prediction, thereby obtaining the final path planning result for the unmanned helicopter.
7. The unmanned helicopter path planning method based on temporal spatial prediction according to claim 6, characterized in that, The method further includes: Set a dropout model after the second TCN time series prediction network layer.
8. The unmanned helicopter path planning method based on temporal spatial prediction according to claim 1, characterized in that, The method further includes: Calculate the root mean square error between the final unmanned helicopter path planning result and the actual unmanned helicopter path; Loss convergence calculations are performed based on root mean square error and label data to optimize the path planning method for unmanned helicopters.
9. The unmanned helicopter path planning method based on temporal spatial prediction according to claim 1, characterized in that, The process of obtaining the disparity map based on the binocular images includes: The disparity map is obtained using the SGBM algorithm based on the stereo image.
10. A path planning system for unmanned helicopters based on temporal spatial prediction, used to implement the path planning method for unmanned helicopters based on temporal spatial prediction as described in any one of claims 1-9, characterized in that, The system includes: The feature extraction module is used to acquire binocular images of the target scene of the unmanned helicopter, extract features from the binocular images, and obtain feature information and two-dimensional position information of obstacles. The binocular ranging module is used to acquire a disparity map based on the binocular image, and to obtain the outline information and three-dimensional position information of the obstacle based on the disparity map, the feature information of the obstacle, and the two-dimensional position information. The dataset construction module is used to obtain time-varying obstacle trajectory information and time-varying distance information between the UAV path and obstacles based on the outline information of obstacles, the three-dimensional position information of obstacles, and the path of the UAV at different times. The path planning module is used to calculate the obstacle trajectory information and the distance information between the unmanned helicopter path and the obstacle information that change over time using a pyramid-based TCN time-series prediction network model. The calculation results are then fused to perform trajectory prediction and obtain the final unmanned helicopter path planning result.