Unmanned aerial vehicle end-to-end planning method and system based on brain-like calculation and guided learning

Abstract

The invention discloses an unmanned aerial vehicle end-to-end planning method and system based on brain-like calculation and guide learning, and the method comprises the steps: designing an unmanned aerial vehicle planning algorithm based on an end-to-end single stage based on a Spike response model of a pulse neural network and a pulse network model based on Poisson coding; a high-order polynomial trajectory representation and a neural network architecture integrating perception, front-end path search and back-end optimization are provided. In order to optimize a network structure more efficiently, guide learning based on an Actor-Critic framework and privileged learning based on an ESDF map numerical gradient are designed. The system comprises a brain-like chip, a calculation module, an unmanned aerial vehicle body, a task load and the like. According to an unmanned aerial vehicle obstacle avoidance algorithm and a module based on brain-like calculation in an environment with an obstacle, through input of a depth image, a current state and a target direction, and end-to-end output of an unmanned aerial vehicle planning track and a control quantity, rapid real-time autonomous planning is realized, a flight obstacle avoidance effect is improved, and calculation power consumption is reduced at the same time.

Description

Technical Field

[0001] This invention belongs to the field of intelligent planning for unmanned aerial vehicles (UAVs) and neuromorphic computing, specifically relating to an end-to-end planning method and system for UAVs based on neuromorphic computing and guided learning. Background Technology

[0002] Brain-inspired planning techniques are gaining increasing attention and have become a hot topic at the intersection of artificial intelligence and neuroscience. A widely used pseudo-neural network in brain-inspired computing is the spiking neural network (SNN). The main difference between artificial neural networks (ANNs) and spiking neural networks (SNNs) lies in the design of their neurons. In ANNs, the function of neurons is described by activation functions; while in spiking neural networks, neurons are called spiking neurons, and their behavior is modeled by an ordinary differential equation (ODE) system. This ODE system describes the spiking neuron as a dynamic system in which membrane potentials accumulate as hidden states. When the membrane potential exceeds a specific threshold on its rising edge, an event (or pulse) is triggered, known as a "pulse." Pulses are the fundamental mode of communication between neurons, and a series of pulses constitutes a pulse sequence. The existence of pulses enables neurons to achieve asynchronous communication. In recent years, progress has been made in the research of brain-inspired navigation mechanisms, algorithm development, and practical applications. Some research teams are dedicated to simulating the neural network structure and information processing mechanisms of biological systems, developing brain-inspired planning systems with autonomous environmental perception and spatial cognition capabilities, and applying them to planning and navigation tasks for vehicles such as unmanned vehicles and drones. Training SNNs is a complex task due to the non-differentiability of the function used to generate pulses, given their inherent complexity. Therefore, researchers have developed various training methods, broadly categorized into three types: 1) bio-precise methods, 2) hybrid methods, and 3) gradient descent-based methods. Within the bio-precise methods, all methods based on SpikeTiming-DependentPlasticity (STDP) fall into this category. These methods suffer from two main problems: 1) they can only train shallow networks, and 2) STDP primarily acts as a switch for connections, limiting the network's expressive power. This training method has been applied to supervised learning tasks; in reinforcement learning, it has solved a simple obstacle avoidance task, where a Lego robot must avoid walls and obstacles in its environment. The second type of training method first trains an Artificial Neural Network (ANN) and then converts it into an Adaptive Neural Network (SNN). This method has solved various tasks, such as performing obstacle avoidance tasks, teaching a network to drive using LiDAR, and solving Atari games on OpenAIGym. However, a drawback of this method is that the expressive potential of SNNs, which is considered to be greater than that of ANNs, is not fully utilized during the training phase. The last class of methods addresses the training problem caused by the non-differentiable activation function of spiking neurons by using backpropagation or its approximation.

[0003] Furthermore, autonomous navigation of quadcopters in obstacle-dense environments has been extensively studied. However, limited computational resources and low-precision airborne sensors still make this task quite challenging. The autonomous navigation problem for unmanned aerial vehicles (UAVs) is typically divided into: 1) perception and mapping, 2) front-end path search, and 3) back-end trajectory optimization. To avoid suboptimal solutions, front-end algorithms such as topology path search are used to find multiple initial paths. Finally, a back-end optimization module further refines the path, making it smooth, safe, and dynamically feasible. This divide-and-conquer pipeline makes the entire system more interpretable but introduces additional latency, which is fatal for high-speed flight. In recent years, learning planners capable of processing raw sensory input and directly executing complex behaviors have shown great potential. Some strategies are trained by imitating privileged experts, while others use reinforcement learning to explore optimal strategies through trial and error. Methods based on supervised learning (imitation learning or behavior cloning) have achieved remarkable results by training a lightweight network to approximate solutions from computationally expensive algorithms. Specifically, the network policy is trained by minimizing the distance between the predicted value and the label shown by the expert. However, evaluation metrics should consider smoothness and safety, not just distance from the optimal demonstration. On the other hand, the upper bound of a network policy is the expert policy, making it highly dependent on expert performance. Conversely, network policies in reinforcement learning are trained by maximizing rewards from the environment, which truly reflects the performance of actions. However, compared to the direct correlation between input and label in supervised learning, the reward signal in reinforcement learning is typically sparse, noisy, and delayed, making convergence more difficult. Furthermore, there are other methods to train networks to predict collisions of predefined motion primitives, which accurately reflect the feasibility of trajectories and account for the multimodal nature of planning problems. However, a limited set of predefined primitives cannot represent complex trajectories or comprehensively cover feasible solutions. Summary of the Invention

[0004] The primary objective of this invention is to provide an end-to-end planning method for unmanned aerial vehicles (UAVs) based on neuromorphic computing and guided learning. The method takes a depth image, current state, and target orientation as input and outputs the planned trajectory and control variables of the UAV directly end-to-end. The end-to-end technology reduces the online planning time of the UAV, and the neuromorphic computing technology reduces the energy consumption of the UAV planning.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: an end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning.

[0006] Preferably, the end-to-end UAV planning algorithm proposes an efficient learning-based one-stage planner that integrates perception, front-end path search, and back-end optimization into a single neural network, generating multiple alternative trajectories to comprehensively explore the solution space. Specifically, it includes using depth image, current state, and target orientation as end-to-end inputs. It designs a Spike response model based on a spiking neural network and a spiking network model based on Poisson coding. It integrates the neural network architecture for perception, front-end path search, and back-end optimization. It employs guided learning based on the Actor-Critic framework and privileged learning training based on ESDF map numerical gradients. The end-to-end output is the UAV planned trajectory and control variables.

[0007] The described neuromorphic computing, based on the overall design principles of neural network architecture, incorporates spiking neurons. During training, it activates integer-valued training data (using a BN normalization module and a spiking activation function), and simultaneously converts these values ​​into binary spikes during inference. Using integer values ​​for training and representing them as equivalent binary spike sequences during inference effectively reduces the quantization error of spiking neurons. Specifically, it includes a Spike response model based on spiking neural networks and a spiking network model based on Poisson coding.

[0008] Preferably, the Spike response model based on a spiking neural network includes constructing a spiking neural network (SNN) model and designing two evolution modes: hard reset and soft reset.

[0009] Preferably, the pulse network model based on Poisson coding includes:

[0010] Poisson-coded spiking neural networks encode information such as velocity, displacement, and target point as state inputs.

[0011] Introducing a lightweight attention mechanism enhances the feature extraction capability and information interaction of the spiking network model, thereby improving the efficiency of dynamic obstacle avoidance.

[0012] Design dynamic energy threshold models and dynamic time threshold models to maintain efficient obstacle navigation decision-making for UAVs.

[0013] Preferably, the end-to-end single-stage planning algorithm includes:

[0014] Noisy depth images are used as input. A lightweight backbone network is employed for perception and feature extraction.

[0015] A set of motion primitives (represented by boundary states) uniformly sampled within the field of view of the depth camera is used as the front end to comprehensively search the solution space.

[0016] Using primitives as anchors, their offsets and scores are predicted in parallel during a single forward propagation. The optimal primitive is selected based on its score, and the offset is applied to the primitive's state as a backend optimization.

[0017] By solving the boundary value problem, the optimal trajectory can be represented as a higher-order polynomial.

[0018] Preferably, the trajectory representation of the higher-order polynomial includes:

[0019] The trajectory in the body coordinate system is represented as three independent one-stage time-parameterized polynomials.

[0020] To provide initial paths with different topologies for local optimization, a set of motion primitives is used as anchors to cover the search space, with each primitive guiding independent optimization during training.

[0021] An end-to-end network is used to optimize the security and smoothness of the initial path.

[0022] Preferably, the guided learning based on the Actor-Critic framework proposes a realistic, accurate, and data-efficient unsupervised learning strategy that directly backpropagates numerical gradients from the environment to the weights of the neural network. This includes:

[0023] A guided learning method based on the Actor-Critic framework is used to train the end-to-end trajectory planning of UAVs.

[0024] The overall cost function is expressed as a weighted sum of the smoothness cost function, the safety cost function, and the temporary target distance cost function.

[0025] The numerical gradient is backpropagated directly to the network parameters using the chain rule, and the score is predicted.

[0026] Preferably, the privileged learning based on ESDF map numerical gradients is trained through privileged learning and integrated into a fully autonomous quadcopter system. This includes:

[0027] Based on the numerical gradient information of the ESDF map, we improve the data-driven method by utilizing the regularity in the training data.

[0028] A dataset of position, pose, and depth images was collected, and invalid depth values ​​were processed using nearest-neighbor interpolation, where position and pose were used only for gradient calculation. The state of the quadcopter was randomly reset in the Flight-mare simulator, and invalid depth values ​​were processed using nearest-neighbor interpolation to avoid confusion with nearby obstacles.

[0029] In the no-expert strategy, various states and targets are randomly initialized for each depth sample to augment the data without re-labeling or re-interacting with the simulator.

[0030] To ensure adequate coverage of the state space, a dataset aggregation strategy (DAgger) is used for further fine-tuning.

[0031] The second objective of this invention is to provide an end-to-end planning system for unmanned aerial vehicles (UAVs) based on a neuromorphic chip with a spiking neural network architecture. The hardware system of the planning system includes a neuromorphic chip and computing module, the UAV body and mission payload, etc.

[0032] Preferably, the neuromorphic chip and computing module based on neuromorphic computing are computing chips and computing modules that mimic the human brain's nervous system based on spiking neural networks (SNNs). They have characteristics such as pulse timing-dependent plasticity, short-term synaptic plasticity, and local steady-state regulation, and have ultra-low power consumption and parallel information processing capabilities. They include, but are not limited to, neuromorphic chips and computing modules based on neuromorphic chips.

[0033] Preferably, the UAV body and its payload include, but are not limited to, small, micro, and micro-mini multi-rotor UAVs and their related optoelectronic payloads. The multi-rotor UAVs include tri-rotor, quadcopter, hexacopter, and octacopter UAVs.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1) Based on the existing end-to-end neural network architecture that integrates perception, front-end path search and back-end optimization, we propose an algorithm based on guided learning of the Actor-Critic framework and privileged learning based on ESDF map numerical gradient, which makes the UAV end-to-end trajectory planning method more efficient.

[0036] 2) Based on the existing end-to-end hardware of UAVs, a brain-like computing method based on spiking neural networks was proposed, and an airborne brain-like chip module was developed accordingly, which makes the power consumption of UAV end-to-end planning lower and the parallel information processing capability stronger. Attached Figure Description

[0037] Figure 1 This is a flowchart of the end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning in this invention;

[0038] Figure 2 This is the Spike response model based on a spiking neural network in this invention;

[0039] Figure 3 This is the pulse network model based on Poisson coding in this invention;

[0040] Figure 4 This is the end-to-end single-stage UAV planning algorithm in this invention;

[0041] Figure 5 This is the trajectory representation based on higher-order polynomials in this invention;

[0042] Figure 6 This is the neural network architecture that integrates perception, front-end path search, and back-end optimization in this invention;

[0043] Figure 7 This invention relates to guided learning based on the Actor-Critic framework.

[0044] Figure 8 This invention relates to privileged learning based on ESDF map numerical gradients.

[0045] Figure 9 This invention relates to a brain-like computing-based unmanned aerial vehicle (UAV) planning hardware system. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0048] Example 1

[0049] See Figure 1 This embodiment provides an end-to-end planning method for unmanned aerial vehicles (UAVs) based on neuromorphic computing and guided learning, including:

[0050] Step 101: Use the depth image acquired by the depth camera, the current state of the UAV, and the target orientation as end-to-end input;

[0051] Step 102: Design a Spike response model based on a spiking neural network and a spiking network model based on Poisson coding;

[0052] Step 103: Integrate a neural network architecture that combines perception, front-end path search, and back-end optimization.

[0053] Step 104: Guided learning based on the Actor-Critic framework and privileged learning training based on the numerical gradient of the ESDF map;

[0054] Step 105: Output the UAV's planned trajectory and control parameters end-to-end.

[0055] See Figure 2 The design of a Spike response model based on a spiking neural network includes the following steps:

[0056] Step 201: Construct a spiking neural network (SNN) model, which can be represented as follows:

[0057]

[0058] Where τ is the time constant, v(t) represents the membrane potential, and I(t) represents the presynaptic current. For ease of implementation of the discrete-time algorithm, equation (1) can be rewritten recursively. dt represents the integration step size of the network simulation, referred to as the "time step". The input current is defined as:

[0059]

[0060] Where n represents the layer, j represents the index of the presynaptic neuron, i represents the index of the postsynaptic neuron, b is the bias (i.e., constant current), w is the weight, and z(t) is the pulse signal emitted by the neuron, i.e., the spike.

[0061] Step 202: After the neuron emits a pulse, two evolutionary modes, hard reset and soft reset, are designed to provide a foundation for subsequent spiking neural network encoding.

[0062] 1) Reset its U(t) to a fixed value, i.e., the resting membrane potential Urest (hard reset):

[0063]

[0064] 2) Subtract the threshold from U(t) (soft reset):

[0065]

[0066] For further details, please refer to [link / reference]. Figure 3 The design of a pulse network model based on Poisson coding includes the following steps:

[0067] Step 301: Use a spiking neural network with Poisson coding to encode information such as velocity, displacement, and target point as state input;

[0068] Step 302: Introduce an attention-based spiking network model to enhance the UAV's ability to process state information in complex scenarios and improve the efficiency of dynamic obstacle avoidance.

[0069] Step 303: Design a dynamic pulse-triggered threshold model, including a dynamic energy threshold model and a dynamic time threshold model, to adapt to different degradation environments and maintain efficient obstacle navigation decision-making for the UAV. The dynamic energy threshold model can be expressed as:

[0070]

[0071] in,

[0072] The dynamic time threshold model can be expressed as:

[0073]

[0074] in,

[0075] See Figure 4 An end-to-end single-stage drone planning algorithm includes the following steps:

[0076] Step 401: Take the noisy depth image as input;

[0077] Step 402 employs a lightweight backbone network for perception and feature extraction. Thanks to its data-driven nature and privileged learning strategy, it exhibits strong robustness to sensor noise without requiring mapping or filtering.

[0078] Step 403: Use a set of motion primitives (represented by boundary states) uniformly sampled within the field of view of the depth camera as the front end.

[0079] Step 404, inspired by the prediction of predefined anchor box offsets and confidence scores by a single-stage target detector, uses primitives as anchors and predicts their offsets and scores in parallel during a single forward propagation to further improve performance.

[0080] Step 405: Select the optimal primitive based on the score and apply the offset to the state of the primitive as backend optimization.

[0081] Step 406: By solving the boundary value problem, the optimal trajectory is represented as a higher-order polynomial.

[0082] Furthermore, end-to-end single-stage planning includes trajectory representation based on high-order polynomials, and a neural network architecture that integrates perception, front-end path search, and back-end optimization.

[0083] For further details, please refer to [link / reference]. Figure 5 Trajectory representation based on higher-order polynomials includes the following steps:

[0084] Step 501: Represent the trajectory in the body coordinate system as three independent one-step time-parameterized polynomials with dimensions μ∈{x, y, z}:

[0085] f μ (t)=a0+a1t+a2t 2 +a3t 3 +...+a n t n (7)

[0086] The coefficient vector is denoted as:

[0087] A=[a0,a1,a2,a3,...,a n (8)

[0088] Step 502: To provide initial paths with different topologies for local optimization, a set of motion primitives P = {p0, p1, p2, ..., p...} is used. M-1 The primitives are used as anchors to cover the search space, with each primitive guiding independent optimization during training. The primitive library is defined in the state lattice space, not in the multinomial coefficients. This spatial separation facilitates the fusion between network predictions and the original anchors. Specifically, M is uniformly sampled within the field of view of the depth sensor. θ Angle and Angle, in volume coordinates, motion primitive p ij Defined as:

[0089]

[0090] Where i and j represent the indices of the basic units in the horizontal and vertical directions, respectively, and r is the radius of the planning range.

[0091] Step 503: An end-to-end network is used to optimize the safety and smoothness of the initial path. For each basic unit, the offset Δθ is predicted. And Δr, as well as the terminal derivatives (velocity and acceleration of the fifth-order polynomial) and the corresponding scores. The refined final state of the ij-th trajectory can be expressed as:

[0092]

[0093] Where r′ ij =r j +Δr ij , θ′ ij =θ j +Δθ ij Generate backward time-domain trajectories with different radii r within a fixed execution time. The current state (given by the state estimator) and the final state (position p predicted by the network) are then compared. ij(and its derivative) are integrated into a vector d, and a constant matrix M is introduced to map it to the coefficients A = M - 1 d:

[0094] For further details, please refer to [link / reference]. Figure 6 The integrated neural network architecture, encompassing perception, front-end path search, and back-end optimization, includes the following steps:

[0095] Step 601 uses the depth image, current state (velocity and acceleration), and target orientation (unit vector) as input to the network to predict the offset, terminal derivative, and score for each basic action. To reduce dimensionality, both input and output are represented in the ontological coordinate system, thus eliminating the need to consider position and pose.

[0096] Step 602, divide the depth image into There are 10 grid cells, each corresponding to a basic action within its view frustum. Each grid cell is responsible for predicting the offset Δθ corresponding to the basic action. And Δr, as well as the terminal derivative and score.

[0097] Step 603, modify ResNet-18 as the backbone network for deep feature extraction. When Furthermore, when the image resolution is 96×96, a downsampling factor of 32 is used. Trajectories are predicted independently at each location in the feature map using 1×1 convolutional layers with shared weights. Unlike object detection tasks, motion planning problems do not possess translation invariance; even with the same features, different prediction results are expected in different grids.

[0098] Step 604, introduce the original coordinate system, and define the rotation matrix from the fuselage coordinate system to the original coordinate system as:

[0099]

[0100] Where R y (·) and R z (·) represents rotation along the y-axis and z-axis, respectively.

[0101] Step 605: Input the transformed original coordinate system and concatenate it with the corresponding positions in the feature map: Here, x represents the vector of the current state and the target direction in the body coordinate system. Furthermore, the terminal derivative of the network prediction is also defined in the original coordinate system. Through this transformation, the input and prediction are decoupled from the mesh position.

[0102] Step 606, the network outputs a shape of The tensor y, where Corresponding to each primitive, and M dThis is the raw representation of the offset, terminal derivative, and score. For a 5th-order polynomial, there are 3 offsets, 3 velocities, 3 accelerations, and 1 score; in this case, M... d =10.

[0103] Step 607: Use the hyperbolic tangent activation function to constrain the output. The offset is then confined to a portion of the solution space surrounding the initial primitives. The prior condition for the local solution space is (-d) θ +d θ ), (- and (-d) r +d r The predicted offset can be calculated as follows:

[0104]

[0105] Step 608, terminal derivative y v and y a The output is constrained by tanh(.) to ensure dynamic feasibility, and is transformed to the body coordinate system as follows:

[0106] υ=R ij tanh(y υ )v max

[0107] a = R ij tanh(y a )a max (13)

[0108] Step 609: Substitute the predicted value with the best score into the solution to solve the rolling time-domain trajectory.

[0109] Furthermore, guided learning based on the Actor-Critic framework and privileged learning based on the numerical gradient of the ESDF map were designed.

[0110] See Figure 7 Guided learning based on the Actor-Critic framework includes the following steps:

[0111] Step 701 employs a guided learning method based on the Actor-Critic framework to train the UAV's end-to-end trajectory planning. The actor-critic reinforcement learning utilizes a critic network to model the environment and act as a differentiable function. This guided learning method, combining a deterministic cost function and a deep learning network, provides an efficient end-to-end trajectory planning solution. The gradient of the cost function is backpropagated, and the gradient is passed to the parameters of the neural network using the chain rule, thus guiding the training.

[0112] Step 702: The overall cost function is expressed as a weighted sum of the smoothness cost function, the safety cost function, and the temporary target distance cost function, and its formula can be expressed as:

[0113] J = λ s J s +λ o J o +λ g J g (14)

[0114] J s J represents the smoothness cost function, used to restrict the integral of the square derivative; o J represents the safety cost function, used to penalize the distance to the obstacle; g The distance cost function to the temporary target is used to guide the quadcopter towards the target; λ i These are the weight parameters used for trade-offs. The state vector d is rearranged using matrix C into fixed blocks dF (the current state) and free blocks dP (the final state):

[0115]

[0116] Step 703, calculate the smoothness cost function J s , can be represented as:

[0117]

[0118] Remember C T M -T QM -1 If C is a matrix B, then it can be written in block form as follows:

[0119]

[0120] Step 704, calculate the security cost function J o First, a fixed time T is used to optimize the free final state of a single polynomial. Then, the safety cost is defined as the line integral of the potential function over the trajectory, which allows the quadcopter to minimize the cost by shortening the planning length. Therefore, the safety cost function J... o Rephrased as a time integral:

[0121]

[0122] Where Γ κ =κ·σt. The potential energy function can be expressed as:

[0123] c(d t )=exp(-(d t =d0) / k) (19)

[0124] Where d t Let represent the distance from p(t) to the nearest obstacle, where d0 and k are predefined scaling factors. Therefore, J in discrete form o The Jacobian matrix is:

[0125]

[0126] Where T = [T 0 κ, T 1 κ, ..., T n κ] T , and L P It is matrix M -1 The right block in C that corresponds to the free derivative.

[0127] Step 705, calculate the target distance cost function J g , can be represented as:

[0128] J g =(d Pp -g) 2 (twenty one)

[0129] Where d Pp It is d P The position component, g, is the projection of the final target onto a sphere of radius around the quadcopter. Therefore, in discrete form, J o The Jacobian matrix can be expressed as:

[0130]

[0131] Step 706: The numerical gradient is backpropagated directly to the network parameters using the chain rule, and the score is predicted. The integral in equation (18) is discretized at different timestamps for numerical calculation. For the total Jacobian matrix, the gradient predicted by the network can be expressed as:

[0132]

[0133] in And ∈{v, a}. Furthermore, the trajectory score is defined as -J, and supervised learning is performed using a smooth L1 loss:

[0134]

[0135] Where y s This is the score predicted by the network policy, which can be calculated using a deep learning framework. The network is then trained using the Adam optimizer, guided by gradients. To avoid the influence of negative trajectories, the predicted scores for all basic actions are optimized, but training is only performed on final states with scores above a threshold.

[0136] For further details, please refer to [link / reference]. Figure 8 Privileged learning based on ESDF map numerical gradients includes the following steps:

[0137] Step 801: Based on the numerical gradient information of the ESDF map, improve the data-driven method by utilizing the regularity in the training data to make it more robust to sensor noise.

[0138] Step 802: Collect a dataset of position, pose, and depth images, and process invalid depth values ​​using nearest neighbor interpolation, where position and pose are used only for gradient calculation. Randomly reset the quadcopter's state in the Flight-mare simulator, processing invalid depth values ​​using nearest neighbor interpolation to avoid confusion with nearby obstacles.

[0139] Step 803, on the pre-collected dataset at 1.5 × 10 -4 The proposed network was trained using a learning rate of 16 and a batch size of 16 for 50 iterations. A threshold of 0.04 was set for trajectory selection during training. A threshold that is too small can cause the network to overfit the safety conditions and become unstable across the entire dataset. Conversely, a threshold that is too large can prevent convergence to satisfactory results due to the influence of negative samples.

[0140] Step 804: In the expert-free strategy, various states and targets are randomly initialized for each depth sample to perform data augmentation without re-labeling or re-interacting with the simulator.

[0141] Step 805: To ensure sufficient coverage of the state space, further fine-tuning is performed using the dataset aggregation strategy (DAgger).

[0142] As an embodiment 2 of the present invention, an end-to-end planning hardware system for unmanned aerial vehicles (UAVs) based on neuromorphic computing is provided. The UAV planning system includes a neuromorphic chip and computing module, the UAV body and mission payload, etc.

[0143] An end-to-end planning hardware system for unmanned aerial vehicles (UAVs) based on neuromorphic computing and guided learning is characterized in that the end-to-end planning hardware system for UAVs includes a neuromorphic chip and computing module, the UAV body and mission payload, etc.

[0144] Furthermore, the neuromorphic chip and computing module based on neuromorphic computing are computing chips and computing modules that mimic the human brain's nervous system based on spiking neural networks (SNNs). They have characteristics such as pulse timing-dependent plasticity, short-term synaptic plasticity, and local steady-state regulation, and have ultra-low power consumption and parallel information processing capabilities. They include, but are not limited to, neuromorphic chips and computing modules based on neuromorphic chips.

[0145] Furthermore, the drone itself and its payload include, but are not limited to, small, micro, and miniature multi-rotor drones and their related optoelectronic payloads. The multi-rotor drones include tri-rotor, quadcopter, hexacopter, and octacopter drones.

[0146] For further details, please refer to [link / reference]. Figure 9 An example of an end-to-end planning hardware system for unmanned aerial vehicles (UAVs) based on neuromorphic computing and guided learning includes:

[0147] 90: Multi-rotor unmanned aerial vehicles and their payloads;

[0148] 91: Computing modules based on neuromorphic computing;

[0149] 901: A computing chip based on neuromorphic computing.

[0150] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. An end-to-end planning method for unmanned aerial vehicles (UAVs) based on neuromorphic computing and guided learning, characterized in that, Includes the following steps: Step 101: Take the depth image, current state, and target orientation as end-to-end inputs; Step 102: Design a Spike response model based on a spiking neural network and a spiking network model based on Poisson coding; Step 103: Integrate a neural network architecture that combines perception, front-end path search, and back-end optimization. Step 104: Guided learning based on the Actor-Critic framework and privileged learning training based on the numerical gradient of the ESDF map; Step 105: Output the UAV's planned trajectory and control parameters end-to-end.

2. The end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning according to claim 1, characterized in that, The design is based on the Spike response model of a spiking neural network and includes the following steps: Step 201: Construct a spiking neural network (SNN) model; Step 202: Design two evolution methods: hard reset and soft reset.

3. The end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning according to claim 1, characterized in that, The design is based on a Poisson-coded pulse network model and includes the following steps: Step 301: Use a spiking neural network with Poisson coding to encode information such as velocity, displacement, and target point as state input; Step 302: Introduce an attention-based spiking network model for feature extraction and information interaction; Step 303: Design a dynamic pulse trigger threshold model, including a dynamic energy threshold model and a dynamic time threshold model, to adapt to different degradation environments and maintain efficient obstacle navigation decision-making for UAVs.

4. The end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning according to claim 1, characterized in that, This includes an end-to-end single-stage planning algorithm, which comprises the following steps: Step 401: Take the noisy depth image as input; Step 402: A lightweight backbone network is used for perception and feature extraction; Step 403: A set of motion primitives (represented by boundary states) uniformly sampled within the depth camera's field of view is used as the front end to comprehensively search the solution space; Step 404: Using primitives as anchors, predict their offsets and scores in parallel during a single forward propagation. Step 405: Select the optimal primitive based on the score and apply the offset to the state of the primitive as backend optimization; Step 406: By solving the boundary value problem, the optimal trajectory is represented as a higher-order polynomial.

5. The end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning according to claim 1, characterized in that, This includes trajectory representation based on higher-order polynomials, which includes the following steps: Step 501: Represent the trajectory in the body coordinate system as three independent one-stage time-parameterized polynomials; Step 502: In order to provide initial paths with different topological structures for local optimization, a set of motion primitives are used as anchors to cover the search space, and each primitive guides independent optimization during training. Step 503: Use an end-to-end network to optimize the security and smoothness of the initial path.

6. The end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning according to claim 1, characterized in that, The neural network architecture integrating perception, front-end path search, and back-end optimization includes the following steps: Step 601: Using the depth image, current state (velocity and acceleration), and target direction (unit vector) as input to the network, predict the offset, terminal derivative, and score for each basic action; Step 602, divide the depth image into Each grid cell corresponds to a basic action within its view frustum, and each grid cell is responsible for predicting the offset Δθ of the corresponding basic action. And Δr, as well as the terminal derivative and score; Step 603: Modify ResNet-18 as the backbone network for deep feature extraction; Step 604: Introduce the original coordinate system and define the rotation matrix from the fuselage coordinate system to the original coordinate system; Step 605: Input the transformed original coordinate system and concatenate it with the corresponding positions in the feature map; Step 606, the network outputs a shape of tensor; Step 607: Use the hyperbolic tangent activation function to constrain the output. And the offset is restricted to a portion of the solution space surrounding the initial primitive; Step 608, terminal derivative y v and y a The output is constrained by tanh(.) to ensure dynamic feasibility and transformed to the body coordinate system; Step 609: Substitute the predicted value with the best score into the solution to solve the rolling time-domain trajectory.

7. The end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning according to claim 1, characterized in that, The guided learning based on the Actor-Critic framework includes the following steps: Step 701: A guided learning method based on the Actor-Critic framework is used to train the end-to-end trajectory planning of the UAV. Step 702: Express the overall cost function as a weighted sum of the smoothness cost function, the safety cost function, and the temporary target distance cost function; Step 703, calculate the smoothness cost function J s ; Step 704, calculate the security cost function J o ; Step 705, calculate the target distance cost function J g ; Step 706: The numerical gradient is backpropagated to the network parameters directly using the chain rule, and the score is predicted.

8. The end-to-end planning method for unmanned aerial vehicles based on neuromorphic computing and guided learning according to claim 1, characterized in that, The privileged learning based on the numerical gradient of the ESDF map includes the following steps: Step 801: Based on the numerical gradient information of the ESDF map, improve the data-driven method by utilizing the regularity in the training data; Step 802: Collect a dataset of position, pose, and depth images, and use nearest neighbor interpolation to process invalid depth values, where position and pose are only used for gradient calculation; randomly reset the state of the quadcopter in the Flight-mare simulator, and process invalid depth values ​​by nearest neighbor interpolation to avoid confusion with nearby obstacles; Step 803, on the pre-collected dataset at 1.5 × 10 -4 The proposed network was trained using a learning rate of 16 and a batch size of 16 for 50 iterations; the threshold for trajectory selection during training was set to 0.

04. Step 804: In the expert-free strategy, various states and targets are randomly initialized for each depth sample to perform data augmentation without re-labeling or re-interacting with the simulator. Step 805: To ensure sufficient coverage of the state space, further fine-tuning is performed using the dataset aggregation strategy (DAgger).

9. An end-to-end planning system for unmanned aerial vehicles (UAVs) based on neuromorphic computing and guided learning, characterized in that, The hardware system of the UAV end-to-end planning system includes a neuromorphic chip and computing module, the UAV body and mission payload; The neuromorphic chip and computing module are computing chips and computing modules based on spiking neural networks that mimic the human brain's nervous system. They possess pulse timing-dependent plasticity, short-term synaptic plasticity, and local steady-state regulation, and have ultra-low power consumption and parallel information processing capabilities. The unmanned aerial vehicle (UAV) itself and its mission payload include, but are not limited to, small, micro, and micro-mini multi-rotor UAVs and their related optoelectronic mission payloads.

10. The system according to claim 9, characterized in that, The neuromorphic chip and computing module include neuromorphic chips and computing modules, and the multi-rotor drone includes tri-rotor, quadcopter, hexacopter, and octacopter drones.

Images