Brain-like obstacle avoidance method for drones based on spiking neuron model and AC network

Abstract

The present invention discloses a brain-like obstacle avoidance method for drones based on a pulse neuron model and an AC network, comprising the steps of: 1) analyzing the response characteristics of pulse neurons and establishing a neuron pulse response model; 2) encoding the input and output of the pulse AC network and completing the parameter update of the timing difference; 3) obtaining drone state information, outputting obstacle avoidance decisions through the neuron pulse response model and the pulse AC network to avoid obstacles; wherein the drone state information includes image information captured by the drone camera. The method of the present invention is short in time consumption, fast in output response, consumes less computing resources and has low energy consumption.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of unmanned aerial vehicles, and in particular to a brain-like obstacle avoidance method for unmanned aerial vehicles based on a pulse neuron model and an AC network. Background Art

[0002] Unmanned aerial vehicles (UAVs) have received unprecedented attention and developed rapidly in both academia and industry. Currently, UAVs are widely used in military fields such as intelligence reconnaissance, military strikes, information confrontation, search and rescue, as well as in civilian fields such as aerial photography, environmental protection, agricultural and forestry operations, cargo delivery, and road traffic monitoring.

[0003] Many drones are involved in flying at low altitudes and in restricted environments, such as aerial combat, rapid crossing of complex environments, and encountering emergencies during cruising. These situations pose a great challenge to the drone's rapid obstacle avoidance. The drone has a limited obstacle recognition distance and number. When facing close obstacles or obstacles with high speed, the drone needs to make very drastic and frequent maneuvers. Accurate and rapid maneuvering decisions can ensure the drone's own safety and greatly improve the drone's survivability.

[0004] With the continuous expansion of applications, drones are increasingly facing complex and dynamic flight environments. In complex and dynamic environments, the possibility of drones encountering unexpected accidents is increasing. In order to complete tasks intelligently and autonomously, drones must have the ability to quickly avoid fast-moving obstacles. Therefore, enabling drones to make rapid maneuvering decisions and quickly avoid close-range or dynamic obstacles is the basis for them to preserve themselves and successfully complete their tasks. Research on rapid decision-making for drone obstacle avoidance has important theoretical significance and application value.

[0005] Drones face the following challenges in achieving rapid obstacle avoidance through action decision-making:

[0006] First, the challenge of rapidity. When faced with rapid obstacle avoidance tasks, drones are faced with targets that move at a fast speed or are relatively close, which requires them to complete rapid obstacle avoidance tasks in a short time. This not only includes rapid response of motion planning and underlying control, but also requires drones to quickly complete top-level decisions for motion planning. This is difficult to achieve for traditional drone motion planning and control methods: on the one hand, motion planning can only be completed quickly when the target's motion state is known, and it is difficult to complete motion planning in a short time when the target's motion state is unknown; on the other hand, multi-layer nested feedback control loops constrain the drone's action execution frequency, making it difficult to meet the needs of rapid obstacle avoidance. Traditional drone decision-making methods are often highly dependent on the preprocessing of environmental information, which often occupies most of the time of the decision-making algorithm. If we can find a way to save this part of time, we can greatly speed up the drone's decision-making.

[0007] Second, onboard resources are limited. Small drones are limited by their own model and payload size and cannot carry much computing resources: complex decision-making algorithms consume a lot of computing resources and rely on high-performance computing devices. However, these are heavy and cannot be used on small drones with limited payloads. This leads to the need for drones to use lightweight algorithms for quick decision-making under the condition of limited onboard resources.

[0008] To address these two major challenges, we need to explore fast obstacle avoidance methods that are time-consuming, consume less computing resources, and consume less energy. When dealing with some complex tasks, the decision-making mechanism of the human brain is very efficient. Therefore, by learning from the decision-making mechanism of the human brain, it is very likely to find a fast decision-making method that is time-consuming, consumes less computing resources, and consumes less energy.

[0009] Spiking Neural Network (SNN), which originated from brain science, is the third generation of artificial neural network. It has attracted the attention of scholars for its rich neurodynamic characteristics in the spatiotemporal domain, diverse coding mechanisms, and event-driven advantages. In recent years, the development of spiking neural networks has greatly developed a new computing paradigm based on and inspired by brain science achievements, namely brain-inspired computing (BIC). Using spiking neural networks to implement brain-inspired decision-making methods can significantly reduce the size of the network, while the high-density information transmission method has the characteristics of reducing resource consumption.

[0010] The research on spiking neural networks in fast decision-making methods can be divided into two strands. The first strand starts from the understanding of the biological brain and is the study of brain-inspired decision-making methods. In the past twenty years, the basal ganglia and thalamus have gradually been considered to be involved in the selection process of motor behavior in the human brain. The second strand starts from the implementation of reinforcement learning algorithms on spiking neural networks, which is the implementation of spiking neural networks in decision-making. The goal of reinforcement learning is to achieve the mapping from state space to action space, which is naturally suitable for application in decision-making. The training methods of spiking neural networks are divided into two categories. One is to map the parameter learning process from traditional artificial neural networks to spiking neural networks according to the corresponding control relationship, and the other is to train directly on spiking neural networks based on STDP.

[0011] In general, in the field of unmanned system decision-making based on spiking neural networks, some research results have emerged, laying a certain theoretical foundation, but there is no universal learning paradigm. At the same time, the existing methods for describing the state space and modeling the state neurons are not yet mature, and the potential of spiking neural networks has not been fully utilized. Summary of the invention

[0012] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides a brain-like obstacle avoidance method for drones based on a pulse neuron model and an AC network with short time consumption and fast response.

[0013] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0014] A UAV brain-like obstacle avoidance method based on a spiking neuron model and an AC network comprises the following steps:

[0015] 1) Analyze the response characteristics of spiking neurons and establish a neuron spiking response model;

[0016] 2) Encode the input and output of the pulse AC network and complete the parameter update of the timing difference;

[0017] 3) Obtain the drone status information and output obstacle avoidance decisions through the neuron pulse response model and pulse AC network to avoid obstacles.

[0018] Preferably, in step 2), the input and output of the pulse AC network are time-series encoded.

[0019] Preferably, in step 2), the process of updating the parameters of the pulse AC network to complete the timing difference is:

[0020] Assume that in a two-layer neural network, the neurons in the input layer are labeled i, the neurons in the output layer are labeled j, and the connection weight between the input layer and the output layer is labeled w. Then, the gradient of the value function is calculated, that is, the partial derivative is calculated.

[0021] Assume that at time t = 0, the neuron is in a static state, and the output of neuron j is t j is the first pulse of neuron j, ignoring the influence of kernel function η(s), then t j satisfy:

[0022]

[0023] Consider the input neuron m, whose connection weight to the output neuron j is w mj , both sides of formula (40) are simultaneously mj Find the partial derivative, combine and move the terms, and we get

[0024]

[0025] Through partial derivatives And formula (41), we get the value function gradient Based on this, the parameters of the Critic network are updated; the parameter update process of the Actor network is the same as that of the Critic network.

[0026] Preferably, in step 1), the specific process of establishing the neuron impulse response model is:

[0027] The discharge time of the spike neuron i is t i f , where f = 1, 2, ...; because the shapes of each pulse are basically similar, the information is encoded in the form of pulse time, and the pulse sequence of neuron i is represented as a discharge time series and expressed in the form of a delta function:

[0028]

[0029] Each time a neuron receives a pulse, it is equivalent to the membrane surface receiving a certain amount of charge at one time. Similarly, each time a neuron releases a pulse, it is considered as the membrane surface removing a certain amount of charge at one time. The charge removed from the membrane surface each time a neuron releases a pulse is recorded as a constant value q r ; After the excitation pulse, the membrane potential changes from the excitation threshold Instantly dropped to u r , then

[0030]

[0031] Then the discharge current generated on the film surface due to the excitation pulse is:

[0032]

[0033] The short pulse current corresponding to the discharge is treated mathematically in the same way as any other time-dependent input current, and the total current consisting of the pulse receiving current and the discharge current of the release pulse is I(t) + I r Substituting (t) into equation (7), we get the following result:

[0034]

[0035] In formula (21), I r Substituting the form of (t) into equation (22), we get:

[0036]

[0037] According to formula (23), when neurons receive and excite pulses, the membrane potential will mutate. After the excitation pulse, the neuron stops working and enters the refractory period, and the membrane potential mutation at this time is negligible. However, when receiving pulses, the membrane potential mutation is inconsistent with the actual situation. Therefore, formula (23) is modified accordingly; two kernel functions η(s) and κ(s) are introduced to rewrite the solution:

[0038]

[0039]

[0040] Then the solution to the differential equation can be written as:

[0041]

[0042] Here, the kernel function η(s) describes the reset of the membrane potential and explains the behavior of neuronal firing pulses; the kernel function κ(s) summarizes the linear electrical properties of the neuronal membrane surface; these two kernel functions, together with the threshold condition, constitute the neuronal response model.

[0043] Preferably, the amount of electricity is used to represent the size of the neuron connection weight, and the input current generated by neuron j is:

[0044]

[0045] The neuron impulse response model is obtained as:

[0046]

[0047] Preferably, the drone status information includes image information captured by a drone camera.

[0048] Preferably, the obstacle avoidance decision includes the movement of the drone in a vertical plane.

[0049] The present invention also discloses a UAV brain-like obstacle avoidance system based on a pulse neuron model and an AC network, comprising:

[0050] The first program module is used to analyze the response characteristics of the pulse neuron and establish a neuron pulse response model;

[0051] The second program module is used to encode the input and output of the pulse AC network and complete the parameter update of the timing difference;

[0052] The third program module is used to obtain the drone status information and output obstacle avoidance decisions through the neuron pulse response model and the pulse AC network to avoid obstacles.

[0053] The present invention further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are executed.

[0054] The present invention also discloses a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is run by the processor, the steps of the method described above are executed.

[0055] Compared with the prior art, the advantages of the present invention are:

[0056] The present invention is based on a neuron pulse response model and a brain-inspired pulse AC network. By analyzing the response characteristics of pulse neurons, a neuron pulse response model is established, the input and output of the brain-inspired pulse AC network are encoded, and the parameter update of the timing difference is completed. Then, the UAV state information is analyzed through the neuron pulse response model and the pulse AC network, and an obstacle avoidance decision is output. The above method takes a short time, has a fast output response, consumes less computing resources and has low energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of neuron pulse response in the present invention.

[0058] Figure 2 Schematic diagram of multi-pulse response and excitation pulse in the present invention.

[0059] Figure 3 Schematic diagram of the neuron dynamics circuit model in the present invention.

[0060] Figure 4 Schematic diagram of the kernel function κ(s) in the present invention.

[0061] Figure 5 This is a diagram of the two-layer pulse AC network structure in the present invention.

[0062] Figure 6 It is a schematic diagram of output time coding in the present invention.

[0063] Figure 7 Schematic diagram of the neuron being excited in the rising phase of the last pulse response received in the present invention.

[0064] Figure 8 Schematic diagram of the change in pulse excitation time caused by the change in connection weight in the present invention.

[0065] Fig. 9 Schematic diagram of the action of the neuron representative selected for the first firing pulse of the present invention.

[0066] Fig.10 Schematic diagram of the camera screen of the drone in the XTDrone simulation platform of the present invention.

[0067] Fig.11 This is a corresponding relationship diagram between the UAV window-crossing mission and the grid map in the present invention.

[0068] Fig.12 It is a schematic diagram of the training results of the frequency input and timing output encoding method of the present invention.

[0069] Fig.13 It is a schematic diagram for comparing the decision speeds of the three encoding methods in the present invention in simulation.

[0070] Fig.14 The present invention is a flowchart of an embodiment of the method. DETAILED DESCRIPTION

[0071] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0072] like Fig.14 As shown, the drone brain-like obstacle avoidance method based on the pulse neuron model and AC network of an embodiment of the present invention includes the following steps:

[0073] 1) Analyze the response characteristics of spiking neurons and establish a neuron spiking response model;

[0074] 2) Encode the input and output of the pulse AC network and complete the parameter update of the timing difference;

[0075] 3) Obtaining drone status information, and outputting obstacle avoidance decisions through a neural pulse response model and a pulse AC network to avoid obstacles; wherein the drone status information includes image information captured by a drone camera.

[0076] The drone brain-like obstacle avoidance method of the present invention is implemented based on a neuron pulse response model and a brain-inspired pulse AC network. By analyzing the response characteristics of pulse neurons, a neuron pulse response model is established, the input and output of the brain-inspired pulse AC network are encoded, and the parameter update of the timing difference is completed. Then, the drone state information is analyzed through the neuron pulse response model and the pulse AC network, and an obstacle avoidance decision is output. The above method takes a short time, has a fast output response, consumes less computing resources and has low energy consumption.

[0077] In a specific embodiment, the specific process of step 1) is:

[0078] 1.1) Analyze the structure and response of spiking neurons in the human brain. In the study of the structure of neuronal cells in the human brain, artificial spiking neurons were established. Spiking neurons are the basic computing units of spiking neural networks. Compared with traditional artificial neurons, spiking neurons perform more complex and highly nonlinear computing tasks, which also makes spiking neural networks have performance that may far exceed traditional artificial neural networks. The following introduces the structure and model of spiking neurons, laying a biological and mathematical foundation for subsequent research on spiking neural networks.

[0079] Spiking neurons transmit information through electrical signals, so the effect of spikes on postsynaptic neurons can be recorded using intracellular electrodes, which can measure the potential difference u(t) between the inside of the cell and its surroundings. This potential difference is called membrane potential. In the absence of any input, the neuron is in a resting state, corresponding to a constant membrane potential u rest After a pulse arrives, the membrane potential changes due to the influence of the pulse, but the pulse is a transient signal. If no new pulse arrives, the membrane potential will eventually decay back to the resting potential, such as Figure 1 shown.

[0080] From the above process, it can be seen that the best parameter to describe the state of spiking neurons is undoubtedly the membrane potential. As long as we can draw a model of how the membrane potential changes over time under the influence of input pulses, we can better describe the work of spiking neurons.

[0081] Next, we examine the changes in membrane potential caused by input pulses. For spiking neuron i, let its membrane potential be u i (t). Before the input pulse arrives, neuron i is in a static state, that is,

[0082] u i (t) = u rest (1)

[0083] When t = t j When t>t jWhen , the response of the spike neuron i is as shown in equation (2) and Figure 1 As shown:

[0084] u i (t)-u rest =ε ij (t) (2)

[0085] The right side of the equation defines the change in postsynaptic potential that can be measured by the intracellular electrode, Figure 1 is a schematic diagram of the response caused by the pulse of neuron j on the synapse of neuron i, where u i (t) is the membrane potential of spiking neuron i, I j (t) is the pulse current output from pulse neuron j to neuron i. After neuron i receives the pulse, its membrane potential rises at a faster rate first and then falls at a slower rate. If the potential difference u i (t)-u rest If it is positive, the postsynaptic potential will be excitatory, and if it is negative, the postsynaptic potential will be inhibitory.

[0086] Consider two presynaptic neurons j=1 and j=2, both of which send a spike to postsynaptic neuron i. Neuron j=1 spikes at t1. 1 , t1 2 , ... and so on; similarly, neuron j = 2 emits pulses at t2 1 , t2 2 , ... and so on. Each pulse causes the postsynaptic potential to change by ε i1 (t) or ε i2 (t). When there are only a few input pulses, the total change in potential is approximately the sum of the changes in the individual postsynaptic potentials, and the membrane potential can generally be considered to respond linearly to the input spikes; e.g. Figure 2 As shown in (a), the responses of the two pulses received by neuron i in the membrane potential are combined in a linear superposition manner.

[0087] On the other hand, if too many input pulses occur in a short time interval, this linear response no longer applies. When , its change shows a behavior completely different from that of simple summation: the membrane potential presents a pulse-like excursion with an amplitude of about 100 mV; Figure 2 As shown in (b), the three pulses received by neuron i are first combined in a linear superposition manner. In the process of responding to the third pulse, the membrane potential crosses the threshold from bottom to top. Just crossing the threshold At the same time, neuron i fires a pulse.

[0088] The amplitude range of the membrane potential response of a single pulse on the postsynaptic neuron is about 1mV. The critical value for the initiation of a neuronal firing pulse is about 20 to 30mV above the resting potential. Therefore, in most neurons, three pulses are often not enough to trigger an action potential. Instead, about 20-50 presynaptic pulses must arrive in a short period of time to trigger a postsynaptic action potential.

[0089] Basically, neuronal dynamics can be considered as an integration process (sometimes also called a "cumulative integration" process), coupled with a special firing mechanism that can be triggered by a membrane potential above a certain critical voltage, thereby firing an excitation pulse. The time when a spiking neuron fires an excitation pulse is usually defined as the moment when the membrane potential reaches a certain threshold from below. In order to establish a dynamic model of the neuronal membrane potential, the threshold voltage can be used. To describe the critical voltage of the excitation pulse. If the membrane potential u i (t) (including the sum effect of all inputs) reaches from below Then neuron i fires a pulse. Triggering time t i f Defined by the moment when the membrane potential exceeds the threshold.

[0090] The action potential of a neuron always has roughly the same form, and the action potential can be simplified as an "event" that occurs at a precise moment. The neuron model that describes the action potential as an event is called the integrate-and-discharge model. This model does not attempt to describe the shape of the action potential. The integrate-and-discharge model has two separate components that describe the membrane potential u and the discharge model. i (t) Equations of evolution and the mechanism of pulse generation.

[0091] The simplest model among the integration model and the excitation model: (1) using linear differential equations to describe the evolution of membrane potential; (2) setting a threshold to generate a discharge pulse. This model is called the leaky integrate-and-fire (LIF) model.

[0092] (1) Leakage Integrated Discharge (LIF) Model

[0093] Variable u i (t) is used to describe the instantaneous value of the membrane potential of neuron i. In the absence of any input, the potential is at its resting value, the resting potential u rest .

[0094] For the input current I(t), the instantaneous value u of the receiving neuron membrane potential iThe equation for (t) can be derived using basic laws of electrical theory. Neurons are surrounded by a cell membrane, which can be considered a capacitor, C. Because the cell membrane is not a perfect insulator, charge will slowly leak through the cell membrane over time, a process that can be modeled by a leakage resistor, R.

[0095] The basic circuit representing the LIF model consists of a capacitor C in parallel with a resistor R driven by a current I(t); Figure 3 If the driving current I(t) disappears, the voltage on the capacitor is reduced by the battery voltage u rest This cell is used to illustrate the resting potential of a neuron.

[0096] Using the law of conservation of current to analyze the circuit, the drive current can be divided into two components:

[0097] I(t)=I R +I C (3)

[0098] The first component is the resistive current I through the linear resistor R R , I R =(u(t)-u rest ) / R. The second component I C The capacitor C is charged, and the capacitive current is I c =d q / d t =Cdu / dt. Therefore, we get the following formula:

[0099]

[0100] Multiply the formula by R and introduce the time constant τ of the "leaky integrator" m =RC, then the basic formula describing the membrane potential u(t) is

[0101]

[0102] Where u(t) is the membrane potential of the neuron, τ m is the membrane time constant of the neuron.

[0103] The solution of this formula is:

[0104]

[0105] By substitution, we can get another solution:

[0106]

[0107] The specific meaning of this solution is analyzed below:

[0108] Assume that at time t = 0, the membrane potential takes the value u rest +Δu; and when t > 0, the input current I(t) = 0. Obviously, after waiting for a long enough time, the membrane potential will relax to its resting value u rest . Therefore, in the absence of input, the membrane potential decays exponentially to its resting value. Then, with the initial value determined as u(t0) = u rest +Δu, the solution of the differential equation is:

[0109]

[0110] The membrane time constant τ m = RC is the characteristic time of decay.

[0111] Assume that the surface of the neuron membrane is stimulated by a constant input current I(t) = I0, which starts at t = 0 and ends at t = Δt; and the membrane potential at t = 0 is at its resting value u(0) = u rest . Under these initial conditions, by integrating the basic formula, the solution of the differential equation within the time range 0 < t < Δt can be obtained as:

[0112]

[0113] If the input current does not stop, then as t → ∞, the membrane potential will asymptote to the value u(∞) = u rest + RI0.

[0114] In reality, the input current often stops within a very short time, that is, a short pulse with a very short duration. For a short pulse, the membrane potential does not reach the steady-state value. At the end of the pulse, the membrane potential at this time can be obtained as:

[0115]

[0116] When the pulse duration Δt < t < τ m , the exponential term can be expanded by the Taylor series and the first-order approximation is taken, then the membrane potential is

[0117]

[0118] This means that the postsynaptic pulse response membrane potential has an approximately linear relationship with the amplitude and duration of the pulse.

[0119] Let the duration Δt of the pulse become shorter and shorter, and at the same time increase the amplitude of the current pulse to the value I0 = q / Δt, so that the integral ∫I(t)dt = q remains unchanged, that is, the total charge q transferred by the current pulse is always the same. Then, no matter how short the pulse is, the membrane potential at the end of the pulse calculated according to Equation (11) remains unchanged. In fact, when taking τ m = RC, the following equation holds:

[0120] u(Δt)-u rest =qR / τ m =q / C (12)

[0121] In this way, the form of the pulse can be defined by the Dirac function δ(t):

[0122]

[0123] Where φ(t) is an arbitrary function defined at time 0. It should be noted that the delta function is a mathematical object and needs to be inserted into the integral to give a meaningful result. Thus, through the delta function, the pulse arriving at time 0 can be written as the input current in the form of:

[0124] I(t)=qδ(t) (14)

[0125] When t>Δt, in the absence of any further input, the membrane potential changes from its new initial value u rest +q / C starts to decay, then t0=Δt, Δu=q / C can be substituted into the formula to solve:

[0126]

[0127] Let Δt←0, then the solution of the membrane potential differential equation under the condition of I(t)=qδ(t) can be obtained as:

[0128]

[0129] Compared with the general form of the solution (7), it can be seen that when the delta function is used to define I(t), the integral is solved, which greatly simplifies the operation.

[0130] The above is the change of membrane potential of the "integrated" part with pulse response in the leakage integrated discharge model.

[0131] The “discharge” part of the leakage integrated discharge model is further introduced. f The moment when a given neuron emits a pulse is called the firing time. The firing time t in the LIF model f It can be precisely defined by the threshold criterion:

[0132]

[0133] After the spiking neuron fires a pulse, the neuron discharges, causing the membrane potential to drop. Corresponding to the LIF model, the membrane potential drops at t f It is then immediately reset to the new value The specific form is as follows:

[0134]

[0135] It should be noted that u r Not only is it smaller than the excitation threshold, it is also smaller than the resting potential u rest .

[0136] For t>t f , the neuron membrane surface enters the leakage integration stage again, and the membrane potential is accumulated and integrated again according to the above kinetic process until the next time the membrane potential crosses the excitation threshold.

[0137] Thus, the combination of leakage integration and discharge excitation pulses defines the leakage integration discharge model.

[0138] 1.2) Establishing a neuron impulse response model

[0139] The discharge time of the spike neuron i is t i f , where f = 1, 2, .... Because the shapes of each pulse are basically similar, the information is encoded in the form of pulse time, so formally, the pulse sequence of neuron i can be expressed as a discharge time series and expressed in the form of a delta function:

[0140]

[0141] Each time a neuron receives a pulse, it is equivalent to the membrane surface receiving a certain amount of charge at one time. Similarly, each time a neuron releases a pulse, it can be regarded as the membrane surface removing a certain amount of charge at one time. The charge removed from the membrane surface each time a neuron releases a pulse is recorded as a constant value q r , note that after the excitation pulse, the membrane potential drops from the excitation threshold Instantly dropped to u r , then

[0142]

[0143] Then the discharge current generated on the film surface due to the excitation pulse is:

[0144]

[0145] The short pulse current corresponding to the "discharge" is treated mathematically in the same way as any other time-dependent input current. The total current I(t)+I consisting of the pulse receiving current and the discharge current of the release pulse r Substituting (t) into equation (7), we get the following result:

[0146]

[0147] In formula (21), I r(t), we get:

[0148]

[0149] According to formula (23), the membrane potential will mutate when the neuron receives and excites pulses. After the excitation pulse, the neuron stops working and enters the refractory period. The mutation of the membrane potential at this time can be ignored; while the mutation of the membrane potential when receiving the pulse is inconsistent with the actual situation. Therefore, it is still necessary to modify formula (23) accordingly. The following introduces two kernel functions η(s) and κ(s) to rewrite the solution:

[0150]

[0151]

[0152] Then the solution to the differential equation can be written as:

[0153]

[0154] Here, the kernel function η(s) describes the resetting of the membrane potential and, more generally, explains the behavior of the neuron's firing spikes. The kernel function κ(s) summarizes the linear electrical properties of the neuron's membrane surface. These two kernel functions, together with special threshold conditions, form the impulse response model of the neuron.

[0155] The impulse response model is more general than the leaky integration model because the kernel functions η(s) and κ(s) are not necessarily required to be exponential, but can have arbitrary shapes. Therefore, a kernel function that is more consistent with the electrical characteristics of neurons can be obtained.

[0156] The kernel function κ(s) is the linear response of the membrane potential to the input current. In the leaky integrated discharge model, the kernel function κ(s) used describes an exponential decay. However, at the moment of pulse arrival, the membrane potential on the surface of the neuron undergoes a sudden change, which is impossible in actual spiking neurons. In fact, the response of the membrane potential to the pulse should follow the principle of starting from zero, rising slowly to reach the peak, and after reaching the peak, decaying to zero at a slower rate. In order to achieve this effect, the difference between two exponential functions can be considered to fit the response of the membrane potential to the received pulse, as shown in the following formula:

[0157]

[0158] Where K is the normalization parameter, let the maximum value of κ(s) be 1, and τ0>τ1. Figure 4 As shown, let τ1 = τ0 / 4, take τ0 = 150ms, τ0 = 250ms and τ0 = 350ms respectively, and the membrane potential response form that meets the actual situation can be obtained.

[0159] The kernel function η(s) describes the standard form of the neuron firing spike, including the negative overshoot that usually occurs after the spike (the post-spike potential u r ). Graphically, whenever the membrane potential reaches the threshold When the kernel function η(s) appears in the membrane potential, the kernel function η(s) appears in the membrane potential. Because the form of the pulse is always the same, the exact time course of the action potential does not contain any information. What is important is whether there is a pulse "event" at the current moment. The complete feature of the event is the pulse excitation time t f .

[0160] In reality, although the discharge process of neurons is not completed in an instant during the process of stimulating pulses, the kernel function η(s) should not retain the mutation component; however, considering that there is an absolute refractory period after the spiking neuron stimulates the pulse, during which the spiking neuron will not stimulate the pulse again, therefore, the excitation threshold is After the pulse, the potential u r The mutation of is not meaningless, and we can choose to retain the kernel function η(s) in the leakage integrated discharge model.

[0161] Because pulse signals are used to transmit signals in pulse neural networks, the current signals received by neurons can also be expressed in the form of the sum of delta functions. Unlike the fixed amount of electricity released when neurons fire pulses, the received pulses need to undergo a conversion from electrical signal to chemical signal to electrical signal when they reach the synapse. Therefore, the amount of electricity received by the pulse on the membrane surface is related to the loss in the conversion, that is, the amount of neurotransmitters in the synapse. This is reflected in the network as different weights of connections between different neurons. In this way, the size of the neuron connection weight can be expressed in terms of electricity, and the input current generated by neuron j can be written as:

[0162]

[0163] Among them, w j is the connection weight of neuron j, is the pulse time. In this way, the impulse response model of the neuron can be obtained:

[0164]

[0165] In a specific embodiment, in step 2), the input and output codes of the pulsed AC network are analyzed:

[0166] (1) Rate coding and timing coding of spiking neurons

[0167] The neural network formed by the interconnection of spiking neurons is a spiking neural network. As the basic unit of a spiking neural network, the signal transmission mechanism of spiking neurons determines that the input and output of a spiking neural network are both pulse sequences. The form of the pulse itself does not carry any information, but the number of pulses and the time when the pulses arrive. Therefore, for a pulse sequence, there are naturally two ways to encode information: one is rate encoding, and the other is timing encoding.

[0168] Still taking the 5*5 grid map as an example, for the 25 states and 4 optional states of each map scene, a two-layer pulse AC network is established, such as Figure 5 shown.

[0169] If encoding is done by rate, it is necessary to count the pulses in the pulse train and then average the number of pulses over time. The expression for the rate r is:

[0170]

[0171] Where T is the sampling time and N is the number of pulses within the sampling time.

[0172] For input, binary judgment can be used to make the pulse sequence frequency of the neuron corresponding to the selected state of the input a certain value, such as r = 40Hz, and the input of the neuron corresponding to the unselected state is 0. For output, the size change of the value function and strategy parameters can be expressed by the size change of r itself.

[0173] In the case of rate coding, the fixed sampling time limits the frequency of coding, so the rate coding always lags behind time. In scenarios that require quick decision-making, the information contained in the pulse sequence cannot be well utilized.

[0174] If the coding is time-series, the time when the pulse arrives or is generated is used as the input and output signal. For the input signal, the information is encoded in the relative time of each pulse, and more significant information about the feature is encoded as the pulse that arrives earlier. In the grid map scene, the input pulse sequence of each state neuron is limited to a fixed time interval (0, 1]. For the selected state neuron, its input pulse is excited earlier, such as t in = 0.5, for the unselected state neurons, their input pulses are fired later, such as t in = 1.5. For the output signal, the information is encoded in the time it takes for a single pulse to be generated.

[0175] (2) Timing output encoding of pulsed AC network

[0176] In the Critic network, the output layer only needs one spiking neuron to encode information to represent the value function of the current state. For spiking neurons, what can be meaningfully encoded is the first pulse generated by the output neuron. The time t when the spiking neuron stimulates the first pulse can be out As the output of the neuron, Figure 6 shown.

[0177] Considering that the kernel function κ(s) rises slowly and then falls more slowly, and that a spiking neuron will only fire a pulse when the membrane potential crosses the threshold from low to high, if a spiking neuron can fire a pulse, the pulse must occur at the rising stage of the last pulse it can receive, such as Figure 7 shown.

[0178] Taking the derivative of the kernel function κ(s) that describes the impulse response shape, we get:

[0179]

[0180] Among them, K is the normalization parameter, the maximum value of κ(s) is set to 1; τ0, τ1 are the time constants of the kernel function.

[0181] Let the derivative of κ(s) be 0, then the time when κ(s) reaches the peak value is:

[0182]

[0183] Let the time when the last pulse received arrives at the neuron be t last , then t out The value range is:

[0184] t last <t out ≤t last +t p (32)

[0185] Considering the speed of decision making, it is necessary to make the pulses generated earlier have higher importance, taking:

[0186] V(s,w)=-logt out (33)

[0187] In this way, the earlier the pulse neuron excites to generate a pulse, the larger the value function it represents. Considering the pulse neural network, the larger the connection weight between two neurons will make the membrane potential response amplitude of the neuron receiving the pulse larger, so that the neuron receiving the pulse will generate a pulse earlier, such as Figure 8 As shown. Therefore, the changing trend of the value function is consistent with the changing trend of the connection weights between neurons.

[0188] In the Actor network, multiple parameters are required to set the action selection strategy. When the action space is discrete and finite, a neuron can represent an action to be selected, and it can be encoded by the first pulse generated by multiple pulse neurons. At this time, the time of the first pulse output by each neuron is the probability of selecting the corresponding action. During the network training process, the softmax strategy can be used to select actions. Similar to the Critic network, the earlier the pulse is generated, the greater the probability. Therefore, the softmax operation is performed on the opposite number of the time when each pulse is generated, as shown in the following formula:

[0189]

[0190] Among them, t i is the time it takes neuron i to generate a spike, a i is the action corresponding to neuron i.

[0191] According to the softmax strategy, actions can be selected according to probability during the training process. After the training, in actual applications, the action represented by the first neuron that generates a pulse is the selected action, such as Fig. 9 shown.

[0192] In the spiking network, the input and output are encoded in a new form:

[0193] Let the parameter of the Critic network be w and the parameter of the Actor network be θ.

[0194] The update formula of the network is:

[0195]

[0196]

[0197] Among them, δ tv and δ tp Since the network output is relatively easy to obtain, the difficulty lies in computing the gradient.

[0198] For the frequency-coded output signal, its gradient is difficult to calculate. In the application scenario of the grid map, the gradient can always be recorded as 1, and its update formula is:

[0199] w←w+α w δ tv (37)

[0200] θ←θ+α θ δ tp (38)

[0201] For the output signal of time series coding, the value function and strategy can be differentiated and the gradient can be calculated with the help of the pulse neuron pulse response model.

[0202] Next, we calculate the value function gradient

[0203] Suppose in a two-layer neural network, the neurons in the input layer are labeled i, the neurons in the output layer are labeled j, and the connection weight between the input layer and the output layer is labeled w ij , then calculate the gradient of the value function, that is, calculate the partial derivative

[0204] Substituting the value function into

[0205]

[0206] Here, t out is the output of neuron j, which can also be written as t j .

[0207] The following calculation

[0208] Assume that at time t = 0, the neuron is in a static state, and at t j is the first pulse of neuron j, ignoring the influence of kernel function η(s), then t j satisfy:

[0209]

[0210] in, is the time when neuron i emits a pulse. Consider neuron m in the input layer, whose connection weight to output neuron j is w mj , both sides of the equation are simultaneously mj Find the partial derivative, combine, and move the terms to get:

[0211]

[0212] in, is the moment when the neuron does not emit a pulse. By using equations (39) and (41), we can get the value function gradient: The parameters of the Critic network can be updated accordingly.

[0213] The parameter update of the Actor network is similar.

[0214] On the basis of the impulse response model, the input and output of the pulse AC network are encoded and the updating mechanism of the timing difference is completed. Taking the timing coding method as an example, a pulse AC network decision algorithm using timing coded input and output signals can be obtained.

[0215]

[0216]

[0217] The above two-layer Actor-Critic network based on the pulse neural network is completed. By encoding the input and output of the pulse AC network and completing the update mechanism of the timing difference, the obstacle avoidance method based on the pulse response model and the pulse AC network is realized. Taking the grid map and the simulation of the drone passing through the window as examples, the decision-making method effects under two different coding methods based on frequency coding and timing coding are demonstrated respectively, and the comparison with the traditional AC network decision-making method is analyzed.

[0218] Specifically, the experimental scenario is set as follows: for the obstacle avoidance process of the rotorcraft UAV aiming at the center of the window during the process of passing through the window, it is simplified into a simulation process of the grid map, the application of the pulse AC network in this scenario is analyzed, and the results are analyzed.

[0219] After setting up the experimental scene, the pulse neural network training was started in the simulation environment. The simulation environment platform selected was the XTDrone drone simulation platform. XTDrone is a customizable multi-rotor drone simulation platform based on ROS, Gazebo and PX4. The platform integrates dynamic models, sensor models, control algorithms, state estimation algorithms and 3D scenes, and supports multiple drones and other robots. In this embodiment, the XTDrone platform is selected to design the drone window penetration scene and train the pulse neural network.

[0220] like Fig.10 As shown in the figure, in the XTDrone simulation platform, an obstacle wall is set in front of the drone, and there is a 40cm*40cm square window on the wall. The drone takes off from the position facing the obstacle wall, obtains status information through the monocular camera, and makes obstacle avoidance decisions based on it. When the drone is facing the window, it is ordered to fly forward through the window.

[0221] like Fig.11 As shown, the upper and lower pictures show the drone's window-crossing mission and the grid ground Figure 1 In the four pictures above, the gray background represents the obstacle wall, and the white square box represents the window; in the four pictures below, the gray dots represent the position of the drone in the grid map. Obviously, when the drone is located at the upper left position, the window observed by the camera in front of it is located at the lower right of the obstacle wall.

[0222] The drone avoids obstacles by moving up, down, left, and right in the vertical plane, aiming at the window on the obstacle wall, and then flying forward through the window. Corresponding to the grid map, the drone moves up, down, left, and right in the grid map and reaches the state of being in the center of the grid map.

[0223] Furthermore, the process of comparing frequency-based and time-based coding methods is as follows:

[0224] The input, output and obstacle avoidance speed of the following three methods are compared: (1) frequency input, frequency output; (2) frequency input, timing output; (3) timing input, timing output. The two methods with timing output use equation (41) to update the network parameters, and the frequency output method uses equations (37) and (38) to update.

[0225] After training in the simulation environment, all three input and output encoding methods can enable the drone to make the fastest obstacle avoidance plan to aim at the window. Taking the frequency input and timing output encoding method as an example, the training results are as follows: Fig.12 As shown in the figure, the value function output by the Critic part increases as the distance between the drone and the intermediate target state decreases, and the action selection strategy output by the Actor part enables the drone to choose the fastest path to the intermediate target state.

[0226] like Fig.13 As shown in the figure, the coding method of frequency input and frequency output, its decision speed depends on the sampling time T. Within the time T, the pulses output by the neuron are counted, and the number of pulses obtained is an integer. Due to the response characteristics of the pulse neuron, the number of pulses it outputs is less than or equal to the number of pulses input to it. Therefore, under this coding method, the output value of the pulse neuron is extremely limited. In order to make the output value of the neuron distinguishable, it is necessary to make the sampling time T as long as possible, so as to further reduce the decision speed. In the simulation experiment, in order to ensure that the output value is meaningful, the sampling time is taken as the simulation time T = 1000ms. Correspondingly, the time for the drone to make an obstacle avoidance plan is also 1000ms.

[0227] The two encoding methods of timing output have a significantly faster decision speed than the frequency input and frequency output methods. Fig.13 As shown in the figure, the decision time of the encoding method of frequency input and timing output is between 140ms-250ms of simulation time, and the decision time of the encoding method of timing input and timing output is between 9ms-30ms of simulation time. For fast decision-making tasks, timing encoding of the output signal of the pulse neuron can effectively improve the decision speed.

[0228] The present invention is based on the neuron pulse response model, and on this basis encodes the input and output signals of the pulse AC network, improves the update mechanism of the pulse AC network, and realizes an obstacle avoidance method based on the neuron pulse response model and the pulse AC network. By comparing the two encoding methods of frequency encoding and timing encoding, as well as the comparison between the pulse AC network decision method and the brain-inspired AC algorithm, the rapidity and effectiveness of the pulse AC network decision method are verified.

[0229] Based on the neuron impulse response model, the present invention designs the input and output encoding and update mechanism of the brain-inspired pulse AC network, obtains a brain-like obstacle avoidance method based on the impulse response model and the pulse AC network, analyzes its back propagation training method in the pulse neural network, practices the combined optimization of the Actor-Critic algorithm and the pulse neural network, and verifies the feasibility of the method through the simulation test of the quadcopter UAV window-penetrating decision task. The method has the advantages of small network scale and strong biological interpretability.

[0230] The embodiment of the present invention also discloses a UAV brain-like obstacle avoidance system based on a spiking neuron model and an AC network, comprising:

[0231] The first program module is used to analyze the response characteristics of the pulse neuron and establish a neuron pulse response model;

[0232] The second program module is used to encode the input and output of the pulse AC network and complete the parameter update of the timing difference;

[0233] The third program module is used to obtain the drone status information and output obstacle avoidance decisions through the neuron pulse response model and the pulse AC network to avoid obstacles.

[0234] The brain-like obstacle avoidance system of the present invention corresponds to the above-mentioned obstacle avoidance method and also has the advantages described in the above method.

[0235] The embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are performed. The embodiment of the present invention further discloses a computer device, including a memory and a processor, on which a computer program is stored, and when the computer program is executed by the processor, the steps of any one of the above methods are performed.

[0236] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A UAV brain-like obstacle avoidance method based on a spiking neuron model and an AC network, characterized in that: Includes steps: 1) Analyze the response characteristics of spiking neurons and establish a neuron spiking response model; 2) Encode the input and output of the pulse AC network and complete the parameter update of the timing difference; 3) Obtain the drone status information and output obstacle avoidance decisions through the neuron pulse response model and pulse AC network to avoid obstacles; In step 1), the specific process of establishing the neuron impulse response model is: The discharge time of the spiking neuron i is denoted as t i f , where f = 1, 2, ...; because the shapes of each pulse are basically similar, the information is encoded in the form of pulse time, and the pulse sequence of neuron i is represented as a discharge time series and expressed in the form of a delta function: Each time a neuron receives a pulse, it is equivalent to the membrane surface receiving a certain amount of charge at one time. Similarly, each time a neuron releases a pulse, it is considered as the membrane surface removing a certain amount of charge at one time. The charge removed from the membrane surface each time a neuron releases a pulse is recorded as a constant value q r ; After the excitation pulse, the membrane potential drops from the excitation threshold θ to u instantaneously r , then q r =C(θ-u r ) (20) Then the discharge current generated on the film surface due to the excitation pulse is: I r (t)=-q r F(t)=-C(θ-u r )∑ f δ(t-t f ) (21) The short pulse current corresponding to the discharge is treated mathematically in the same way as any other time-dependent input current, and the total current consisting of the pulse receiving current and the discharge current of the release pulse is I(t) + I r Substituting (t) into equation (7), we get the following result: In formula (21), I r Substituting the form of (t) into equation (22), we get: According to formula (23), when neurons receive and excite pulses, the membrane potential will mutate. After the excitation pulse, the neuron stops working and enters the refractory period, and the membrane potential mutation at this time is negligible. However, when receiving pulses, the membrane potential mutation is inconsistent with the actual situation. Therefore, formula (23) is modified accordingly; two kernel functions η(s) and κ(s) are introduced to rewrite the solution: Then the solution to the differential equation can be written as: Here, the kernel function η(s) describes the reset of the membrane potential and explains the behavior of neuronal firing pulses; the kernel function κ(s) summarizes the linear electrical properties of the neuronal membrane surface; these two kernel functions, together with the threshold condition, constitute the neuronal response model.

2. The drone brain-like obstacle avoidance method based on the pulse neuron model and AC network according to claim 1 is characterized in that: In step 2), the input and output of the pulse AC network are time-coded.

3. The drone brain-like obstacle avoidance method based on the pulse neuron model and AC network according to claim 2 is characterized in that: In step 2), the process of updating the parameters of the pulse AC network to complete the timing difference is: Assume that in a two-layer neural network, the neurons in the input layer are labeled i, the neurons in the output layer are labeled j, and the connection weight between the input layer and the output layer is labeled w. Then, the gradient of the value function is calculated, that is, the partial derivative is calculated. Assume that at time t = 0, the neuron is in a static state, and the output of neuron j is t j is the first pulse of neuron j, ignoring the influence of kernel function η(s), then t j satisfy: in, is the moment when neuron i emits a pulse; Consider the input neuron m, whose connection weight to the output neuron j is w mj , both sides of formula (40) are simultaneously mj Find the partial derivative, combine and move the terms, and we get: in, It is the moment when the neuron does not fire; Through partial derivatives And formula (41), we get the value function gradient ▽ w V(s t ,w), and update the parameters of the Critic network accordingly; the parameter update process of the Actor network is the same as that of the Critic network.

4. The drone brain-like obstacle avoidance method based on the pulse neuron model and AC network according to claim 1 is characterized in that: The power is used to represent the size of the neuron connection weight, and the input current generated by neuron j is: Among them, w j is the connection weight of neuron j, is the pulse time; The neuron impulse response model is obtained as:

5. The drone brain-based obstacle avoidance method based on a spiking neuron model and an AC network according to any one of claims 1 to 3, characterized in that: The drone status information includes image information captured by the drone camera.

6. The drone brain-based obstacle avoidance method based on a spiking neuron model and an AC network according to any one of claims 1 to 3, characterized in that: The obstacle avoidance decision includes the movement of the UAV in the vertical plane.

7. A UAV brain-like obstacle avoidance system based on a spiking neuron model and an AC network, used to execute the steps of a UAV brain-like obstacle avoidance method based on a spiking neuron model and an AC network as claimed in any one of claims 1 to 6, characterized in that: include: The first program module is used to analyze the response characteristics of the pulse neuron and establish a neuron pulse response model; The second program module is used to encode the input and output of the pulse AC network and complete the parameter update of the timing difference; The third program module is used to obtain the drone status information and output obstacle avoidance decisions through the neuron pulse response model and the pulse AC network to avoid obstacles.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program executes the steps of the method according to any one of claims 1 to 6 when executed by a processor.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: The computer program executes the steps of the method according to any one of claims 1 to 6 when executed by a processor.

Images