Unmanned obstacle avoidance method for brain-like pulse enhanced evolution and related device

Abstract

The invention discloses a brain-like pulse enhanced evolution unmanned obstacle avoidance method and a related device, and the method comprises the steps: generating an obstacle avoidance action at a current moment according to a state sequence of an unmanned vehicle at the current moment and a cortex-basal ganglion-thalamus network; and optimizing the cortex-basal ganglion-thalamus network at the next moment based on the obstacle avoidance action at the current moment, in the optimization process of the cortex-basal ganglion-thalamus network, according to the award difference value at the adjacent moments, accumulating the vigilance value of the cortex-basal ganglion-thalamus network to the environment, and when the accumulated vigilance value does not exceed a second threshold value, determining that the awarded value does not exceed the second threshold value. And when the accumulated vigilance value exceeds a first threshold value, optimizing the cortex-basal ganglion-thalamus network based on a peak time sequence dependency plasticity mechanism and a dopamine regulation mechanism in sequence, and when the accumulated vigilance value exceeds a second threshold value, optimizing the cortex-basal ganglion-thalamus network based on an offline evolution method of a genetic algorithm, the obstacle avoidance method which can quickly adapt to different complex environments and is accurate in decision making is realized.

Description

Technical Field

[0001] The present invention relates to an unmanned driving obstacle avoidance method and related device based on brain-like pulse enhanced evolution, belonging to the field of unmanned driving. Background Art

[0002] The core of unmanned driving technology is to realize autonomous decision-making and path planning of vehicles in complex dynamic environments. Among them, obstacle avoidance technology, as an important component to ensure the safety and reliability of unmanned vehicles, has always been the focus and difficulty of research.

[0003] Although deep reinforcement learning, such as Deep Q-Network (DQN), shows potential in policy learning, there is still a high demand for computing resources and training samples, especially in dynamic and complex environments, which require longer training time to achieve better policies.

[0004] Spiking Neural Network (SNN) has attracted much attention because it simulates the spiking behavior of neurons in the human brain. Compared with traditional deep learning models, SNN has higher computing efficiency and energy efficiency, and is considered a potential solution to improve the real-time performance of unmanned driving systems. However, based on spiking neural networks, there is currently no obstacle avoidance method that can quickly adapt to different complex environments and make accurate decisions. Summary of the invention

[0005] The present invention provides an unmanned driving obstacle avoidance method and related devices based on brain-like pulse enhanced evolution, which solves the problems disclosed in the background technology.

[0006] According to one aspect of the present disclosure, a method for autonomous driving obstacle avoidance based on brain-like pulse enhanced evolution is provided, comprising:

[0007] Obtain the state sequence of the driverless vehicle in the lane scene at the current moment;

[0008] Generate the obstacle avoidance action of the unmanned vehicle at the current moment according to the state sequence and the spiking neural network; wherein the spiking neural network is a cortex-basal ganglia-thalamus network;

[0009] The obstacle avoidance action is interacted with the current environment to generate a reward given by the current environment; wherein the current environment is the environment in the lane scene at the current moment;

[0010] If the difference between the reward at the current moment and the reward at the previous moment exceeds the first threshold, the current moment environment is copied to the environment playback pool, and the accumulated alert value up to the current moment is calculated; wherein the alert value is the alert value of the pulse neural network to the environment;

[0011] If the accumulated alert value up to the current moment does not exceed the second threshold, optimizing the spiking neural network based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn; wherein the optimized spiking neural network is used to generate the obstacle avoidance action of the unmanned vehicle at the next moment;

[0012] If the accumulated alert value up to the current moment exceeds the second threshold, the pulse neural network is optimized based on the offline evolution method of the genetic algorithm according to the current moment environment in the environment playback pool.

[0013] Furthermore, the spiking neuron A in the cortex-basal ganglia-thalamus network was constructed based on the gene regulatory network A1 and the Izhikevich neuron model A2;

[0014] Izhikevich neuron model A2 is used to model the firing behavior of spiking neuron A;

[0015] Gene regulatory network A1 is used to generate synaptic weights between spiking neuron A and spiking neuron B. The input of gene regulatory network A1 is the synaptic current of input spiking neuron A. Among them, spiking neuron B is a spiking neuron that receives the output of spiking neuron A.

[0016] Furthermore, the reward given by the current environment is generated using the formula:

[0017] ;

[0018] ;

[0019] ;

[0020] In the formula, r t is the reward given by the environment at the current moment t, s t is the state sequence of the unmanned vehicle at the current time t, R b (s t ) is s t The corresponding total discrete reward, Eva(s t ) is s t The corresponding continuous reward, α is a fixed coefficient, d safe is the preset safety distance between the unmanned vehicle and the obstacle, d current is the distance between the unmanned vehicle and the obstacle at the current moment, v current is the speed of the driverless vehicle at the current moment, R collision (s t ) is the discrete reward when the unmanned vehicle is in a collision state with an obstacle, R speed (s t ) is the discrete reward calculated based on the speed interval of the unmanned vehicle at the current moment, R obstacle (st ) is the discrete reward for whether the driverless vehicle maintains a safe distance from the obstacle, R lane-change (s t ) is the discrete reward for the driverless vehicle to successfully change lanes to avoid an obstacle, R fre-change (s t ) is the discrete reward for the unmanned vehicle to change lanes continuously, y1 and y2 are preset values, and C1~C4 are the states of four different unmanned vehicles.

[0021] Furthermore, the accumulated alert value up to the current moment is calculated using the formula:

[0022] ;

[0023] In the formula, is the accumulated alert value up to the current moment t, β is the accumulated alert value up to the previous moment, μ is the update rate of δ(t), and δ(t) is the difference between the reward at the current moment and the reward at the previous moment.

[0024] Furthermore, the spiking neural network is optimized based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism, including:

[0025] The optimized spiking neural network is optimized for the first time based on the spike timing-dependent plasticity mechanism; wherein the first optimization is to optimize the synaptic weights between the dorsolateral prefrontal cortex and the striatal D1 dopamine receptors, and the synaptic weights between the dorsolateral prefrontal cortex and the striatal D2 dopamine receptors in the spiking neural network;

[0026] If the difference between the current reward and the previous reward is greater than 0, based on the first optimization, the synaptic weight between the dorsolateral prefrontal cortex and the striatal D1 dopamine receptor is increased, and the synaptic weight between the dorsolateral prefrontal cortex and the striatal D2 dopamine receptor is reduced;

[0027] If the difference between the current reward and the previous reward is not greater than 0, based on the first optimization, the synaptic weight between the dorsolateral prefrontal cortex and the striatal D1 dopamine receptors is reduced, and the synaptic weight between the dorsolateral prefrontal cortex and the striatal D2 dopamine receptors is increased.

[0028] Furthermore, according to the current environment in the environment playback pool, the spiking neural network is optimized by an offline evolution method based on a genetic algorithm, including:

[0029] S1) obtaining spiking neurons of the dorsolateral prefrontal cortex of the spiking neural network to be optimized, and using the obtained spiking neurons as the initial nucleus; S2) randomly expanding the initial core group to obtain multiple new core groups, using the new core groups to replace the initial core group in the pulse neural network to be optimized, and obtaining multiple new pulse neural networks to be optimized; S3) Traverse all the spiking neural networks to be optimized, obtain the rewards corresponding to the elite network and the rewards corresponding to the spiking neural network to be optimized in the current environment, and take the difference between the rewards corresponding to the initial elite network and the rewards corresponding to the spiking neural network to be optimized as the rewards of the corresponding nuclei, and calculate the rewards of each spiking neuron in the nuclei according to the rewards of the nuclei and the synaptic weights of each spiking neuron in the nuclei; among them, the initial elite network is the spiking neural network with the highest reward in the historical moment; S4) If the number of iterations reaches the third threshold, the first N spiking neurons with high rewards are used as the final nucleus to obtain the optimized spiking neural network. If the number of iterations does not reach the third threshold, the first N spiking neurons with high rewards are mutated and propagated, and the spiking neurons after the mutation and propagation operations are used as the initial nucleus, and then go to S2); where N is the number of spiking neurons in the dorsolateral prefrontal cortex of the spiking neural network that is not optimized using the offline evolution method.

[0030] According to another aspect of the present disclosure, there is provided an unmanned driving obstacle avoidance device with brain-like pulse enhanced evolution, comprising:

[0031] An acquisition module obtains the state sequence of the unmanned vehicle in the lane scene at the current moment;

[0032] The obstacle avoidance action generation module generates the obstacle avoidance action of the unmanned vehicle at the current moment according to the state sequence and the spiking neural network; wherein the spiking neural network is a cortex-basal ganglia-thalamus network;

[0033] The reward generation module interacts the obstacle avoidance action with the current environment to generate a reward given by the current environment; wherein the current environment is the environment in the lane scene at the current moment;

[0034] A vigilance value accumulation module, if the difference between the reward at the current moment and the reward at the previous moment exceeds the first threshold, copies the current moment environment to the environment playback pool, and calculates the vigilance value accumulated to the current moment; wherein the vigilance value is the vigilance value of the pulse neural network to the environment;

[0035] The first optimization module optimizes the spiking neural network based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn if the accumulated alert value up to the current moment does not exceed the second threshold; wherein the optimized spiking neural network is used to generate the obstacle avoidance action of the unmanned vehicle at the next moment;

[0036] The second optimization module optimizes the pulse neural network based on the offline evolution method of the genetic algorithm according to the current environment in the environment playback pool if the accumulated alert value up to the current moment exceeds the second threshold.

[0037] Furthermore, in the reward generation module, the reward given by the current environment is generated, and the formula is:

[0038] ;

[0039] ;

[0040] ;

[0041] In the formula, r t is the reward given by the environment at the current moment t, s t is the state sequence of the unmanned vehicle at the current time t, R b (s t ) is s t The corresponding total discrete reward, Eva(s t ) is s t The corresponding continuous reward, α is a fixed coefficient, d safe is the preset safety distance between the unmanned vehicle and the obstacle, d current is the distance between the unmanned vehicle and the obstacle at the current moment, v current is the speed of the driverless vehicle at the current moment, R collision (s t ) is the discrete reward when the unmanned vehicle is in a collision state with an obstacle, R speed (s t ) is the discrete reward calculated based on the speed interval of the unmanned vehicle at the current moment, R obstacle (s t ) is the discrete reward for whether the driverless vehicle maintains a safe distance from the obstacle, R lane-change (s t ) is the discrete reward for the driverless vehicle to successfully change lanes to avoid an obstacle, R fre-change (s t ) is the discrete reward for the unmanned vehicle to change lanes continuously, y1 and y2 are preset values, and C1~C4 are the states of four different unmanned vehicles.

[0042] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes an unmanned driving obstacle avoidance method based on brain-like pulse enhancement evolution.

[0043] According to another aspect of the present disclosure, a computer device is provided, comprising one or more processors and one or more memories, wherein one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing an unmanned driving obstacle avoidance method based on brain-like pulse enhancement evolution.

[0044] The beneficial effects achieved by the present invention are as follows: the present invention generates an obstacle avoidance action at the current moment according to the state sequence of the unmanned vehicle at the current moment and the cortex-basal ganglia-thalamus network, and optimizes the cortex-basal ganglia-thalamus network at the next moment based on the obstacle avoidance action at the current moment; in the optimization process of the cortex-basal ganglia-thalamus network, the alertness value of the cortex-basal ganglia-thalamus network to the environment is accumulated according to the reward difference at adjacent moments; when the accumulated alertness value does not exceed the second threshold, the cortex-basal ganglia-thalamus network is optimized based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn; when the accumulated alertness value exceeds the second threshold, the cortex-basal ganglia-thalamus network is optimized based on an offline evolution method based on a genetic algorithm, which can effectively improve the rapid adaptation and decision-making accuracy of the cortex-basal ganglia-thalamus network to complex environments, thereby realizing an obstacle avoidance method that can rapidly adapt to different complex environments and make accurate decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flowchart of the autonomous driving obstacle avoidance method evolved by brain-like pulse enhancement;

[0046] Figure 2 Schematic diagram of gene regulatory network;

[0047] Figure 3 It is a schematic diagram of the structure of the cortex-basal ganglia-thalamus network;

[0048] Figure 4 The overall schematic diagram for optimizing the spiking neural network;

[0049] Figure 5 Flowchart for optimizing spiking neural networks based sequentially on spike-timing-dependent plasticity mechanisms and dopamine-modulatory mechanisms;

[0050] Figure 6 Flowchart for optimizing the spiking neural network using an offline evolutionary method based on a genetic algorithm according to the current moment environment in the environment playback pool;

[0051] Figure 7 Block diagram of an unmanned driving obstacle avoidance device evolved for brain-like pulse enhancement. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. It is obvious that the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0053] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0054] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0055] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0056] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0057] It should be noted that similar symbols and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it does not require further discussion in subsequent figures.

[0058] Meanwhile, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. Thus, the features defined as "first" or "second" may explicitly or implicitly include one or more features.

[0059] The spiking neural network is a new generation of artificial neural network model inspired by biology. It is a subset of deep learning and has a strong biological basis. The spiking neural network contains neuron nodes with timing dynamics, synaptic structures with homeostasis-plasticity balance, and functionally specific network loops. It draws heavily on the biological optimization methods of local unsupervised (such as pulse timing-dependent plasticity, short-term synaptic plasticity, local homeostasis regulation, etc.) and global weak supervision (such as dopamine reward learning, energy-based function optimization, etc.) inspired by biology. Therefore, it has powerful spatiotemporal information representation, asynchronous event information processing, network self-organization learning, etc.

[0060] The present invention provides an unmanned driving obstacle avoidance method based on a pulse neural network, aiming to perform unmanned driving obstacle avoidance by constructing and optimizing a pulse neural network that can quickly adapt to different complex environments and make accurate decisions. Unmanned driving obstacle avoidance is the core of unmanned driving technology. The unmanned driving obstacle avoidance method can be executed by an obstacle avoidance device, which can be a terminal device or a server. Among them, the terminal device can include but is not limited to a computer, an intelligent vehicle-mounted device, etc., and the present invention does not limit it; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, big data and artificial intelligence platforms, etc., and the present invention does not limit it. Optionally, the unmanned driving obstacle avoidance method can also be collaboratively executed by multiple electronic devices with computing power. For the sake of convenience, the subsequent embodiments are described as being executed by an obstacle avoidance device.

[0061] See also Figure 1 , Figure 1 The flowchart of the unmanned driving obstacle avoidance method of brain-like pulse enhanced evolution provided by the present invention can be performed by an obstacle avoidance device, and the unmanned driving obstacle avoidance method can at least include the following steps:

[0062] like Figure 1 As shown, step 1 of the embodiment is to obtain the state sequence of the unmanned vehicle in the lane scene at the current moment.

[0063] It should be noted that the lane scene refers to the scene of an unmanned vehicle driving on the road. The lane scene can be collected through the camera of the unmanned vehicle. Further, through the existing video image recognition technology, the environmental information of the unmanned vehicle can be obtained, such as whether there are other vehicles within 100 meters of the unmanned vehicle, the distance between other vehicles and itself, etc., combined with the position, speed, acceleration, etc. collected by the on-board system of the unmanned vehicle, they can together constitute the state sequence of the unmanned vehicle.

[0064] Assume that the state information of the driverless vehicle in the lane scene at the current time t is (S1, S2, …, S n ), S1~S n It represents n state information, which may include the current position, speed, acceleration, whether there are other vehicles within the range (such as 100 meters), the distance between other vehicles and the driver, etc. In order to facilitate subsequent processing, these state information need to be preprocessed. Specifically, the continuous values ​​in the state information can be linearly normalized and the normalized data can be encoded. The encoding formula can be expressed as Encode(S i ), thereby obtaining the state pulse sequence set at the current time t ;in, is the pulse sequence of the ith state, S i is the i-th state information of the unmanned vehicle, i∈[1,n], Encode() is the encoding function, through which the state pulse sequence can be obtained.

[0065] return Figure 1 , step 2 of the embodiment, generates the obstacle avoidance action of the unmanned vehicle at the current moment according to the state sequence and the pulse neural network; wherein the pulse neural network is a cortex-basal ganglia-thalamus network.

[0066] It should be noted that simple spiking neurons only have the ability to forward calculate pulse signals and do not have evolutionary functions in the biological sense. Therefore, in some embodiments, the spiking neurons in the cortex-basal ganglia-thalamus network are evolved spiking neurons (hereinafter referred to as "spiking neurons"). Such spiking neurons have the ability to evolve through the gene regulatory network, which is conducive to the subsequent optimization of spiking neurons by evolutionary calculations.

[0067] Taking spiking neuron A as an example, spiking neuron A is constructed based on gene regulatory network A1 and Izhkevich neuron model A2; wherein, Izhkevich neuron model A2 is used to model the firing behavior of spiking neuron A; gene regulatory network A1 is used to generate the synaptic weight between spiking neuron A and spiking neuron B, and the input of gene regulatory network A1 is the synaptic current of input spiking neuron A; wherein, spiking neuron B is a spiking neuron that receives the output of spiking neuron A. Similarly, taking spiking neuron B as an example, it is constructed based on gene regulatory network B1 and Izhkevich neuron model B2.

[0068] The above-mentioned Izhikevich neuron model is an existing neuron model (EMIzhikevich's paper, "Simple model of spiking neurons," IEEE Transactions on neural networks, vol.14, no. 6, pp. 1569–1572, 2003), which is used to model the firing behavior of spiking neurons and describe the dynamic changes of the membrane potential and recovery variables of spiking neurons. It can be expressed as follows:

[0069] ;

[0070] ;

[0071] Where I is the synaptic current input to the spiking neuron, v and u are the membrane potential and recovery variable of the spiking neuron, respectively, and a and b are the parameters used to control the speed and recovery characteristics of the Izhkevich neuron model, respectively.

[0072] The maximum voltage of the pulse neuron can be set to 30mv. When the pulse neuron reaches 30mv, it will reset. The formula can be expressed as:

[0073] ;

[0074] In the formula, c is the reset amount and d is the increase amount, which means that it is reset when v is greater than or equal to 30mv, that is, c is assigned to v, and d is added to the original u.

[0075] It should be noted that the gene regulatory network is a gene regulatory network based on artificial genome encoding, see Figure 2 , based on a fixed number of genes, it contains an artificial genome, specifically an artificial genome constructed by promoter, regulatory region and gene coding region; among them, the main function of the promoter region is to preliminarily process the input signal and provide the initial activity level for gene expression; the regulatory region is the middle layer of gene expression, responsible for integrating and adjusting the signal according to the input signal and gene expression value; the gene coding region is responsible for the actual gene expression calculation, and its output result will be used to calculate the synaptic weight between pulse neurons. The gene regulatory network includes input nodes, regulatory nodes ( Figure 2 The solid line in the middle is the primary regulatory node, and the dotted line is the secondary regulatory node to be developed) and the output node; among them, the input node (corresponding to the promoter) determines the initial activity of gene expression, and the regulatory node can realize the regulation between gene expressions. The synaptic current of the input pulse neuron is used as the input of the gene regulatory network, and the output is the synaptic weight connected to the current pulse neuron. The synaptic weight is jointly determined by the synaptic current, promoter and regulatory region.

[0076] Gene expression is affected by the promoter, regulatory matrix and synaptic current. The regulatory effect value is calculated through the regulatory matrix and synaptic current, and the gene expression is calculated based on the promoter. The gene expression value is normalized and updated, and the gene expression value is mapped to the synaptic weight, which can be expressed as follows:

[0077] V = R × I;

[0078] ΔE=P×V;

[0079] ΔE norm =(ΔE-μ ΔE ) / σ ΔE ;

[0080] W = tanh(ΔE norm );

[0081] Where V is the regulatory effect value, R is the regulatory matrix, tanh() is the hyperbolic tangent function, ΔE is the initial (unnormalized) gene expression value, P is the promoter influence matrix, ΔE normis the normalized gene expression value, μ ΔE is the mean expression value of all genes output by the spiking neuron, σ ΔE is the variance of all gene expression values, and W is ΔE norm The final gene expression value obtained after the hyperbolic tangent function.

[0082] It should be noted that the cortex-basal ganglia-thalamus network is a hybrid model that imitates the cortex-basal ganglia-thalamus structure of the brain and combines reinforcement learning with biological inspiration.

[0083] See also Figure 3 The cortical-basal ganglia-thalamus network includes the dorsolateral prefrontal cortex, orbitofrontal cortex, substantia nigra compacta / ventral tegmental area, striatal D1 dopamine receptors, striatal D2 dopamine receptors, subthalamic nucleus, lateral globus pallidus, substantia nigra reticulata / internal globus pallidus, thalamus and primary motor area; among them, the output of the substantia nigra compacta / ventral tegmental area is excitatory connected with the input of the orbitofrontal cortex, and the output of the dorsolateral prefrontal cortex is excitatory connected with the input of the striatal D1 dopamine receptors, the input of the striatal D2 dopamine receptors and the input of the subthalamic nucleus, respectively. The excitability of the orbitofrontal cortex affects the excitatory connection between the dorsolateral prefrontal cortex and the striatal D1 dopamine receptors, and the excitatory connection between the dorsolateral prefrontal cortex and the subthalamic nucleus. The output of the striatal D1 dopamine receptors is inhibitoryly connected to the input of the substantia nigra reticulata / globus pallidus internal part, the output of the striatal D2 dopamine receptors is inhibitoryly connected to the input of the globus pallidus lateralis, the output of the globus pallidus lateralis is excitatory connected to the input of the substantia nigra reticulata / globus pallidus internalis, the output of the subthalamic nucleus is excitatory connected to the input of the substantia nigra reticulata / globus pallidus internalis, the output of the subthalamic nucleus is excitatory connected to the input of the globus pallidus lateralis, the output of the globus pallidus lateralis is inhibitory connected to the input of the subthalamic nucleus, the output of the substantia nigra reticulata / globus pallidus internalis is excitatory connected to the input of the thalamus, the output of the thalamus is excitatory connected to the input of the primary motor area, and the output of the dorsolateral prefrontal cortex is excitatory connected to the input of the primary motor area.

[0084] In the cortical-basal ganglia-thalamus network, the dorsolateral prefrontal cortex receives state pulse trains and transmits them to the basal ganglia. The basal ganglia include direct pathway, indirect pathway and hyperdirect pathway. The direct pathway includes striatal D1 dopamine receptors and substantia nigra reticularis / globus pallidus internus. Striatal D1 dopamine receptors directly inhibit substantia nigra reticularis / globus pallidus internus. The indirect pathway includes striatal D2 dopamine receptors, lateral globus pallidus and substantia nigra reticularis / globus pallidus internus. D2 dopamine receptors inhibit substantia nigra reticularis / globus pallidus internus by inhibiting lateral globus pallidus. The hyperdirect pathway includes subthalamic nucleus, lateral globus pallidus and substantia nigra The reticularis pars / globus pallidus medialis and the dorsolateral prefrontal cortex activate the subthalamic nucleus, which further excites the lateral part of the globus pallidus and the substantia nigra reticularis / globus pallidus medialis. The three pathways converge in the substantia nigra reticularis / globus pallidus medialis, that is, the substantia nigra reticularis / globus pallidus medialis is the output area of ​​the basal ganglia, which regulates the inhibitory output of the substantia nigra reticularis / globus pallidus medialis to the thalamus, and then integrates the excitatory input of the dorsolateral prefrontal cortex, affecting the excitatory output of the thalamus to the primary motor area, that is, the substantia nigra reticularis / globus pallidus medialis produces obstacle avoidance action selection, the thalamus receives the projection of the substantia nigra reticularis / globus pallidus medialis and transmits the obstacle avoidance action to the primary motor area and outputs the obstacle avoidance action.

[0085] The above-mentioned cortical-basal ganglia-thalamus network takes into account the signal response mechanism of the orbital cortex, in which the orbitofrontal cortex receives the difference in rewards between adjacent moments, that is, it receives the difference in rewards between adjacent moments through the substantia nigra pars compacta / ventral tegmental area.

[0086] According to the top-down bias effect of the temporal difference error on the basal ganglia, the direct pathway and the superdirect pathway are responsible for the obstacle avoidance action, while the indirect pathway is an inhibitor that inhibits the occurrence of erroneous obstacle avoidance actions.

[0087] return Figure 1 In step 3 of the embodiment, the obstacle avoidance action is interacted with the current environment to generate a reward given by the current environment; wherein the current environment is the environment in the lane scene at the current moment.

[0088] It should be noted that, in some embodiments, there are two types of rewards, one is discrete reward and the other is continuous reward. A single continuous reward cannot accurately represent the reward information from the environment. Considering the discreteness of the state, both discrete rewards and continuous rewards are set. This is conducive to better representing the rewards based on environmental feedback while ensuring the stability of the reward value.

[0089] If the driverless vehicle collides with an obstacle (specifically a stationary obstacle), a very large negative reward should be given to strongly punish this unsafe behavior; in order to encourage the driverless vehicle to maintain a suitable speed during driving, rewards can be given based on the speed; in order to encourage the driverless vehicle to actively avoid obstacles, at each time step, if the driverless vehicle maintains a sufficiently safe distance from the obstacle in front, a positive reward can be given; if the car successfully changes lanes to avoid an obstacle, a positive reward can be given to encourage it to learn to change lanes when appropriate, but in order to avoid unnecessary risks caused by frequent lane changes by the car, a small amount of negative rewards can be set to punish unnecessary lane changes.

[0090] Therefore, the mathematical formula for discrete rewards can be expressed as:

[0091] R collision (s t )=y2;

[0092] ;

[0093] ;

[0094] R lane-change (s t )=y6;

[0095] R fre-change (s t )=y4;

[0096] ;

[0097] In the formula, R b (s t ) is s t The corresponding total discrete reward, s t is the state sequence of the unmanned vehicle at the current time t, R collision (s t ) is the discrete reward when the unmanned vehicle is in a collision state with an obstacle, R speed (s t ) is the discrete reward calculated based on the speed interval of the unmanned vehicle at the current moment, R obstacle (s t ) is the discrete reward for whether the driverless vehicle maintains a safe distance from the obstacle, R lane-change (s t ) is the discrete reward for the driverless vehicle to successfully change lanes to avoid an obstacle, R fre-change (s t) is the discrete reward for the driverless vehicle to change lanes continuously, y1~y6 are preset values, which can be 10, -1000, 5, -5, -10, 20 respectively. These values ​​can be determined according to the actual situation. min and v max are the lower and upper limits of the speed of the driverless vehicle, for example, they are set to 5km / h and 50km / h respectively, D safe It is the preset safe distance between the driverless vehicle and obstacles, which can be set to 20 meters.

[0098] Based on the continuous state sequence of the unmanned vehicle, it is divided into four ranges of discrete states, and C1~C4 are defined as four different states of the unmanned vehicle; among them, C1 means that the distance between the unmanned vehicle and the obstacle is less than the safe distance, and this state requires a negative reward; C2 means that there is an obstacle within the range of the unmanned vehicle (i.e., the sensor range, such as 100 meters), and its relative position relative to the obstacle needs to be evaluated; C3 means that when the unmanned vehicle maintains a safe distance and there is no special danger, the reward is y1 (i.e., 10), indicating that the current state is neutral and there is no need for special rewards or penalties; C4 means that if the unmanned vehicle collides with an obstacle, this state requires a very large negative reward.

[0099] Therefore, the mathematical formula for continuous rewards can be expressed as:

[0100] ;

[0101] ;

[0102] In the formula, r t is the reward given by the environment at the current moment t, Eva(s t ) is s t The corresponding continuous reward, α is a fixed coefficient, d safe is the preset safety distance between the unmanned vehicle and the obstacle, d current is the distance between the unmanned vehicle and the obstacle at the current moment, v current is the speed of the driverless vehicle at the current moment.

[0103] return Figure 1 In step 4 of the embodiment, if the difference between the reward at the current moment and the reward at the previous moment exceeds the first threshold, the current moment environment is copied to the environment playback pool, and the accumulated alert value up to the current moment is calculated; wherein the alert value is the alert value of the pulse neural network to the environment.

[0104] It should be noted that the current reward is delivered to the substantia nigra compacta / ventral tegmental area, and the difference between the current reward and the previous reward is calculated, and this difference is defined as the reinforcement signal δ t , the formula can be expressed as:

[0105] δ(t)=r t -r t-1 ;

[0106] In the formula, δ(t) is the difference between the reward at the current moment and the reward at the previous moment, r t-1 It is the reward given by the environment at the previous moment t-1.

[0107] The anterior cingulate cortex (anterior cingulate cortex model outside the cortex-basal ganglia-thalamus network) determines whether the reinforcement signal exceeds a first threshold, which can be set to 10.

[0108] When δ t When it does not exceed the first threshold, it means that the pulse neural network does not need to be alert to the current environment, that is, there is no alert value at the current moment, and there is no need to copy the current environment to the environment playback pool, and the accumulated alert value remains unchanged.

[0109] See also Figure 4 , when δ t When the first threshold is exceeded, it means that the spiking neural network needs to be alert to the current environment, that is, the current environment needs to be copied to the environment playback pool, the current alert value needs to be calculated and accumulated, which can be expressed as:

[0110] ;

[0111] In the formula, is the accumulated alert value up to the current moment t, β is the accumulated alert value up to the previous moment, μ is the update rate of δ(t), δ(t) is the difference between the reward at the current moment and the reward at the previous moment, μ is the update rate, which can be set to 0.5.

[0112] Based on the vigilance threshold mechanism of the anterior cingulate cortex, an environmental replay pool is constructed, and the vigilance environment can be used to evolve and optimize the spiking neural network using a genetic algorithm. This can explore a larger strategy space and obtain a better unmanned driving obstacle avoidance strategy.

[0113] return Figure 1 In step 5 of the embodiment, if the accumulated alert value up to the current moment does not exceed the second threshold, the pulse neural network is optimized based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn; if the accumulated alert value up to the current moment exceeds the second threshold, the pulse neural network is optimized based on the offline evolution method of the genetic algorithm according to the current moment environment in the environment playback pool; wherein the optimized pulse neural network is used to generate the obstacle avoidance action of the unmanned vehicle at the next moment.

[0114] See also Figure 4When the accumulated alert value does not exceed the second threshold (which can be set to 100), the spiking neural network is optimized based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn, specifically, the synaptic weights between the spiking neurons in the dorsolateral prefrontal cortex and the D1 dopamine receptor spiking neurons in the striatum are optimized, and the synaptic connection weights between the spiking neurons in the dorsolateral prefrontal cortex and the D2 dopamine receptor spiking neurons in the striatum are optimized.

[0115] In some embodiments, see Figure 5 , the above optimization method may include:

[0116] S51) The optimized spiking neural network is optimized for the first time based on the spike timing-dependent plasticity mechanism; wherein the first optimization is to optimize the synaptic weights between the spiking neurons in the dorsolateral prefrontal cortex and the D1 dopamine receptor spiking neurons in the striatum, and to optimize the synaptic connection weights between the spiking neurons in the dorsolateral prefrontal cortex and the D2 dopamine receptor spiking neurons in the striatum.

[0117] It should be noted that, using the spike timing-dependent plasticity mechanism, if the spike of the pre-pulse neuron occurs before the spike of the post-pulse neuron, the synaptic weight will be enhanced, and if the spike of the pre-pulse neuron occurs after the spike of the post-pulse neuron, the synaptic weight will be weakened; it can be expressed as:

[0118] ;

[0119] Where Δw is the update amount of synaptic weight, A + A is the constant proportionality factor for the magnitude of the increase in synaptic weight. - is the proportional factor of the amplitude constant of the synaptic weight reduction, Δt is the time difference between the pre-spiking neuron and the post-spiking neuron, specifically, the time interval between the pre-spiking neuron firing signal and the post-spiking neuron firing signal, is the time constant, which determines how quickly the inhibitory effect decays over time.

[0120] S52) Determine whether the difference between the current reward and the previous reward is greater than 0, if so, go to S53), if not, go to S54).

[0121] S53) Based on the first optimization, the synaptic weight between the dorsolateral prefrontal cortex and the striatum D1 dopamine receptors is increased, and the synaptic weight between the dorsolateral prefrontal cortex and the striatum D2 dopamine receptors is reduced.

[0122] S54) If the difference between the reward at the current moment and the reward at the previous moment is not greater than 0, based on the first optimization, the synaptic weight between the dorsolateral prefrontal cortex and the striatal D1 dopamine receptors is reduced, and the synaptic weight between the dorsolateral prefrontal cortex and the striatal D2 dopamine receptors is increased.

[0123] The above synaptic weight increase and decrease can be expressed as follows:

[0124] ;

[0125] ;

[0126] ;

[0127] ;

[0128] In the formula, DA D1 is the increase or decrease coefficient of the synaptic weight between the dorsolateral prefrontal cortex and the striatum D1 dopamine receptor, DA D2 is the increase or decrease coefficient of the synaptic weight between the dorsolateral prefrontal cortex and the striatal D2 dopamine receptor, The synaptic weights between the dorsolateral prefrontal cortex and striatal D1 dopamine receptors before and after the increase and decrease, respectively. These are the synaptic weights between the dorsolateral prefrontal cortex and striatal D2 dopamine receptors before and after increases and decreases, respectively.

[0129] The spike timing-dependent plasticity mechanism is a commonly used weight update mechanism in spiking neural networks, which is similar to the backpropagation mechanism in neural networks. Combined with the dopamine regulation mechanism, it can effectively amplify the differences between direct pathways and indirect pathways, and help to clarify action selection.

[0130] See also Figure 4 When the accumulated alert value exceeds the second threshold, the pulse neural network is optimized based on the offline evolution method of genetic algorithm according to the current environment in the environment playback pool. The niche sharing mechanism is adopted in the offline evolution method, so that the pulse neural network can obtain more accurate rewards.

[0131] In some embodiments, see Figure 6 , the above optimization method may include:

[0132] S1) obtaining spiking neurons of the dorsolateral prefrontal cortex of the spiking neural network to be optimized, and using the obtained spiking neurons as the initial nucleus.

[0133] S2) randomly expanding the initial nuclear cluster to obtain multiple new nuclear clusters, and using the new nuclear clusters to replace the initial nuclear cluster in the pulse neural network to be optimized to obtain multiple new pulse neural networks to be optimized.

[0134] In the above steps, random expansion is to randomly modify the synaptic weights of the spiking neurons in the initial nuclei, thereby expanding multiple new nuclei. The above two steps are equivalent to population initialization, randomly initializing the spiking neurons of the dorsolateral prefrontal cortex as a population. Each spiking neuron contains a promoter and a regulatory matrix. The promoter determines the activation rules of the spiking neuron, and the regulatory matrix describes the distribution and dynamic changes of the synaptic weights.

[0135] S3) Traverse all the spiking neural networks to be optimized, obtain the rewards corresponding to the elite network and the rewards corresponding to the spiking neural network to be optimized in the current environment, and take the difference between the rewards corresponding to the initial elite network and the rewards corresponding to the spiking neural network to be optimized as the rewards of the corresponding nuclear group, and calculate the rewards of each spiking neuron in the nuclear group according to the rewards of the nuclear group and the synaptic weights of each spiking neuron in the nuclear group; among them, the initial elite network is the spiking neural network with the highest reward in the historical moment.

[0136] The elite network and the pulse neural network to be optimized interact in the replicated environment (i.e., the replicated current environment) to obtain their respective rewards, and the difference between the two rewards is used as the reward of the corresponding core cluster. For example, if the pulse neural network to be optimized is the pulse neural network to be optimized corresponding to core cluster H, then the difference is the reward of core cluster H.

[0137] A spiking neuron niche sharing mechanism based on synaptic weights is introduced to distribute the overall reward of nucleus cluster H to each spiking neuron in nucleus cluster H. Specifically, the reward of each spiking neuron is obtained according to the reward of nucleus cluster H and the synaptic weight of each spiking neuron, ensuring that the adaptability and rewards of different spiking neurons in competition can be reasonably evaluated.

[0138] For spiking neurons, their activation patterns can be directly defined as responses to input stimuli (i.e., the generated pulse outputs). The pulse train of each spiking neuron can be represented by a vector that records the occurrence of pulses within a time window. When allocating rewards, the functional overlap of spiking neurons can be further considered. The activation patterns of spiking neurons (i.e., responses to input stimuli) are used to measure the functional overlap between spiking neurons. When allocating rewards, rewards are shared for highly overlapping spiking neurons based on the inner product to reduce homogeneous competition.

[0139] The above niche sharing mechanism can be expressed as follows:

[0140] ;

[0141] ;

[0142] ;

[0143] ;

[0144] In the formula, is the reward of the jth spiking neuron, w j is the synaptic weight of the j-th spiking neuron, w k is the synaptic weight of the k-th spiking neuron, To adjust the parameters of reward distribution intensity, , R LPFC As a reward for the nucleus, is the functional overlap between the jth and kth spiking neurons (measured by the activation pattern between spiking neurons), are the activation sequences of the j-th and k-th spike neurons at the current time t, respectively.

[0145] S4) Determine whether the number of iterations reaches a third threshold (which can be set to 1000). If so, go to S5). If not, go to S6.

[0146] S5) The first N spiking neurons with high rewards are taken as the final nucleus to obtain the optimized spiking neural network; wherein N is the number of spiking neurons in the dorsolateral prefrontal cortex of the spiking neural network that is not optimized by the offline evolutionary method. Assuming that the number of spiking neurons in the dorsolateral prefrontal cortex before optimization is 4422, then N here is also 4422.

[0147] S6) Perform mutation and propagation operations on the first N spiking neurons with high rewards, use the spiking neurons after mutation and propagation operations as the initial nucleus, and go to S2).

[0148] It should be noted that mutation operations include changing the initial activity of genes by mutation of promoter genes, and regulating the mutation of regional genes to adjust synaptic weights or dynamic change patterns. Through mutation and reproduction operations, new spiking neurons are generated to form the next generation population.

[0149] The above method generates an obstacle avoidance action at the current moment according to the state sequence of the unmanned vehicle at the current moment and the cortex-basal ganglia-thalamus network, and optimizes the cortex-basal ganglia-thalamus network at the next moment based on the obstacle avoidance action at the current moment. In the optimization process of the cortex-basal ganglia-thalamus network, the vigilance value of the cortex-basal ganglia-thalamus network to the environment is accumulated according to the reward difference at adjacent moments. When the accumulated vigilance value does not exceed the second threshold, the cortex-basal ganglia-thalamus network is optimized based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn. When the accumulated vigilance value exceeds the second threshold, the cortex-basal ganglia-thalamus network is optimized based on the offline evolution method of the genetic algorithm, which can effectively improve the rapid adaptation and decision-making accuracy of the cortex-basal ganglia-thalamus network to complex environments, thereby realizing an obstacle avoidance method that can quickly adapt to different complex environments and make accurate decisions.

[0150] See also Figure 7 , is an unmanned driving obstacle avoidance device with brain-like pulse enhancement evolution provided by the present disclosure, Figure 7 The embodiment is a virtual device that can be loaded and executed by a computer device, and the computer device can include the above-mentioned obstacle avoidance device, Figure 7 The unmanned driving obstacle avoidance device with enhanced evolution of brain-like pulses shown may include an acquisition module, an obstacle avoidance action generation module, a reward generation module, a vigilance value accumulation module, a first optimization module and a second optimization module, which, when used to execute the above-mentioned unmanned driving obstacle avoidance method with enhanced evolution of brain-like pulses, can:

[0151] The acquisition module is configured to acquire a state sequence of the driverless vehicle in the lane scene at a current moment.

[0152] The obstacle avoidance action generation module is configured to generate the obstacle avoidance action of the unmanned vehicle at the current moment according to the state sequence and the spiking neural network; wherein the spiking neural network is a cortex-basal ganglia-thalamus network.

[0153] The reward generation module is configured to interact the obstacle avoidance action with the current environment to generate a reward given by the current environment; wherein the current environment is the environment in the lane scene at the current moment.

[0154] In the reward generation module, the reward given by the current environment is generated. The formula is:

[0155] ;

[0156] ;

[0157] ;

[0158] In the formula, r tis the reward given by the environment at the current moment t, s t is the state sequence of the unmanned vehicle at the current time t, R b (s t ) is s t The corresponding total discrete reward, Eva(s t ) is s t The corresponding continuous reward, α is a fixed coefficient, d safe is the preset safety distance between the unmanned vehicle and the obstacle, d current is the distance between the unmanned vehicle and the obstacle at the current moment, v current is the speed of the driverless vehicle at the current moment, R collision (s t ) is the discrete reward when the unmanned vehicle is in a collision state with an obstacle, R speed (s t ) is the discrete reward calculated based on the speed interval of the unmanned vehicle at the current moment, R obstacle (s t ) is the discrete reward for whether the driverless vehicle maintains a safe distance from the obstacle, R lane-change (s t ) is the discrete reward for the driverless vehicle to successfully change lanes to avoid an obstacle, R fre-change (s t ) is the discrete reward for the unmanned vehicle to change lanes continuously, y1 and y2 are preset values, and C1~C4 are the states of four different unmanned vehicles.

[0159] The alert value accumulation module copies the current environment to the environment playback pool and calculates the alert value accumulated to the current moment if the difference between the reward at the current moment and the reward at the previous moment exceeds the first threshold; wherein the alert value is the alert value of the pulse neural network to the environment.

[0160] The first optimization module is configured to optimize the pulse neural network based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in sequence if the accumulated alert value up to the current moment does not exceed the second threshold; wherein the optimized pulse neural network is used to generate the obstacle avoidance action of the unmanned vehicle at the next moment.

[0161] The second optimization module is configured to optimize the pulse neural network based on an offline evolution method of a genetic algorithm according to the current environment in the environment playback pool if the accumulated alert value up to the current moment exceeds the second threshold.

[0162] The above-mentioned device generates an obstacle avoidance action at the current moment according to the state sequence of the unmanned vehicle at the current moment and the cortex-basal ganglia-thalamus network, and optimizes the cortex-basal ganglia-thalamus network at the next moment based on the obstacle avoidance action at the current moment. In the process of optimizing the cortex-basal ganglia-thalamus network, the vigilance value of the cortex-basal ganglia-thalamus network to the environment is accumulated according to the reward difference at adjacent moments. When the accumulated vigilance value does not exceed the second threshold, the cortex-basal ganglia-thalamus network is optimized based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn. When the accumulated vigilance value exceeds the second threshold, the cortex-basal ganglia-thalamus network is optimized based on the offline evolution method of the genetic algorithm, which can effectively improve the rapid adaptation and decision-making accuracy of the cortex-basal ganglia-thalamus network to complex environments, thereby realizing an obstacle avoidance method that can quickly adapt to different complex environments and make accurate decisions.

[0163] The present disclosure also relates to a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes an unmanned driving obstacle avoidance method based on brain-like pulse enhancement evolution.

[0164] The present disclosure also relates to a computer device, comprising one or more processors and one or more memories, wherein one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing an unmanned driving obstacle avoidance method that is evolved by brain-like pulse enhancement.

[0165] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. An unmanned driving obstacle avoidance method based on brain-like pulse enhancement evolution, characterized in that: include: Obtain the state sequence of the driverless vehicle in the lane scene at the current moment; Generate the obstacle avoidance action of the unmanned vehicle at the current moment according to the state sequence and the spiking neural network; wherein the spiking neural network is a cortex-basal ganglia-thalamus network; The obstacle avoidance action is interacted with the current environment to generate a reward given by the current environment; wherein the current environment is the environment in the lane scene at the current moment; If the difference between the reward at the current moment and the reward at the previous moment exceeds the first threshold, the current moment environment is copied to the environment playback pool, and the accumulated alert value up to the current moment is calculated; wherein the alert value is the alert value of the pulse neural network to the environment; If the accumulated alert value up to the current moment does not exceed the second threshold, optimizing the spiking neural network based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn; wherein the optimized spiking neural network is used to generate the obstacle avoidance action of the unmanned vehicle at the next moment; If the accumulated alert value up to the current moment exceeds the second threshold, the pulse neural network is optimized based on the offline evolution method of the genetic algorithm according to the current moment environment in the environment playback pool.

2. The method according to claim 1, characterized in that Spiking neurons A in the cortex-basal ganglia-thalamus network were constructed based on the gene regulatory network A1 and the Izhikevich neuron model A2; Izhikevich neuron model A2 is used to model the firing behavior of spiking neuron A; Gene regulatory network A1 is used to generate synaptic weights between spiking neuron A and spiking neuron B. The input of gene regulatory network A1 is the synaptic current of input spiking neuron A. Among them, spiking neuron B is a spiking neuron that receives the output of spiking neuron A.

3. The method according to claim 1, characterized in that Generate the reward given by the current environment. The formula is: ； ； ； In the formula, r t is the reward given by the environment at the current moment t, s t is the state sequence of the unmanned vehicle at the current time t, R b (s t ) is s t The corresponding total discrete reward, Eva(s t ) is s t The corresponding continuous reward, α is a fixed coefficient, d safe is the preset safety distance between the unmanned vehicle and the obstacle, d current is the distance between the unmanned vehicle and the obstacle at the current moment, v current is the speed of the driverless vehicle at the current moment, R collision (s t ) is the discrete reward when the unmanned vehicle is in a collision state with an obstacle, R speed (s t ) is the discrete reward calculated based on the speed interval of the unmanned vehicle at the current moment, R obstacle (s t ) is the discrete reward for whether the unmanned vehicle maintains a safe distance from the obstacle, R lane-change (s t ) is the discrete reward for the driverless vehicle to successfully change lanes to avoid an obstacle, R fre-change (s t ) is the discrete reward for the unmanned vehicle to change lanes continuously, y1 and y2 are preset values, and C1~C4 are the states of four different unmanned vehicles.

4. The method according to claim 1, characterized in that: Calculate the accumulated alert value up to the current moment, the formula is: ； In the formula, is the accumulated alert value up to the current moment t, β is the accumulated alert value up to the previous moment, μ is the update rate of δ(t), and δ(t) is the difference between the reward at the current moment and the reward at the previous moment.

5. The method according to claim 1, characterized in that: The spiking neural network is optimized based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism, including: The optimized spiking neural network is optimized for the first time based on the spike timing-dependent plasticity mechanism; wherein the first optimization is to optimize the synaptic weights between the dorsolateral prefrontal cortex and the striatal D1 dopamine receptors, and the synaptic weights between the dorsolateral prefrontal cortex and the striatal D2 dopamine receptors in the spiking neural network; If the difference between the current reward and the previous reward is greater than 0, based on the first optimization, the synaptic weight between the dorsolateral prefrontal cortex and the striatal D1 dopamine receptor is increased, and the synaptic weight between the dorsolateral prefrontal cortex and the striatal D2 dopamine receptor is reduced; If the difference between the current reward and the previous reward is not greater than 0, based on the first optimization, the synaptic weight between the dorsolateral prefrontal cortex and the striatal D1 dopamine receptors is reduced, and the synaptic weight between the dorsolateral prefrontal cortex and the striatal D2 dopamine receptors is increased.

6. The method according to claim 1, characterized in that According to the current environment in the environment playback pool, the spiking neural network is optimized based on the offline evolution method of genetic algorithm, including: S1) obtaining spiking neurons of the dorsolateral prefrontal cortex of the spiking neural network to be optimized, and using the obtained spiking neurons as the initial nucleus; S2) randomly expanding the initial core group to obtain multiple new core groups, using the new core groups to replace the initial core group in the pulse neural network to be optimized, and obtaining multiple new pulse neural networks to be optimized; S3) Traverse all the spiking neural networks to be optimized, obtain the rewards corresponding to the elite network and the rewards corresponding to the spiking neural network to be optimized in the current environment, and take the difference between the rewards corresponding to the initial elite network and the rewards corresponding to the spiking neural network to be optimized as the rewards of the corresponding nuclei, and calculate the rewards of each spiking neuron in the nuclei according to the rewards of the nuclei and the synaptic weights of each spiking neuron in the nuclei; among them, the initial elite network is the spiking neural network with the highest reward in the historical moment; S4) If the number of iterations reaches the third threshold, the first N spiking neurons with high rewards are used as the final nucleus to obtain the optimized spiking neural network. If the number of iterations does not reach the third threshold, the first N spiking neurons with high rewards are mutated and propagated, and the spiking neurons after the mutation and propagation operations are used as the initial nucleus, and then go to S2); where N is the number of spiking neurons in the dorsolateral prefrontal cortex of the spiking neural network that is not optimized using the offline evolution method.

7. An unmanned driving obstacle avoidance device based on brain-like pulse enhancement evolution, characterized in that: include: An acquisition module obtains the state sequence of the unmanned vehicle in the lane scene at the current moment; The obstacle avoidance action generation module generates the obstacle avoidance action of the unmanned vehicle at the current moment according to the state sequence and the spiking neural network; wherein the spiking neural network is a cortex-basal ganglia-thalamus network; The reward generation module interacts the obstacle avoidance action with the current environment to generate a reward given by the current environment; wherein the current environment is the environment in the lane scene at the current moment; A vigilance value accumulation module, if the difference between the reward at the current moment and the reward at the previous moment exceeds the first threshold, copies the current moment environment to the environment playback pool, and calculates the vigilance value accumulated to the current moment; wherein the vigilance value is the vigilance value of the pulse neural network to the environment; The first optimization module optimizes the spiking neural network based on the spike timing-dependent plasticity mechanism and the dopamine regulation mechanism in turn if the accumulated alert value up to the current moment does not exceed the second threshold; wherein the optimized spiking neural network is used to generate the obstacle avoidance action of the unmanned vehicle at the next moment; The second optimization module optimizes the pulse neural network based on the offline evolution method of the genetic algorithm according to the current environment in the environment playback pool if the accumulated alert value up to the current moment exceeds the second threshold.

8. The device according to claim 7, characterized in that In the reward generation module, the reward given by the current environment is generated. The formula is: ； ； ； In the formula, r t is the reward given by the environment at the current moment t, s t is the state sequence of the unmanned vehicle at the current time t, R b (s t ) is s t The corresponding total discrete reward, Eva(s t ) is s t The corresponding continuous reward, α is a fixed coefficient, d safe is the preset safety distance between the unmanned vehicle and the obstacle, d current is the distance between the unmanned vehicle and the obstacle at the current moment, v current is the speed of the driverless vehicle at the current moment, R collision (s t ) is the discrete reward when the unmanned vehicle is in a collision state with an obstacle, R speed (s t ) is the discrete reward calculated based on the speed interval of the unmanned vehicle at the current moment, R obstacle (s t ) is the discrete reward for whether the unmanned vehicle maintains a safe distance from the obstacle, R lane-change (s t ) is the discrete reward for the driverless vehicle to successfully change lanes to avoid an obstacle, R fre-change (s t ) is the discrete reward for the unmanned vehicle to change lanes continuously, y1 and y2 are preset values, and C1~C4 are the states of four different unmanned vehicles.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes any one of the methods of claims 1 to 6.

10. A computer device, characterized in that: include: One or more processors, and one or more memories, one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods described in claims 1 to 6.

Images