End-to-end automatic driving method and system based on brain-like neural circuit trajectory guidance

Abstract

This application relates to the field of autonomous driving technology, and particularly to an end-to-end autonomous driving method and system based on neuromorphic neural circuit trajectory guidance. The method includes the following steps: extracting effective traffic information from the environment; simulating a nematode neural network to establish a neuromorphic neural circuit network, using the neuromorphic neural circuit network as the motion branch; and cyclically constructing the instruction units of the motion branch to obtain the trajectory branch; simultaneously inputting the effective traffic information from the environment into the motion branch and the trajectory branch, outputting the vehicle's control commands and predicted trajectory points at different future times; and obtaining the vehicle's final control strategy based on the vehicle's control commands and the predicted trajectory points at different future times. The method provided in this application uses a ResNet encoder and an MLP encoder to extract effective traffic information from the environment, and simulates a nematode neural system to establish a neuromorphic neural network. Through supervised imitation learning, it trains the vehicle's intelligent agent behavior to complete the end-to-end autonomous driving task.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an end-to-end autonomous driving method and system based on neuromorphic neural circuit trajectory guidance. Background Technology

[0002] Currently, with the development of artificial intelligence, the development of autonomous driving is imminent and has become one of the major directions of national development. Current autonomous driving paradigms can be divided into two types: modular and end-to-end. Modular architecture, with its clear division of sub-tasks, is widely adopted in the industry, but it suffers from high computational load and high error accumulation. End-to-end architecture, as a paradigm of general artificial intelligence, has received widespread attention from academia.

[0003] End-to-end autonomous driving currently employs two main control outputs: one directly outputs control commands such as throttle and steering wheel angle; the other outputs a predicted future multi-step trajectory, which is then followed by control algorithms like PID controllers. However, the different details addressed by each control output lead to inconsistent performance across various scenarios. For instance, control commands often suffer from delays, making them prone to loss of control during sudden traffic changes; while trajectory outputs perform poorly under conditions like sharp turns. Furthermore, when training a single output using supervised imitation learning, limitations in the network structure can easily result in suboptimal fitting and poor generalization. Summary of the Invention

[0004] This application provides an end-to-end autonomous driving method and system based on neuromorphic neural circuit trajectory guidance. The method uses the vehicle's forward-looking camera and related measurement information as input, applies a ResNet encoder and an MLP encoder to extract effective traffic information from the environment, and establishes a neuromorphic neural network by simulating the nematode neural system. Through supervised imitation learning, the behavior of the vehicle's intelligent agent is trained to complete the end-to-end autonomous driving task.

[0005] To address the aforementioned technical problems, in a first aspect, embodiments of this application provide an end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance, comprising: first, extracting effective traffic information from the environment; then, simulating a nematode neural network to establish a neuromorphic neural circuit network, using the neuromorphic neural circuit network as a motion branch; and cyclically constructing the instruction units of the motion branch to obtain a trajectory branch; next, simultaneously inputting the effective traffic information from the environment into the motion branch and the trajectory branch, outputting vehicle control commands and predicted trajectory points at different future times; and finally, obtaining the vehicle's final control strategy based on the vehicle's control commands and the predicted trajectory points at different future times.

[0006] In some exemplary embodiments, extracting effective traffic information in the environment includes: taking a time-series image from a forward-looking camera as input and obtaining a first branch feature vector through a ResNet encoder; taking sensor measurement information of the vehicle as input and obtaining a second branch feature vector through an MLP encoder; and fusing the first branch feature vector and the second branch feature vector to obtain effective traffic information in the environment.

[0007] In some exemplary embodiments, the vehicle's sensor measurement information includes vehicle speed information and vehicle position information.

[0008] In some exemplary embodiments, the brain-like neural circuit network includes four layers of neurons; wherein, the four layers of neurons are: N s One sensory neuron, N i N internal neurons c N instruction neurons, m n motor neurons; between any two consecutive layers, for any source neuron, insert n so-t n synapses; of which, n so-t Satisfy: n so-t ≤N t The synaptic polarity follows a Bernoulli distribution, where N t n represents the number of target neurons. so-t The target neurons are randomly selected using a binomial distribution.

[0009] Between any two consecutive layers, any target neuron j without a synapse is inserted into m. so-t One synapse, m so-t satisfy: in, Let m be the number of synapses connected to the target neuron i, and let m be the number of synapses whose polarity follows a Bernoulli distribution. so-t Each source neuron is randomly selected using a binomial distribution.

[0010] Circular connections exist between instruction neurons; for any instruction neuron, insert l so-t There are synapses, of which l so-t Satisfy: l so-t ≤N c The synaptic polarity follows a Bernoulli distribution, where N c Indicates the number of instruction neurons, l so-t Each source neuron is randomly selected using a binomial distribution.

[0011] In some exemplary embodiments, each neuron is modeled as follows, based on the characteristics of the current transmission between neuronal synapses:

[0012]

[0013] Where x(t) represents the neuronal synaptic current, I(t) represents the external input to the synapse, A is the bias matrix, and f I Let τ represent a neural network, and τ represent the time constant.

[0014] In some exemplary embodiments, loss functions for the motion branch and the trajectory branch are defined respectively, and brain-like neural circuit networks are used to model the motion branch and the trajectory branch respectively.

[0015] In some exemplary embodiments, the loss function for the motion branch is defined as:

[0016]

[0017] Where Beta(a) represents the Beta distribution parameter represented by the corresponding predicted distribution; This represents an expert distribution from the dataset; KL divergence is used to measure the similarity between predictive controls. This represents the supervisory signal from expert data at the current moment. and prediction signal The L2 measure of feature loss between, λ F This indicates an adjustable loss weight.

[0018] In some exemplary embodiments, the loss function for trajectory branching is defined as:

[0019]

[0020] Among them, wpt and These represent the values ​​of the predicted trajectory point and the actual trajectory point at time t, respectively. This represents the supervisory signal from expert data at the current moment. and prediction signal The L2 measure of feature loss between, λ F This indicates an adjustable loss weight.

[0021] Secondly, embodiments of this application also provide an end-to-end autonomous driving system based on neuromorphic neural circuit trajectory guidance, including a feature vector extraction module, a model building module, and a final action output module connected in sequence; the feature vector extraction module is used to extract effective traffic information of the environment; the model building module includes a motion branch unit and a trajectory branch unit; the model building module is used to simulate a nematode neural network, establish a neuromorphic neural circuit network, and construct the motion branch unit; and iteratively construct the instruction unit of the motion branch unit to obtain the trajectory branch unit; the motion branch unit is used to output vehicle control commands based on the effective traffic information of the environment; the trajectory branch unit is used to obtain predicted trajectory points at different future times based on the effective traffic information of the environment; the final action output module is used to obtain the final control strategy of the vehicle based on the vehicle control commands and the predicted trajectory points at different future times.

[0022] In some exemplary embodiments, the feature vector extraction module includes a first branch unit, a second branch unit, and a fusion unit; the first branch unit is used to obtain a first branch feature vector by taking the temporal image of the foreseeing camera as input and passing it through a ResNet encoder; the second branch unit is used to obtain a second branch feature vector by taking the sensor measurement information of the vehicle as input and passing it through an MLP encoder; the fusion unit is used to fuse the first branch feature vector and the second branch feature vector to obtain effective traffic information of the environment.

[0023] The technical solution provided in this application has at least the following advantages:

[0024] This application provides an end-to-end autonomous driving method and system based on neuromorphic neural circuit trajectory guidance. The method includes: first, extracting effective traffic information from the environment; then, simulating a nematode neural network to establish a neuromorphic neural circuit network, using the neuromorphic neural circuit network as a motion branch; and cyclically constructing the instruction units of the motion branch to obtain a trajectory branch; next, simultaneously inputting the effective traffic information from the environment into the motion branch and the trajectory branch, and outputting the vehicle's control commands and predicted trajectory points at different future times; finally, based on the vehicle's control commands and the predicted trajectory points at different future times, obtaining the vehicle's final control strategy.

[0025] This application proposes an end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance. This method utilizes the vehicle's forward-looking camera and related vehicle measurement information as input, applying a ResNet encoder and an MLP encoder to extract effective traffic information from the environment. The temporal image input from the forward-looking camera is processed by the ResNet encoder to form one branch of the feature vector; the other branch is the feature vector formed by processing vehicle speed, position, and other sensor information through the MLP encoder. The two branches are fused to form the final perception information, which is then fed into a trajectory branch and a motion branch. The motion branch uses a neuromorphic neural circuit architecture built to simulate a nematode neural network, outputting vehicle control commands; the trajectory branch iteratively constructs the command units of the motion branch to output predicted trajectory points at different future times. The trajectory is fused with the control commands generated by dual PID controllers and the control commands from the control branch to form the vehicle's final control strategy. Attached Figure Description

[0026] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments, and unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0027] Figure 1 A flowchart illustrating an end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance provided in an embodiment of this application;

[0028] Figure 2 A schematic diagram of the structure of an end-to-end autonomous driving system based on neuromorphic neural circuit trajectory guidance provided in an embodiment of this application;

[0029] Figure 3 An architectural flowchart of an end-to-end autonomous driving system based on neuromorphic neural circuit trajectory guidance provided in an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of a brain-like neural circuit architecture provided in an embodiment of this application. Detailed Implementation

[0031] As the background technology shows, existing end-to-end autonomous driving control methods focus on different details in their single control output, resulting in inconsistent performance in different scenarios. For example, control commands often have delays, making them prone to loss of control when traffic conditions change abruptly; while trajectory outputs perform poorly under conditions such as sharp turns. Furthermore, when training a single output through supervised imitation learning, limitations in the network structure can easily lead to suboptimal fitting and difficulty in generalization.

[0032] Control commands and trajectory outputs have different application focuses. Combining them, using trajectory-guided control outputs, can easily combine their advantages to handle more complex scenarios. Furthermore, biological intelligence can be introduced to modify neural network structures and simulate biological intelligence to improve the model's convergence rate and performance.

[0033] To address the aforementioned technical problems, this application provides an end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance, comprising: first, extracting effective traffic information from the environment; then, simulating a nematode neural network to establish a neuromorphic neural circuit network, using the neuromorphic neural circuit network as a motion branch; and cyclically constructing the instruction units of the motion branch to obtain a trajectory branch; next, simultaneously inputting the effective traffic information from the environment into the motion branch and the trajectory branch, outputting vehicle control commands and predicted trajectory points at different future times; finally, based on the vehicle control commands and the predicted trajectory points at different future times, obtaining the vehicle's final control strategy. This application provides an end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance. This method utilizes the vehicle's forward-looking camera and related measurement information as input, applies a ResNet (deep residual neural network model) encoder and an MLP (multilayer perceptron) encoder to extract effective traffic information from the environment, and simulates a nematode neural system to establish a neuromorphic neural network. Through supervised imitation learning, the behavior of the vehicle's intelligent agent is trained to complete the end-to-end autonomous driving task.

[0034] The embodiments of this application will now be described in detail with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0035] See Figure 1 This application provides an end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance, including:

[0036] Step S1: Extract valid traffic information from the environment.

[0037] Step S2: Simulate the nematode neural network, establish a brain-like neural circuit network, and use the brain-like neural circuit network as the motor branch; and cyclically construct the instruction unit of the motor branch to obtain the trajectory branch.

[0038] Step S3: Simultaneously input the effective traffic information of the environment into the motion branch and the trajectory branch, and output the vehicle control command and the predicted trajectory points at different future times.

[0039] Step S4: Based on the vehicle's control commands and predicted trajectory points at different future times, the final control strategy for the vehicle is obtained.

[0040] In some embodiments, step S1 extracts valid environmental traffic information, including:

[0041] Step S101: Using the temporal image from the foreseeing camera as input, obtain the first branch feature vector through a ResNet encoder.

[0042] Step S102: Using the vehicle's sensor measurement information as input, the second branch feature vector is obtained through the MLP encoder.

[0043] Step S103: Fuse the first branch feature vector and the second branch feature vector to obtain effective traffic information in the environment.

[0044] It should be noted that steps S102 and S103 can be performed simultaneously or separately. This application does not limit the order of steps S101 and S102.

[0045] In some embodiments, the vehicle's sensor measurement information includes vehicle speed information and vehicle position information.

[0046] See Figure 2 This application also provides an end-to-end autonomous driving system based on neuromorphic neural circuit trajectory guidance, including a feature vector extraction module 101, a model building module 102, and a final action output module 103 connected in sequence. The feature vector extraction module 101 is used to extract effective traffic information in the environment. The model building module 102 includes a motion branch unit 1021 and a trajectory branch unit 1022. The model building module 102 is used to simulate a nematode neural network, establish a neuromorphic neural circuit network, and construct the motion branch unit. The instruction unit of the motion branch unit 1021 is cyclically constructed to obtain the trajectory branch unit 1022. The motion branch unit 1021 is used to output vehicle control commands based on the effective traffic information in the environment. The trajectory branch unit 1022 is used to obtain predicted trajectory points at different future times based on the effective traffic information in the environment. The final action output module 103 is used to obtain the final control strategy of the vehicle based on the vehicle control commands and the predicted trajectory points at different future times.

[0047] Please continue reading. Figure 2In some embodiments, the feature vector extraction module 101 includes a first branch unit 1011, a second branch unit 1012, and a fusion unit 103; the first branch unit 1011 is used to obtain a first branch feature vector by taking the temporal image of the foreseeing camera as input and passing it through a ResNet encoder; the second branch unit 1012 is used to obtain a second branch feature vector by taking the sensor measurement information of the vehicle as input and passing it through an MLP encoder; the fusion unit 103 is used to fuse the first branch feature vector and the second branch feature vector to obtain effective traffic information of the environment.

[0048] It should be noted that the ResNet encoder is a residual neural network encoder; the MLP encoder is a multilayer perceptron encoder.

[0049] The model architecture diagram used in this application is as follows: Figure 3 As shown, from Figure 3 As can be seen, the perception module receives two branches of information from the environment. The temporal image input from the forward-looking camera is encoded into a feature vector by a ResNet encoder; the other branch is the feature vector formed by sensor information such as vehicle speed and position after being encoded by an MLP encoder. These two branches are fused to form the final perception information (feature vector), which is then fed into the subsequent trajectory and motion branches. The motion branch uses a brain-like neural circuit architecture modeled after a nematode neural network, including perception units, internal units, command units, and motion units. Each unit ultimately outputs the vehicle's control commands. The trajectory branch is constructed by iteratively building upon the command units of the motion branch. It includes perception units, internal units, and multiple command units, each corresponding to a different motion unit, used to output predicted trajectory points at different future times. Finally, the trajectory is fused with the control commands generated by the dual PID controller and the control commands from the control branch to form the vehicle's final control strategy.

[0050] This application enhances the learning ability of a model by establishing a brain-like neural circuit network that mimics the neural system of nematodes, replacing traditional neural networks. This application establishes a brain-like neural circuit architecture that mimics the activation mode of neurons in *Caenorhabditis elegans* and their communication via electrical impulses. Figure 4 As shown.

[0051] See Figure 4 The brain-like neural circuit network consists of four layers of neurons; the four layers of neurons are: N s One sensory neuron, N i N internal neurons c N instruction neurons, m n motor neurons; between any two consecutive layers, for any source neuron, insert n so-t n synapses; of which, n so-tSatisfy: n so-t ≤N t The synaptic polarity follows a Bernoulli distribution, where N t n represents the number of target neurons. so-t The target neurons are randomly selected using a binomial distribution.

[0052] Between any two consecutive layers, any target neuron j without a synapse is inserted into m. so-t One synapse, m so-t satisfy: in, Let m be the number of synapses connected to the target neuron i, and let m be the number of synapses whose polarity follows a Bernoulli distribution. so-t Each source neuron is randomly selected using a binomial distribution.

[0053] Circular connections exist between instruction neurons; for any instruction neuron, insert l so-t There are synapses, of which l so-t Satisfy: l so-t ≤N c The synaptic polarity follows a Bernoulli distribution, where N c Indicates the number of instruction neurons, l so-t Each source neuron is randomly selected using a binomial distribution.

[0054] In some exemplary embodiments, each neuron is modeled as follows, based on the characteristics of the current transmission between neuronal synapses:

[0055]

[0056] Where x(t) represents the neuronal synaptic current, I(i) represents the external input to the synapse, A is the bias matrix, and f I Let τ represent a neural network, and τ represent the time constant.

[0057] Imitation learning employs a supervised learning strategy. First, it uses expert strategies to collect and save various types of information, including information from forward-looking cameras, vehicle sensors, trajectory, and control, in different environments, thereby creating a dataset for training.

[0058] During training, assuming the current time is t0, the time interval from t0 to t... TThe model is trained using environmental data input. Input from the forward-looking camera and sensor information such as speed and position are fed into different encoders. The temporal image input from the forward-looking camera is encoded into one branch of a feature vector by a ResNet encoder, while the other branch uses the vehicle's speed, position, and other sensor information encoded by an MLP encoder to form a feature vector. These two are then merged into a new environmental representation feature vector. This vector outputs vehicle control commands and motion trajectories through a motion branch and a trajectory branch, respectively. Both branches are modeled using the aforementioned neuromorphic neural network architecture and have their own loss functions.

[0059] In some embodiments, loss functions for the motor branch and trajectory branch are defined respectively, and a brain-like neural circuit network is used to model the motor branch and trajectory branch respectively. The loss function for the motor branch is defined as follows:

[0060]

[0061] Where Beta(a) represents the Beta distribution parameter represented by the corresponding predicted distribution; This represents an expert distribution from the dataset; KL divergence is used to measure the similarity between predictive controls. This represents the supervisory signal from expert data at the current moment. and prediction signal The L2 measure of feature loss between, λ F This indicates an adjustable loss weight.

[0062] In some embodiments, the loss function for trajectory branching is defined as:

[0063]

[0064] Among them, wp t and These represent the values ​​of the predicted trajectory point and the actual trajectory point at time t, respectively. This represents the supervisory signal from expert data at the current moment. and prediction signal The L2 measure of feature loss between, λ F This indicates an adjustable loss weight.

[0065] The overall model loss function is the weighted sum of the loss functions of the two branches (motion branch and trajectory branch), as shown in equation (4) below. When the model loss function converges to a certain level, autonomous driving control of the vehicle can be achieved.

[0066]

[0067] in, These are the loss functions for the trajectory branch and the motion branch, respectively.

[0068] Based on the above technical solutions, this application provides an end-to-end autonomous driving method and system based on neuromorphic neural circuit trajectory guidance. The method includes: first, extracting effective traffic information from the environment; then, simulating a nematode neural network to establish a neuromorphic neural circuit network, using the neuromorphic neural circuit network as a motion branch; and cyclically constructing the instruction units of the motion branch to obtain a trajectory branch; next, simultaneously inputting the effective traffic information from the environment into the motion branch and the trajectory branch, and outputting the vehicle's control commands and predicted trajectory points at different future times; finally, based on the vehicle's control commands and the predicted trajectory points at different future times, obtaining the vehicle's final control strategy.

[0069] This application proposes an end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance. This method utilizes the vehicle's forward-looking camera and related vehicle measurement information as input, applying a ResNet encoder and an MLP encoder to extract effective traffic information from the environment. The temporal image input from the forward-looking camera is processed by the ResNet encoder to form one branch of the feature vector; the other branch is the feature vector formed by processing vehicle speed, position, and other sensor information through the MLP encoder. The two branches are fused to form the final perception information, which is then fed into a trajectory branch and a motion branch. The motion branch uses a neuromorphic neural circuit architecture built to simulate a nematode neural network, outputting vehicle control commands; the trajectory branch iteratively constructs the command units of the motion branch to output predicted trajectory points at different future times. The trajectory is fused with the control commands generated by dual PID controllers and the control commands from the control branch to form the vehicle's final control strategy.

[0070] Those skilled in the art will understand that the above-described embodiments are specific examples of implementing this application, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of this application. Any person skilled in the art can make their own modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.

Claims

1. An end-to-end autonomous driving method based on brain-like neural circuit trajectory guidance, characterized in that, Includes the following steps: Extract effective traffic information from the environment; A brain-like neural circuit network is established by simulating a nematode neural network, and the brain-like neural circuit network is used as a motion branch; and the instruction units of the motion branch are cyclically constructed to obtain the trajectory branch. The effective traffic information of the environment is simultaneously input into the motion branch and the trajectory branch, and the vehicle control command and the predicted trajectory points at different future times are output. Based on the vehicle's control commands and predicted trajectory points at different future times, the final control strategy of the vehicle is obtained. The extraction of effective traffic information from the environment includes: The first branch feature vector is obtained by taking the temporal image from the front-view camera as input and passing it through a ResNet encoder; the second branch feature vector is obtained by taking the sensor measurement information of the vehicle as input and passing it through an MLP encoder. The first branch feature vector and the second branch feature vector are fused to obtain effective traffic information in the environment. The brain-like neural circuit network comprises four layers of neurons; wherein... The four layers of neurons are as follows: One sensory neuron, One internal neuron, One instruction neuron, One motor neuron; Between any two consecutive layers, for any source neuron, insertion One synapse; among them satisfy: The synaptic polarity follows a Bernoulli distribution, where, Each target neuron is randomly selected using a binomial distribution; Between any two consecutive layers, any target neuron without a synapse insert One synapse, satisfy: ;in, To synapse the target neuron The number of synapses and their polarity follow a Bernoulli distribution. Each source neuron is randomly selected using a binomial distribution; Circular connections exist between instruction neurons; for any instruction neuron, insertion... One synapse, of which satisfy: Synaptic polarity follows a Bernoulli distribution. Each source neuron is randomly selected using a binomial distribution; The loss functions for the motion branch and the trajectory branch are defined respectively, and the brain-like neural circuit network is used to model the motion branch and the trajectory branch respectively.

2. The end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance according to claim 1, characterized in that, The sensor measurement information of the vehicle includes vehicle speed information and vehicle position information.

3. The end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance according to claim 1, characterized in that, Based on the characteristics of current transmission between neuronal synapses, each neuron is modeled as follows: （1） in, Represents neuronal synaptic currents. A represents the external input to the synapse, and A is the deviation matrix. Represents a neural network. This represents the time constant.

4. The end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance according to claim 1, characterized in that, The loss function of the motion branch is defined as: （2） in, This indicates that the corresponding predicted distribution represents Distribution parameters; This represents an expert distribution from the dataset; Divergence is used to measure the similarity between predictive controls; This represents the supervisory signal from expert data at the current moment. and prediction signal Feature loss between measure, This indicates an adjustable loss weight.

5. The end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance according to claim 1, characterized in that, The loss function for the trajectory branch is defined as: （3） in, and They represent The values ​​of the predicted trajectory points and the values ​​of the actual trajectory points are constantly being calculated. This represents the supervisory signal from expert data at the current moment. and prediction signal Feature loss between measure, This indicates an adjustable loss weight.

6. An end-to-end autonomous driving system based on neuromorphic neural circuit trajectory guidance, the system being used to implement the end-to-end autonomous driving method based on neuromorphic neural circuit trajectory guidance as described in any one of claims 1 to 5, characterized in that, It includes a feature vector extraction module, a model building module, and a final action output module connected in sequence; The feature vector extraction module is used to extract effective traffic information from the environment. The model building module includes a motion branch unit and a trajectory branch unit; the model building module is used to simulate the nematode neural network, establish a brain-like neural circuit network, and construct the motion branch unit; and to cyclically construct the instruction unit of the motion branch unit to obtain the trajectory branch unit; The motion branch unit is used to output vehicle control commands based on the effective traffic information of the environment; The trajectory branch unit is used to obtain predicted trajectory points at different future times based on the effective traffic information of the environment. The final action output module is used to obtain the final control strategy of the vehicle based on the vehicle's control commands and predicted trajectory points at different future times.

7. The end-to-end autonomous driving system based on neuromorphic neural circuit trajectory guidance according to claim 6, characterized in that, The feature vector extraction module includes a first branch unit, a second branch unit, and a fusion unit; The first branch unit is used as input to the temporal image of the foreseeing camera and obtains the first branch feature vector through the ResNet encoder; The second branch unit is used to obtain the second branch feature vector by taking the vehicle's sensor measurement information as input and passing it through an MLP encoder; The fusion unit is used to fuse the first branch feature vector and the second branch feature vector to obtain effective traffic information in the environment.

Images