Velocity control method and apparatus for autonomous vehicle, and device and medium

Abstract

The present application is applicable to the technical field of autonomous driving. Provided are a velocity control method and apparatus for an autonomous vehicle, and a device and a medium. The method comprises: acquiring the current driving information data of a target vehicle, the current driving information data of surrounding vehicles, and trajectory data of a human-driven vehicle; constructing a simulated driving environment of the target vehicle; on the basis of the driving information data of the target vehicle and the driving information data of the surrounding vehicles, using an SAC algorithm to generate a synthesized trajectory, and running the synthesized trajectory in the simulated driving environment; on the basis of the running result and the trajectory data of the human-driven vehicle, calculating an assistance reward function value, a discriminator calculating a reward function value, and then optimizing the SAC algorithm; using the optimized SAC algorithm to generate a final synthesized trajectory; and on the basis of the final synthesized trajectory, controlling an acceleration and a velocity direction of the target vehicle. The method in the present application can improve the rationality of control over autonomous vehicles.

Description

A speed control method, device, equipment and medium for an autonomous driving vehicle Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a speed control method, device, equipment and medium for an autonomous driving vehicle. Background Art

[0002] Autonomous driving, a current research hotspot, is expected to improve road safety, mobility, and reduce energy consumption, becoming a reality in the future. The main technical approaches for autonomous vehicles include traffic environment perception, driving behavior decision-making and planning, and vehicle response and control. The driving behavior decision-making and planning module is a key component in the research and application of autonomous vehicle micro-behaviors.

[0003] In situations where autonomous vehicles coexist with human-driven vehicles, their ability to generate anthropomorphic motion decisions and planning can help them better adapt to diverse traffic scenarios and complex road conditions. This can also make their behavior more predictable, making it easier for other road users to understand and interact with them, thereby helping to improve road safety and traffic efficiency. However, traditional autonomous driving motion planning models have certain drawbacks in practical applications, such as low accuracy and poor generalization, which results in low rationality in autonomous vehicle control.

[0004] Summary of the Invention

[0005] An embodiment of the present application provides a speed control method for an autonomous driving vehicle, which can solve the problem of low rationality of autonomous driving vehicle control.

[0006] In a first aspect, an embodiment of the present application provides a speed control method for an autonomous driving vehicle, the speed control method comprising:

[0007] Obtaining current driving information data of a target vehicle, current driving information data of multiple surrounding vehicles, and trajectory data of human-driven vehicles; the target vehicle is an autonomous vehicle, the surrounding vehicles are vehicles within the target vehicle's sensor sensing range, and the human-driven vehicle trajectory data includes location information of at least one human-driven vehicle when it arrives at the target vehicle's current destination;

[0008] Constructing a simulated driving environment for the target vehicle based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles;

[0009] Based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, the SAC algorithm is used to generate the current synthetic trajectory of the target vehicle, and the synthetic trajectory is run in the simulated driving environment;

[0010] Based on the running results of the synthetic trajectory and the trajectory data of the human-driven vehicle, the auxiliary reward function value of the SAC algorithm is calculated, and the reward function value is calculated using the discriminator network. The auxiliary reward function value and the reward function value are used to optimize the SAC algorithm;

[0011] The optimized SAC algorithm is used to generate the final synthetic trajectory of the target vehicle;

[0012] The acceleration and velocity direction of the target vehicle are controlled according to the final synthesized trajectory.

[0013] Optionally, the auxiliary reward function value of the SAC algorithm is calculated based on the running results of the synthetic trajectory and the human-driven vehicle trajectory data, including:

[0014] By formula:

[0015] Calculate the auxiliary reward function value;

[0016] Among them, R represents the auxiliary reward function value, and FDE represents the final displacement error:

[0017] in, represents the horizontal coordinate of the target vehicle at the Tth moment after running the synthetic trajectory, T represents the moment when the target vehicle reaches the destination of the synthetic trajectory, x T Indicates the horizontal coordinate when the human-driven vehicle in the human-driven vehicle trajectory data reaches the destination of the synthetic trajectory, Indicates the ordinate of the target vehicle at time T after running the synthetic trajectory, y Y Indicates the vertical coordinate when the human-driven vehicle in the human-driven vehicle trajectory data reaches the destination of the synthetic trajectory.

[0018] Optionally, the auxiliary reward function value and the reward function value are used to optimize the SAC algorithm, including:

[0019] Add the auxiliary reward function value to the reward function value to obtain the comprehensive reward function value;

[0020] The parameters of the SAC algorithm are updated and the current driving information data of the target vehicle is returned. The SAC algorithm is used to generate the current synthetic trajectory of the target vehicle, and the synthetic trajectory is run in a simulated driving environment. At the same time, the discriminator network is updated based on the synthetic trajectory and the human driving trajectory. The SAC algorithm and the discriminator network are updated alternately until both networks converge.

[0021] Optionally, update the parameters of the SAC algorithm, including:

[0022] Calculate the soft state value gradient based on the state and action of the target vehicle at each moment, and update the parameters of the soft state value network of the SAC algorithm according to the soft state value gradient;

[0023] Update the parameters of the target value function network of the SAC algorithm according to the target state update formula;

[0024] The Q function gradient is calculated based on the state and action of the target vehicle at each moment, and the parameters of each Q function network of the SAC algorithm are updated according to the Q function gradient.

[0025] Optionally, a soft state value gradient is calculated based on the state and action of the target vehicle at each moment, including:

[0026] By formula:

[0027] Calculating soft state value gradients

[0028] in, represents the soft state value gradient operation, represents the state set, V ψ () represents the soft state value function network, represents the expectation at the tth moment, Q θ () represents the Q function network, π θ () represents the policy network, s t represents the state of the target vehicle at the tth moment, a t It represents the action of the target vehicle at the tth moment when running the synthetic trajectory, t = 1, 2, ..., T, and the Tth moment represents the moment when the target vehicle arrives at the destination.

[0029] Optionally, the target state update formula is:

[0030] in, represents the updated parameters in the target value function network, ψ represents the parameters in the target value function network, and τ represents the soft update parameters.

[0031] Optionally, the Q function gradient is calculated based on the state and action of the target vehicle at each moment, including:

[0032] By formula:

[0033] Calculate the Q function gradient

[0034] in, represents the Q function gradient operation, Represents a state set, represents any Q function network, r() represents the reward operation, γ represents the discount coefficient, represents the target value function network, s t+1 represents the state of the target vehicle at the t+1th moment, s t represents the state of the target vehicle at the tth moment, a t represents the action of the target vehicle at time t when running the synthetic trajectory, t = 1, 2, ..., T, where T represents the time when the target vehicle arrives at the destination;

[0035] Calculate the policy gradient based on the state of the target vehicle at each moment, including:

[0036] By formula:

[0037] Calculating policy gradients

[0038] in, represents the policy gradient operation, represents Gaussian distribution, ∈ t represents noise, π φ () represents the policy network, f φ () represents the reparameterized function, Q θ () represents the Q function network;

[0039] The discriminator network is updated based on the synthetic and human driving trajectories, including:

[0040] By formula:

[0041] Calculate the discriminator gradient

[0042] Among them, D() represents the discriminator network, represents the discriminator gradient operation, represents the expected operation, λ0 and λ represent hyperparameters, η(π) represents the state-based reward, H(π) represents the discounted causal entropy, s represents the state set of the synthetic trajectory and the human driving trajectory, and a represents the action set of the synthetic trajectory and the human driving trajectory.

[0043] In a second aspect, an embodiment of the present application provides a speed control device for an autonomous driving vehicle, comprising:

[0044] an acquisition module for acquiring current driving information data of a target vehicle, current driving information data of multiple surrounding vehicles, and trajectory data of human-driven vehicles; the target vehicle is an autonomous vehicle, the surrounding vehicles are vehicles within the target vehicle's sensor sensing range, and the human-driven vehicle trajectory data includes the location information of at least one human-driven vehicle when it arrives at the target vehicle's current destination;

[0045] A construction module, which constructs a simulated driving environment of the target vehicle based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles;

[0046] An operation module generates a synthetic trajectory of the target vehicle using a SAC algorithm based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, and runs the synthetic trajectory in a simulated driving environment;

[0047] The optimization module calculates the auxiliary reward function value of the SAC algorithm based on the running results of the synthetic trajectory and the human driving vehicle trajectory data, and uses the discriminator network to calculate the reward function value. The auxiliary reward function value and the reward function value are used to optimize the SAC algorithm;

[0048] The generation module uses the optimized SAC algorithm to generate the final synthetic trajectory of the target vehicle;

[0049] The control module controls the acceleration and velocity direction of the target vehicle according to the final synthetic trajectory.

[0050] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned speed control method for the autonomous driving vehicle when executing the above-mentioned computer program.

[0051] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned speed control method of the autonomous driving vehicle.

[0052] The above solution of the present application has the following beneficial effects:

[0053] In an embodiment of the present application, by obtaining the current driving information data of the target vehicle, the current driving information data of multiple surrounding vehicles, and the trajectory data of a human-driven vehicle, a simulated driving environment of the target vehicle is constructed based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, and then based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, a SAC algorithm is used to generate the current synthetic trajectory of the target vehicle, and the synthetic trajectory is run in the simulated driving environment, and then based on the running result of the synthetic trajectory and the trajectory data of the human-driven vehicle, an auxiliary reward function value of the SAC algorithm is calculated, and a reward function value is calculated using a discriminator network, and the SAC algorithm is optimized using the auxiliary reward function value and the reward function value, and then the optimized SAC algorithm is used to generate the current final synthetic trajectory of the target vehicle, and finally the acceleration and velocity direction of the target vehicle are controlled according to the final synthetic trajectory. Among them, optimizing the SAC algorithm according to the trajectory data of the human-driven vehicle can improve the quality of the SAC algorithm, thereby improving the accuracy of the synthetic trajectory while anthropomorphizing the synthetic trajectory, and the target vehicle controls its own acceleration and velocity direction according to the anthropomorphic final synthetic trajectory, which can improve the rationality of the control of the autonomous driving vehicle.

[0054] In addition, anthropomorphizing the synthetic trajectory can enable autonomous vehicles to adapt to various traffic environments and make the control of autonomous vehicles predictable according to human logic, making it easier for other road participants to understand and interact with the autonomous vehicles, thereby improving traffic safety and traffic efficiency.

[0055] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0057] FIG1 is a schematic flow chart of a method for controlling the speed of an autonomous driving vehicle according to an embodiment of the present application;

[0058] FIG2 is a schematic diagram of the structure of a speed control device for an autonomous driving vehicle according to an embodiment of the present application;

[0059] FIG3 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0061] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0062] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0063] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0064] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0065] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0066] To address the problem of low rationality of existing autonomous vehicle control, an embodiment of the present application provides a speed control method for an autonomous vehicle. The speed control method obtains current driving information data of a target vehicle, current driving information data of multiple surrounding vehicles, and trajectory data of a human-driven vehicle, and then constructs a simulated driving environment for the target vehicle based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles. Then, based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, a SAC algorithm is used to generate the current synthetic trajectory of the target vehicle, and the synthetic trajectory is run in the simulated driving environment. Then, based on the running results of the synthetic trajectory and the trajectory data of the human-driven vehicle, an auxiliary reward function value of the SAC algorithm is calculated, and a discriminator network is used to calculate the reward function value. The SAC algorithm is optimized using the auxiliary reward function value and the reward function value, and the optimized SAC algorithm is used to generate the current final synthetic trajectory of the target vehicle. Finally, the acceleration and speed direction of the target vehicle are controlled according to the final synthetic trajectory. Among them, optimizing the SAC algorithm based on human-driven vehicle trajectory data can improve the quality of the SAC algorithm, thereby improving the accuracy of the synthesized trajectory while making the synthesized trajectory anthropomorphic. The target vehicle controls its own acceleration and speed direction according to the anthropomorphic final synthesized trajectory, which can improve the rationality of the control of the autonomous driving vehicle.

[0067] In addition, anthropomorphizing the synthetic trajectory can enable autonomous vehicles to adapt to various traffic environments and make the control of autonomous vehicles predictable according to human logic, making it easier for other road participants to understand and interact with the autonomous vehicles, thereby improving traffic safety and traffic efficiency.

[0068] Next, an exemplary description is given of the speed control method for an autonomous driving vehicle provided in this application.

[0069] As shown in FIG1 , the speed control method of the autonomous driving vehicle provided in this application includes the following steps:

[0070] Step 11: Obtain the current driving information data of the target vehicle, the current driving information data of multiple surrounding vehicles, and the trajectory data of human-driven vehicles.

[0071] The target vehicle is an autonomous vehicle, and the surrounding vehicles are vehicles within the sensor sensing range of the target vehicle. The human-driven vehicle trajectory data includes the location information of at least one human-driven vehicle when it arrives at the current destination of the target vehicle.

[0072] It should be noted that the above-mentioned driving information data includes the speed information and position information of the target vehicle or surrounding vehicles, and the trajectory data of at least one human-driven vehicle in the above-mentioned human-driven vehicle trajectory data. According to the setting of the current destination of the target vehicle, the trajectory data of the human-driven vehicle passing the destination is selected from the human-driven vehicle trajectory data, and then the position of the human-driven vehicle when it arrives at the current destination of the target vehicle is obtained.

[0073] For example, trajectory data collection devices such as drones and radars can be used to collect and extract trajectory data of human-driven vehicles on the road, and data collection devices such as sensors of the target vehicle can be used to obtain the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles.

[0074] Step 12: constructing a simulated driving environment for the target vehicle based on the current driving information data of the target vehicle and the current driving information data of a plurality of surrounding vehicles.

[0075] For example, a simulated driving environment for a target vehicle can be constructed using computer software for simulated driving, such as SUMMIT (Simulator for Urban Driving in Massive Mixed Traffic) or Virtual Test Drive (VTD).

[0076] Step 13: Based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, a SAC algorithm is used to generate a current synthetic trajectory of the target vehicle, and the synthetic trajectory is run in a simulated driving environment.

[0077] In some embodiments of the present application, the steps of generating the current synthetic trajectory of the target vehicle using a soft actor-critic (SAC) algorithm based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, and running the synthetic trajectory in a simulated driving environment are specifically as follows:

[0078] First, the current driving information data of the target vehicle and multiple surrounding vehicles are calculated to obtain the longitudinal and lateral distances between the target vehicle and each surrounding vehicle, the relative speed between the target vehicle and each surrounding vehicle, the displacement of the target vehicle at the current moment relative to the previous moment, and the speed of the target vehicle. The four types of data obtained above are integrated into a state matrix. The state matrix is ​​then input into the strategy network of the SAC algorithm, and the destination is set. The strategy network outputs an action matrix including the acceleration and angular data of the target vehicle at T moments. Then, based on the action matrix, a synthetic trajectory with the end point being the above destination is generated. Finally, the synthetic trajectory is run in a simulated driving environment.

[0079] For example, computer software such as Python can be used to run the SAC algorithm and generate a synthetic trajectory.

[0080] It should be noted that the destination of the synthetic trajectory is the same as the current destination of the target vehicle.

[0081] It is worth mentioning that the SAC algorithm can analyze the state of the target vehicle at each moment, generate corresponding actions, and then generate a synthetic trajectory, providing a basis for controlling the acceleration and direction angle of the autonomous driving vehicle.

[0082] In step 14, based on the running results of the synthetic trajectory and the human-driven vehicle trajectory data, the auxiliary reward function value of the SAC algorithm is calculated, and the reward function value is calculated using the discriminator network. The auxiliary reward function value and the reward function value are used to optimize the SAC algorithm.

[0083] In some embodiments of the present application, the steps of calculating the auxiliary reward function value of the SAC algorithm based on the running results of the synthetic trajectory and the human-driven vehicle trajectory data, calculating the reward function value using the discriminator network, and optimizing the SAC algorithm using the auxiliary reward function value and the reward function value specifically include:

[0084] The first step is through the formula:

[0085] Calculate the auxiliary reward function value R;

[0086] Where PDE represents the final displacement error:

[0087] in, represents the horizontal coordinate of the target vehicle at the Tth moment after running the synthetic trajectory, T represents the moment when the target vehicle reaches the destination of the synthetic trajectory, x T Indicates the horizontal coordinate when the human-driven vehicle in the human-driven vehicle trajectory data reaches the destination of the synthetic trajectory, Indicates the ordinate of the target vehicle at time T after running the synthetic trajectory, y T Indicates the vertical coordinate when the human-driven vehicle in the human-driven vehicle trajectory data reaches the destination of the synthetic trajectory.

[0088] It should be noted that the horizontal and vertical coordinates of the above-mentioned target vehicle and human-driven vehicle are coordinates in a plane rectangular coordinate system, and the plane rectangular coordinate system takes the target vehicle as the origin.

[0089] In the second step, the discriminator network is used to calculate the reward function value.

[0090] Exemplarily, the above-mentioned discriminator network can be a discriminator network in a network such as a generative adversarial network.

[0091] The third step is to add the auxiliary reward function value to the reward function value to obtain the comprehensive reward function value.

[0092] In the fourth step, the parameters of the SAC algorithm are updated, and the current driving information data of the target vehicle is returned. The SAC algorithm is used to generate the current synthetic trajectory of the target vehicle, and the synthetic trajectory is run in a simulated driving environment. At the same time, the discriminator network is updated based on the synthetic trajectory and the human driving trajectory. The SAC algorithm and the discriminator network are updated alternately until each network converges.

[0093] It should be noted that the above steps for updating the parameters of the SAC algorithm are specifically as follows:

[0094] First, the soft state value gradient is calculated based on the state and action of the target vehicle at each moment, and the parameters of the soft state value network of the SAC algorithm are updated according to the soft state value gradient.

[0095] Specifically, through the formula:

[0096] Calculating soft state value gradients

[0097] in, represents the soft state value gradient operation, represents the state set, V ψ () represents the soft state value function network, represents the expectation at the tth moment, Q θ () represents the Q function network, π θ () represents the policy network, s t represents the state of the target vehicle at the tth moment, a t It represents the action of the target vehicle at the tth moment when running the synthetic trajectory, t = 1, 2, ..., T, and the Tth moment represents the moment when the target vehicle arrives at the destination.

[0098] It should be noted that the above-mentioned action is the acceleration and speed direction of the target vehicle, the above-mentioned state is physical information such as the speed of the target vehicle and the distance between the target vehicle and other vehicles, and the above-mentioned process of updating the parameters of the soft state value network of the SAC algorithm based on the soft state value gradient is: according to the direction and size of the soft state value gradient, the parameters in the soft state value network are adjusted by corresponding amplitudes.

[0099] Then, the parameters of the target value function network of the SAC algorithm are updated according to the target state update formula.

[0100] The above target state update formula is:

[0101] in, represents the updated parameters in the target value function network, ψ represents the parameters in the target value function network, and τ represents the soft update parameters.

[0102] Then, the Q function gradient is calculated based on the state and action of the target vehicle at each moment, and the parameters of each Q function network of the SAC algorithm are updated according to the Q function gradient.

[0103] Specifically, through the formula:

[0104] Calculate the Q function gradient

[0105] in, represents the Q function gradient operation, Represents a state set, represents any Q function network, r() represents the reward operation, γ represents the discount coefficient, represents the target value function network, s t+1 represents the state of the target vehicle at the t+1th moment, s t represents the state of the target vehicle at the tth moment, a t represents the action of the target vehicle at time t when running the synthetic trajectory, t = 1, 2, ..., T, and T represents the time when the target vehicle arrives at the destination.

[0106] It should be noted that the above process of updating the parameters of each Q function network of the SAC algorithm according to the Q function gradient is: adjusting the parameters in each Q function network by a corresponding amplitude according to the direction and magnitude of the Q function gradient.

[0107] Finally, the policy gradient is calculated based on the state of the target vehicle at each moment, and the parameters of the policy network of the SAC algorithm are updated according to the policy gradient.

[0108] Specifically, through the formula:

[0109] Calculating policy gradients

[0110] in, represents the policy gradient operation, represents Gaussian distribution, ∈ t represents noise, π φ () represents the policy network, f φ () represents the reparameterized function, Q θ () represents the Q function network.

[0111] It should be noted that the above process of updating the parameters of the policy network of the SAC algorithm based on the policy gradient is: adjusting the parameters in the policy network by corresponding amplitudes according to the direction and magnitude of the policy gradient.

[0112] The steps for updating the discriminator network based on the synthetic trajectory and the human driving trajectory are as follows:

[0113] By formula:

[0114] Calculate the discriminator gradient

[0115] Among them, D() represents the discriminator network, represents the discriminator gradient operation, represents the expected operation, λ0 and λ represent hyperparameters, η(π) represents the state-based reward, H(π) represents the discounted causal entropy, s represents the state set of the synthetic trajectory and the human driving trajectory, and a represents the action set of the synthetic trajectory and the human driving trajectory.

[0116] For example, when the loss functions of each network tend to be stable and convergent, and the value of the comprehensive reward function also eventually tends to be stable and convergent, the SAC algorithm reaches the expected quality and is an optimized SAC algorithm.

[0117] It is worth mentioning that optimizing the SAC algorithm based on human-driven vehicle trajectory data can improve the quality of the SAC algorithm. At the same time, learning the trajectory of human-driven vehicles can make the generated synthetic trajectory humanized.

[0118] Step 15: Generate the final synthetic trajectory of the target vehicle using the optimized SAC algorithm.

[0119] Exemplarily, a state matrix is ​​obtained based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, and then the state matrix is ​​input into the optimized SAC algorithm, and a destination is set. The optimized SAC algorithm generates a final synthetic trajectory with the end point as the destination based on the above state matrix.

[0120] It is worth mentioning that the performance of the optimized SAC algorithm meets expectations. The final synthetic trajectory generated by the high-performance SAC algorithm has high accuracy and good anthropomorphism.

[0121] Step 16: Control the acceleration and velocity direction of the target vehicle according to the final synthesized trajectory.

[0122] It should be noted that the final synthetic trajectory gives the acceleration and velocity direction of the target vehicle at each moment.

[0123] For example, the acceleration and speed direction of the target vehicle may be controlled by using a vehicle speed control system, equipment, or the like, such as an Intelligent Speed ​​Adaptation (ISA) control system.

[0124] It is worth mentioning that the target vehicle controls its own acceleration and velocity direction according to the anthropomorphic final synthetic trajectory, which can improve the rationality of the control of the autonomous driving vehicle.

[0125] In addition, anthropomorphizing the synthetic trajectory can enable autonomous vehicles to adapt to various traffic environments and make the control of autonomous vehicles predictable according to human logic, making it easier for other road participants to understand and interact with the autonomous vehicles, thereby improving traffic safety and traffic efficiency.

[0126] The speed control device for the autonomous driving vehicle provided in this application is exemplified below.

[0127] As shown in FIG2 , an embodiment of the present application provides a speed control device for an autonomous driving vehicle. The speed control device 200 for an autonomous driving vehicle includes:

[0128] Acquisition module 201 acquires current driving information data of a target vehicle, current driving information data of multiple surrounding vehicles, and trajectory data of human-driven vehicles; the target vehicle is an autonomous vehicle, the surrounding vehicles are vehicles within the sensor sensing range of the target vehicle, and the human-driven vehicle trajectory data includes location information of at least one human-driven vehicle when it arrives at the target vehicle's current destination;

[0129] A construction module 202 constructs a simulated driving environment of the target vehicle based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles;

[0130] An operation module 203 generates a current synthetic trajectory of the target vehicle using a SAC algorithm based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, and runs the synthetic trajectory in a simulated driving environment;

[0131] an optimization module 204 that calculates an auxiliary reward function value of the SAC algorithm based on the running results of the synthetic trajectory and the human-driven vehicle trajectory data, calculates the reward function value using the discriminator network, and optimizes the SAC algorithm using the auxiliary reward function value and the reward function value;

[0132] The generation module 205 generates the final synthetic trajectory of the target vehicle using the optimized SAC algorithm;

[0133] The control module 206 controls the acceleration and velocity direction of the target vehicle according to the final synthesized trajectory.

[0134] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0135] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0136] As shown in Figure 3, an embodiment of the present application provides a terminal device. The terminal device D10 of this embodiment includes: at least one processor D100 (only one processor is shown in Figure 3), a memory D101, and a computer program D102 stored in the memory D101 and capable of running on the at least one processor D100. When the processor D100 executes the computer program D102, the steps in any of the above-mentioned method embodiments are implemented.

[0137] Specifically, when the processor D100 executes the computer program D102, it obtains the current driving information data of the target vehicle, the current driving information data of multiple surrounding vehicles, and the trajectory data of human-driven vehicles, and then constructs a simulated driving environment for the target vehicle based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles. Then, based on the current driving information data of the target vehicle and the current driving information data of multiple surrounding vehicles, it uses the SAC algorithm to generate the current synthetic trajectory of the target vehicle, and runs the synthetic trajectory in the simulated driving environment. Then, based on the running result of the synthetic trajectory and the trajectory data of human-driven vehicles, it calculates the auxiliary reward function value of the SAC algorithm, and uses the discriminator network to calculate the reward function value. The SAC algorithm is optimized using the auxiliary reward function value and the reward function value, and then the optimized SAC algorithm is used to generate the current final synthetic trajectory of the target vehicle. Finally, the acceleration and speed direction of the target vehicle are controlled according to the final synthetic trajectory. Among them, optimizing the SAC algorithm based on human-driven vehicle trajectory data can improve the quality of the SAC algorithm, thereby improving the accuracy of the synthesized trajectory while making the synthesized trajectory anthropomorphic. The target vehicle controls its own acceleration and speed direction according to the anthropomorphic final synthesized trajectory, which can improve the rationality of the control of the autonomous driving vehicle.

[0138] In addition, anthropomorphizing the synthetic trajectory can enable autonomous vehicles to adapt to various traffic environments and make the control of autonomous vehicles predictable according to human logic, making it easier for other road participants to understand and interact with the autonomous vehicles, thereby improving traffic safety and traffic efficiency.

[0139] The processor D100 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0140] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, Smart Media Card), a secure digital (SD, Secure Digital) card, a flash card, etc. equipped on the terminal device D10. Furthermore, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0141] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0142] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the speed control method device / terminal device of the autonomous vehicle, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, USB flash drive, mobile hard disk, magnetic disk or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0144] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0145] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0146] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A speed control method for an autonomous driving vehicle, characterized in that: include: Obtaining current driving information data of the target vehicle, current driving information data of multiple surrounding vehicles, and trajectory data of human-driven vehicles; The target vehicle is an autonomous vehicle, the surrounding vehicles are vehicles within the sensor sensing range of the target vehicle, and the human-driven vehicle trajectory data includes location information of at least one human-driven vehicle when it arrives at the current destination of the target vehicle; constructing a simulated driving environment for the target vehicle based on the current driving information data of the target vehicle and the current driving information data of the plurality of surrounding vehicles; generating a current synthetic trajectory of the target vehicle using a SAC algorithm based on the current driving information data of the target vehicle and the current driving information data of the plurality of surrounding vehicles, and running the synthetic trajectory in the simulated driving environment; Calculating an auxiliary reward function value of the SAC algorithm based on the running result of the synthetic trajectory and the human-driven vehicle trajectory data, calculating a reward function value using a discriminator network, and optimizing the SAC algorithm using the auxiliary reward function value and the reward function value; Generate the final synthetic trajectory of the target vehicle using the optimized SAC algorithm; The acceleration and velocity direction of the target vehicle are controlled according to the final composite trajectory.

2. The speed control method according to claim 1, characterized in that: The calculating of the auxiliary reward function value of the SAC algorithm according to the running result of the synthetic trajectory and the human-driven vehicle trajectory data includes: By formula: Calculating the auxiliary reward function value R; Where FDE represents the final displacement error: in, represents the horizontal coordinate of the target vehicle at the Tth moment after running the synthetic trajectory, T represents the moment when the target vehicle reaches the destination of the synthetic trajectory, x T represents the horizontal coordinate when the human-driven vehicle in the human-driven vehicle trajectory data reaches the destination of the synthetic trajectory, express The ordinate of the target vehicle at time T after running the synthetic trajectory, y T The vertical coordinate represents the time when the human-driven vehicle in the human-driven vehicle trajectory data reaches the destination of the synthetic trajectory.

3. The speed control method according to claim 2, characterized in that: The optimizing the SAC algorithm by using the auxiliary reward function value and the reward function value includes: Adding the auxiliary reward function value to the reward function value to obtain a comprehensive reward function value; The SAC algorithm is parameter updated, and the current driving information data of the target vehicle is returned, the current synthetic trajectory of the target vehicle is generated by the SAC algorithm, and the synthetic trajectory is run in the simulated driving environment. At the same time, a discriminator network is updated based on the synthetic trajectory and the human driving trajectory, and the SAC algorithm and the discriminator network are alternately updated until each network converges.

4. The speed control method according to claim 3, characterized in that: The updating of parameters of the SAC algorithm includes: Calculating a soft state value gradient based on the state and action of the target vehicle at each moment, and updating parameters of a soft state value network of the SAC algorithm according to the soft state value gradient; Update the parameters of the target value function network of the SAC algorithm according to the target state update formula; Calculating a Q function gradient based on the state and action of the target vehicle at each moment, and updating parameters of each Q function network of the SAC algorithm according to the Q function gradient; A policy gradient is calculated based on the state of the target vehicle at each moment, and parameters of the policy network of the SAC algorithm are updated according to the policy gradient.

5. The speed control method according to claim 4, characterized in that: The calculating of the soft state value gradient based on the state and action of the target vehicle at each moment includes: By formula: Calculate the soft state value gradient in, represents the soft state value gradient operation, represents the state set, V ψ () represents the soft state value function network, represents the expectation at the tth moment, Q θ () represents the Q function network, π θ () represents the policy network, s t represents the state of the target vehicle at the tth moment, a t represents the action of the target vehicle at the tth moment when running the synthetic trajectory, t=1, 2, ..., T, and the Tth moment represents the moment when the target vehicle arrives at the destination.

6. The speed control method according to claim 4, characterized in that: The target state update formula is: in, represents the updated parameters in the target value function network, ψ represents the parameters in the target value function network, and τ represents the soft update parameters.

7. The speed control method according to claim 4, characterized in that: The calculating of the Q function gradient based on the state and action of the target vehicle at each moment includes: By formula: Calculate the Q function gradient in, represents the Q function gradient operation, Represents a state set, represents any Q function network, r() represents the reward operation, γ represents the discount coefficient, represents the target value function network, s t+1 represents the state of the target vehicle at the t+1th moment, s t represents the state of the target vehicle at the tth moment, a t represents the action of the target vehicle at time t when running the synthetic trajectory, t=1, 2, ..., T, where T represents the time when the target vehicle arrives at the destination; The calculating of the policy gradient based on the state of the target vehicle at each moment includes: By formula: Calculate the policy gradient in, represents the policy gradient operation, represents Gaussian distribution, ∈ t represents noise, π φ () represents the policy network, f φ () represents the reparameterized function, Q θ () represents the Q function network; The updating of the discriminator network based on the synthetic trajectory and the human driving trajectory includes: By formula: Calculate the discriminator gradient Wherein, D() represents the discriminator network, represents the discriminator gradient operation, represents the expected operation, λ0 and λ represent hyperparameters, η(π) represents the state-based reward, H(π) represents the discounted causal entropy, s represents the state set of the synthetic trajectory and the human driving trajectory, and a represents the action set of the synthetic trajectory and the human driving trajectory.

8. A method for controlling the speed of an autonomous vehicle, characterized in that: include: An acquisition module acquires the current driving information data of the target vehicle, the current driving information data of multiple surrounding vehicles, and the trajectory data of human-driven vehicles; The target vehicle is an autonomous vehicle, the surrounding vehicles are vehicles within the sensor sensing range of the target vehicle, and the human-driven vehicle trajectory data includes location information of at least one human-driven vehicle when it arrives at the current destination of the target vehicle; A construction module, constructing a simulated driving environment of the target vehicle based on the current driving information data of the target vehicle and the current driving information data of the plurality of surrounding vehicles; an operating module, which generates a current synthetic trajectory of the target vehicle using a SAC algorithm based on the current driving information data of the target vehicle and the current driving information data of the plurality of surrounding vehicles, and operates the synthetic trajectory in the simulated driving environment; an optimization module, calculating an auxiliary reward function value of the SAC algorithm based on the running result of the synthetic trajectory and the human-driven vehicle trajectory data, calculating a reward function value using a discriminator network, and optimizing the SAC algorithm using the auxiliary reward function value and the reward function value; A generation module generates the final synthetic trajectory of the target vehicle using the optimized SAC algorithm; A control module controls the acceleration and velocity direction of the target vehicle according to the final composite trajectory.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the speed control method for the autonomous driving vehicle according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speed control method of the autonomous driving vehicle as described in any one of claims 1 to 7 is implemented.

Images