Autonomous driving control methods, devices, electronic equipment and computer storage media

Abstract

This invention relates to an autonomous driving control method, device, electronic device, and computer storage medium, belonging to the field of autonomous driving technology. The autonomous driving control method includes acquiring scene information of the vehicle's location, including vehicle state information and road environment information of the road where the vehicle is located; analyzing the vehicle state information and road environment information using a deep reinforcement learning-based vehicle energy-saving control model to determine a vehicle control strategy. The vehicle energy-saving control model includes a reward function, which represents the balance between safe driving and energy efficiency under the vehicle control strategy, with the reward function corresponding to the vehicle control strategy having the maximum reward value; and controlling the vehicle to drive autonomously according to the vehicle control strategy. This invention can effectively reduce vehicle energy consumption and save energy.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving control method, device, electronic device, and computer storage medium. Background Technology

[0002] In recent years, the rapid development of artificial intelligence technology has had a profound impact on various industries, such as image recognition, speech recognition, and natural language processing. These technologies provide strong technical support for autonomous driving, enabling cars to better perceive their surroundings, understand and cope with complex and ever-changing road conditions. Major automakers have showcased their own schools of autonomous driving perception technologies.

[0003] In existing autonomous driving technologies, for safety and stability reasons, vehicle control is relatively rigid. In different driving environments, there may be continuous speed changes, rapid acceleration, sudden braking, sudden lane changes, etc., which will increase vehicle energy consumption and reduce passenger riding experience. In particular, for complex driving environments, vehicle energy consumption is not taken into account, resulting in energy waste.

[0004] This shows that existing autonomous driving technologies can cause technical problems such as excessive vehicle energy consumption. Summary of the Invention

[0005] In view of this, it is necessary to provide an autonomous driving control method, device, electronic device, and computer storage medium to solve the technical problem that existing autonomous driving technologies cause excessive vehicle energy consumption.

[0006] To address the above problems, the present invention provides an autonomous driving control method, comprising:

[0007] Obtain scene information of the vehicle's location, including vehicle status information and road environment information of the road where the vehicle is located;

[0008] A deep reinforcement learning-based vehicle energy-saving control model is used to analyze the vehicle state information and road environment information to determine the vehicle control strategy. The vehicle energy-saving control model includes a reward function, which represents the balance between safe driving and energy efficiency under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized.

[0009] The vehicle is controlled to operate automatically according to the vehicle control strategy.

[0010] In one possible implementation, obtaining scene information about the vehicle's location includes:

[0011] Vehicle status information is collected using onboard sensors;

[0012] The system uses image acquisition devices and vehicle-mounted radar devices to collect road environment information of the road where the vehicle is located.

[0013] In one possible implementation, the acquisition of road environment information of the road where the vehicle is located using an image acquisition device and an onboard radar device includes:

[0014] The image acquisition device is used to acquire image information of the vehicle within a first preset range;

[0015] The vehicle-mounted radar device is used to collect radar point cloud information within a second preset range of the vehicle;

[0016] The image information and the radar point cloud information are fused to obtain the environmental information of the road where the vehicle is located.

[0017] In one possible implementation, the training process of the deep reinforcement learning-based vehicle energy-saving control model includes:

[0018] The vehicle energy-saving control model is trained in the first round using sample data from the high sample pool. Sample data whose reward value of the reward function is greater than the preset reward threshold in the first round of training are added to the high sample pool, and the high sample pool is updated.

[0019] According to a preset ratio, sample data is randomly selected from the updated high sample pool and the standard sample pool to conduct a second round of training on the vehicle energy-saving control model until the vehicle energy-saving control model achieves the expected effect.

[0020] In one possible implementation, the step of waiting until the vehicle energy-saving control model achieves the desired effect includes:

[0021] Calculate the reward value of the reward function for each training result and the evaluation value of the current value network, wherein the evaluation value of the value network is used to indicate the score of the vehicle control strategy in the training result;

[0022] The target value of the target value network is calculated based on the reward value of the reward function of each training result and the current value network evaluation value.

[0023] A loss function is constructed based on the evaluation value of the current value network and the target value of the target value network, and the vehicle energy-saving control model is updated to minimize the loss function.

[0024] In one possible implementation, calculating the evaluation value of the current value network for each training result includes:

[0025] Calculate the first evaluation value of the first current value network and the second evaluation value of the second current value network for each training result;

[0026] The smaller of the first evaluation value and the second evaluation value is used as the evaluation value of the current value network of the training result.

[0027] In one possible implementation, the calculation of the reward value of the reward function includes:

[0028] The energy consumption score is calculated based on the vehicle's motor speed and motor torque under the aforementioned vehicle control strategy.

[0029] The vehicle speed score is calculated based on the relationship between the vehicle speed and the vehicle's permissible range under the aforementioned vehicle control strategy.

[0030] A safety score is determined based on the vehicle's safety under the aforementioned vehicle control strategy;

[0031] The reward value of the reward function is calculated based on the energy consumption score, the vehicle speed score, and the safety score according to preset weights.

[0032] The present invention also provides an automatic driving control device, comprising:

[0033] An environmental information acquisition module is used to acquire environmental information of the scene where the vehicle is located, including vehicle status information and environmental information of the road where the vehicle is located.

[0034] The control strategy determination module is used to analyze the vehicle state information and the road environment information using a vehicle energy-saving control model based on deep reinforcement learning to determine the vehicle control strategy. The vehicle energy-saving control model includes a reward function, which represents the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized.

[0035] The control strategy execution module is used to control the vehicle to drive automatically according to the vehicle control strategy.

[0036] The present invention also provides an electronic device, including a memory and a processor, wherein,

[0037] The memory is used to store programs;

[0038] The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the autonomous driving control method described in any of the above embodiments.

[0039] The present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps in the autonomous driving control method described in any of the above embodiments.

[0040] The beneficial effects of this invention are as follows: The autonomous driving control method provided by this invention acquires vehicle state information and road information of the road where the vehicle is located, and uses a vehicle energy-saving control model based on deep reinforcement learning to analyze the vehicle state information and road environment information to determine the vehicle control strategy. The vehicle energy-saving control model includes a reward function, which represents the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized. By using the vehicle control strategy with the maximum reward value of the reward function to control the vehicle for autonomous driving, vehicle energy consumption can be effectively reduced and energy can be saved. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating an autonomous driving control method provided in an embodiment of the present invention;

[0043] Figure 2 This is a flowchart illustrating a scene information acquisition method provided in an embodiment of the present invention;

[0044] Figure 3 A flowchart illustrating one implementation method of S202 provided in this embodiment of the invention;

[0045] Figure 4 A flowchart illustrating a vehicle energy-saving control model training method provided in an embodiment of the present invention;

[0046] Figure 5 A flowchart illustrating a vehicle energy-saving control model optimization method provided in an embodiment of the present invention;

[0047] Figure 6 A flowchart illustrating a value network calculation method provided in an embodiment of the present invention;

[0048] Figure 7 This is a control principle diagram of a vehicle energy-saving model according to an embodiment of the present invention;

[0049] Figure 8 This is a diagram illustrating the training process of a vehicle energy-saving model provided in an embodiment of the present invention.

[0050] Figure 9 This is a flowchart illustrating a method for calculating the reward value of a reward function provided in an embodiment of the present invention.

[0051] Figure 10 a is a schematic diagram of an overtaking scenario provided in an embodiment of the present invention;

[0052] Figure 10 b is an engine power consumption diagram for overtaking under an embodiment of the present invention;

[0053] Figure 11 a is a schematic diagram of a following vehicle scenario provided in an embodiment of the present invention;

[0054] Figure 11 b is an engine power consumption diagram under following vehicle conditions provided in an embodiment of the present invention;

[0055] Figure 12 This is a schematic diagram of the structure of an automatic driving control device provided in an embodiment of the present invention;

[0056] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0057] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which constitute a part of the present invention and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0058] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0059] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0060] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0061] A specific embodiment of the present invention, such as Figure 1 As shown, an autonomous driving control method is disclosed, including:

[0062] S101, Obtain scene information of the vehicle's location, including vehicle status information and road environment information of the road where the vehicle is located.

[0063] S102, a vehicle energy-saving control model based on deep reinforcement learning is used to analyze vehicle state information and road environment information to determine vehicle control strategy. The vehicle energy-saving control model includes a reward function, which is used to represent the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized.

[0064] S103 controls the vehicle's autonomous driving according to the vehicle control strategy.

[0065] In this embodiment of the invention, the autonomous driving method is provided primarily for controlling autonomous vehicles, which may be cars, trucks, vans, buses, motorcycles, or other suitable vehicles.

[0066] In this embodiment of the invention, the scene information of the vehicle's location includes vehicle status information and road environment information of the road where the vehicle is located. The vehicle status information includes, but is not limited to, the vehicle's current speed, acceleration, steering wheel angle, and motor torque. The road information includes, but is not limited to, road condition information, traffic light signal information, and information on vehicles, pedestrians, and obstacles in front of and behind the vehicle. The information on vehicles, pedestrians, and obstacles in front of and behind the vehicle includes position information and movement trend information. The specific method for obtaining the scene information of the vehicle's location will be described later in this invention.

[0067] In this embodiment of the invention, the vehicle energy-saving control model based on deep reinforcement learning is a pre-set neural network model used to determine the vehicle control strategy based on scene information of the vehicle's location. The reward function represents the balance between safe driving and energy-saving efficiency under the vehicle control strategy. Maximizing the reward value of the reward function indicates that the vehicle's energy-saving efficiency is maximized while ensuring driving safety. Optionally, based on vehicle state information and road environment information of the road where the vehicle is located, the vehicle energy-saving control model will provide an optimized vehicle control strategy to save vehicle energy while ensuring driving safety. The specific process of determining the vehicle control strategy by this vehicle energy-saving control model will be described later in this invention.

[0068] In this embodiment of the invention, the vehicle control strategy includes, but is not limited to, the control of vehicle actions, such as the control of the vehicle steering wheel angle, accelerator, brake, gear, lights, and horn.

[0069] The autonomous driving control method provided by this invention acquires vehicle state information and road information of the road where the vehicle is located, and uses a vehicle energy-saving control model based on deep reinforcement learning to analyze the vehicle state information and road environment information to determine the vehicle control strategy. The vehicle energy-saving control model includes a reward function, which represents the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized. By using the vehicle control strategy with the maximum reward value of the reward function to control the vehicle for autonomous driving, vehicle energy consumption can be effectively reduced and energy can be saved.

[0070] As one possible embodiment of the present invention, in this embodiment, such as Figure 2 As shown, the scene information of the vehicle's location is obtained, including:

[0071] S201 uses onboard sensors to collect vehicle status information;

[0072] S202 uses image acquisition devices and vehicle-mounted radar devices to collect road environment information of the road where the vehicle is located.

[0073] In this embodiment of the invention, the autonomous vehicle includes an information collection device, which comprises onboard sensors, an image acquisition device, and an onboard radar device. Optionally, the onboard sensors include vehicle speed sensors, acceleration sensors, torque sensors, etc., the image acquisition device includes cameras, video cameras, etc., and the onboard radar device includes ultrasonic radar and lidar, etc. The information collection device collects vehicle status information and road environment information of the road where the vehicle is located.

[0074] In this embodiment of the invention, the driving route of the autonomous vehicle is first determined. Vehicle positioning can be achieved through high-precision maps and global positioning systems. At the same time, the driving route is planned to be globally optimal. When the vehicle is driving along the driving route, the vehicle's current speed, acceleration, steering wheel angle, motor torque, etc. are collected through vehicle sensors. Traffic lights, road conditions, and other information of the road where the vehicle is located are obtained through V2X (vehicle to X, vehicle wireless communication technology). Image information of vehicles, pedestrians, and obstacles in front of and behind the vehicle is collected through cameras and other means. Radar point cloud information of vehicles, pedestrians, and obstacles in front of and behind the vehicle is obtained through lidar and / or ultrasonic radar.

[0075] The embodiments of the present invention collect vehicle status information and road environment information of the road where the vehicle is located through various information collection devices, which can provide richer scenario information for the vehicle energy-saving control model.

[0076] As one possible embodiment of the present invention, in this embodiment, such as Figure 3As shown, road environment information of the road where the vehicle is located is collected using an image acquisition device and an onboard radar device, including:

[0077] S301, uses an image acquisition device to acquire image information of the vehicle within a first preset range;

[0078] S302 uses an onboard radar device to collect radar point cloud information within a second preset range of the vehicle;

[0079] S303 fuses image information and radar point cloud information to obtain environmental information about the road where the vehicle is located.

[0080] In this embodiment of the invention, multiple image acquisition devices can be installed on the autonomous vehicle, such as four image acquisition devices facing different directions installed around the autonomous vehicle, for acquiring image information within a first preset range of the autonomous vehicle. The image information includes at least image information of vehicles, pedestrians, obstacles, etc. within the first preset range of the autonomous vehicle. Similarly, multiple vehicle-mounted radar devices can be installed on the autonomous vehicle for acquiring radar point cloud information of vehicles, pedestrians, obstacles, etc. within a second preset range of the autonomous vehicle. For the acquired image information and radar point cloud information, feature extraction can be performed first, and then the extracted feature maps can be fused to obtain the road environment information of the road where the vehicle is located.

[0081] This invention integrates image information and radar point cloud information to obtain richer and more accurate scene information, which facilitates the vehicle energy-saving control model to provide better vehicle control strategies.

[0082] As one possible embodiment of the present invention, in this embodiment, such as Figure 4 As shown, the training process of the vehicle energy-saving control model based on deep reinforcement learning includes:

[0083] S401, Initialize the high sample pool and the standard sample pool, wherein the reward value of the reward function corresponding to the vehicle control strategy in the high sample pool is greater than the preset reward threshold.

[0084] S402, the vehicle energy-saving control model is trained for the first round using sample data from the high sample pool. Sample data whose reward value of the reward function is greater than the preset reward threshold in the first round of training are added to the high sample pool and the high sample pool is updated.

[0085] S403: Randomly select sample data from the updated high-sample pool and standard sample pool according to the preset ratio to conduct a second round of training on the vehicle energy-saving control model until the vehicle energy-saving control model achieves the expected results.

[0086] In this embodiment of the invention, the sample data in the high sample pool and the standard sample pool include scene information of the current scene in which the vehicle is located, the vehicle control strategy determined by the vehicle energy-saving control model based on the scene information of the current scene, the reward value of the reward function of the vehicle control strategy, and the scene information of the next scene in which the vehicle is located under the vehicle control strategy. In the sample data in the high sample pool, the reward value of the reward function is greater than a preset reward threshold. Because vehicle control strategies that are detrimental to vehicle safety may emerge in the early stages of training the vehicle energy-saving control model, resulting in a small number of positive samples and low training efficiency, sample data from the high sample pool is collected in the early stages of training by manually driving the vehicle. For example, experienced drivers can drive the vehicle for a long distance to collect scene information and vehicle actions during the journey, generating sample data in the initial high sample pool. Only the sample data from this initial high sample pool is used for the first round of training of the vehicle energy-saving control model. For each training result in the first round, sample data whose reward value of the reward function is greater than the preset reward threshold is added to the high sample pool to enrich the sample data in the high sample pool and increase its capacity. The number of training iterations in this first round should reach a preset number, such as 1,000 or 10,000, or the sample data capacity in the high sample pool should reach its maximum value.

[0087] Furthermore, after the first round of training, sample data is randomly selected from the updated high sample pool and standard sample pool according to a preset ratio to conduct a second round of training on the vehicle energy-saving control model. Specifically, the preset ratio can be adjusted to train the vehicle energy-saving control model multiple times in order to improve the energy-saving effect of the vehicle energy-saving control model.

[0088] Furthermore, such as Figure 5 As shown, until the vehicle energy-saving control model achieves the expected results, including:

[0089] S501, calculate the reward value of the reward function for each training result and the evaluation value of the current value network. The evaluation value of the value network is used to indicate the score of the vehicle control strategy in the training result.

[0090] S502, calculate the target value of the target value network based on the reward value of the reward function of each training result and the current value network evaluation value;

[0091] S503: Based on the current value network's evaluation value and the target value network's target value, construct a loss function and update the vehicle energy-saving control model to minimize the loss function.

[0092] In this embodiment of the invention, for the training process of the vehicle energy-saving control model, a value network can be used to indicate the quality of each training result. For each training session, the input of the vehicle energy-saving control model is the scene information of the vehicle's current environment, and the output is the vehicle control strategy. Based on this scene information and the vehicle's control strategy, the current evaluation value of the vehicle's value network is calculated. Then, based on the reward value of the reward function under the vehicle control strategy and the scene information of the vehicle's location at the next moment, and the vehicle control strategy for the next moment given by the vehicle energy-saving control model based on the scene information of the vehicle's location at the next moment, the target value of the target value network is calculated based on the reward value of the reward function, the scene information of the vehicle's location at the next moment, and the vehicle control strategy for the next moment given by the vehicle energy-saving control model based on the scene information of the vehicle's location at the next moment. Then construct the loss function. Loss As shown in formula (1):

[0093] (1)

[0094] in, N This represents the number of sample data.

[0095] By adjusting the parameters of the vehicle energy-saving control model to minimize the loss function, a well-trained vehicle energy-saving control model can be obtained.

[0096] Furthermore, such as Figure 6 As shown, the evaluation value of the current value network is calculated for each training result, including:

[0097] S601, calculate the first evaluation value of the first current value network and the second evaluation value of the second current value network for each training result;

[0098] S602, the smaller of the first evaluation value and the second evaluation value is used as the evaluation value of the current value network of the training result.

[0099] In this embodiment of the invention, the training process of the vehicle energy-saving control model is a deep learning process. The value network can indicate the score of each training result, and the target value network is calculated by the current value network and the reward value of the future reward function. Since the neural network may have errors when fitting the target value function, these errors will accumulate continuously during the learning process, leading to the overestimation of the target value. To prevent the above situation, the original current value network is divided into two current value networks, and the evaluation of the vehicle control strategy is implemented using two different current value networks. The current value network with the smaller estimate is selected as the evaluation object of the target value network, thereby avoiding the overestimation problem caused by a single network.

[0100] Optional, see reference Figure 7 and Figure 8 This document presents a schematic diagram of the vehicle energy-saving control model principle and a flowchart of the vehicle energy-saving control model training process provided in this embodiment of the invention. In the initial stage of training, the vehicle energy-saving control model is trained using manually driven vehicles and sample data from an initial high-sample pool. The energy-saving control is continuously optimized, and sample data with reward values ​​exceeding a preset threshold are added to the high-sample pool. After training reaches a certain level, a certain proportion of sample data from both the high-sample pool and the standard sample pool is sampled to continue training the vehicle energy-saving control model. Then, two current value networks and a target value network are used to update the model, ensuring it meets the requirements. Specifically, during the manual vehicle sample data collection process, network parameters and an experience pool are initialized. Under the premise of ensuring vehicle safety, vehicle actions are collected each time the driver operates the vehicle, and corresponding reward values ​​are obtained based on different driving states such as overtaking and following. Samples with reward values ​​exceeding a preset reward threshold are added to the high-sample pool. When the sample data in the high-sample pool reaches its maximum, the vehicle energy-saving control model is trained proportionally using sample data from both the high-sample pool and the standard sample pool until training is complete.

[0101] In this embodiment of the invention, the vehicle energy-saving control model is continuously updated and optimized through a loss function to improve the energy-saving effect of the vehicle energy-saving control model.

[0102] As one possible embodiment of the present invention, in this embodiment, such as Figure 9 As shown, the calculation of the reward value of the reward function includes:

[0103] S901, calculates energy consumption score based on vehicle motor speed and motor torque under vehicle control strategy;

[0104] S901, calculates the vehicle speed score based on the relationship between the vehicle speed and the vehicle's allowed range under the vehicle control strategy.

[0105] S903, determining the safety score based on the vehicle's safety under the vehicle control strategy;

[0106] S904 calculates the reward value of the reward function based on the energy consumption score, vehicle speed score, and safety score according to preset weights.

[0107] In this embodiment of the invention, the reward function measures the degree of "good" or "bad" achieved by the agent after performing an action in the environment, guiding the agent to adjust its strategy to maximize future cumulative rewards, thereby making effective decisions and achieving its goals in complex environments. In this embodiment of the invention, in order to minimize the energy consumption of the vehicle while ensuring vehicle driving safety, the reward function includes energy consumption score, vehicle speed score, and safety score.

[0108] Specifically, the energy consumption score will be explained using several specific examples, such as... Figure 10 As shown in diagram a, this is an overtaking scenario. Car A intends to overtake car B to position C. There are several routes to achieve the overtaking maneuver. The first route is overtaking via route 1. The overtaking process involves a smooth acceleration, constant speed, and then smooth deceleration. During this process, the engine's operating point is... Figure 10 In section b, there is route 1; the second method is to overtake using overtaking route 2, which involves rapid acceleration and deceleration. During this process, the engine's operating point is... Figure 10 Area 2 of Route b, where... Figure 10 In diagram b, the horizontal axis represents motor speed, and the vertical axis represents motor torque. Higher speed and torque indicate higher motor energy consumption. Therefore, from... Figure 10 b shows that route 1 is more economical than route 2.

[0109] For example, such as Figure 11 As shown in Figure a, this is a map depicting a car-following scenario. Due to environmental constraints, car A can only follow car B. L represents the following distance. Car B's speed fluctuates due to the influence of car C in front, alternating between fast and slow. Several strategies can be employed to achieve this following: The first is to use an adaptive following distance L, allowing the vehicle to travel at a fixed, economical speed while ensuring safety and efficiency. Figure 11 Following strategy 1 speed curve, at which point the engine operating point is... Figure 11 Strategy 1 in area b; the second method is to achieve following distance L, in which case the speed of car A is the same as the speed of car B, such as... Figure 11 Following strategy 2 speed curve, at this time the engine operating point is... Figure 11 In strategy 2 area b, by Figure 11 b shows that following strategy 1 is more economical than strategy 2.

[0110] Based on the two embodiments above, it can be clearly seen that the energy consumption at time t... The score is obtained from the motor speed and motor torque, and is generally between -50 and 50.

[0111] Furthermore, the vehicle speed score at time t. It is used to indicate the optimal speed to be achieved under permissible driving and speed limits, and can be calculated using formula (2):

[0112] (2)

[0113] in, Let be the vehicle's actual speed at time t. Let t be the maximum permissible speed of the vehicle at time t.

[0114] Furthermore, the safety score at time t is When the vehicle is not involved in an accident, the safety score is 0; when the vehicle is involved in an accident, the safety score is -300.

[0115] Based on this, the reward value of the reward function can be calculated using formula (3):

[0116] (3)

[0117] in, Let be the reward value of the reward function at time t. , , These are the weighting coefficients for energy consumption score, vehicle speed score, and safety score, respectively.

[0118] This invention calculates the reward value of the reward function by using energy consumption score, vehicle speed score, and safety score, thereby ensuring vehicle safety while minimizing vehicle energy consumption.

[0119] To better implement the autonomous driving control method in the embodiments of the present invention, based on the autonomous driving control method, correspondingly, as follows: Figure 12 As shown, this embodiment of the invention also provides an autonomous driving control device, the autonomous driving control device 1200 including:

[0120] The environmental information acquisition module 1201 is used to acquire environmental information of the scene where the vehicle is located. The environmental information includes vehicle status information and environmental information of the road where the vehicle is located.

[0121] The control strategy determination module 1202 is used to analyze vehicle state information and road environment information using a vehicle energy-saving control model based on deep reinforcement learning to determine the vehicle control strategy. The vehicle energy-saving control model includes a reward function, which is used to represent the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized.

[0122] The control strategy execution module 1203 is used to control the vehicle's autonomous driving according to the vehicle control strategy.

[0123] The autonomous driving control device 1200 provided in the above embodiments can realize the technical solutions described in the above autonomous driving control method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above autonomous driving control method embodiments, and will not be repeated here.

[0124] The autonomous driving control device provided by this invention acquires vehicle state information and road information of the road where the vehicle is located, and uses a vehicle energy-saving control model based on deep reinforcement learning to analyze the vehicle state information and road environment information to determine the vehicle control strategy. The vehicle energy-saving control model includes a reward function, which represents the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized. By using the vehicle control strategy with the maximum reward value of the reward function to control the vehicle for autonomous driving, vehicle energy consumption can be effectively reduced and energy can be saved.

[0125] like Figure 13 As shown, the present invention also provides an electronic device 1300. The electronic device 1300 includes a processor 1301, a memory 1302, and a display 1303. Figure 13 Only some components of the electronic device 1300 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0126] In some embodiments, processor 1301 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in memory 1302 or process data, such as the autonomous driving control method of the present invention.

[0127] In some embodiments, processor 1301 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 1301 may be local or remote. In some embodiments, processor 1301 may be implemented on a cloud platform. In some embodiments, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-cloud, or any combination thereof.

[0128] In some embodiments, memory 1302 may be an internal storage unit of electronic device 1300, such as a hard disk or memory of electronic device 1300. In other embodiments, memory 1302 may also be an external storage device of electronic device 1300, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 1300.

[0129] Furthermore, the memory 1302 may include both internal storage units of the electronic device 1300 and external storage devices. The memory 1302 is used to store application software and various types of data installed on the electronic device 1300.

[0130] In some embodiments, display 1303 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 1303 is used to display information from electronic device 1300 and to display a visual user interface. Components 1301-1303 of electronic device 1300 communicate with each other via a system bus.

[0131] In some embodiments, when the processor 1301 executes the autonomous driving control program in the memory 1302, the following steps may be implemented:

[0132] Obtain scene information of the vehicle's location, including vehicle status information and road environment information of the road where the vehicle is located;

[0133] A vehicle energy-saving control model with a reward function is adopted to determine the vehicle control strategy based on vehicle state information and road environment information. The reward function is used to calculate the energy-saving efficiency of the vehicle under the control strategy, and the reward value of the reward function corresponding to the vehicle control strategy is maximized.

[0134] The vehicle is controlled to operate automatically according to the vehicle control strategy.

[0135] It should be understood that when the processor 1301 executes the automatic driving control program in the memory 1302, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0136] Furthermore, the embodiments of the present invention do not specifically limit the type of the electronic device 1300 mentioned. The electronic device 1300 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 1300 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0137] Accordingly, embodiments of the present invention also provide a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions in the autonomous driving control methods provided in the above-described method embodiments.

[0138] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0139] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An automatic driving control method, characterized in that, include: Obtain scene information of the vehicle's location, including vehicle status information and road environment information of the road where the vehicle is located; A vehicle energy-saving control model based on deep reinforcement learning is used to analyze the vehicle state information and the road environment information to determine the vehicle control strategy. The vehicle energy-saving control model includes a reward function, which is used to represent the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized. The vehicle is controlled to operate automatically according to the vehicle control strategy. The training process of the vehicle energy-saving control model based on deep reinforcement learning includes: Initialize a high-sample pool and a standard sample pool, wherein the reward value of the reward function corresponding to the vehicle control strategy in the high-sample pool is greater than a preset reward threshold; The vehicle energy-saving control model is trained in the first round using sample data from the high sample pool. Sample data whose reward value of the reward function is greater than the preset reward threshold in the first round of training are added to the high sample pool, and the high sample pool is updated. According to a preset ratio, sample data is randomly selected from the updated high sample pool and the standard sample pool to conduct a second round of training on the vehicle energy-saving control model until the vehicle energy-saving control model achieves the expected effect. The process until the vehicle energy-saving control model achieves the expected results includes: Calculate the reward value of the reward function for each training result and the evaluation value of the current value network, wherein the evaluation value of the value network is used to indicate the score of the vehicle control strategy in the training result; The target value of the target value network is calculated based on the reward value of the reward function of each training result and the current value network evaluation value. A loss function is constructed based on the evaluation value of the current value network and the target value of the target value network, and the vehicle energy-saving control model is updated to minimize the loss function.

2. The automatic driving control method according to claim 1, characterized in that, The acquisition of scene information of the vehicle's location includes: Vehicle status information is collected using onboard sensors; The system uses image acquisition devices and vehicle-mounted radar devices to collect road environment information of the road where the vehicle is located.

3. The automatic driving control method according to claim 2, characterized in that, The method of collecting road environment information of the road where the vehicle is located using an image acquisition device and an on-board radar device includes: The image acquisition device is used to acquire image information of the vehicle within a first preset range; The vehicle-mounted radar device is used to collect radar point cloud information within a second preset range of the vehicle; The image information and the radar point cloud information are fused to obtain the environmental information of the road where the vehicle is located.

4. The automatic driving control method according to claim 1, characterized in that, Calculate the current evaluation value of the network for each training result, including: Calculate the first evaluation value of the first current value network and the second evaluation value of the second current value network for each training result; The smaller of the first evaluation value and the second evaluation value is used as the evaluation value of the current value network of the training result.

5. The automatic driving control method according to claim 1, characterized in that, The calculation of the reward value of the reward function includes: The energy consumption score is calculated based on the vehicle's motor speed and motor torque under the aforementioned vehicle control strategy. The vehicle speed score is calculated based on the relationship between the vehicle speed and the vehicle's permissible range under the aforementioned vehicle control strategy. A safety score is determined based on the vehicle's safety under the aforementioned vehicle control strategy; The reward value of the reward function is calculated based on the energy consumption score, the vehicle speed score, and the safety score according to preset weights.

6. An automatic driving control device, applicable to the automatic driving control method according to any one of claims 1-5, characterized in that, include: An environmental information acquisition module is used to acquire environmental information of the scene where the vehicle is located, including vehicle status information and environmental information of the road where the vehicle is located. The control strategy determination module is used to analyze the vehicle state information and the road environment information using a vehicle energy-saving control model based on deep reinforcement learning to determine the vehicle control strategy. The vehicle energy-saving control model includes a reward function, which represents the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is maximized. The control strategy execution module is used to control the vehicle to drive automatically according to the vehicle control strategy.

7. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the autonomous driving control method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the autonomous driving control method according to any one of claims 1 to 5.

Images