Automatic driving control method and device, electronic equipment and computer storage medium

Abstract

The invention relates to an automatic driving control method and device, electronic equipment and a computer storage medium, and belongs to the technical field of automatic drive.The automatic driving control method comprises the steps that scene information of a scene where a vehicle is located is obtained, and the scene information comprises vehicle state information and road environment information of a road where the vehicle is located; a vehicle energy-saving control model based on deep reinforcement learning is adopted to analyze the vehicle state information and the road environment information, a vehicle control strategy is determined, the vehicle energy-saving control model comprises a reward function, and the reward function is used for representing the balance of safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy; the reward value of the reward function corresponding to the vehicle control strategy is maximum; and controlling the vehicle to automatically drive according to the vehicle control strategy. The vehicle energy consumption can be effectively reduced, and energy is saved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an autonomous driving control method, device, electronic equipment and computer storage medium. Background Art

[0002] In recent years, the rapid development of artificial intelligence technology has had a profound impact on all walks of life, such as image recognition, speech recognition, natural language processing, etc. These technologies provide strong technical support for autonomous driving, enabling cars to better perceive the surrounding environment, understand and respond to complex and changing road conditions. Major car companies have demonstrated their own autonomous driving perception technology.

[0003] In existing autonomous driving technologies, vehicle control is relatively rigid for safety and stability reasons. In different vehicle driving environments, operations such as continuous speed changes, sudden acceleration, sudden braking, and sudden lane changes may occur, which increases vehicle energy consumption and reduces the passenger riding experience. Especially in complex driving environments, vehicle energy consumption is not taken into consideration, resulting in energy waste.

[0004] It can be seen from this that the existing autonomous driving technology will cause technical problems such as excessive vehicle energy consumption. Summary of the invention

[0005] In view of this, it is necessary to provide an automatic driving control method, device, electronic device and computer storage medium to solve the technical problem that the existing automatic driving technology may cause excessive energy consumption of the vehicle.

[0006] In order to solve the above problems, the present invention provides an automatic driving control method, comprising: Acquire scene information of a scene in which the vehicle is located, wherein the scene information includes vehicle state information and road environment information of a road in which the vehicle is located; A vehicle energy-saving control model based on deep reinforcement learning is used to analyze the vehicle state information and the road environment information to determine a vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, which is used to represent the balance between the safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy, and the reward value of the reward function corresponding to the vehicle control strategy is the maximum. Controlling the automatic driving of the vehicle according to the vehicle control strategy.

[0007] In a possible implementation manner, the acquiring scene information of the scene in which the vehicle is located includes: Use vehicle-mounted sensors to collect vehicle status information; An image acquisition device and a vehicle-mounted radar device are used to collect road environment information of the road where the vehicle is located.

[0008] In a possible implementation, the using of an image acquisition device and a vehicle-mounted radar device to collect road environment information of a road where the vehicle is located includes: Using an image acquisition device to acquire image information within a first preset range of the vehicle; Using a vehicle-mounted radar device to collect radar point cloud information within a second preset range of the vehicle; The image information and the radar point cloud information are fused to obtain environmental information of the road where the vehicle is located.

[0009] In a possible implementation, the training process of the vehicle energy-saving control model based on deep reinforcement learning includes: Using sample data in the high sample pool to perform a first round of training on the vehicle energy-saving control model, adding sample data whose reward value of the reward function in the first round of training is greater than the preset reward threshold to the high sample pool, and updating the high sample pool; The vehicle energy-saving control model is trained for the second round by randomly selecting sample data from the updated high sample pool and the standard sample pool according to a preset ratio until the vehicle energy-saving control model achieves the expected effect.

[0010] In a possible implementation manner, the step of achieving the expected effect of the vehicle energy-saving control model includes: Calculating a reward value of a reward function for each training result and an evaluation value of a current value network, wherein the evaluation value of the value network is used to indicate a score of a vehicle control strategy in the training result; Calculate the target value of the target value network based on the reward value of the reward function of each training result and the current value network evaluation value; A loss function is constructed based on the evaluation value of the current value network and the target value of the target value network, and the vehicle energy-saving control model is updated to minimize the loss function.

[0011] In a possible implementation, the calculating of the evaluation value of the current value network of each training result includes: Calculate the first evaluation value of the first current value network and the second evaluation value of the second current value network for each training result respectively; The smaller value between the first evaluation value and the second evaluation value is used as the evaluation value of the current value network of the training result.

[0012] In a possible implementation, the calculation of the reward value of the reward function includes: Calculating an energy consumption score based on a motor speed and a motor torque of the vehicle under the vehicle control strategy; Calculating a vehicle speed score based on a relationship between the vehicle speed and the vehicle allowable range of the vehicle under the vehicle control strategy; Determining a safety score based on the safety of the vehicle under the vehicle control strategy; The reward value of the reward function is calculated based on the energy consumption score, the vehicle speed score and the safety score according to preset weights.

[0013] The present invention also provides an automatic driving control device, comprising: An environmental information acquisition module is used to acquire environmental information of the scene in which the vehicle is located, wherein the environmental information includes vehicle status information and environmental information of the road in which the vehicle is located; A control strategy determination module, used to analyze the vehicle state information and the road environment information using a vehicle energy-saving control model based on deep reinforcement learning to determine a vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, and the reward function is used to represent the balance between the safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy, and the reward value of the reward function corresponding to the vehicle control strategy is the largest; A control strategy execution module is used to control the automatic driving of the vehicle according to the vehicle control strategy.

[0014] The present invention also provides an electronic device, comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the automatic driving control method described in any of the above embodiments.

[0015] The present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the automatic driving control method described in any of the above embodiments.

[0016] The beneficial effect of the present invention is as follows: the automatic driving control method provided by the present invention obtains vehicle status information and road information of the road where the vehicle is located, and adopts a vehicle energy-saving control model based on deep reinforcement learning to analyze the vehicle status information and road environment information to determine the vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, and the reward function is used to represent the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is the largest. By adopting the vehicle control strategy with the largest reward value of the reward function to control the vehicle for automatic driving, the vehicle energy consumption can be effectively reduced and energy can be saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A schematic diagram of a flow chart of an automatic driving control method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a flow chart of a scene information collection method provided by an embodiment of the present invention; Figure 3 A flowchart of an implementation method of S202 provided in an embodiment of the present invention; Figure 4 A flow chart of a vehicle energy-saving control model training method provided by an embodiment of the present invention; Figure 5 A schematic flow chart of a vehicle energy-saving control model optimization method provided by an embodiment of the present invention; Figure 6 A flow chart of a value network calculation method provided by an embodiment of the present invention; Figure 7 A control principle diagram of a vehicle energy-saving model adopted by an embodiment of the present invention; Figure 8 A vehicle energy-saving model training process diagram provided by an embodiment of the present invention; Fig. 9 A process diagram of a method for calculating a reward value of a reward function provided by an embodiment of the present invention; Fig.10 a is a schematic diagram of an overtaking scenario provided by an embodiment of the present invention; Fig.10 b is an engine power consumption diagram under overtaking conditions provided by an embodiment of the present invention; Fig.11 a is a schematic diagram of a vehicle following scenario provided by an embodiment of the present invention; Fig.11 b is an engine power consumption diagram in a vehicle following condition provided by an embodiment of the present invention; Fig.12 A schematic diagram of the structure of an automatic driving control device provided by an embodiment of the present invention; Fig.13 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of the present invention and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise specified, "multiple" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone.

[0021] The descriptions of "first", "second", etc. involved in the embodiments of the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0022] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] A specific embodiment of the present invention, as Figure 1 As shown, a method for controlling an automatic driving is disclosed, comprising: S101, obtaining scene information of a scene in which a vehicle is located, where the scene information includes vehicle state information and road environment information of a road in which the vehicle is located; S102, using a vehicle energy-saving control model based on deep reinforcement learning to analyze vehicle state information and road environment information to determine a vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, and the reward function is used to represent the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy, and the reward value of the reward function corresponding to the vehicle control strategy is the largest; S103, controlling the automatic driving of the vehicle according to the vehicle control strategy.

[0024] In an embodiment of the present invention, an autonomous driving method is provided mainly for controlling an autonomous driving vehicle, which may be a car, a truck, a van, a bus, a motorcycle, or other suitable vehicles.

[0025] In the embodiment of the present invention, the scene information of the scene where the vehicle is located includes the vehicle state information and the road environment information of the road where the vehicle is located, wherein the vehicle state information includes but is not limited to the current speed, acceleration, steering wheel angle, motor torque, etc. of the vehicle, and the road information of the road where the vehicle is located includes but is not limited to the road condition information of the road, the road traffic light signal information, the information of the vehicles in front and behind the vehicle, pedestrians, and obstacles, wherein the information of the vehicles in front and behind the vehicle, pedestrians, and obstacles includes location information, movement trend information, etc. The specific method for obtaining the scene information of the scene where the vehicle is located will be described later in the present invention.

[0026] In an embodiment of the present invention, the vehicle energy-saving control model based on deep reinforcement learning is a pre-set neural network model for determining the vehicle control strategy based on the scene information of the scene in which the vehicle is located. The reward function is used to represent the balance between the safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The maximum reward value of the reward function means that the vehicle has the maximum energy-saving efficiency while ensuring driving safety. Optionally, based on the vehicle state information and the road environment information of the road where the vehicle is located, the vehicle energy-saving control model will give the most optimized vehicle control strategy to save vehicle energy while ensuring the safety of vehicle driving. Among them, the specific determination process of the vehicle control strategy by the vehicle energy-saving control model will be described later in the present invention.

[0027] In the embodiment of the present invention, the vehicle control strategy includes but is not limited to the control of vehicle actions, such as the control of the vehicle steering wheel steering angle, throttle, brake, gear, lights and horn.

[0028] The automatic driving control method provided by the present invention obtains vehicle status information and road information of the road where the vehicle is located, and adopts a vehicle energy-saving control model based on deep reinforcement learning to analyze the vehicle status information and road environment information to determine the vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, and the reward function is used to represent the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is the largest. By adopting the vehicle control strategy with the largest reward value of the reward function to control the vehicle for automatic driving, the vehicle energy consumption can be effectively reduced and energy can be saved.

[0029] As a possible implementation mode of the present invention, in this implementation mode, Figure 2 As shown, the scene information of the scene where the vehicle is located is obtained, including: S201, collecting vehicle status information using vehicle-mounted sensors; S202, using an image acquisition device and a vehicle-mounted radar device to collect road environment information of the road where the vehicle is located.

[0030] In an embodiment of the present invention, the autonomous driving vehicle includes an information collection device, which includes a vehicle-mounted sensor, an image collection device, and a vehicle-mounted radar device. Optionally, the vehicle-mounted sensor includes a vehicle speed sensor, an acceleration sensor, a torque sensor, etc., the image collection device includes a camera, a camera, etc., and the vehicle-mounted radar device includes an ultrasonic radar and a laser radar, etc. The vehicle status information and the road environment information of the road where the vehicle is located are collected through the above-mentioned information collection device.

[0031] In an embodiment of the present invention, the driving route of the autonomous driving vehicle is first confirmed, and the vehicle positioning can be achieved through high-precision maps and global positioning systems, and the driving route is planned to be globally optimal. When the vehicle travels according to the driving route, the vehicle's current speed, acceleration, steering wheel angle, motor torque, etc. are collected through vehicle sensors, and traffic lights, road conditions and other information on the road where the vehicle is located are obtained through V2X (vehicle to X, vehicle wireless communication technology). Image information of vehicles, pedestrians and obstacles in front and behind the vehicle is collected through cameras and cameras, and radar point cloud information of vehicles, pedestrians and obstacles in front and behind the vehicle is obtained through lidar and / or ultrasonic radar.

[0032] The embodiment of the present invention collects vehicle status information and road environment information of the road where the vehicle is located through multiple information collection devices, and can provide richer scene information for the vehicle energy-saving control model.

[0033] As a possible implementation mode of the present invention, in this implementation mode, Figure 3 As shown, an image acquisition device and a vehicle-mounted radar device are used to collect road environment information of the road where the vehicle is located, including: S301, using an image acquisition device to acquire image information within a first preset range of the vehicle; S302, using a vehicle-mounted radar device to collect radar point cloud information within a second preset range of the vehicle; S303, fusing the image information and the radar point cloud information to obtain environmental information of the road where the vehicle is located.

[0034] In an embodiment of the present invention, a plurality of image acquisition devices may be installed on the autonomous driving vehicle, such as image acquisition devices facing four directions installed around the autonomous driving vehicle respectively, for acquiring image information within a first preset range of the autonomous driving vehicle, where the image information at least includes image information of vehicles, pedestrians, obstacles, etc. within the first preset range of the autonomous driving vehicle. Similarly, a plurality of on-board radar devices may be installed on the autonomous driving vehicle for acquiring radar point cloud information of vehicles, pedestrians, obstacles, etc. within a second preset range of the autonomous driving vehicle. For the acquired image information and radar point cloud information, feature extraction may be performed first, and then the extracted feature maps may be fused to obtain road environment information of the road where the vehicle is located.

[0035] The embodiment of the present invention obtains richer and more accurate scene information by fusing image information and radar point cloud information, which facilitates the vehicle energy-saving control model to provide a better vehicle control strategy.

[0036] As a possible implementation mode of the present invention, in this implementation mode, Figure 4 As shown in the figure, the training process of the vehicle energy-saving control model based on deep reinforcement learning includes: S401, initializing a high sample pool and a standard sample pool, wherein the reward value of the reward function corresponding to the vehicle control strategy in the high sample pool is greater than a preset reward threshold; S402, using the sample data in the high sample pool to perform a first round of training on the vehicle energy-saving control model, adding the sample data whose reward value of the reward function in the first round of training is greater than a preset reward threshold to the high sample pool, and updating the high sample pool; S403, randomly selecting sample data from the updated high sample pool and standard sample pool according to a preset ratio to perform a second round of training on the vehicle energy-saving control model until the vehicle energy-saving control model achieves the expected effect.

[0037] In an embodiment of the present invention, the sample data in the high sample pool and the standard sample pool include scene information of the scene in which the vehicle is currently located, the vehicle control strategy determined by the vehicle energy-saving control model based on the scene information of the scene in which the vehicle is currently located, the reward value of the reward function of the vehicle control strategy, and the scene information of the scene in which the vehicle is located at the next moment when it is traveling under the vehicle control strategy, wherein the reward value of the reward function in the sample data in the high sample pool is greater than a preset reward threshold. Since vehicle control strategies that are not conducive to vehicle safety may appear in the early stage of training the vehicle energy-saving control model, resulting in a small number of positive samples and a low efficiency of the training process, in the early stage of training, sample data in the high sample pool is collected by manually driving the vehicle. For example, an experienced driver drives the vehicle for a long distance, and the vehicle collects scene information and vehicle actions during driving to generate sample data in the initial high sample pool. Only the sample data in the initial high sample pool is used to perform the first round of training on the vehicle energy-saving control model. For each training result of the first round of training, sample data whose reward value of the reward function is greater than the preset reward threshold is added to the high sample pool to enrich the sample data in the high sample pool and increase the capacity of the sample data in the high sample pool. The number of training times for the first round of training should reach a preset number, such as 1,000 times, 10,000 times, etc., or the sample data capacity in the high sample pool reaches a maximum value.

[0038] Furthermore, after the first round of training is completed, sample data is randomly selected from the updated high sample pool and standard sample pool according to a preset ratio to perform a second round of training on the vehicle energy-saving control model. Specifically, the preset ratio can be adjusted and the vehicle energy-saving control model can be trained multiple times to improve the energy-saving effect of the vehicle energy-saving control model.

[0039] Further, such as Figure 5 As shown, until the vehicle energy-saving control model achieves the expected effect, including: S501, calculating the reward value of the reward function of each training result and the evaluation value of the current value network, where the value network evaluation value is used to indicate the score of the vehicle control strategy in the training result; S502, calculating the target value of the target value network based on the reward value of the reward function of each training result and the current value network evaluation value; S503, constructing a loss function based on the evaluation value of the current value network and the target value of the target value network, and updating the vehicle energy-saving control model to minimize the loss function.

[0040] In the embodiment of the present invention, for the training process of the vehicle energy-saving control model, a value network can be used to indicate the quality of each training result. For each training, the input of the vehicle energy-saving control model is the scene information of the scene in which the vehicle is located, and the output result is the vehicle control strategy. Based on the scene information and the vehicle control strategy, the evaluation value of the vehicle current value network is calculated. Then, based on the reward value of the reward function under the vehicle control strategy and the scene information of the scene where the vehicle is located at the next moment, and the vehicle control strategy for the next moment given by the vehicle energy-saving control model based on the scene information of the scene where the vehicle is located at the next moment, the target value of the target value network is calculated based on the reward value of the reward function, the scene information of the scene where the vehicle is located at the next moment, and the vehicle control strategy for the next moment given by the vehicle energy-saving control model based on the scene information of the scene where the vehicle is located at the next moment , and then construct the loss function Loss , as shown in formula (1): (1) in, N is the number of sample data.

[0041] By adjusting the parameters of the vehicle energy-saving control model to minimize the loss function, a trained vehicle energy-saving control model can be obtained.

[0042] Further, such as Figure 6 As shown, the evaluation value of the current value network of each training result is calculated, including: S601, respectively calculating a first evaluation value of a first current value network and a second evaluation value of a second current value network for each training result; S602: Taking the smaller value between the first evaluation value and the second evaluation value as the evaluation value of the current value network of the training result.

[0043] In an embodiment of the present invention, the training process of the vehicle energy-saving control model is a deep learning process, and the score of each training result can be indicated by the value network, and the target value network is calculated by the reward value of the current value network and the future reward function. Since there may be errors in the neural network when fitting the target value function, such errors will continue to accumulate during the learning process, resulting in the target value being overestimated. To prevent the above situation, the original current value network is divided into two current value networks, and the evaluation of the vehicle control strategy is implemented using two different current value networks. The current value network with a smaller estimate is selected as the evaluation object of the target value network, thereby avoiding the over-estimation problem caused by a single network.

[0044] Optional, reference Figure 7 and Figure 8, which is a schematic diagram of the principle of the vehicle energy-saving control model provided by the embodiment of the present invention and a schematic diagram of the process of training the vehicle energy-saving control model. In the early stage of training the vehicle energy-saving control model, the vehicle is manually driven and the sample data in the initial high sample pool are used to perform the first round of training on the vehicle energy-saving control model, and the vehicle energy-saving control is continuously optimized. At the same time, the sample data whose reward value of the reward function is greater than the preset threshold is added to the high sample pool. After training to a certain extent, the sample data in the high sample pool and the standard sample pool are sampled in a certain proportion, and the vehicle energy-saving control model is continued to be trained. Then, the model is updated using two current value networks and target value networks, so that the vehicle energy-saving control model meets the requirements. In the process of manually driving the vehicle to collect sample data, the network parameters and the experience pool are initialized first. Under the premise of ensuring vehicle safety, the vehicle action is collected each time the driver operates the vehicle, and the corresponding reward value is obtained based on different driving states such as overtaking and following the vehicle, and the sample whose reward value is greater than the preset reward threshold is added to the high sample pool. When the sample data in the high sample pool is the largest, the vehicle energy-saving control model is trained with the sample data in the high sample pool and the standard sample pool in proportion until the training is completed.

[0045] The embodiment of the present invention continues to update and optimize the vehicle energy-saving control model through the loss function, thereby improving the energy-saving effect of the vehicle energy-saving control model.

[0046] As a possible implementation mode of the present invention, in this implementation mode, Fig. 9 As shown, the calculation of the reward value of the reward function includes: S901, calculating an energy consumption score based on a motor speed and a motor torque of the vehicle under a vehicle control strategy; S901, calculating a vehicle speed score based on a relationship between a vehicle speed and an allowable vehicle range of the vehicle under a vehicle control strategy; S903, determining a safety score based on the safety of the vehicle under the vehicle control strategy; S904: Calculate a reward value of a reward function based on the energy consumption score, the vehicle speed score, and the safety score according to preset weights.

[0047] In the embodiment of the present invention, the reward function guides the agent to adjust its strategy to maximize the future cumulative rewards by measuring the degree of "good" or "bad" obtained after the agent performs a certain action in the environment, so as to make effective decisions and achieve its goals in a complex environment. In the embodiment of the present invention, under the premise of ensuring the safety of vehicle driving, in order to reduce the energy consumption of the vehicle as much as possible, the reward function set includes energy consumption score, speed score, and safety score.

[0048] Specifically, the energy consumption score is described with several specific embodiments, such as Fig.10As shown in a, this is an overtaking scenario. Car A intends to overtake Car B to position C. There are multiple routes to achieve overtaking. The first is to overtake via overtaking route 1. The overtaking process is a process of steady acceleration, constant speed, and steady deceleration. During this process, the engine operating point is Fig.10 b) Route 1 area; the second is to overtake with overtaking route 2. The overtaking process is a process of rapid acceleration and deceleration. During this process, the engine operating point is Fig.10 b Route 2 area, where Fig.10 In b, the horizontal axis is the motor speed, and the vertical axis is the motor torque. The higher the speed and the greater the torque, the higher the motor energy consumption. Therefore, Fig.10 b It can be seen that route 1 is more economical than route 2.

[0049] For example, Fig.11 As shown in a, it is a following scene map. Due to environmental restrictions, car A can only follow car B. L is the following distance. Car B is affected by the front car C and its speed is always fast and slow. At this time, there are many strategies to achieve following: the first is to achieve following with an adaptive following distance L, so that the vehicle can travel at a fixed economic speed while ensuring safety and efficiency, such as Fig.11 In a, the speed curve of the following strategy 1, the engine operating point is Fig.11 b in strategy 1 area; the second is to achieve following the car with a fixed following distance L, then the speed of car A is consistent with the speed of car B, such as Fig.11 In a, the speed curve of the following strategy 2 is as follows: Fig.11 The strategy 2 region in b is composed of Fig.11 b It can be seen that following strategy 1 is more economical than strategy 2.

[0050] Based on the above two embodiments, it can be clearly seen that the energy consumption at time t is The score is obtained by motor speed and motor torque, and is generally between -50 and 50.

[0051] Furthermore, the speed score at time t It is used to indicate that the optimal speed can be achieved as much as possible under the conditions of permissible driving and speed limit. It can be calculated using formula (2): (2) in, is the actual speed of the vehicle at time t, is the maximum permissible speed of the vehicle at time t.

[0052] Furthermore, the safety score at time t is When the vehicle has no accident, the safety score is 0, and when the vehicle has an accident, the safety score is -300.

[0053] Based on this, the reward value of the reward function can be calculated by formula (3): (3) in, is the reward value of the reward function at time t, , , They are the weight coefficients of energy consumption score, vehicle speed score and safety score respectively.

[0054] The embodiment of the present invention calculates the reward value of the reward function through the energy consumption score, the vehicle speed score and the safety score, thereby ensuring the safety of the vehicle while minimizing the energy consumption of the vehicle.

[0055] In order to better implement the automatic driving control method in the embodiment of the present invention, based on the automatic driving control method, correspondingly, Fig.12 As shown, an embodiment of the present invention further provides an automatic driving control device, and the automatic driving control device 1200 includes: The environment information acquisition module 1201 is used to acquire the environment information of the scene where the vehicle is located, and the environment information includes the vehicle state information and the environment information of the road where the vehicle is located; A control strategy determination module 1202 is used to analyze vehicle state information and road environment information using a vehicle energy-saving control model based on deep reinforcement learning to determine a vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, and the reward function is used to represent the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy, and the reward value of the reward function corresponding to the vehicle control strategy is the largest; The control strategy execution module 1203 is used to control the automatic driving of the vehicle according to the vehicle control strategy.

[0056] The automatic driving control device 1200 provided in the above embodiment can implement the technical solution described in the above automatic driving control method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above automatic driving control method embodiment, which will not be repeated here.

[0057] The automatic driving control device provided by the present invention obtains vehicle status information and road information of the road where the vehicle is located, and adopts a vehicle energy-saving control model based on deep reinforcement learning to analyze the vehicle status information and road environment information to determine the vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, and the reward function is used to represent the balance between safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy. The reward value of the reward function corresponding to the vehicle control strategy is the largest. By adopting the vehicle control strategy with the largest reward value of the reward function to control the vehicle for automatic driving, the vehicle energy consumption can be effectively reduced and energy can be saved.

[0058] like Fig.13 As shown, the present invention also provides an electronic device 1300. The electronic device 1300 includes a processor 1301, a memory 1302 and a display 1303. Fig.13 Only some components of the electronic device 1300 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0059] In some embodiments, the processor 1301 can be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 1302, such as the automatic driving control method in the present invention.

[0060] In some embodiments, processor 1301 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, processor 1301 may be local or remote. In some embodiments, processor 1301 may be implemented in a cloud platform. In some embodiments, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination thereof.

[0061] In some embodiments, the memory 1302 may be an internal storage unit of the electronic device 1300, such as a hard disk or memory of the electronic device 1300. In other embodiments, the memory 1302 may also be an external storage device of the electronic device 1300, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 1300.

[0062] Furthermore, the memory 1302 may include both an internal storage unit of the electronic device 1300 and an external storage device. The memory 1302 is used to store application software installed in the electronic device 1300 and various data.

[0063] In some embodiments, the display 1303 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 1303 is used to display information of the electronic device 1300 and to display a visual user interface. The components 1301-1303 of the electronic device 1300 communicate with each other via a system bus.

[0064] In some embodiments, when the processor 1301 executes the autonomous driving control program in the memory 1302, the following steps may be implemented: Obtaining scene information of the scene where the vehicle is located, the scene information includes vehicle status information and road environment information of the road where the vehicle is located; A vehicle energy-saving control model with a reward function is used to determine a vehicle control strategy based on vehicle state information and road environment information, wherein the reward function is used to calculate the energy-saving efficiency of the vehicle under the control strategy, and the reward value of the reward function corresponding to the vehicle control strategy is the largest; Control the vehicle's automatic driving according to the vehicle control strategy.

[0065] It should be understood that when the processor 1301 executes the automatic driving control program in the memory 1302, in addition to the above functions, other functions can also be implemented. For details, please refer to the description of the corresponding method embodiment above.

[0066] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 1300 mentioned, and the electronic device 1300 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 1300 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0067] Correspondingly, an embodiment of the present invention also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps or functions in the automatic driving control method provided by the above-mentioned method embodiments can be implemented.

[0068] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0069] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An automatic driving control method, characterized in that: include: Acquire scene information of a scene in which the vehicle is located, wherein the scene information includes vehicle state information and road environment information of a road in which the vehicle is located; A vehicle energy-saving control model based on deep reinforcement learning is used to analyze the vehicle state information and the road environment information to determine a vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, and the reward function is used to represent the balance between the safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy, and the reward value of the reward function corresponding to the vehicle control strategy is the largest; Controlling the automatic driving of the vehicle according to the vehicle control strategy.

2. The automatic driving control method according to claim 1, characterized in that: The acquiring of scene information of the scene in which the vehicle is located includes: Use vehicle-mounted sensors to collect vehicle status information; An image acquisition device and a vehicle-mounted radar device are used to collect road environment information of the road where the vehicle is located.

3. The automatic driving control method according to claim 2, characterized in that: The method of using an image acquisition device and a vehicle-mounted radar device to collect road environment information of the road where the vehicle is located includes: Using an image acquisition device to acquire image information within a first preset range of the vehicle; Using a vehicle-mounted radar device to collect radar point cloud information within a second preset range of the vehicle; The image information and the radar point cloud information are fused to obtain environmental information of the road where the vehicle is located.

4. The automatic driving control method according to claim 1, characterized in that: The training process of the vehicle energy-saving control model based on deep reinforcement learning includes: Initializing a high sample pool and a standard sample pool, wherein a reward value of a reward function corresponding to a vehicle control strategy in the high sample pool is greater than a preset reward threshold; Using sample data in the high sample pool to perform a first round of training on the vehicle energy-saving control model, adding sample data whose reward value of the reward function in the first round of training is greater than the preset reward threshold to the high sample pool, and updating the high sample pool; The vehicle energy-saving control model is trained for the second round by randomly selecting sample data from the updated high sample pool and the standard sample pool according to a preset ratio until the vehicle energy-saving control model achieves the expected effect.

5. The automatic driving control method according to claim 4, characterized in that: The step of: until the vehicle energy-saving control model achieves the expected effect includes: Calculating a reward value of a reward function for each training result and an evaluation value of a current value network, wherein the evaluation value of the value network is used to indicate a score of a vehicle control strategy in the training result; Calculate the target value of the target value network based on the reward value of the reward function of each training result and the current value network evaluation value; A loss function is constructed based on the evaluation value of the current value network and the target value of the target value network, and the vehicle energy-saving control model is updated to minimize the loss function.

6. The automatic driving control method according to claim 5, characterized in that: Calculate the evaluation value of the current value network for each training result, including: Calculate the first evaluation value of the first current value network and the second evaluation value of the second current value network for each training result respectively; The smaller value between the first evaluation value and the second evaluation value is used as the evaluation value of the current value network of the training result.

7. The automatic driving control method according to claim 1, characterized in that: The calculation of the reward value of the reward function includes: Calculating an energy consumption score based on a motor speed and a motor torque of the vehicle under the vehicle control strategy; Calculating a vehicle speed score based on a relationship between the vehicle speed and the vehicle allowable range of the vehicle under the vehicle control strategy; Determining a safety score based on the safety of the vehicle under the vehicle control strategy; The reward value of the reward function is calculated based on the energy consumption score, the vehicle speed score and the safety score according to preset weights.

8. An automatic driving control device, characterized in that: include: An environmental information acquisition module is used to acquire environmental information of the scene in which the vehicle is located, wherein the environmental information includes vehicle status information and environmental information of the road in which the vehicle is located; A control strategy determination module, used to analyze the vehicle state information and the road environment information using a vehicle energy-saving control model based on deep reinforcement learning to determine a vehicle control strategy, wherein the vehicle energy-saving control model includes a reward function, and the reward function is used to represent the balance between the safe driving and energy-saving efficiency of the vehicle under the vehicle control strategy, and the reward value of the reward function corresponding to the vehicle control strategy is the largest; A control strategy execution module is used to control the automatic driving of the vehicle according to the vehicle control strategy.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the automatic driving control method described in any one of claims 1 to 7 above.

10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the automatic driving control method described in any one of claims 1 to 7.

Images