Intelligent automobile driving decision-making method considering attachment condition

Abstract

The embodiment of the invention discloses an intelligent vehicle driving decision-making method considering an attachment condition. The method comprises the steps of obtaining a current expected vehicle speed, a speed error item, an attachment coefficient and front and rear images of a vehicle of an intelligent vehicle; the comfortable deceleration in the IDM car-following model is corrected according to the adhesion coefficient, and the longitudinal acceleration of the vehicle is generated through the corrected comfortable deceleration; extracting image features from the front and rear images of the vehicle by using a feature extractor, and inputting the image features, the expected vehicle speed, the speed error term and the adhesion coefficient into a DQN model; the DQN model calculates a reward value according to the corrected comfortable deceleration, and generates a transverse lane changing action of the vehicle according to the reward value and an anti-collision action mask; and the longitudinal acceleration and the transverse lane changing action jointly form the final action of the vehicle. According to the embodiment, the adaptability of the vehicle running at a high speed to different attachment conditions is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of automatic driving of intelligent vehicles, and in particular to a driving decision-making method of an intelligent vehicle taking adhesion conditions into consideration. Background Art

[0002] In recent years, the level of automobile electrification and intelligence has been continuously improved. While the total number of vehicles has maintained an upward trend, road traffic safety faces severe challenges, especially in rainy and snowy weather, where a large number of road traffic accidents occur. As smart cars move towards L4 level high automation, the core position of the behavior decision system as the "brain" of the vehicle is becoming increasingly prominent. Therefore, how to establish an intelligent decision-making system that takes into account the adhesion conditions of the road surface has become an important issue to break through the bottleneck of all-weather operation of autonomous driving and improve driving safety under low adhesion conditions.

[0003] In the prior art, driving decision models that consider adhesion coefficients are mainly rule-based decision models and reinforcement learning models. The rule-based decision model focuses on longitudinal driving behavior decisions, and pays less attention to lateral driving behavior decisions under different adhesion coefficients; while the rule-based decision model based on expert knowledge has strong interpretability and logic, and performs well when the state transfer signal is clear, but the generalization ability is insufficient, and it is difficult to adapt to traffic scenarios that require dynamic interaction. The decision-making process of the reinforcement learning driving decision model is difficult to explain, and there are problems with insufficient safety and stability. Dangerous actions may be searched during training, and there are also safety risks. Patent application CN116946162A provides a safe driving decision-making method for intelligent connected commercial vehicles that considers road adhesion conditions, and CN114030485A provides a human-like lane change decision-making planning method for autonomous driving vehicles that considers adhesion coefficients, neither of which can solve the above problems.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] The embodiment of the present invention provides a smart car driving decision method taking adhesion conditions into consideration to solve the above technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a driving decision method for an intelligent vehicle taking adhesion conditions into consideration, comprising:

[0007] Obtain the current expected speed, speed error term, adhesion coefficient and front and rear images of the intelligent vehicle;

[0008] Correcting the comfort deceleration in the IDM following model according to the adhesion coefficient, and generating the longitudinal acceleration of the vehicle using the corrected comfort deceleration;

[0009] Extracting image features from the front and rear images of the vehicle using a feature extractor, and inputting the image features, the expected vehicle speed, the speed error term, and the adhesion coefficient into a DQN model;

[0010] The DQN model calculates a reward value according to the corrected comfortable deceleration, and generates a lateral lane changing action of the vehicle according to the reward value and an anti-collision action mask;

[0011] The longitudinal acceleration and the lateral lane-changing action together constitute the final action of the vehicle.

[0012] In a second aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0013] one or more processors;

[0014] A memory for storing one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent vehicle driving decision-making method considering adhesion conditions as described in any embodiment.

[0016] The embodiment of the present invention proposes a reinforcement learning driving decision model based on action mask considering the adhesion coefficient. First, the comfortable deceleration of the model is corrected according to the adhesion coefficient, and then a friction-aware DQN decision model considering the adhesion coefficient is established, and the state space, action space, reward function and network structure are constructed. On this basis, an action mask based on rule constraints is constructed for the reinforcement learning driving decision model. The intelligent intelligent car driving decision method based on this model improves the safety of rule driving decision and the reliability of reinforcement learning driving decision by integrating rule decision and reinforcement learning decision, and enhances the adaptability of high-speed vehicles to different adhesion conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a framework diagram of a Maskable friction-aware DQN model provided by an embodiment of the present invention;

[0019] Figure 2 is a schematic diagram of the distance between a main vehicle and surrounding vehicles provided by an embodiment of the present invention;

[0020] Figure 3 is a flow chart of a smart car driving decision method considering adhesion conditions provided by an embodiment of the present invention;

[0021] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0023] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0024] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] The embodiment of the present invention provides a driving decision method for an intelligent vehicle considering adhesion conditions. In order to illustrate the method, a driving decision model for an intelligent vehicle considering adhesion conditions that supports the method is introduced first. Specifically, in order to better consider the impact of road adhesion conditions on intelligent vehicles, the embodiment corrects the comfortable deceleration of the IDM (Intelligent Driver Model) and the expected spacing of the IDM, so that the IDM outputs the longitudinal acceleration according to the current adhesion conditions; and introduces the adhesion coefficient into the reinforcement learning lateral lane change model, establishes a DQN (Deep Q-Learning) decision model considering the adhesion coefficient (hereinafter referred to as friction-aware DQN), constructs the state space, action space, and reward function of the model, and on this basis, constructs a rule-constrained action mask for the friction-aware DQN, and proposes a maskable friction-aware DQN decision model considering the adhesion coefficient. Each part is described in detail below.

[0026] The first is the IDM car-following model that takes the adhesion coefficient into consideration. The traditional IDM model includes an acceleration term in a free state and a deceleration term to avoid collision with the vehicle in front. It can describe static start, balanced flow state, and emergency braking state at high speed. It has the advantages of few model parameters and strong interpretability. Its expression is:

[0027] (1)

[0028] (2)

[0029] In the formula, For IDM following model The vehicle's acceleration at that moment, is the expected spacing, is the maximum acceleration, For comfortable deceleration, For IDM following model The speed of the vehicle at the moment, is the speed in free flow state, is the acceleration index, is the minimum safe stopping distance, For IDM following model The actual distance between the vehicle in front and the vehicle ahead at the time. A safe time interval.

[0030] When driving at high speed and affected by surrounding vehicles (such as on highways), when the adhesion condition is poor, the maximum deceleration (positive value) of the vehicle will be lower than the comfort deceleration (positive value), making the vehicle calculated according to the original comfort deceleration If the acceleration is too small, the acceleration output by the model will be too high, which will further increase the driving risk under poor adhesion conditions. In order to improve the adaptability of IDM to low adhesion conditions, this embodiment introduces the road adhesion coefficient in formula (2): By improving it, we get:

[0031] (3)

[0032] (4)

[0033] In the formula, is the corrected expected spacing, is the corrected comfortable deceleration, is the acceleration due to gravity. That is, the deceleration under normal circumstances can reach the comfortable deceleration (faster deceleration), but due to the small road adhesion coefficient, the comfortable deceleration cannot be achieved and can only reach (It decreases slowly).

[0034] The above-mentioned IDM following model considering the adhesion coefficient is used for longitudinal control of intelligent vehicles, while the Friction-aware DQN decision model is used for lateral control of intelligent vehicles. The main innovations of the DQN algorithm include the introduction of deep neural networks, experience replay mechanism, and separation of target networks. By approximating the Q-value function through neural networks, features can be extracted from the input high-dimensional state space, enabling the algorithm to handle more complex decision-making tasks; by adopting the experience replay mechanism, DQN can reuse historical experience data, reduce dependence on environmental interactions, and improve sample efficiency; DQN uses a dual network structure, namely the Q-value network and the target network, in which the parameters of the target network are updated regularly, reducing the fluctuation of the target value and making the training process more stable.

[0035] This embodiment reconstructs the state space, action space, reward function and network structure of the DQN model for the intelligent car driving decision-making scenario considering the adhesion condition. In a specific implementation, the state space of the DQN model can be expressed as:

[0036] (5)

[0037] In the formula, is the state variable, is the expected vehicle speed, is the speed error term, Indicates images in front of and behind the vehicle.

[0038] Optionally, the speed error can be Indicates that the vehicle is Speed ​​at the moment and The difference, that is:

[0039] (6)

[0040] In order to facilitate the decision model to capture the trend of vehicle speed changes and maintain the stability of speed changes, Can include , , The velocity error at three moments is expressed as:

[0041] (7)

[0042] The front view image and rear view image obtained by the on-board camera at the moment are and ,but It can be expressed as:

[0043] (8)

[0044] As for the action space of the DQN model, since the DQN algorithm can only handle discrete action space problems, the action space can include three actions: changing lanes to the left, keeping lanes, and changing lanes to the right.

[0045] Accordingly, this embodiment also redesigns the reward function of DQN. Specifically, the goal of training the DQN lane-changing model is to enable the vehicle to learn how to maximize the expected probability of cumulative rewards. The ideal state of vehicle driving is to have high traffic efficiency while ensuring safety, while maintaining good riding comfort. In addition, the vehicle should also travel as far as possible to the maximum round length in a round to obtain a higher mileage. In order to guide the decision model to achieve this goal, this embodiment constructs the following reward function :

[0046] (9)

[0047] In the formula, As a reward for driving efficiency, is the reward for lane changing. As a reward for maintaining a comfortable ride, As a reward for mileage, is the reward for the number of simulation steps in the episode, is the penalty item for collision. The following describes the specific forms of each item.

[0048] In order to improve the driving efficiency of the vehicle, the vehicle should be Driving, that is, keeping a small , so the following can be used :

[0049] (10)

[0050] In the formula, Indicates the preset maximum speed of the vehicle. Indicates the preset minimum speed of the vehicle. Represents the exponential function to base e.

[0051] Lane changing behavior helps improve the driving efficiency of individual vehicles, but frequent lane changing behavior will interfere with the driving status of surrounding vehicles, causing the speed of surrounding vehicles to decrease and reducing the overall traffic efficiency. Lane changing at high speeds can also easily increase the risk of collision with other vehicles. Therefore, the reward function for vehicle lane changing behavior should include a reward item for lane changing benefits and a penalty item for safety risks caused by lane changing behavior. It can be expressed as:

[0052] (11)

[0053] In the formula, , , are the acceleration gain coefficients of the main control vehicle, the vehicle behind the main control vehicle, and the vehicle behind the main control vehicle after it changes lanes to the target lane. , They respectively represent the vehicle behind the master vehicle and the vehicle behind the master vehicle after it changes lanes to the target lane. The acceleration of time, , , They respectively represent the acceleration of the master vehicle after the master vehicle completes the lane change, the vehicle behind the master vehicle, and the vehicle behind the master vehicle after the master vehicle changes lanes to the target lane.

[0054] Excessive acceleration or braking deceleration when the vehicle is driving will have a negative impact on the driving experience of the passengers in the car. In order to improve the comfort of the passengers in the car, a reward function can be designed for the acceleration of the vehicle. , whose expression is:

[0055] (12)

[0056] In the formula, express The acceleration of the vehicle (i.e. the above-mentioned master vehicle) at the moment, Indicates the vehicle's limit braking deceleration.

[0057] In order to guide the vehicle to travel as far as possible in a limited round length, a reward function is designed for the vehicle's accumulated mileage in the current round. , whose expression is:

[0058] (13)

[0059] In the formula, express The cumulative mileage of the vehicle in the current round at the moment, L represents the maximum mileage of a single round, and optionally, L=2200 meters. Furthermore, when building the model, the model performance can be verified through simulation calculations. In simulation calculations, a round can refer to a simulation round; after the model is verified, it can be applied to actual vehicle decisions on the roadside. In actual vehicle decisions, a round can refer to a time period corresponding to a simulation round. Through this round, it is possible to check whether the vehicle has traveled a long distance continuously in different time periods.

[0060] When a vehicle collides at the beginning of driving, it is difficult for the vehicle to learn useful driving strategies because the round ends too early. At the same time, maintaining safe driving for a long time in the round is a prerequisite for improving mileage. Therefore, a reward function is designed based on the number of simulation steps experienced by the vehicle in the current round. , whose expression is:

[0061] (14)

[0062] In the formula, express The number of simulation steps that the vehicle has experienced in the current round, Indicates the number of simulation steps included in a preset single round, and a decision is made in each simulation step. Optional, =90. The meaning of a round is the same as that of formula (13). In simulation calculation, and Corresponding to the number of simulation steps; in actual vehicle decision-making, Indicated in The number of decisions the vehicle has made in the current round. Indicates the number of decisions included in a preset single round.

[0063] The reward function should penalize collision accidents, thereby guiding the decision-making model to adjust its strategy during the learning process and learn how to take actions to avoid collisions, thereby improving the safety and reliability of decision-making. It can be expressed as:

[0064] (15)

[0065] It should be noted that the normal constants 3, 0.5, 0.1 and 5 in the above-mentioned reward function terms can be respectively expressed by set normal constants A, B, C and D, satisfying D>A>B>C. Different values ​​will affect the convergence speed and adjustment effect of the model. At the same time, the above-mentioned reward functions can also be calculated using other forms of expressions. Different expressions also affect the convergence speed and adjustment accuracy of the model, but ultimately a usable stable model can be obtained. The above-mentioned various transformation combinations all belong to the protection scope of this embodiment.

[0066] Based on the above state space, action space and reward function, we can construct Figure 1 The network structure shown in Figure 2 serves as a smart car driving decision model that takes into account the adhesion conditions. Figure 1 , the model first calculates Perform Min-Max normalization, and then use a feature extractor containing four layers of convolution to extract image information Extract features from , and After normalization Concatenate and pass them into the DQN model. Optionally, the feature extractor consists of a ReLU (Rectified Linear Unit) activation function, a convolution layer, and a pooling layer. The convolution kernel size is 3×3, the step size is 1, the padding is 1, and the pooling kernel size is 2×2. Optionally, DQN contains two fully connected layers. Fully connected layer 1 contains 512 neurons and the activation function is ReLU. Fully connected layer 2 contains 256 neurons and the activation function is ReLU.

[0067] Based on the above network structure, an action mask for reinforcement learning can be further constructed. Specifically, an action mask is a way to limit the actions that an agent can choose under a specific state, which can help the agent avoid choosing unsafe or unreasonable actions, thereby improving the safety and interpretability of the decision model. At each decision, the action mask dynamically shields some unfeasible or unreasonable actions according to the current state, allowing the agent to choose safer actions from the remaining valid actions. In order to reduce the interference of the lane changing action output by DQN to other surrounding vehicles and reduce the probability of collision, this embodiment adds an action mask based on the distance constraints between the main control vehicle and the vehicles before and after the current lane and the target lane to the DQN lane changing decision model.

[0068] In a specific embodiment, it is possible to define The distance between the main control vehicle and the vehicle in front of the current lane at the moment is , the distance from the vehicle behind in the current lane is , the longitudinal distance from the front vehicle in the target lane is , and the longitudinal distance from the vehicle behind the target lane is ,like Figure 2 As shown. When the distance between the two lanes is greater than the expected distance, the lane change can be performed, otherwise the lane change action cannot be performed. In addition, when the vehicle is in the leftmost lane, the action mask will block the lane change action to the left, and when the vehicle is in the rightmost lane, the action mask will block the lane change action to the right. Optionally, the constraints of the action mask are:

[0069] (16)

[0070] (17)

[0071] (18)

[0072] (19)

[0073] In the formula, is the minimum safe stopping distance, is the body length, is the time required for lane change, is the current speed of the main control vehicle, is the speed of the vehicle in front of the current lane, is the speed of the vehicle behind in the current lane, is the speed of the front vehicle in the target lane, is the speed of the vehicle behind in the target lane.

[0074] The above is the main technical content of each part of the intelligent vehicle driving decision model considering adhesion conditions within the framework of this embodiment. Based on this model, Figure 3 is a flow chart of a smart car driving decision method considering adhesion conditions provided by an embodiment of the present invention. The method is executed by an electronic device, such as Figure 3 As shown, the specific steps include:

[0075] S110, obtaining the current expected speed, speed error term, adhesion coefficient, and front and rear images of the intelligent vehicle.

[0076] S120 . Correcting the comfort deceleration in the IDM following model according to the adhesion coefficient, and generating the longitudinal acceleration of the vehicle using the corrected comfort deceleration.

[0077] Optionally, the comfortable deceleration in the IDM following model can be corrected according to formula (4), and the corrected comfortable deceleration Substituting into formula (3), we get the corrected expected distance ; then As Substituting into formula (1), we get the longitudinal acceleration .

[0078] S130, extracting image features from the front and rear images of the vehicle using a feature extractor, and inputting the image features, expected vehicle speed, speed error term, and adhesion coefficient into a DQN model.

[0079] Specifically, according to Figure 1 The network structure in the state space is first Normalize; then use the feature extractor to extract state information Extract image features ,Will After normalization Concatenate input into the DQN model.

[0080] S140: The DQN model calculates a reward value according to the corrected comfortable deceleration, and generates a lateral lane changing action of the vehicle according to the reward value and an anti-collision action mask.

[0081] The DQN model is based on the reinforcement learning algorithm. It calculates the immediate reward of each action in the action space according to formula (9) and determines the Q value of each action based on the immediate reward. Then, according to the action formulas (16)-(19) and other mask constraints, the candidate lateral lane change actions are screened and the action that satisfies all constraints and has the largest Q value is selected as the final lateral lane change action.

[0082] S150: The longitudinal speed and the final lateral lane-changing action together constitute a final action of the vehicle.

[0083] Among them, the longitudinal acceleration from the improved IDM model is used to control the longitudinal driving condition of the vehicle, and the final lateral lane changing action from the Maskable friction-aware DQN is used to control the lateral driving condition of the vehicle, jointly completing the driving decision of the intelligent car.

[0084] Furthermore, in order to verify the effectiveness of the intelligent car driving decision model and intelligent car driving decision method considering the adhesion conditions proposed in this embodiment, a simulation environment is also built for verification in this embodiment. In a specific implementation, CARLA can be used to verify the effectiveness of the model and method. CARLA is an open source autonomous driving simulation software designed for the development, training and verification of autonomous driving systems, providing a highly realistic simulation environment. The traffic environment includes urban layout, buildings, vehicles and pedestrians. The main advantage of CARLA is its flexible modular interface, which supports multiple programming languages ​​and facilitates user customization and expansion of functions. In addition, it also provides rich sensor simulations such as cameras, lidars and global positioning systems, as well as a variety of traffic scenarios and weather conditions, making the test environment more diverse and realistic.

[0085] Optionally, the simulation scenario is a straight section of a one-way three-lane highway. In addition to the master vehicle, 20 other environment vehicles are generated. The longitudinal acceleration and deceleration behaviors of all vehicles are controlled by the IDM model, the lateral lane changing behavior of the master vehicle is controlled by the DQN model, and the lateral lane changing behavior of the environment vehicles is controlled by MOBIL (Minimizing Overall Braking Induced by Lane Changes). At the initialization of each round, the target speed of the master vehicle is randomly set from 26.39-31.94 m / s (i.e. 95-115km / h), and the target speed of the environment vehicle is randomly set from 23.61-31.94m / s (i.e. 85-115km / h). At the initialization, the speed of all vehicles is set to 23.61m / s (i.e. 85km / h), and the adhesion coefficient is randomly set between [0.2, 0.4, 0.6, 0.8, 1.0]. The maximum length of a round is 90 simulation steps. The round termination condition is that the maximum round length is reached or the main control vehicle collides.

[0086] Optionally, use Python programming language and PyTorch framework, the operating system is Ubuntu 22.04.5 LTS, the processor is Intel Xeon(R) Sliver 4210R, which contains 40 processing cores and a main frequency of 2.40 GHz, and the GPU is NVIDIA GeForce RTX 4080, with 67 GB of memory and 16 GB of video memory.

[0087] Based on the above environment, this embodiment tests the model under different adhesion coefficients and target vehicle speeds, where the adhesion coefficients include five scenarios of 0.2, 0.4, 0.6, 0.8, and 1.0, and the target vehicle speeds include three scenarios of 26.39 m / s (95 km / h), 29.17 m / s (105 km / h), and 31.94 m / s (115 km / h). Different adhesion coefficients and vehicle speed combinations form 15 test scenarios, and each test scenario is tested for 300 rounds.

[0088] The evaluation indicators include average vehicle speed and collision rate. Average vehicle speed refers to the average value of vehicle speed in each simulation step of the round. Collision rate refers to the proportion of rounds in which collisions occur in this scenario. The above indicators are averaged after 300 rounds of testing.

[0089] First, the same test was performed on the friction-aware DQN decision model, MOBIL, and DQN without considering the adhesion coefficient to verify the performance of the friction-aware DQN decision model. Then, the Maskable friction-aware DQN was tested and compared with the test results of the friction-aware DQN.

[0090] The comparison results show that as the adhesion coefficient increases, the average speeds of friction-aware DQN, DQN and MOBIL all show an upward trend. MOBIL has the highest average speed, followed by friction-aware DQN, and DQN has the lowest average speed. Although MOBIL has the highest average speed, it also has the highest collision rate. Friction-aware DQN not only significantly reduces the collision rate compared to MOBIL, but also improves driving efficiency based on DQN.

[0091] By designing a rule-based action mask for the output action of reinforcement learning, the driving safety of high-speed vehicles under various adhesion conditions is further enhanced when the average vehicle speed is slightly reduced.

[0092] In summary, the existing rule-based driving decision model considering the adhesion coefficient has problems such as insufficient generalization ability in dynamic environments, focusing on longitudinal driving behavior decisions and insufficient consideration of lateral lane change decisions, and the reinforcement learning driving decision model has problems such as lack of interpretability and safety. To overcome the above problems, this embodiment proposes a reinforcement learning driving decision model based on action mask considering the adhesion coefficient. First, the comfortable deceleration of the model is corrected according to the adhesion coefficient, and then a friction-aware DQN decision model considering the adhesion coefficient is established, and the state space, action space, reward function and network structure are constructed. On this basis, an action mask based on rule constraints is constructed for the reinforcement learning driving decision model. The intelligent intelligent car driving decision method based on the model improves the safety of rule-based driving decisions and the reliability of reinforcement learning driving decisions by integrating rule-based decisions and reinforcement learning decisions, and enhances the adaptability of high-speed vehicles to different adhesion conditions.

[0093] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 4 A processor 60 is taken as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0094] The memory 61 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the intelligent vehicle driving decision method considering adhesion conditions in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 61, that is, realizing the above-mentioned intelligent vehicle driving decision method considering adhesion conditions.

[0095] The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include a memory remotely arranged relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0097] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent vehicle driving decision-making method considering adhesion conditions of any embodiment.

[0098] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0099] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0100] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0101] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A driving decision method for an intelligent vehicle considering adhesion conditions, characterized in that: include: Obtain the current expected speed, speed error term, adhesion coefficient and front and rear images of the intelligent vehicle; Correcting the comfort deceleration in the IDM following model according to the adhesion coefficient, and generating the longitudinal acceleration of the vehicle using the corrected comfort deceleration; Extracting image features from the front and rear images of the vehicle using a feature extractor, and inputting the image features, the expected vehicle speed, the speed error term, and the adhesion coefficient into a DQN model; The DQN model calculates a reward value according to the corrected comfortable deceleration, and generates a lateral lane changing action of the vehicle according to the reward value and an anti-collision action mask; The longitudinal acceleration and the lateral lane-changing action together constitute the final action of the vehicle.

2. The method according to claim 1, characterized in that: The step of correcting the comfort deceleration in the IDM following model according to the adhesion coefficient and generating the longitudinal acceleration of the vehicle using the corrected comfort deceleration includes: The comfortable deceleration in the IDM following model is corrected according to the following formula: ， in, is the corrected comfortable deceleration, is the comfortable deceleration before correction, is the adhesion coefficient, is the acceleration due to gravity; The longitudinal acceleration of the vehicle is generated according to the following formula: ， ， In the formula, For IDM following model The vehicle's acceleration at that moment, is the corrected expected spacing, is the maximum acceleration, For IDM following model The speed of the vehicle at the moment, For IDM following model The speed of the vehicle in front of you at that moment, is the speed in free flow state, is the acceleration index, is the minimum safe stopping distance, For IDM following model The actual distance between the vehicle and the vehicle ahead at the moment, A safe time interval.

3. The method according to claim 1, characterized in that The state space of the DQN model is: ， in, is the state variable, is the expected vehicle speed, is the speed error term, is the adhesion coefficient, An image showing the front and rear of the vehicle; The action space includes left lane change, lane keeping, and right lane change; The feature extractor is used to extract image features from the front and rear images of the vehicle, and the image features are input into the DQN model together with the expected vehicle speed, the speed error term, and the adhesion coefficient, including: Normalize; use feature extractor from Extract image features ,Will After normalization Concatenate input into the DQN model.

4. The method according to claim 1, characterized in that: The DQN model calculates a reward value according to the modified comfortable deceleration, including: The DQN model uses the following reward function to calculate the reward value : ， in, As a reward for driving efficiency, is the reward for lane changing. As a reward for maintaining a comfortable ride, As a reward for mileage, is the reward for the number of simulation steps in the episode, is the penalty term for collision.

5. The method according to claim 4, characterized in that ， in, express The speed of the vehicle at the moment, Indicates the expected vehicle speed, express and The difference, Indicates the preset maximum speed of the vehicle. Indicates the preset minimum speed of the vehicle. is a normal number set, is an exponential function with base e.

6. The method according to claim 4, characterized in that ， in, , , are the acceleration gain coefficients of the main control vehicle, the vehicle behind the main control vehicle, and the vehicle behind the main control vehicle after it changes lanes to the target lane. , , They represent the main control vehicle, the vehicle behind the main control vehicle, and the vehicle behind the main control vehicle after it changes lanes to the target lane. The acceleration of time, , , They respectively represent the acceleration of the master vehicle after the master vehicle completes the lane change, the vehicle behind the master vehicle, and the vehicle behind the master vehicle after the master vehicle changes lanes to the target lane. is a normal number set, is an exponential function with base e.

7. The method according to claim 4, characterized in that ， In the formula, express The vehicle's acceleration at that moment, Indicates the vehicle's limit braking deceleration. is the corrected comfortable deceleration, is the maximum acceleration, is a set normal number.

8. The method according to claim 4, characterized in that ， in, express The cumulative mileage of the vehicle in the current round at the moment, L represents the preset maximum mileage of a single round, Represents a set normal number, is an exponential function with base e; ， in, express The number of decisions the vehicle has made in the current round. Indicates the number of decisions included in a preset single round.

9. The method according to claim 1, characterized in that: The step of generating a lateral lane-changing action of the vehicle according to the reward value and the anti-collision action mask comprises: The DQN model calculates the reward value of each action in the action space according to the current state, and calculates the Q value of each action according to each reward value; According to the constraints of the following action mask, each candidate lateral lane change action is screened, and the action that meets the constraints and has the maximum Q value is used as the lateral lane change action: ， ， ， ， in, , , and They are The distance between the main control vehicle and the vehicle in front of the current lane, the distance between the main control vehicle and the vehicle behind the current lane, the longitudinal distance between the main control vehicle and the vehicle in front of the target lane, and the longitudinal distance between the main control vehicle and the vehicle behind the target lane. is the minimum safe stopping distance, is the body length, is the maximum acceleration, is the corrected comfortable deceleration, is the time required for lane change, is the current speed of the main control vehicle, is the speed of the vehicle in front of the current lane, is the speed of the vehicle behind in the current lane, is the speed of the front vehicle in the target lane, is the speed of the vehicle behind in the target lane.

10. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent vehicle driving decision-making method considering adhesion conditions as described in any one of claims 1-9.

Images