Intelligent driving decision-making method and device, equipment, storage medium and program product

Abstract

The invention provides an intelligent driving decision-making method and device, equipment, a storage medium and a program product, and relates to the field of intelligent driving. The method comprises the following steps: acquiring driving information and surrounding environment information of a current vehicle; according to the acquired driving information and surrounding environment information of the current vehicle, generating a decision instruction of the current vehicle in response to triggering a first decision condition based on a rule, and if the first decision condition based on the rule is not triggered, generating a decision instruction of the current vehicle based on a second decision condition based on reinforcement learning; and generating a torque instruction of the current vehicle according to the decision instruction. The method can adapt to intelligent driving decisions of complex and changeable real scenes, and the intelligence, safety and real-time performance of the system are balanced.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and more particularly to an intelligent driving decision-making method, device, equipment, storage medium, and program product. Background Technology

[0002] Intelligent driving is a complex system that integrates environmental perception, decision-making and planning, and dynamic control. Decision-making methods are mainly divided into two major directions: rule-based and reinforcement learning-based.

[0003] Rule-based decision-making methods require extensive manual rule-making, which involves defining vehicle behavior patterns for various scenarios. However, driving scenarios are inherently uncertain, and vehicles themselves exhibit nonlinearity, making it difficult to adapt to dynamically changing traffic environments. This approach neglects global efficiency and safety, easily leading to local optima. In contrast, learning-based decision-making methods, such as deep reinforcement learning, can utilize neural networks to achieve end-to-end control from sensor input to action output. However, multi-agent systems need to learn strategies for all possible scenarios, making the training process extremely complex and difficult to converge. The resulting strategies are often local optima among multiple competing objectives, failing to achieve optimal performance in specific scenarios (such as emergency braking, economical cruising, and congested following).

[0004] In summary, the decision-making logic in existing technologies is difficult to adapt to complex and ever-changing real-world scenarios, and it is difficult to balance the intelligence, security, and real-time performance of the system. Summary of the Invention

[0005] This application provides an intelligent driving decision-making method, device, equipment, storage medium, and program product to improve the decision-making logic to better adapt to complex real-world scenarios and balance intelligence with safety and real-time performance.

[0006] In a first aspect, embodiments of this application provide an intelligent driving decision-making method, including: acquiring current vehicle driving information and surrounding environment information;

[0007] Based on the obtained driving information of the current vehicle and the surrounding environment information, a decision instruction for the current vehicle is generated in response to the triggering of the first decision condition based on rules. If the first decision condition is not triggered, a decision instruction for the current vehicle is generated based on the second decision condition based on reinforcement learning.

[0008] The torque command for the current vehicle is generated based on the decision-making instructions.

[0009] In one possible implementation, the first decision condition includes a safety gating decision condition and a scenario selection decision condition. The above-mentioned generation of a decision instruction for the current vehicle in response to triggering the rule-based first decision condition, based on the acquired driving information of the current vehicle and surrounding environment information, includes:

[0010] Based on the acquired driving information of the current vehicle and the surrounding environment information, a decision command for the current vehicle is generated in response to the triggering of the safety gating decision condition.

[0011] Otherwise, based on the obtained driving information of the current vehicle and the surrounding environment information, a decision instruction for the current vehicle is generated in response to the triggering scenario selection decision conditions.

[0012] In one possible implementation, generating the current vehicle torque command based on the decision instruction includes:

[0013] Match the target agent corresponding to the decision instruction;

[0014] Generate torque commands for the current vehicle using the target intelligent agent.

[0015] In one possible implementation, the above-mentioned generation of the torque command for the current vehicle using the target intelligent agent includes:

[0016] Determine the candidate torque commands for the current vehicle and calculate the reward value of the candidate torque commands based on the reward function;

[0017] Based on the reward value of the candidate torque commands, the optimal torque command for the current vehicle is generated.

[0018] In one possible implementation, generating the optimal torque command for the current vehicle based on the reward value of the candidate torque command includes:

[0019] The candidate torque command with the highest reward value is determined as the optimal torque command for the current vehicle.

[0020] In one possible implementation, the reward function described above is determined based on the reward items corresponding to the factors considered, the weight coefficients, and the penalty coefficients.

[0021] Secondly, embodiments of this application provide an intelligent driving decision-making device, comprising:

[0022] The acquisition module is used to acquire the current vehicle's driving information and surrounding environment information;

[0023] The first processing module is used to generate a decision instruction for the current vehicle based on the obtained driving information of the current vehicle and the surrounding environment information, in response to the triggering of a rule-based first decision condition. If the first decision condition is not triggered, a decision instruction for the current vehicle is generated based on a second decision condition based on reinforcement learning.

[0024] The second processing module is used to generate the torque command for the current vehicle based on the decision instructions.

[0025] Thirdly, embodiments of this application provide an intelligent driving decision-making device, including: a memory and a processor;

[0026] The memory stores the instructions that the computer executes;

[0027] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0028] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor or the like, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0029] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor or the like, implements the first aspect and / or various possible implementations of the first aspect.

[0030] Sixthly, embodiments of this application provide a vehicle, including a vehicle body and an intelligent driving decision-making device as described in the third aspect above.

[0031] This application provides an intelligent driving decision-making method, device, equipment, storage medium, and program product. During the operation of an intelligent driving vehicle, by collecting the vehicle's driving information and surrounding environmental information, it first makes a judgment based on a rule-based first decision condition. If the first decision condition is triggered, the current vehicle's decision command is directly output. In this case, rule-based decision-making can prioritize identification and decision-making in the first instance, resulting in better reliability. Secondly, if the first decision condition is not triggered, it indicates that the vehicle's driving state is good. In this case, a second decision condition based on reinforcement learning is used to output the corresponding vehicle's decision command, resulting in better intelligence. Therefore, this application can better handle decision-making in complex and ever-changing real-world scenarios, and better balance intelligence, safety, and real-time performance. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0033] Figure 1 A schematic diagram of an intelligent driving scenario provided in an embodiment of this application;

[0034] Figure 2 A flowchart illustrating the intelligent driving decision-making method provided in the embodiments of this application. Figure 1 ;

[0035] Figure 3An application system block diagram of the intelligent driving decision-making method provided in the embodiments of this application;

[0036] Figure 4 A flowchart illustrating the intelligent driving decision-making method provided in the embodiments of this application. Figure 2 ;

[0037] Figure 5 This is a schematic diagram of the structure of the intelligent driving decision-making device provided in the embodiments of this application;

[0038] Figure 6 This is a schematic diagram of the structure of the intelligent driving decision-making device provided in the embodiments of this application.

[0039] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0041] Figure 1 This is a schematic diagram of an intelligent driving scenario provided in an embodiment of this application. Figure 1 As shown, the specific application scenario of this application is in the field of intelligent driving. In intelligent driving vehicle control, three steps are typically required: first, relevant information is collected through environmental perception; second, operational instructions are determined through decision planning; and third, the specific driving behavior of the vehicle is executed through power control. Decision planning is the central system of the entire intelligent driving system. Existing technologies usually adopt a rule base based on human presets (such as traffic regulations and driving experience), and realize decision-making through the logic of "condition judgment - action output". Essentially, it encodes human driving knowledge into executable rules. There are also methods based on massive amounts of data (simulation data, real vehicle road test data), which automatically learn decision-making rules through reinforcement learning models without manual coding. However, both of these methods have certain limitations and cannot balance the decision-making needs in complex scenarios.

[0042] Based on the above problems, this application provides an intelligent driving decision-making scheme, which makes decisions in layers according to the intelligent driving scenario. First, decisions are made through rule-based means. If a rule is triggered, the corresponding decision command is directly output. If no rule is triggered, the corresponding decision command is generated through reinforcement learning. This solves the problem that the decision logic in the prior art is difficult to adapt to complex and ever-changing real-world scenarios, and it is difficult to achieve a good balance between the intelligence, safety, and real-time performance of the system.

[0043] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0044] Figure 2 A flowchart illustrating the intelligent driving decision-making method provided in the embodiments of this application. Figure 1 .like Figure 2 As shown, the method includes:

[0045] S201. Obtain the current vehicle's driving information and surrounding environment information.

[0046] In this step, driving information refers to the current operating status, attitude, and torque commands of the vehicle itself, such as speed, acceleration, acceleration derivative (jerk), and heading angle, which can be directly obtained through the vehicle bus, inertial measurement unit (IMU), positioning system, etc.

[0047] Surrounding environment information refers to information covering the roads, obstacles, traffic participants, and traffic rules around the current vehicle, such as distance to the vehicle in front, lane lines, road curvature, slope, and traffic lights. This information can be obtained through multi-sensor fusion (LiDAR, cameras, millimeter-wave radar, ultrasonic radar).

[0048] S202. Based on the obtained driving information of the current vehicle and the surrounding environment information, generate a decision instruction for the current vehicle in response to the triggering of the first decision condition based on rules. If the first decision condition is not triggered, generate a decision instruction for the current vehicle based on the second decision condition based on reinforcement learning.

[0049] In this step, the current vehicle driving information and surrounding environment information obtained in step S201 are input into the decision layer for judgment. The judgment conditions include a rule-based first decision condition and a reinforcement learning-based second decision condition. The rule-based first decision condition is the highest priority and most critical trigger condition in the decision system, usually for safety and compliance scenarios or core driving tasks, such as emergency obstacle avoidance and cruise control. In such cases, if the trigger condition is met, the corresponding decision instruction is directly generated. For example, if the collision time of the obstacle ahead is determined to be less than a safety threshold based on the current vehicle driving information and surrounding environment information, an emergency obstacle avoidance instruction will be directly generated. The reinforcement learning-based second decision condition is a supplementary decision condition with a lower priority than the rule-based first decision condition. For example, a comfort optimization condition, without triggering the rule-based decision condition, outputs recommended modes such as "comfort," "sport," and "economy" based on the current vehicle driving information and surrounding environment information.

[0050] S203. Generate the torque command for the current vehicle based on the decision command.

[0051] In this step, a decision instruction is obtained by executing step S202. The decision instruction is then used to generate executable instructions for the current vehicle. For example, if the decision instruction is to avoid an obstacle, then a torque instruction for the vehicle's obstacle avoidance torque is generated.

[0052] This application provides an intelligent driving decision-making method. During the operation of an intelligent driving vehicle, by collecting the vehicle's driving information and surrounding environmental information, it first makes a judgment based on a rule-based first decision condition. If the first decision condition is triggered, the current vehicle's decision command is directly output. In this case, rule-based decision-making can prioritize identification and decision-making in the first instance, resulting in better reliability. Secondly, if the first decision condition is not triggered, it indicates that the vehicle's driving state is good. In this case, a second decision condition based on reinforcement learning is used to output the corresponding vehicle's decision command, resulting in better intelligence. Therefore, this application can better handle decision-making in complex and ever-changing real-world scenarios, and better balance intelligence, safety, and real-time performance.

[0053] Based on the above, one possible implementation is as follows: Figure 3As shown, the vehicle is equipped with a perception layer and a decision layer. The perception layer collects driving information and environmental data, which is then input into the decision layer. The decision layer sets a first decision condition based on rules for safety scenarios and a second decision condition based on reinforcement learning for comfort scenarios. First, the real-time collected information is evaluated based on the first decision condition. If the first decision condition is triggered, a decision command is directly output, generating the vehicle's torque command. This allows for rapid response to safety control needs, avoiding delays and the instability of artificial intelligence. If the first decision condition is not triggered, the second decision condition based on reinforcement learning is used, recommending a suitable driving mode through an intelligent agent. This reinforcement learning approach ensures continued intelligence. Therefore, by deeply integrating rules to ensure safety and learning to optimize performance, the system achieves a balance between intelligence, safety, and real-time performance.

[0054] In one specific implementation, the above-mentioned generation of the current vehicle torque command based on the decision instruction includes:

[0055] Based on the decision instruction, match the target agent corresponding to the decision instruction;

[0056] Generate torque commands for the current vehicle using the target intelligent agent.

[0057] In this implementation, multiple intelligent agents can be deployed in the vehicle, such as an emergency obstacle avoidance agent, an economy cruise agent, a comfort following agent, and a predictive parking agent, each performing a single function. (Reference) Figure 3 The vehicle also incorporates a multi-agent function selection layer. After the decision-making layer determines the specific decision command, the multi-agent function selection layer further selects an agent that matches the decision command as the target agent. This target agent can focus on a single scenario, generating torque commands based on real-time driving information and surrounding environmental information, and then sending them to the corresponding vehicle actuators for execution. These actuators include, for example, drive motors and braking systems, enabling precise control of the vehicle. For instance, if the decision command is emergency obstacle avoidance, the emergency obstacle avoidance agent is activated. This agent can output different torques in different scenarios. For example, when facing an emergency braking obstacle avoidance on a straight road, its torque target is to quickly provide maximum braking torque to shorten the braking distance. When avoiding obstacles on a curve, it needs to coordinate steering torque and drive torque to prevent the vehicle from skidding.

[0058] Optionally, all other agents besides the target agent are in a dormant state to reduce computational power consumption. The selected target agent is activated, receives detailed state information, and outputs the optimal torque command through forward computation via a neural network based on a reward function specific to the scenario.

[0059] Furthermore, still referencing Figure 3 The information collected by the perception layer can also include the driver's intentions, such as selecting a mode, accelerating, and steering, which serve as the information basis for the decision layer to generate decision instructions.

[0060] In this embodiment, the torque command of the vehicle is generated by multiple intelligent agents, which can bring out the ultimate performance in this scenario. At the same time, the modular design is compatible with scalability. Without changing the existing decision-making logic, other intelligent agents can be easily added to enhance the system's adaptability and scalability.

[0061] In one specific implementation, the above-mentioned generation of the torque command of the current vehicle using the target intelligent agent may further include: using the target intelligent agent to perform the following process to generate the torque command of the current vehicle: determining candidate torque commands of the current vehicle, calculating the reward value of the candidate torque commands based on the reward function; and generating the optimal torque command of the current vehicle based on the reward value of the candidate torque commands.

[0062] In this implementation, firstly, the target agent can generate a batch of candidate torque commands that meet basic constraints based on the current vehicle's driving information and surrounding environment information. These candidate torque commands cover different magnitudes of driving torque, braking torque, and steering assist torque. Secondly, a dedicated reward function is constructed based on the target agent's core objective. Finally, for each candidate torque command, the vehicle state changes after its execution are simulated, and the reward value is calculated by substituting it into the reward function. The optimal torque command is then selected based on the reward value.

[0063] In this embodiment, by introducing a reward function, different reward functions are customized for different scenarios, thereby guiding the agent to make optimal decisions.

[0064] In one specific implementation, the reward function can be determined based on the reward items corresponding to the considered factors, weight coefficients, and penalty coefficients. As an example, the reward function can be defined as a weighted multi-objective reward function:

[0065]

[0066] In the above formula: R represents the total reward; The weighting coefficients, where n takes values ​​of 1, 2, 3, 4, ..., are used to balance the importance of different considerations (i.e., reward items), and typically satisfy ∑ =1, and ≥0; Factors to consider can include safety factors, comfort factors, energy consumption factors, and follow-through performance factors, etc., correspondingly, The reward items corresponding to safety factors are called safety items, which are used to measure the safety of vehicle behavior, such as avoiding collisions, maintaining a safe distance, and obeying traffic rules. The corresponding reward items for comfort factors, namely comfort items, are used to measure the passenger's comfort experience, such as reducing violent operations such as sudden acceleration, sudden braking, and sharp turns; The reward items corresponding to energy consumption factors, namely energy items, are used to measure the efficiency of energy consumption, such as reducing fuel or battery consumption and encouraging smooth driving. This is a reward item corresponding to the performance factor, used to measure the vehicle's following accuracy to a reference path or reference speed, such as path following error, speed error, etc.

[0067] For example, in an emergency obstacle avoidance agent, its primary function is to handle unexpected scenarios, such as sudden braking by the vehicle in front or a vehicle forcibly cutting in. The goal is to avoid collisions; comfort and fuel economy are ignored or given very little weight. In other words, the focus is on... The design is as follows:

[0068]

[0069] in This is an indicator function representing a collision distance less than a threshold. Its value is 1 when the condition is true, indicating a dangerous collision scenario. Once this occurs, a large fixed penalty (-1000) is imposed, which typically results in extremely low cumulative rewards for the entire episode, strongly incentivizing the agent to avoid this situation. TTC is the collision time. It is indicated that as TTC decreases, the penalty increases dramatically and non-linearly, incentivizing the agent to take increasingly decisive avoidance actions when danger approaches.

[0070] For example, in an economic cruise agent, the primary tasks are highway cruising and cruising on open suburban roads. The core objective is to maximize energy recovery efficiency and significantly improve driving range while ensuring absolute safety. Correspondingly, the reward items in the above reward function are as follows:

[0071]

[0072] in, This represents the dynamically calculated safe distance from the vehicle in front. This represents the actual distance to the vehicle in front. We will consider both our own vehicle speed and the speed of the vehicle in front. Used to determine if TTC is less than If yes, then I takes the value 1; otherwise, I takes the value 0, where TTC is the collision time. The safe collision threshold is defined. An exponential penalty term is used for distance to ensure that the penalty increases sharply when the distance to the vehicle is much smaller than the safe distance, strongly encouraging the agent to maintain a safe following distance; a penalty coefficient is also designed. This indicates that the penalty for dangerous TTCs is much greater than the distance penalty.

[0073]

[0074] Where 'a' is the current vehicle acceleration and 'jerk' is the derivative of the current vehicle acceleration. The design uses negative values ​​to penalize 'a' and 'jerk', ensuring smoothness during economical cruising and avoiding unpleasant jerking sensations in pursuit of fuel economy. The weight of the jerk is relatively high because passengers are more sensitive to the jerk.

[0075]

[0076] in This is a scaling factor used to adjust the energy value to match the order of magnitude of other terms in the reward function; The energy actually recovered and stored in the battery within a time step will take into account both charging power and motor recovery power. To consider the adaptive weight function of SOC, it is designed as follows: , The midpoint of the SOC where the weights begin to decrease significantly. It is a parameter that controls the steepness of the transition. It's a sigmoid function that provides a smoother weight transition. When the SOC is low (e.g., 0.2), the weight is close to 1, strongly encouraging recycling; when the SOC is close to SOC_mid, the weight is approximately 0.5; and when the SOC is high (e.g., 0.9), the weight is close to 0, avoiding ineffective recycling. This smooth transition is more consistent with practical battery energy management strategies than a piecewise linear function and is also more conducive to agent learning.

[0077]

[0078] in Ideal speed; This is a weighting factor. It encourages the vehicle to stay close to the set ideal speed, but in this mode, the weight is usually set relatively small, allowing the speed to fluctuate within a small range to find the optimal energy recovery point.

[0079] The reward function described above can be used to calculate reward values ​​for different torques, and then the optimal torque command can be determined based on calibration or custom filtering rules.

[0080] In the embodiments of this application, the original design of each reward item in the reward function, such as the adaptive weight in the energy item, the jerk threshold penalty in the comfort item, and the nonlinear TTC penalty in the safety item, guides the agent not only to learn how to complete the task, but also to learn how to complete the task in the optimal way.

[0081] In one specific implementation, generating the optimal torque command for the current vehicle based on the reward value of the candidate torque commands may include: determining the candidate torque command with the largest reward value as the optimal torque command for the current vehicle.

[0082] In this embodiment, the reward values ​​of each candidate torque command are sorted by magnitude. Following the principle that the higher the reward value, the better the corresponding candidate torque command, the final torque command is selected from the sorted candidate torques. For example, the candidate torque command corresponding to the largest or second largest reward value can be selected as the optimal torque command from the sorted candidate torques; alternatively, the candidate torque command corresponding to a preset reward value condition can be selected as the optimal torque command from the sorted candidate torques. The preset reward value condition can be determined based on historical experience or safe driving factors.

[0083] Optionally, the first decision condition may further include a safety gating decision condition and a scenario selection decision condition. The above-mentioned generation of a decision instruction for the current vehicle in response to triggering the rule-based first decision condition, based on the acquired driving information and surrounding environment information, includes: generating a decision instruction for the current vehicle in response to triggering the safety gating decision condition based on the acquired driving information and surrounding environment information; otherwise, generating a decision instruction for the current vehicle in response to triggering the scenario selection decision condition based on the acquired driving information and surrounding environment information.

[0084] The following section, in conjunction with the accompanying diagram, explains the first decision conditions, including the security gating decision conditions and the scenario selection decision conditions.

[0085] Figure 4 A flowchart illustrating the intelligent driving decision-making method provided in the embodiments of this application. Figure 2 .like Figure 4 As shown, the following steps are performed based on the above embodiments:

[0086] S301. Obtain the current vehicle's driving information and surrounding environment information.

[0087] In this step, the method for obtaining the current vehicle's driving information and surrounding environment information is the same as the method described above, and will not be repeated here.

[0088] S302. Based on the obtained current vehicle driving information and surrounding environment information, determine whether the safety gate control decision conditions are triggered.

[0089] If step S303 is not executed, then step S304 is executed to generate the decision instruction for the current vehicle.

[0090] In this step, the first decision condition is further divided into safety gating decision conditions and scenario selection decision conditions. Safety gating decision conditions are used for emergency obstacle avoidance commands, and their core purpose is to deal with sudden obstacles (such as pedestrians suddenly appearing out of nowhere, vehicles cutting in, objects falling from the road, etc.) and static obstacles to avoid collisions. They have the highest priority. First, the safety gating decision condition is judged. If triggered, the current vehicle's decision command is directly output; if not triggered, the judgment on whether the scenario selection decision condition is triggered continues.

[0091] S303. Determine whether the scenario selection decision condition has been triggered.

[0092] If step S305 is not executed, then step S304 is executed to generate the decision instruction for the current vehicle.

[0093] In this step, the scenario selection decision conditions are based on most driving conditions, including urban roads, highways, and industrial parks, encompassing scenarios such as turning, pre-stopping, following, and cruising. Next, it is determined whether the scenario selection decision conditions are triggered. If triggered, the current vehicle's decision command is directly output; otherwise, step S305 is executed.

[0094] S305. Generate decision instructions for the current vehicle based on the second decision condition of reinforcement learning.

[0095] In this step, feature vectors can be formed by considering factors such as SOC (State of Charge), forward slope, and driver intent. Lightweight GBDT (Gradient Boosting Decision Trees) models like LightGBM can be used to output recommended modes such as "Comfort," "Sport," and "Economy."

[0096] S306. Generate the torque command for the current vehicle based on the decision command.

[0097] In this step, based on the decision instructions determined in the preceding steps, the decision instructions are further used to generate executable instructions for the current vehicle, such as torque instructions for emergency obstacle avoidance.

[0098] When the vehicle's status changes, a new round of perception data enters the system, forming a closed-loop control.

[0099] In this embodiment, by further distinguishing the first decision conditions, the judgment order can be set according to the level of security requirements, which better ensures the reliability of security-related controls.

[0100] Figure 5 This is a schematic diagram of the intelligent driving decision-making device provided in an embodiment of this application. Figure 5As shown, the intelligent driving decision-making device 40 provided in this application embodiment includes:

[0101] The acquisition module 401 is used to acquire the current vehicle's driving information and surrounding environment information;

[0102] The first processing module 402 is used to generate a decision instruction for the current vehicle in response to triggering a rule-based first decision condition based on the obtained driving information of the current vehicle and surrounding environment information. If the first decision condition is not triggered, a decision instruction for the current vehicle is generated based on a second decision condition based on reinforcement learning.

[0103] The second processing module 403 is used to generate the torque command for the current vehicle based on the decision command.

[0104] Based on the above embodiments, the first decision condition includes a security gating decision condition and a scenario selection decision condition. The first processing module 402 is further configured to:

[0105] Based on the acquired driving information of the current vehicle and the surrounding environment information, a decision command for the current vehicle is generated in response to the triggering of the safety gating decision condition.

[0106] Otherwise, based on the obtained driving information of the current vehicle and the surrounding environment information, a decision instruction for the current vehicle is generated in response to the triggering scenario selection decision conditions.

[0107] Optionally, the second processing module 403 is specifically used for: matching the target intelligent agent corresponding to the decision instruction according to the decision instruction; and generating the torque instruction of the current vehicle using the target intelligent agent.

[0108] Optionally, the second processing module 403 is further configured to: determine the candidate torque command of the current vehicle, and calculate the reward value of the candidate torque command based on the reward function;

[0109] Based on the reward value of the candidate torque commands, the optimal torque command for the current vehicle is generated.

[0110] Optionally, the second processing module 403 is also used to: determine the candidate torque command with the largest reward value as the optimal torque command for the current vehicle.

[0111] Optionally, the above reward function is determined based on the reward items, weight coefficients, and penalty coefficients corresponding to the factors considered.

[0112] The intelligent driving decision-making device provided in this application embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0113] Figure 6 This is a schematic diagram of the structure of the intelligent driving decision-making device provided in an embodiment of this application. Figure 6As shown, the intelligent driving decision-making device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the intelligent driving decision-making device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus.

[0114] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0115] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0116] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0117] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0118] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0119] This application also provides a vehicle, including a vehicle body and an intelligent driving decision-making device as described above.

[0120] This application also provides a computer program product, including a computer program that, when executed by a processor or the like, implements the above-described method.

[0121] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor or the like, implement the above-described method.

[0122] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0123] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0124] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0127] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0128] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0129] Finally, it should be noted that other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and alterations may be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An intelligent driving decision-making method, characterized in that, Obtain current vehicle driving information and surrounding environment information; Based on the driving information and the surrounding environment information, a decision instruction for the current vehicle is generated in response to triggering a rule-based first decision condition; if the first decision condition is not triggered, a decision instruction for the current vehicle is generated based on a reinforcement learning-based second decision condition. The torque command for the current vehicle is generated based on the decision command.

2. The method according to claim 1, characterized in that, The first decision condition includes a safety gating decision condition and a scene selection decision condition. The step of generating a decision instruction for the current vehicle in response to triggering the rule-based first decision condition, based on the driving information and the surrounding environment information, includes: Based on the driving information and the surrounding environment information, a decision command for the current vehicle is generated in response to triggering a safety gating decision condition; Otherwise, based on the driving information and the surrounding environment information, a decision instruction for the current vehicle is generated in response to the triggering scenario selection decision conditions.

3. The method according to claim 1 or 2, characterized in that, The step of generating the current vehicle torque command based on the decision command includes: Match the target agent corresponding to the decision instruction according to the decision instruction; Generate torque commands for the current vehicle using the target intelligent agent.

4. The method according to claim 3, characterized in that, The step of generating the torque command for the current vehicle using the target intelligent agent includes: The target agent executes the following process to generate the torque command for the current vehicle: Determine the candidate torque commands for the current vehicle and calculate the reward value of the candidate torque commands based on the reward function; Based on the reward value of the candidate torque commands, the optimal torque command for the current vehicle is generated.

5. The method according to claim 4, characterized in that, The step of generating the optimal torque command for the current vehicle based on the reward value of the candidate torque commands includes: The candidate torque command with the highest reward value is determined as the optimal torque command for the current vehicle.

6. The method according to claim 4, characterized in that, The reward function is determined based on the reward items corresponding to the factors considered, the weight coefficients, and the penalty coefficients.

7. An intelligent driving decision-making device, characterized in that, include: The acquisition module is used to acquire the current vehicle's driving information and surrounding environment information; The first processing module is configured to generate a decision instruction for the current vehicle in response to triggering a rule-based first decision condition based on the driving information and the surrounding environment information; if the first decision condition is not triggered, generate a decision instruction for the current vehicle based on a second decision condition based on reinforcement learning. The second processing module is used to generate the torque command for the current vehicle based on the decision command.

8. An intelligent driving decision-making device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed, implements the method according to any one of claims 1-6.

Images