Intelligent driving implementation method and related equipment

Abstract

The invention discloses an intelligent driving implementation method and related equipment. The embodiment of the invention can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. In the embodiment of the invention, a traffic simulation environment is constructed, and a plurality of traffic agents with different behavior types are configured in the traffic simulation environment; selecting a target traffic agent under the behavior type from the traffic simulation environment; performing confrontation training on the target traffic agent and the reference traffic agent in the traffic simulation environment to update the agent control strategy of the behavior type; and through the updated intelligent agent control strategy of each behavior type, driving guidance is carried out on a vehicle in a real road scene. In the embodiment of the invention, through the updated agent control strategy of each behavior type, the vehicle in the real road scene can be guided to flexibly cope with various traffic scenes. According to the scheme, the accuracy and timeliness of coping with a complex traffic scene by the vehicle can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, in particular to an intelligent driving implementation method and related equipment. The related equipment can include an intelligent driving implementation apparatus, an electronic device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] An intelligent driving system refers to a system that uses advanced technology to enable vehicles to drive autonomously. It combines sensors (such as radar, lidar, and cameras), artificial intelligence, and machine learning algorithms to perceive the environment, make decisions, and control the vehicle, with the goal of improving safety, efficiency, and driving experience. Intelligent driving systems are generally divided into different automation levels, from assisted driving to fully autonomous driving.

[0003] Current intelligent driving systems mainly rely on pre-set strategies, which are developed and updated based on common traffic scenarios and engineering experience. However, since traffic scenarios are complex and diverse, these strategies cover limited traffic scenarios, making it difficult for current intelligent driving systems to respond flexibly when dealing with complex traffic scenarios. SUMMARY

[0004] Embodiments of the present application provide an intelligent driving implementation method and related equipment, which can include an intelligent driving implementation apparatus, an electronic device, a computer readable storage medium, and a computer program product, which can improve the accuracy and timeliness of vehicles responding to complex traffic scenarios.

[0005] Embodiments of the present application provide an intelligent driving implementation method, which includes:

[0006] Obtaining traffic data in a real road scene, the traffic data including traffic data of a plurality of traffic participants;

[0007] Performing clustering analysis on the traffic data of the plurality of traffic participants to obtain at least one traffic agent of a behavior type;

[0008] Constructing a traffic simulation environment for training the traffic agent, and configuring a plurality of traffic agents of different behavior types in the traffic simulation environment;

[0009] For each behavior type, selecting a target traffic agent of the behavior type from the traffic simulation environment;

[0010] Based on the current agent control strategy of the behavior type, performing adversarial training on the target traffic agent and a reference traffic agent in the traffic simulation environment to obtain adversarial information, and updating the agent control strategy of the behavior type according to the adversarial information, the reference traffic agent being a traffic agent in the traffic simulation environment;

[0011] By updating the intelligent agent control strategy for each behavior type, driving guidance is provided for vehicles in real road scenarios.

[0012] This application embodiment also provides an intelligent driving implementation device, including:

[0013] The acquisition unit is used to acquire traffic data in real road scenarios, including traffic data from multiple traffic participants.

[0014] Clustering units are used to perform cluster analysis on traffic data from multiple traffic participants to obtain traffic intelligent agents with at least one behavioral type.

[0015] A configuration unit is built to construct a traffic simulation environment for training traffic agents, and to configure multiple traffic agents with different behavior types in the traffic simulation environment.

[0016] The agent selection unit is used to select the target traffic agent for each behavior type from the traffic simulation environment.

[0017] The strategy update unit is used to conduct adversarial training between the target traffic agent and the reference traffic agent in a traffic simulation environment based on the current agent control strategy of the behavior type, to obtain adversarial information, and to update the agent control strategy of the behavior type according to the adversarial information. The reference traffic agent is the traffic agent in the traffic simulation environment.

[0018] The vehicle guidance unit is used to provide driving guidance to vehicles in real road scenarios through updated intelligent agent control strategies for various behavior types.

[0019] In some embodiments, the apparatus further includes a cooperative countermeasure unit for:

[0020] By analyzing the adversarial interactions between various traffic agents in a traffic simulation environment, cooperative traffic agents are selected from among them.

[0021] Based on the current agent control strategies of the cooperative traffic agent and the target traffic agent, adversarial training is conducted on the cooperative traffic agent and the target traffic agent in a traffic simulation environment to obtain cooperative adversarial information.

[0022] Based on the cooperative adversarial information, the current agent control strategy of the cooperative traffic agent is updated.

[0023] In some embodiments, the cooperative adversarial unit is configured to:

[0024] The coordinated traffic intelligent agent is triggered to identify policy vulnerabilities of the target traffic intelligent agent through the current intelligent agent control strategy, and obtain information on policy vulnerabilities to be countered.

[0025] The collaborative traffic agent is triggered to conduct adversarial training with the target traffic agent in a traffic simulation environment based on the vulnerability information of the strategy to be challenged, thereby obtaining collaborative adversarial information.

[0026] In some embodiments, the cooperative adversarial unit is configured to:

[0027] Determine the alliance of traffic intelligent agents in a traffic simulation environment;

[0028] By observing the adversarial interactions between the various traffic intelligent agents in the traffic intelligent agent alliance, we can obtain information on the alliance adversarial results among the traffic intelligent agents in the traffic intelligent agent alliance.

[0029] Based on the results of the alliance confrontation, cooperative traffic intelligent agents are selected from the traffic intelligent agents in the traffic intelligent agent alliance.

[0030] In some embodiments, the apparatus further includes a training adversarial unit for:

[0031] Randomly select a training traffic intelligent agent from among the traffic intelligent agents in the Traffic Intelligent Agent Alliance to be a collaborating traffic intelligent agent;

[0032] Based on the current agent control strategies of the cooperative traffic agent and the training traffic agent, adversarial training is conducted on the cooperative traffic agent and the training traffic agent in a traffic simulation environment to obtain training adversarial information.

[0033] Based on the training adversarial information, the current agent control strategies of the cooperative traffic agent and the training traffic agent are updated.

[0034] In some embodiments, the adversarial unit is trained for:

[0035] Identify the target traffic intelligent agent and the adversary traffic intelligent agent;

[0036] Based on the current agent control strategies of the adversary traffic agent and the target traffic agent, adversarial training is conducted on the adversarial traffic agent and the target traffic agent in a traffic simulation environment to obtain defensive adversarial information.

[0037] Based on the defense and confrontation information, the current agent control strategy of the adversary traffic agent and the current agent control strategy of the target traffic agent are updated.

[0038] In some embodiments, the adversarial unit is trained for:

[0039] By observing the adversarial interactions between various traffic intelligent agents in a traffic simulation environment, information on the adversarial outcomes of each agent is obtained.

[0040] Based on the adversarial outcome information, adversarial traffic agents are identified from among the various traffic agents.

[0041] In some embodiments, the adversarial unit is trained for:

[0042] The adversarial traffic agent identifies policy vulnerabilities of the target traffic agent through the current agent control strategy and obtains policy vulnerability information to be referenced.

[0043] The adversarial traffic agent adjusts its current agent control strategy based on the vulnerability information of the policy to be referenced, thus obtaining the adjusted agent control strategy of the adversarial traffic agent.

[0044] Based on the adjusted agent control strategy of the adversarial traffic agent and the current agent control strategy of the target traffic agent, adversarial training is conducted on the adversarial traffic agent and the target traffic agent in a traffic simulation environment to obtain defensive adversarial information.

[0045] In some embodiments, the vehicle guidance unit is configured to:

[0046] Collect real-time traffic information of the actual road scene where the target vehicle is located;

[0047] Based on real-time traffic information, select the target traffic agent control strategy from the updated agent control strategies for each behavior type.

[0048] The target traffic intelligent agent control strategy provides driving guidance to target vehicles in real road scenarios.

[0049] In some embodiments, the policy update unit is configured to:

[0050] Configure reward strategy information for traffic intelligent agents of various behavior types;

[0051] Based on the current intelligent agent control strategy according to the behavior type, adversarial training is carried out between the target traffic intelligent agent and the reference traffic intelligent agent in a traffic simulation environment to obtain adversarial information.

[0052] Based on the adversarial information and reward strategy information, determine the reward value information of the target traffic intelligent agent;

[0053] Update the agent control strategy based on the reward value information of the target traffic agent.

[0054] In some embodiments, the policy update unit is configured to:

[0055] Based on the current agent control strategy of the behavior type, when conducting adversarial training between the target traffic agent and the reference traffic agent in the traffic simulation environment, the current simulation environment information of the traffic simulation environment is obtained.

[0056] Based on the current simulation environment information, the interaction behavior between the target traffic agent and the reference traffic agent is evaluated and processed to obtain adversarial information.

[0057] In some embodiments, the vehicle guidance unit is configured to:

[0058] Obtain strategy evaluation metrics;

[0059] Based on the policy evaluation index, the updated agent control policy for each behavior type is evaluated to obtain the policy evaluation information of the updated agent control policy for each behavior type.

[0060] Based on the policy evaluation information, the updated agent control policies for each behavior type are screened to obtain the screened agent control policies.

[0061] The selected intelligent agent control strategy provides driving guidance for vehicles in real road scenarios.

[0062] This application also provides an electronic device, including a processor and a memory, wherein the memory stores multiple instructions; the processor loads instructions from the memory to execute steps in any of the intelligent driving implementation methods provided in this application.

[0063] This application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute steps in any of the intelligent driving implementation methods provided in this application.

[0064] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in any of the intelligent driving implementation methods provided in this application.

[0065] This application embodiment can acquire traffic data from real road scenarios, including traffic data from multiple traffic participants; perform cluster analysis on the traffic data of multiple traffic participants to obtain traffic agents of at least one behavior type; construct a traffic simulation environment for training traffic agents, and configure multiple traffic agents of different behavior types in the traffic simulation environment; for each behavior type, select a target traffic agent under the behavior type from the traffic simulation environment; based on the current agent control strategy for the behavior type, conduct adversarial training between the target traffic agent and a reference traffic agent in the traffic simulation environment to obtain adversarial information, and update the agent control strategy for the behavior type according to the adversarial information, where the reference traffic agent is the traffic agent in the traffic simulation environment; and provide driving guidance to vehicles in real road scenarios through the updated agent control strategies for each behavior type.

[0066] In this application, a traffic simulation environment can be used to simulate traffic scenarios faced by a target traffic agent and participating traffic agents under a specific behavior type, enabling adversarial training. The acquired adversarial information is used to update the current agent control strategy for that behavior type, allowing the traffic agent to cope with the simulated traffic scenarios. Because the traffic simulation environment can simulate various traffic scenarios that the target traffic agent under that behavior type may face, it can conduct adversarial training in these scenarios to continuously adjust its current agent control strategy, thereby enhancing its ability to cope with complex traffic scenarios. Since the traffic agents are obtained through cluster analysis of traffic participant data, traffic agents of different behavior types can represent different traffic participants. The updated agent control strategies for each behavior type can guide vehicles in real-world road scenarios to flexibly respond to various traffic situations. This improves the accuracy and timeliness of vehicles in dealing with complex traffic scenarios. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1a This is a schematic diagram of a scenario for the intelligent driving implementation method provided in the embodiments of this application;

[0069] Figure 1b This is a flowchart illustrating the intelligent driving implementation method provided in the embodiments of this application;

[0070] Figure 2a This is a flowchart illustrating the application of the intelligent driving implementation method provided in this application embodiment in a personalized intelligent driving system scenario;

[0071] Figure 2b This is a flowchart illustrating the clustering analysis process provided in the application's implementation examples;

[0072] Figure 2c This is a schematic diagram of the process of conducting adversarial training on the target traffic intelligent agent provided in the application embodiment;

[0073] Figure 2d This is a schematic diagram of the process of adversarial training for a collaborative traffic intelligent agent provided in the application embodiment;

[0074] Figure 2eThis is a schematic diagram of the process for adversarial training of an adversarial traffic intelligent agent provided in the application embodiment;

[0075] Figure 3 This is a schematic diagram of the intelligent driving implementation device provided in the embodiments of this application;

[0076] Figure 4 This is a schematic diagram of the electronic device structure provided in the embodiments of this application. Detailed Implementation

[0077] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0078] This application provides a method for implementing intelligent driving and related equipment. The related equipment may include an intelligent driving implementation device, an electronic device, a computer-readable storage medium, and a computer program product.

[0079] Specifically, the intelligent driving realization device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.

[0080] In some embodiments, the intelligent driving implementation device can also be integrated into multiple electronic devices. For example, the intelligent driving implementation device can be integrated into multiple servers, and the intelligent driving implementation method of this application can be implemented by multiple servers.

[0081] In some embodiments, the terminal can also be used as a server to implement some or all of the functions of a server.

[0082] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0083] It is understood that in the specific implementation of this application, traffic data and other related data of traffic participants are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0084] For example, refer to Figure 1a This electronic device can acquire traffic data from real road scenarios, including traffic data from multiple traffic participants; perform cluster analysis on the traffic data of multiple traffic participants to obtain traffic agents of at least one behavior type; construct a traffic simulation environment for training traffic agents, and configure multiple traffic agents of different behavior types in the traffic simulation environment; for each behavior type, select a target traffic agent under that behavior type from the traffic simulation environment; based on the current agent control strategy for the behavior type, conduct adversarial training between the target traffic agent and a reference traffic agent in the traffic simulation environment to obtain adversarial information, and update the agent control strategy for the behavior type based on the adversarial information, with the reference traffic agent being the traffic agent in the traffic simulation environment; and provide driving guidance to vehicles in real road scenarios through the updated agent control strategies for each behavior type.

[0085] This application utilizes a traffic simulation environment to simulate traffic scenarios faced by target traffic agents and participating traffic agents under different behavior types. This allows for adversarial training, and the acquired adversarial information updates the agent's control strategy for that behavior type, enabling the agent to cope with simulated traffic scenarios. Because the traffic simulation environment can simulate various traffic scenarios that a target traffic agent under that behavior type might face, it can conduct adversarial training within these scenarios to continuously adjust its control strategy, thereby enhancing its ability to handle complex traffic scenarios. Since the traffic agents are obtained through cluster analysis of traffic participant data, different behavior types can represent different traffic participants. The updated control strategies for each behavior type can guide vehicles in real-world road scenarios to flexibly respond to various traffic situations. This improves the accuracy and timeliness of vehicles in handling complex traffic scenarios.

[0086] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0087] In this embodiment, a method for implementing intelligent driving is provided, such as... Figure 1b As shown, the specific process of this intelligent driving implementation method can be as follows:

[0088] 101. Obtain traffic data from real-world road scenarios, including traffic data from multiple traffic participants.

[0089] Real-world road scenarios refer to traffic situations on actual roads. For example, real-world road scenarios can include busy city streets, long-distance driving on highways, changes in traffic signals at intersections, pedestrians crossing the road, and slippery road surfaces due to inclement weather.

[0090] Traffic data refers to various types of data that affect actual road traffic. For example, traffic data includes traffic environment data, motor vehicle driving data, non-motor vehicle driving data, pedestrian activity data, animal activity data, etc. These data work together to affect vehicle movement and traffic flow, forming a complex and dynamic traffic scenario in actual driving.

[0091] Traffic participants refer to various individuals who engage in traffic activities on actual roads, such as pedestrians, animals, motor vehicle drivers, and non-motor vehicle drivers. These individuals have different traffic behavior characteristics. These characteristics can include traffic behavior style (e.g., aggressive, conservative), driving experience, and historical driving behavior records.

[0092] Traffic data of traffic participants may include information on traffic behavior exhibited by traffic participants in a traffic environment, or information on traffic behavior exhibited by traffic participants in a traffic environment and data on the vehicles driven by traffic participants.

[0093] For example, if a traffic participant is a motor vehicle driver or a non-motor vehicle driver, then the traffic data of that traffic participant refers to the traffic data generated by the traffic participant based on their own traffic behavior characteristics and the vehicle they are driving when facing a traffic environment (which may include traffic flow, road type, weather conditions, etc.). The traffic data is affected by vehicle data, which may include parameters such as vehicle size, acceleration, braking performance, maximum speed, and turning radius.

[0094] If the traffic participant is a pedestrian or an animal, then the traffic data of that traffic participant refers to the data on the traffic behavior characteristics of the traffic participant when facing the traffic environment. These traffic behavior characteristics may include movement patterns and travel habits.

[0095] 102. Perform cluster analysis on traffic data from multiple traffic participants to obtain traffic intelligent agents with at least one behavioral type.

[0096] Among them, behavior type describes the regular behavior of traffic participants in the traffic environment and the types of traffic participants.

[0097] Traffic agents are virtual models used to simulate traffic participants. Their behavior types correspond to the types of actual traffic participants. Different behavior types of traffic agents represent different types of traffic participants, and the types of traffic participants correspond to their own traffic data.

[0098] For example, when the type of traffic participant is a motor vehicle driver or a non-motor vehicle driver, the traffic data of the traffic participant is associated with traffic behavior characteristics and the vehicle they are driving. When the type of traffic participant is a pedestrian or an animal, the traffic data of the traffic participant is associated with traffic behavior characteristics.

[0099] For example, at least one type of traffic intelligent agent may include fast and aggressive traffic intelligent agents, conservative and slow traffic intelligent agents, large and slow traffic intelligent agents, traffic violation traffic intelligent agents, non-motorized vehicle violation intelligent agents, and traffic intelligent agents that normally abide by the rules.

[0100] Among them, the fast and aggressive traffic intelligent agent can be used to simulate traffic participants who exhibit high speed, high acceleration, and frequent lane changes. The traffic data of this type of traffic participant has characteristics such as high average speed and high turning frequency.

[0101] Conservative and slow-moving traffic agents can be used to simulate traffic participants who exhibit low speed, high stability, and low lane-changing frequency. Traffic data of these participants are characterized by low average speed and low acceleration.

[0102] Large-volume, slow-moving traffic agents can be used to simulate traffic participants whose vehicles are large in size, accelerate slowly, and have long braking distances. The traffic data of these participants are characterized by low average speed and low acceleration.

[0103] Traffic violation-type traffic agents can be used to simulate traffic participants who exhibit a high probability of violations and a tendency to violate traffic rules. The traffic data of these traffic participants are characterized by a high frequency of violations and the selection of abnormal routes.

[0104] Traffic intelligent agents that simulate non-motorized vehicle violations can be used to simulate traffic participants who exhibit random behavior and high probability of violations, such as non-motorized vehicle drivers and their actions. The traffic data of these traffic participants are characterized by irregular movement paths and frequent speed changes.

[0105] Traffic agents that normally abide by the rules can be used to simulate traffic participants who exhibit compliance with traffic rules and stable behavior. Traffic data from such participants are characterized by low frequency of violations and stable speed and acceleration.

[0106] In some embodiments, in order to obtain at least one type of traffic agent from a large amount of traffic data, cluster analysis is performed on the traffic data of multiple traffic participants to obtain at least one type of traffic agent, including:

[0107] Traffic data from multiple traffic participants is cleaned to obtain cleaned traffic data.

[0108] Select key features from the cleaned traffic data;

[0109] Based on key features, the cleaned traffic data is subjected to dimensionality reduction processing to obtain feature data for multiple traffic participants under each key feature.

[0110] Based on the feature data of multiple traffic participants under each key feature, cluster analysis is performed on multiple traffic participants to obtain traffic intelligent agents with at least one behavior type.

[0111] Among them, traffic data after data cleaning refers to the elimination of inaccurate and inconsistent information in the original traffic data by removing outliers and duplicates, correcting errors, and standardizing data formats.

[0112] Key features are the main attributes in the cleaned traffic data that can be analyzed and identified to determine the types of behavior. For example, key features may include speed, acceleration, turning frequency, driving path, and frequency of violations.

[0113] Feature data refers to the numerical values ​​corresponding to key features in the traffic data after data cleaning.

[0114] For example, principal component analysis or t-distributed random neighborhood embedding (t-SNE) can be used to reduce the dimensionality of cleaned traffic data, obtaining feature data for multiple traffic participants under each key feature. Using the K-means clustering algorithm, based on the feature data for each key feature, the multiple traffic participants are divided into K different clusters, resulting in traffic participants for each cluster, with different clusters representing different behavior types. Based on the traffic participants in each cluster, traffic agents corresponding to their respective behavior types can be created.

[0115] When dividing multiple traffic participants into K distinct clusters, the K-means clustering algorithm (K-means++ algorithm) is used to select initial cluster centers to ensure a uniform distribution of the initial cluster centers. When K is set to 6, traffic participants exhibit six distinct groups with the aforementioned characteristics.

[0116] 103. Construct a traffic simulation environment for training traffic agents, and configure multiple traffic agents with different behavior types in the traffic simulation environment.

[0117] The traffic simulation environment is a simulation environment that mimics real-world traffic. For example, a traffic simulation environment may include a virtual road network corresponding to actual roads, virtual weather conditions, traffic lights, traffic signs, etc.

[0118] In a traffic simulation environment, by configuring one or more traffic agents of each behavior type, various virtual traffic scenarios can be simulated to reflect the actual traffic scenarios that different types of traffic participants may encounter. This configuration enables the simulation environment to reproduce the dynamics of multiple types of traffic participants, thereby more accurately reproducing the interactions and behaviors of traffic agents and providing a more realistic traffic scenario.

[0119] 104. For each behavior type, select the target traffic agent under the behavior type from the traffic simulation environment.

[0120] In this context, the target traffic agent is a traffic agent randomly selected from the traffic simulation environment that requires focused policy optimization. For example, the traffic simulation environment may contain a target traffic agent focused on a specific task, or it may assign a dedicated target traffic agent to each behavior type, and so on.

[0121] 105. Based on the current agent control strategy of behavior type, conduct adversarial training between the target traffic agent and the reference traffic agent in a traffic simulation environment to obtain adversarial information. Based on the adversarial information, update the agent control strategy of behavior type. The reference traffic agent is the traffic agent in the traffic simulation environment.

[0122] Here, the current agent control strategy for each behavior type refers to the agent control strategy currently used by traffic agents of each behavior type. For example, the current agent control strategy for each behavior type could be the agent control strategy that the traffic agents of each behavior type have not yet optimized, or it could be the agent control strategy used by the traffic agents of each behavior type in the current iterative optimization process.

[0123] The participating traffic agents can be any traffic agent in the traffic simulation environment. For example, a reference traffic agent can have the same or similar intelligent control strategies as the target traffic agent. Similar agent control strategies can be historical versions of the target traffic agent's agent control strategies, or they can be traffic agents with different behavior types than the target traffic agent. Through these reference traffic agents, the target traffic agent can engage in self-play.

[0124] Adversarial information refers to the relevant behavioral data and interaction information between the target traffic agent and the reference traffic agent during adversarial training. For example, adversarial information can be the interaction data between the target traffic agent and the reference traffic agent in situations such as meeting oncoming traffic, driving side by side, overtaking on curves, changing lanes, and emergency braking.

[0125] In some embodiments, in order to evaluate the performance of a traffic agent in a traffic simulation environment and specify the update direction of its agent control policy, adversarial training is performed on the target traffic agent and a reference traffic agent in the traffic simulation environment based on the current agent control policy for the behavior type to obtain adversarial information. This adversarial information is then used to update the agent control policy for the behavior type, including:

[0126] Configure reward strategy information for traffic intelligent agents of various behavior types;

[0127] Based on the current intelligent agent control strategy according to the behavior type, adversarial training is carried out between the target traffic intelligent agent and the reference traffic intelligent agent in a traffic simulation environment to obtain adversarial information.

[0128] Based on the adversarial information and reward strategy information, determine the reward value information of the target traffic intelligent agent;

[0129] Update the agent control strategy based on the reward value information of the target traffic agent.

[0130] Among them, the reward strategy information is used to evaluate the performance of traffic agents in the traffic simulation environment. Different traffic agents with different behavior types correspond to different reward strategy information. The reward strategy information can guide the update direction of the agent control strategy of the corresponding traffic agent.

[0131] For example, if the target traffic agent's behavior type is traffic rule compliance, then the reward strategy settings should follow the following reward design principles: 1) Encourage rule-compliant driving: Emphasize compliance with traffic rules and avoid violations. 2) Improve traffic efficiency: Maintain appropriate speed and avoid overly conservative driving behavior that affects traffic flow. 3) Ensure driving safety: Encourage safe driving, maintain a reasonable following distance, and avoid collisions.

[0132] The reward value information is the score in the reward strategy information that matches the adversarial information, used to evaluate the performance of the target traffic agent in adversarial situations. These scores can help adjust and optimize the agent control strategy of the target traffic agent to improve its behavior in these adversarial situations.

[0133] In some embodiments, it also includes:

[0134] When the reward value of the target traffic agent is lower than the preset value, update the agent control strategy for the behavior type.

[0135] When the reward value of the target traffic agent is greater than or equal to the preset value, the agent control strategy for the behavior type is stopped from being updated.

[0136] The preset value is a pre-set score used to measure reward information. The preset value can limit the update of the agent control policy for the behavior type.

[0137] In some embodiments, in order to obtain adversarial information generated during adversarial training between the target traffic agent and the reference traffic agent in a traffic simulation environment, adversarial training is performed on the target traffic agent and the reference traffic agent in the traffic simulation environment based on the current agent control policy of the behavior type to obtain adversarial information, including:

[0138] Based on the current agent control strategy of the behavior type, when conducting adversarial training between the target traffic agent and the reference traffic agent in the traffic simulation environment, the current simulation environment information of the traffic simulation environment is obtained.

[0139] Based on the current simulation environment information, the interaction behavior between the target traffic agent and the reference traffic agent is evaluated and processed to obtain adversarial information.

[0140] Among them, the current simulation environment information can reflect the environmental changes of the target traffic agent and the reference traffic agent during the current adversarial training. Through these environmental changes, the interaction behavior between the target traffic agent and the reference traffic agent can be evaluated, and adversarial information can be obtained.

[0141] For example, in a traffic simulation environment, adversarial training between the target traffic agent and the reference traffic agent covers their interactions during oncoming traffic. By acquiring information about the current simulation environment during oncoming traffic, the interactive behaviors of the target and reference traffic agents can be analyzed, and corresponding adversarial information can be extracted.

[0142] 106. By updating the intelligent agent control strategy for each behavior type, driving guidance is provided for vehicles in real road scenarios.

[0143] Among them, the updated agent control strategy enables the target traffic agent to effectively handle scenarios related to adversarial information.

[0144] Real-world road scenarios refer to traffic conditions that occur on actual roads, including road layout, traffic signs, vehicle flow, and pedestrian activities. By updating intelligent agent control strategies based on various behavior types, driving guidance for vehicles can be effectively provided.

[0145] In some embodiments, in order to improve the collaborative ability of the target agent with other traffic agents in the traffic simulation environment under the current agent control strategy, before providing driving guidance to vehicles in real road scenarios through the updated agent control strategy for each behavior type, the method further includes:

[0146] By analyzing the adversarial interactions between various traffic agents in a traffic simulation environment, cooperative traffic agents are selected from among them.

[0147] Based on the current agent control strategies of the cooperative traffic agent and the target traffic agent, adversarial training is conducted on the cooperative traffic agent and the target traffic agent in a traffic simulation environment to obtain cooperative adversarial information.

[0148] Based on the cooperative adversarial information, the current agent control strategy of the cooperative traffic agent is updated.

[0149] In this context, a cooperative traffic agent refers to a traffic agent that utilizes and benefits from the resources provided by a traffic agent alliance within a traffic simulation environment. Cooperative traffic agents can leverage these resources to engage in adversarial training against a target traffic agent, which helps identify problems that arise between the target and cooperative traffic agents during the adversarial process.

[0150] For example, a collaborative traffic agent can be a user of the traffic agent alliance, or a traffic agent that can utilize the resources of the traffic agent alliance, and so on.

[0151] A traffic intelligent agent alliance refers to an alliance composed of traffic intelligent agents with common goals. For example, a traffic intelligent agent alliance can be composed of traffic intelligent agents with the same behavior type and consistent goals, or it can be composed of traffic intelligent agents with different behavior types but common goals, and so on.

[0152] Cooperative adversarial information refers to the relevant behavioral data and interaction information between the target traffic agent and the cooperative traffic agent during adversarial training.

[0153] In some embodiments, based on the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent, adversarial training is performed on the cooperative traffic agent and the target traffic agent in a traffic simulation environment to obtain cooperative adversarial information, including:

[0154] The coordinated traffic intelligent agent is triggered to identify policy vulnerabilities of the target traffic intelligent agent through the current intelligent agent control strategy, and obtain information on policy vulnerabilities to be countered.

[0155] The collaborative traffic agent is triggered to conduct adversarial training with the target traffic agent in a traffic simulation environment based on the vulnerability information of the strategy to be challenged, thereby obtaining collaborative adversarial information.

[0156] Specifically, the vulnerability information of the strategy to be countered refers to the vulnerability information in the current agent control strategy of the target traffic agent identified by the collaborative traffic agent. For example, the vulnerability information may include scenario information about policy weaknesses, such as poor performance of the target traffic agent under high-density traffic or severe weather conditions. It may also include policy flaws that may lead to insufficient robustness of the target traffic agent's behavior, making it vulnerable to countermeasures. Furthermore, it may include the countermeasure effect, indicating that the target traffic agent may perform poorly in the face of rapidly changing traffic conditions or make inaccurate decisions in complex traffic scenarios, and so on.

[0157] For example, on a highway in a traffic simulation environment, the target traffic agent reacts slowly when changing lanes. Based on this vulnerability in the adversarial strategy, the cooperating traffic agent uses its current agent control strategy to specifically target the target traffic agent's lane-changing vulnerability and conducts adversarial training against the target traffic agent. This yields detailed cooperative adversarial information, such as the target traffic agent's specific behavioral patterns and response strategies in this situation.

[0158] In some embodiments, to evaluate the actual performance of individual traffic agents in a traffic agent alliance and select the cooperative traffic agents that can most effectively utilize alliance resources, the cooperative traffic agents engage in adversarial training against the target traffic agent. This strategy helps to identify and improve the performance of traffic agents, thereby enhancing the overall effectiveness of the entire traffic agent alliance. Selecting cooperative traffic agents from among the traffic agents through adversarial training in a traffic simulation environment includes:

[0159] Determine the alliance of traffic intelligent agents in a traffic simulation environment;

[0160] By observing the adversarial interactions between the various traffic intelligent agents in the traffic intelligent agent alliance, we can obtain information on the alliance adversarial results among the traffic intelligent agents in the traffic intelligent agent alliance.

[0161] Based on the results of the alliance confrontation, cooperative traffic intelligent agents are selected from the traffic intelligent agents in the traffic intelligent agent alliance.

[0162] Among them, the alliance adversarial result information refers to the result data obtained through adversarial training between different traffic intelligent agents in the traffic intelligent agent alliance. The alliance adversarial result information may include at least one of the following: win rate information, collaborative effect, resource utilization, and environmental adaptability of each traffic intelligent agent in the traffic intelligent agent alliance.

[0163] For example, if the information on the outcome of the alliance confrontation includes the win rate information of each traffic intelligent agent in the traffic intelligent agent alliance, then the traffic intelligent agent with the highest win rate information can be selected as the cooperative traffic intelligent agent.

[0164] In some embodiments, to enable the target traffic agent to conduct adversarial training with other traffic agents having the same objective, which is beneficial for the target traffic agent to deal with opponents with similar objectives, before providing driving guidance to vehicles in real road scenarios through updated agent control strategies for various behavior types, the method further includes:

[0165] Based on the current agent control strategy according to the behavior type, adversarial training is carried out between the target traffic agent and traffic agents in the traffic agent alliance in a traffic simulation environment to obtain alliance adversarial information.

[0166] Based on the alliance confrontation information, the current agent control strategies of the traffic intelligent agents in the traffic intelligent agent alliance and the current agent control strategies of the target traffic intelligent agent are updated.

[0167] Among them, alliance adversarial information refers to the relevant behavioral data and interaction information between the target traffic agent and traffic agents in the traffic agent alliance during adversarial training.

[0168] In some embodiments, prior to obtaining cooperative adversarial information by conducting adversarial training on the cooperative traffic agent and the target traffic agent in a traffic simulation environment based on the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent, the process includes:

[0169] Randomly select a training traffic intelligent agent from among the traffic intelligent agents in the Traffic Intelligent Agent Alliance to be a collaborating traffic intelligent agent;

[0170] Based on the current agent control strategies of the cooperative traffic agent and the training traffic agent, adversarial training is conducted on the cooperative traffic agent and the training traffic agent in a traffic simulation environment to obtain training adversarial information.

[0171] Based on the training adversarial information, the current agent control strategies of the cooperative traffic agent and the training traffic agent are updated.

[0172] Among them, the training traffic agents are the other traffic agents in the traffic agent alliance, excluding the cooperative traffic agents. Through adversarial training between the training and cooperative traffic agents, the current traffic agent control strategy of the cooperative traffic agents is updated. This updated control strategy allows the cooperative traffic agents to perform better when adversarially against the target traffic agent.

[0173] Training adversarial information refers to the relevant behavioral data and interaction information between the coordinating traffic intelligent agent and the training traffic intelligent agent during adversarial training.

[0174] In some embodiments, in order to continuously update and optimize the agent control strategy during adversarial learning, after updating the current agent control strategy and the current agent control strategy of the cooperative traffic agent based on cooperative adversarial information, the method further includes:

[0175] When the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent meet the preset update conditions, continue to conduct adversarial training between the cooperative traffic agent and the target traffic agent in the traffic simulation environment.

[0176] When the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent do not meet the preset update conditions, the adversarial training between the cooperative traffic agent and the target traffic agent in the traffic simulation environment is stopped.

[0177] Among them, the preset update conditions are the pre-set update conditions for restricting the agent control strategy. These preset update conditions can be used to determine whether the current agent control strategy for each behavior type needs to be adjusted.

[0178] In some embodiments, in order to enable a traffic agent with superior adversarial performance to conduct adversarial training with a target traffic agent in order to update the target traffic agent's agent control policy, the method further includes the following before providing driving guidance to vehicles in real road scenarios through the updated agent control policy for each behavior type:

[0179] Identify the target traffic intelligent agent and the adversary traffic intelligent agent;

[0180] Based on the current agent control strategies of the adversary traffic agent and the target traffic agent, adversarial training is conducted on the adversarial traffic agent and the target traffic agent in a traffic simulation environment to obtain defensive adversarial information.

[0181] Based on the defense and confrontation information, the current agent control strategy of the adversary traffic agent and the current agent control strategy of the target traffic agent are updated.

[0182] Among them, adversarial traffic agents are traffic agents primarily responsible for optimizing the agent control strategies of the target traffic agent. For example, an adversarial traffic agent could be one of the traffic agents with the best adversarial performance in a traffic simulation environment, or it could be a traffic agent that can effectively counter the target traffic agent, and so on.

[0183] Defense adversarial information refers to the relevant behavioral data and interaction information between adversarial traffic intelligent agents and target traffic intelligent agents during adversarial training.

[0184] In some embodiments, in order to screen out the traffic agents primarily responsible for promoting the optimization of the agent control strategy of the target traffic agent, and to determine the adversarial traffic agents of the target traffic agent, the following steps are included:

[0185] By observing the adversarial interactions between various traffic intelligent agents in a traffic simulation environment, information on the adversarial outcomes of each agent is obtained.

[0186] Based on the adversarial outcome information, adversarial traffic agents are identified from among the various traffic agents.

[0187] The adversarial outcome information refers to the result data obtained from adversarial training between various traffic agents in a traffic simulation environment. This information can include at least one of the following: win rate, collaborative effect, resource utilization, and environmental adaptability of each traffic agent. By analyzing the adversarial outcome information, traffic agents with the best adversarial performance, or those capable of effectively adversarially defeating the target traffic agent, can be identified.

[0188] In some embodiments, to enable the adversarial traffic agent to effectively counter the target traffic agent, adversarial training is conducted on both the adversarial and target traffic agents in a traffic simulation environment based on the adversarial agent's current agent control strategy and the target traffic agent's current agent control strategy, to obtain defensive adversarial information, including:

[0189] The adversarial traffic agent identifies policy vulnerabilities of the target traffic agent through the current agent control strategy and obtains policy vulnerability information to be referenced.

[0190] The adversarial traffic agent adjusts its current agent control strategy based on the vulnerability information of the policy to be referenced, thus obtaining the adjusted agent control strategy of the adversarial traffic agent.

[0191] Based on the adjusted agent control strategy of the adversarial traffic agent and the current agent control strategy of the target traffic agent, adversarial training is conducted on the adversarial traffic agent and the target traffic agent in a traffic simulation environment to obtain defensive adversarial information.

[0192] Among them, the vulnerability information of the reference strategy refers to the vulnerability information in the current agent control strategy of the target traffic agent identified by the adversarial traffic agent.

[0193] The adjusted agent control strategy of the adversarial traffic agent enables the adversarial traffic agent to launch an adversarial attack against the target traffic agent based on the vulnerability information of the reference strategy.

[0194] In some embodiments, in order to continuously update and optimize the agent control strategy during adversarial learning, after updating the agent control strategy of the adversarial traffic agent and the current agent control strategy of the cooperative traffic agent based on the defensive adversarial information, the method further includes:

[0195] When the current agent control strategy of the adversary traffic agent and the current agent control strategy of the target traffic agent meet the preset update conditions, continue to conduct adversarial training on the adversary traffic agent and the target traffic agent in the traffic simulation environment.

[0196] When the current agent control strategy of the adversary traffic agent and the current agent control strategy of the target traffic agent do not meet the preset update conditions, the adversarial training between the adversary traffic agent and the target traffic agent in the traffic simulation environment is stopped.

[0197] The aforementioned traffic intelligent agent of at least one behavior type may include: at least one traffic intelligent agent of pedestrian behavior type, at least one traffic intelligent agent of motor vehicle behavior type, and at least one traffic intelligent agent of non-motor vehicle behavior type.

[0198] Taking at least one type of pedestrian behavior traffic agent as an example, traffic agents that comply with traffic rules can be selected as target traffic agents (such as walking on sidewalks and obeying traffic lights), traffic agents that are good at exploiting loopholes in traffic rules can be selected as cooperative traffic agents (such as being able to quickly cross the road when traffic volume is low and not completely relying on traffic lights), and traffic agents that target the weaknesses of motor vehicles can be selected as adversarial traffic agents (such as being able to identify vehicle blind spots and use these blind spots to quickly pass through the lane).

[0199] Taking at least one type of motor vehicle behavior as an example, traffic intelligent agents that comply with traffic rules can be selected as target traffic intelligent agents (such as stopping at a red light or yielding to pedestrians), traffic intelligent agents that are good at utilizing favorable conditions in traffic flow can be selected as cooperative traffic intelligent agents (such as accelerating through a yellow light or finding the optimal path in traffic congestion), and traffic intelligent agents that specifically react to the behavior of pedestrians or non-motorized vehicles can be selected (such as being able to recognize pedestrians suddenly crossing the road and quickly take evasive measures).

[0200] Taking at least one type of non-motorized vehicle behavior as an example, traffic intelligent agents that comply with traffic rules can be selected as target traffic intelligent agents (such as driving in dedicated lanes and obeying traffic signals), traffic intelligent agents that can find the best path in traffic flow can be selected as cooperative traffic intelligent agents (such as those that may use gaps between motor vehicles or ride on sidewalks to avoid congestion), and traffic intelligent agents that specifically utilize the non-motorized vehicle behavior of motor vehicles and pedestrians can be selected (such as those that can identify opportunity windows in traffic flow and quickly cross complex intersections).

[0201] Among them, the target traffic agent can represent a typical traffic agent that follows conventional rules.

[0202] Collaborative traffic agents can represent traffic agents that excel at discovering and utilizing favorable conditions in traffic simulation environments;

[0203] Adversarial traffic agents can represent traffic agents that are specifically optimized to exploit the weaknesses of other traffic agents.

[0204] The selection of target traffic agents, cooperative traffic agents, and adversarial traffic agents in the traffic simulation environment is not limited here. That is, target traffic agents, cooperative traffic agents, and adversarial traffic agents can belong to the same behavior type or different behavior types.

[0205] In some embodiments, in order to provide driving guidance for vehicles in real-world road scenarios, updated agent control strategies for various behavior types are used to provide driving guidance for vehicles in real-world road scenarios, including:

[0206] Collect real-time traffic information of the actual road scene where the target vehicle is located;

[0207] Based on real-time traffic information, select the target traffic agent control strategy from the updated agent control strategies for each behavior type.

[0208] The target traffic intelligent agent control strategy provides driving guidance to target vehicles in real road scenarios.

[0209] The target vehicle refers to a vehicle in a real road scenario, and the model of the target vehicle is not limited here.

[0210] Real-time traffic information reflects the traffic conditions and road surface conditions encountered by a target vehicle when driving in a real-world road scenario. For example, real-time traffic information can cover factors such as weather, traffic flow, the surrounding environment of the vehicle, and road conditions in a real-world road scenario.

[0211] The target traffic agent control strategy refers to at least one traffic agent control strategy that a target vehicle can adopt when responding to real-time path information.

[0212] In some embodiments, in order to apply high-performing agent control strategies to vehicle driving guidance, updated agent control strategies for each behavior type are used to provide driving guidance for vehicles in real-world road scenarios, including:

[0213] Obtain strategy evaluation metrics;

[0214] Based on the policy evaluation index, the updated agent control policy for each behavior type is evaluated to obtain the policy evaluation information of the updated agent control policy for each behavior type.

[0215] Based on the policy evaluation information, the updated agent control policies for each behavior type are screened to obtain the screened agent control policies.

[0216] The selected intelligent agent control strategy provides driving guidance for vehicles in real road scenarios.

[0217] Among them, the strategy evaluation index is an indicator for evaluating the agent's control strategy. The agent's control strategy is evaluated by using strategy evaluation (such as win rate and efficiency) to select the best performing strategy.

[0218] The policy evaluation information consists of feedback data obtained after evaluating the updated agent control policies for each behavior type. This policy evaluation data is used to measure and compare the performance of different updated agent control policies, including the effectiveness and efficiency of each updated agent control policy in a specific behavior type.

[0219] The selected agent control strategy refers to an agent control strategy that can counter high performance and can be applied to vehicle driving guidance.

[0220] In some embodiments, in order to introduce new environmental variables when conducting adversarial training on the traffic agent, after constructing the traffic simulation environment for training the traffic agent, the method further includes:

[0221] The traffic simulation environment is updated to obtain the updated traffic simulation environment.

[0222] In some embodiments, to introduce new strategy types and improve the diversity and adaptability of traffic agents in traffic simulation environments, the method further includes:

[0223] Obtain the agent control strategy corresponding to the new strategy type, so that traffic agents of each behavior type can conduct adversarial training in a traffic simulation environment through the agent control strategy corresponding to the new strategy type.

[0224] As can be seen from the above, the embodiments of this application can acquire traffic data in real road scenarios, including traffic data of multiple traffic participants; perform cluster analysis on the traffic data of multiple traffic participants to obtain traffic intelligent agents of at least one behavior type; construct a traffic simulation environment for training traffic intelligent agents, and configure multiple traffic intelligent agents of different behavior types in the traffic simulation environment; for each behavior type, select a target traffic intelligent agent under the behavior type from the traffic simulation environment; based on the current intelligent agent control strategy of the behavior type, conduct adversarial training between the target traffic intelligent agent and a reference traffic intelligent agent in the traffic simulation environment to obtain adversarial information, and update the intelligent agent control strategy of the behavior type according to the adversarial information, where the reference traffic intelligent agent is the traffic intelligent agent in the traffic simulation environment; and provide driving guidance to vehicles in real road scenarios through the updated intelligent agent control strategies of each behavior type.

[0225] Therefore, this solution utilizes a traffic simulation environment to simulate traffic scenarios faced by target traffic agents and participating traffic agents under different behavior types. This allows for adversarial training, and the acquired adversarial information updates the agent's control strategy for that behavior type, enabling the agent to cope with simulated traffic scenarios. Because the traffic simulation environment can simulate various traffic scenarios that a target traffic agent under that behavior type might face, it can conduct adversarial training within these scenarios to continuously adjust its control strategy, enhancing its ability to handle complex traffic scenarios. Since traffic agents are obtained through cluster analysis of traffic participant data, different behavior types of traffic agents can represent different traffic participants. The updated agent control strategies for each behavior type can guide vehicles in real-world road scenarios to flexibly respond to various traffic situations. This improves the accuracy and timeliness of vehicles in dealing with complex traffic scenarios.

[0226] The method described in the above embodiments will be further described in detail below.

[0227] In this embodiment, a personalized intelligent driving system will be used as an example to describe the method of this application embodiment in detail.

[0228] like Figure 2a As shown, the specific process of an intelligent driving implementation method is as follows:

[0229] 201. Obtain traffic data from real road scenarios, including traffic data from multiple traffic participants, and perform cluster analysis on the traffic data of multiple traffic participants to obtain traffic intelligent agents with at least one behavioral type.

[0230] likeFigure 2b As shown, traffic data can be data collected from sensors and then filtered by traffic detection equipment.

[0231] In some embodiments, cluster analysis is performed on traffic data from multiple traffic participants to obtain traffic agents of at least one behavioral type, including:

[0232] Traffic data from multiple traffic participants is cleaned to obtain cleaned traffic data.

[0233] Select key features from the cleaned traffic data;

[0234] Based on key features, the cleaned traffic data is subjected to dimensionality reduction processing to obtain feature data for multiple traffic participants under each key feature.

[0235] Based on the feature data of multiple traffic participants under each key feature, cluster analysis is performed on multiple traffic participants to obtain traffic intelligent agents with at least one behavior type.

[0236] In some embodiments, such as Figure 2b As shown, a suitable clustering algorithm can also be selected to perform clustering analysis on multiple traffic participants based on the feature data corresponding to each key feature, thereby obtaining traffic intelligent agents with at least one behavior type.

[0237] 202. Construct a traffic simulation environment for training traffic agents, and configure multiple traffic agents with different behavior types in the traffic simulation environment.

[0238] The design of multiple traffic intelligent agents with different behavioral types can be achieved by classifying various types of driving behaviors in real-world road scenarios. Six types of agents were designed, including those that obey traffic rules, those that are fast and aggressive, those that are conservative and slow, those that are large and slow, those that violate traffic rules, and those that violate non-motorized vehicle rules. These agents represent different driving styles and vehicle characteristics in reality. Through training in a simulation environment, they can better simulate and understand complex traffic conditions.

[0239] In some embodiments, it also includes:

[0240] Initialize the position and state of all traffic agents in the traffic simulation environment.

[0241] Construction of a traffic simulation environment:

[0242] 1) Traffic Scenario Simulation: Based on real urban traffic data, simulation tools are used to construct realistic traffic environments. Different weather conditions, road conditions, traffic flow, and other scenarios are simulated to cover diverse driving situations.

[0243] 2) Traffic agent interaction design: Configure traffic agents with different behavior types in the traffic simulation environment, and define the interaction rules and physical constraints between traffic agents.

[0244] Among them, traffic intelligent agents of different behavior types can represent vehicles, pedestrians, bicycles, etc.

[0245] 3) Data interface: The interface enables data exchange between the traffic simulation environment and the traffic intelligent agent, providing real-time status information and control commands.

[0246] 203. Obtain user driving needs, and select target traffic agents from multiple traffic agents with different behavior types in the traffic simulation environment based on user driving needs.

[0247] Among them, user driving needs include driving style information and driving preference information. Driving style information refers to the habits and behavioral patterns that users exhibit during driving, such as rapid acceleration and smooth driving. Driving preference information refers to the user's personal preferences for the driving experience, such as whether the user prefers high-speed driving or economical driving.

[0248] In some embodiments, it also includes:

[0249] Based on the goals and behavioral characteristics of traffic intelligent agents of various behavior types, reward strategy information is set for traffic intelligent agents of various behavior types.

[0250] The reward mechanism can be designed by tailoring detailed reward mechanisms to the behavioral characteristics and goals of traffic agents with different behavior types. For example, traffic agents that comply with traffic rules improve the overall safety of the intelligent driving system through reward strategies for obeying traffic lights, maintaining lanes, and keeping safe distances. Meanwhile, training traffic agents that violate traffic rules in a simulated environment helps to identify and address potential dangerous driving behaviors, thereby improving the system's robustness.

[0251] 204. Based on the target traffic intelligent agent, determine the traffic intelligent agent alliance, the cooperative traffic intelligent agents in the traffic intelligent agent alliance, and the adversarial traffic intelligent agents from each traffic intelligent agent.

[0252] The target traffic agent's task is to execute the core agent control strategy, serving as the main control unit in the intelligent driving system. Strategy updates involve continuously optimizing its agent control strategy through self-play and adversarial learning with other alliance agents.

[0253] During self-play, the target traffic agent engages in combat with itself or other traffic agents with similar control strategies, collects state, action, and reward information during the combat, and updates the target traffic agent's control strategy.

[0254] The task of collaborative traffic agents is to identify the weaknesses in the agent control strategies of the trainee traffic agents within the traffic agent alliance and develop targeted strategies. Strategy updates involve adjusting the agent control strategy of each traffic agent in the alliance to maximize its own advantages.

[0255] When a target traffic agent and a cooperative traffic agent engage in alliance-based adversarial combat, the cooperative traffic agent attempts to identify policy vulnerabilities in the target traffic agent to obtain information on policy vulnerabilities to be countered. Based on this information, the cooperative traffic agent then engages in adversarial combat against the target traffic agent. The target traffic agent learns and adapts to the cooperative traffic agent's agent control strategy.

[0256] The task of adversarial traffic agents is to optimize their control strategies against specific agents and to identify and exploit weaknesses in their strategies. Strategy updates involve continuously adjusting and optimizing one's own agent control strategy through adversarial interactions with the target traffic agent and other agents in the traffic agent alliance.

[0257] In adversarial traffic agents, when confronting a target traffic agent, the adversarial agent explores and exploits the target agent's strategic weaknesses. The target traffic agent, through adversarial action, optimizes its own agent control strategy to reduce the likelihood of its strategy being exploited.

[0258] 205. Based on the current agent control strategy of the target traffic agent and the current agent control strategy of the reference traffic agent, the target traffic agent and the reference traffic agent are trained to play against each other in a traffic simulation environment to obtain adversarial information. Based on the adversarial information, the current agent control strategies of the target traffic agent and the reference traffic agent are updated. The reference traffic agent is the traffic agent in the traffic simulation environment.

[0259] The reference traffic agent can be a traffic agent in a traffic agent alliance or a traffic agent similar to the target traffic agent.

[0260] In some embodiments, such as Figure 2c As shown, the target traffic agent can update its agent control strategy through adversarial training via alliance training and / or self-play.

[0261] 206. Based on the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent, conduct alliance adversarial training on the cooperative traffic agent and the target traffic agent in a traffic simulation environment to obtain cooperative adversarial information, so as to update the current agent control strategy of the cooperative traffic agent and the target traffic agent according to the cooperative adversarial information.

[0262] In some embodiments, prior to obtaining cooperative adversarial information by conducting adversarial training on the cooperative traffic agent and the target traffic agent in a traffic simulation environment based on the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent, the process includes:

[0263] Randomly select a training traffic intelligent agent from among the traffic intelligent agents in the Traffic Intelligent Agent Alliance to be a collaborating traffic intelligent agent;

[0264] Based on the current agent control strategies of the cooperative traffic agent and the training traffic agent, adversarial training is conducted on the cooperative traffic agent and the training traffic agent in a traffic simulation environment to obtain training adversarial information.

[0265] Based on the training adversarial information, the current agent control strategies of the cooperative traffic agent and the training traffic agent are updated.

[0266] In some embodiments, such as Figure 2d As shown, cooperative traffic agents can update their agent control strategies through adversarial training with target traffic agents and alliance training.

[0267] 207. Based on the current agent control strategy of the adversarial traffic agent and the current agent control strategy of the target traffic agent, conduct adversarial training on the adversarial traffic agent and the target traffic agent in a traffic simulation environment to obtain defense adversarial information; and update the current agent control strategy of the adversarial traffic agent and the target traffic agent based on the defense adversarial information.

[0268] In some embodiments, such as Figure 2e As shown, adversarial traffic agents can update their own agent control strategies by engaging in adversarial training with target traffic agents.

[0269] Self-play and Alliance Training: By employing self-play and alliance training strategies, traffic agents learn and optimize their control strategies through diverse adversarial interactions, thereby acquiring stronger decision-making and adaptability. Through multiple strategic confrontations with various traffic agents, the traffic agents can quickly make optimal decisions when facing different traffic scenarios and emergencies.

[0270] In the process of updating the agent control strategy:

[0271] a) Repeat the above adversarial learning process to continuously update and optimize the agent's control strategy;

[0272] b) Regularly introduce new strategy types and environmental variables for traffic simulation environments to improve the diversity and adaptability of traffic agents.

[0273] Adversarial training enables traffic agents to respond efficiently to emergencies. In particular, traffic simulation environments can simulate various emergency traffic scenarios, such as pedestrians crossing the road or vehicles suddenly changing lanes. Traffic agents learn how to make quick and accurate responses in complex and ever-changing environments, thereby greatly improving the ability of intelligent driving systems to respond to emergencies.

[0274] Traffic agents with different behavioral types can improve traffic flow without compromising safety through diversified learning and optimization of different strategies. For example, the design of a fast-aggressive traffic agent aims to increase traffic flow speed and reduce unnecessary delays, while the conservative-slow traffic agent aims to avoid traffic congestion caused by excessive conservatism while ensuring safety.

[0275] Self-play and alliance training enable the control strategies of agents of various behavior types to be continuously updated and iterated. As the number of training sessions increases, traffic agents can better adapt to different traffic flows and situations in complex environments, thereby improving overall traffic efficiency.

[0276] This application employs cluster analysis to categorize different types of traffic participants (including various vehicles and pedestrians) into six major categories of traffic agents. This method not only adapts to the current traffic environment but can also adjust clustering parameters to accommodate new types of transportation or driving behaviors that may emerge in the future.

[0277] The modular design of intelligent driving systems allows for easy integration into existing autonomous driving platforms, increasing their adaptability. As traffic scenarios change, intelligent driving systems can be modularly upgraded to address new challenges and demands.

[0278] The safety, smoothness, and efficiency of intelligent driving systems directly enhance the user's driving experience, making them more competitive in the autonomous driving market. Users can experience smarter and more reliable autonomous driving services, further promoting market recognition and acceptance of the technology. Self-play and alliance training enable the agent's strategy to be continuously updated and iterated. With the increase in training times, the agent can better adapt to different traffic flows and situations in complex environments, improving overall traffic efficiency.

[0279] 208. By updating the intelligent agent control strategy for each behavior type, provide driving guidance for vehicles that match the user's driving needs in real road scenarios.

[0280] In some embodiments, it also includes:

[0281] The updated agent control policies for each behavior type are stored in the policy library;

[0282] By using policy evaluation metrics, the updated agent control policies for each behavior type are evaluated to obtain policy evaluation information for each behavior type.

[0283] Based on the policy evaluation information, the updated agent control policies for each behavior type are screened to obtain the screened agent control policies, which are the best performing agent control policies.

[0284] The selected agent control strategy is applied to the target traffic agent.

[0285] In some embodiments, driving guidance for vehicles in real-world road scenarios is provided through updated agent control strategies for various behavior types, including:

[0286] Collect real-time traffic information of the actual road scene where the target vehicle is located;

[0287] Based on real-time traffic information, select the target traffic agent control strategy from the updated agent control strategies for each behavior type.

[0288] The target traffic intelligent agent control strategy provides driving guidance to target vehicles in real road scenarios.

[0289] Real-time traffic information can be obtained by continuously monitoring the environment around the target vehicle. The intelligent driving system can provide immediate feedback to the driver of the target vehicle and offer warnings and suggestions in dangerous situations. Such functionality can help drivers react quickly in emergency situations and avoid accidents.

[0290] In addition, by predicting the behavior of other road users (such as pedestrians, non-motorized vehicle drivers and drivers of other vehicles), intelligent driving systems can identify potential dangerous behaviors in advance, such as traffic violations or sudden lane changes, and thus take preventive measures.

[0291] In some embodiments, a target traffic agent control strategy is selected from updated agent control strategies for each behavior type based on real-time traffic information, including:

[0292] Based on real-time traffic information and the preference information of traffic participants driving the target vehicle, the target traffic agent control strategy is selected from the updated agent control strategies for each behavior type.

[0293] For example, it can automatically adjust between aggressive, conservative, and customizable agent control strategies to adapt to different driving situations.

[0294] In some embodiments, it also includes:

[0295] By learning users' driving habits through machine learning algorithms, the intelligent driving system can gradually optimize the intelligent agent control strategy recommended by the system, providing a driving experience that better meets user expectations.

[0296] In some embodiments, it also includes:

[0297] Deep learning methods based on multimodal sensor fusion utilize data sources from different sensors (such as cameras, LiDAR, radar, and ultrasonic sensors) and perform comprehensive analysis and processing of the data through deep learning models to achieve a comprehensive understanding of the environment and accurate decision-making.

[0298] This application enhances the environmental perception capabilities of intelligent driving systems by integrating advanced sensor technologies and deep learning algorithms. It utilizes multimodal sensor (such as cameras, radar, and lidar) fusion technology, combined with deep learning models, to achieve real-time perception and recognition of complex traffic scenarios, thereby improving the decision-making ability of intelligent driving systems in dynamic environments.

[0299] In some embodiments, it also includes:

[0300] Multi-agent collaborative learning based on game theory: Game theory is used to study the strategic interaction between traffic agents. By simulating the game process between different traffic participants, collaborative and competitive learning among traffic agents with multiple behavior types can be achieved, thereby optimizing the overall performance of the intelligent driving system.

[0301] In some embodiments, it further includes: a behavior cloning method based on adaptive learning, behavior cloning being a method of learning using human driving data, wherein the traffic agent learns appropriate agent control strategies by imitating human driving behavior and continuously optimizes its performance through an adaptive learning mechanism.

[0302] In some embodiments, it also includes:

[0303] Real-time intelligent decision-making systems based on edge computing move computing tasks from the cloud to the network edge, utilizing computing resources inside and outside the vehicle to achieve rapid data processing and decision-making, in order to address the high real-time requirements of autonomous driving.

[0304] This application employs a multi-strategy joint optimization method, introducing a multi-level training strategy for traffic intelligent agents, including target traffic intelligent agents, adversarial traffic intelligent agents, and cooperative traffic intelligent agents. This multi-strategy training framework enhances the diversity and flexibility of the intelligent driving system, enabling it to make better decisions in complex scenarios. Through these improvements, this application effectively addresses the main shortcomings of existing technologies, significantly improving the adaptability and safety of intelligent driving systems in complex traffic scenarios, and laying a solid foundation for realizing more intelligent autonomous driving technology.

[0305] By employing reinforcement learning (RL) strategies, including self-play and league training methods, intelligent driving systems acquire adaptive learning capabilities. These systems can simulate different traffic scenarios and engage in adversarial learning with themselves or other traffic agents to optimize their control strategies, achieving adaptive adjustments to complex traffic environments.

[0306] This application introduces traffic intelligent agents with multiple behavior types, classifying different types of vehicles and driving styles through cluster analysis, and adaptively adjusting driving strategies. Through this approach, the system can identify and adapt to different types of driving behaviors, thereby providing a more personalized and accurate assisted driving experience.

[0307] By introducing a traffic intelligent agent with characteristics of traffic violations, specifically designed to simulate and handle violation scenarios, the system can predict and effectively respond to violations, reducing the possibility of misjudgments and improving the accuracy and timeliness of the response.

[0308] The technical solution presented in this application not only comprehensively improves the intelligent driving system in terms of intelligence, safety, robustness, and traffic efficiency, but also significantly enhances its adaptability and scalability. This multi-dimensional improvement provides a solid technical foundation for the practical application and promotion of autonomous driving technology, meeting the needs of future intelligent transportation development.

[0309] This application proposes an innovative multi-class adaptive intelligent driving system based on clustering and alliance training. By introducing traffic agents with different behavioral types, including factors such as vehicle type, driving style, and traffic violation behavior, reinforcement learning is carried out using self-play and alliance training methods. This enables the intelligent driving system to adapt to complex and ever-changing traffic environments and improves the decision-making ability and safety performance of the intelligent driving system.

[0310] As can be seen from the above, the innovative application of this application in the autonomous driving assistance system achieves an efficient and intelligent autonomous driving assistance solution by combining cluster analysis, reinforcement learning, self-play, and alliance training strategies. This system not only copes with complex traffic scenarios but also improves safety and driving experience through continuous optimization and adaptation.

[0311] To better implement the above methods, this application also provides an intelligent driving implementation device, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster composed of multiple servers.

[0312] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the intelligent driving realization device as specifically integrated into an electronic device.

[0313] For example, such as Figure 3 As shown, the intelligent driving implementation device may include an acquisition unit 301, a clustering unit 302, a configuration construction unit 303, an agent selection unit 304, a policy update unit 305, and a vehicle guidance unit 306, as follows:

[0314] (I) Acquisition Unit 301.

[0315] The acquisition unit 301 is used to acquire traffic data in real road scenarios, including traffic data of multiple traffic participants.

[0316] (ii) Clustering unit 302.

[0317] Clustering unit 302 is used to perform cluster analysis on traffic data of multiple traffic participants to obtain traffic intelligent agents of at least one behavior type.

[0318] (III) Construct configuration unit 303.

[0319] The configuration unit 303 is used to construct a traffic simulation environment for training traffic agents and to configure multiple traffic agents with different behavior types in the traffic simulation environment.

[0320] (iv) Agent selection unit 304.

[0321] The agent selection unit 304 is used to select a target traffic agent for each behavior type from the traffic simulation environment.

[0322] (V) Strategy Update Unit 305.

[0323] The strategy update unit 305 is used to perform adversarial training on the target traffic agent and the reference traffic agent in a traffic simulation environment based on the current agent control strategy of the behavior type, to obtain adversarial information, and to update the agent control strategy of the behavior type according to the adversarial information. The reference traffic agent is the traffic agent in the traffic simulation environment.

[0324] In some embodiments, the apparatus further includes a cooperative countermeasure unit for:

[0325] By analyzing the adversarial interactions between various traffic agents in a traffic simulation environment, cooperative traffic agents are selected from among them.

[0326] Based on the current agent control strategies of the cooperative traffic agent and the target traffic agent, adversarial training is conducted on the cooperative traffic agent and the target traffic agent in a traffic simulation environment to obtain cooperative adversarial information.

[0327] Based on the cooperative adversarial information, the current agent control strategy of the cooperative traffic agent is updated.

[0328] In some embodiments, the cooperative adversarial unit is configured to:

[0329] The coordinated traffic intelligent agent is triggered to identify policy vulnerabilities of the target traffic intelligent agent through the current intelligent agent control strategy, and obtain information on policy vulnerabilities to be countered.

[0330] The collaborative traffic agent is triggered to conduct adversarial training with the target traffic agent in a traffic simulation environment based on the vulnerability information of the strategy to be challenged, thereby obtaining collaborative adversarial information.

[0331] In some embodiments, the cooperative adversarial unit is configured to:

[0332] Determine the alliance of traffic intelligent agents in a traffic simulation environment;

[0333] By observing the adversarial interactions between the various traffic intelligent agents in the traffic intelligent agent alliance, we can obtain information on the alliance adversarial results among the traffic intelligent agents in the traffic intelligent agent alliance.

[0334] Based on the results of the alliance confrontation, cooperative traffic intelligent agents are selected from the traffic intelligent agents in the traffic intelligent agent alliance.

[0335] In some embodiments, the apparatus further includes a training adversarial unit for:

[0336] Randomly select a training traffic intelligent agent from among the traffic intelligent agents in the Traffic Intelligent Agent Alliance to be a collaborating traffic intelligent agent;

[0337] Based on the current agent control strategies of the cooperative traffic agent and the training traffic agent, adversarial training is conducted on the cooperative traffic agent and the training traffic agent in a traffic simulation environment to obtain training adversarial information.

[0338] Based on the training adversarial information, the current agent control strategies of the cooperative traffic agent and the training traffic agent are updated.

[0339] In some embodiments, the adversarial unit is trained for:

[0340] Identify the target traffic intelligent agent and the adversary traffic intelligent agent;

[0341] Based on the current agent control strategies of the adversary traffic agent and the target traffic agent, adversarial training is conducted on the adversarial traffic agent and the target traffic agent in a traffic simulation environment to obtain defensive adversarial information.

[0342] Based on the defense and confrontation information, the current agent control strategy of the adversary traffic agent and the current agent control strategy of the target traffic agent are updated.

[0343] In some embodiments, the adversarial unit is trained for:

[0344] By observing the adversarial interactions between various traffic intelligent agents in a traffic simulation environment, information on the adversarial outcomes of each agent is obtained.

[0345] Based on the adversarial outcome information, adversarial traffic agents are identified from among the various traffic agents.

[0346] In some embodiments, the adversarial unit is trained for:

[0347] The adversarial traffic agent identifies policy vulnerabilities of the target traffic agent through the current agent control strategy and obtains policy vulnerability information to be referenced.

[0348] The adversarial traffic agent adjusts its current agent control strategy based on the vulnerability information of the policy to be referenced, thus obtaining the adjusted agent control strategy of the adversarial traffic agent.

[0349] Based on the adjusted agent control strategy of the adversarial traffic agent and the current agent control strategy of the target traffic agent, adversarial training is conducted on the adversarial traffic agent and the target traffic agent in a traffic simulation environment to obtain defensive adversarial information.

[0350] In some embodiments, the policy update unit is configured to:

[0351] Configure reward strategy information for traffic intelligent agents of various behavior types;

[0352] Based on the current intelligent agent control strategy according to the behavior type, adversarial training is carried out between the target traffic intelligent agent and the reference traffic intelligent agent in a traffic simulation environment to obtain adversarial information.

[0353] Based on the adversarial information and reward strategy information, determine the reward value information of the target traffic intelligent agent;

[0354] Update the agent control strategy based on the reward value information of the target traffic agent.

[0355] In some embodiments, the policy update unit is configured to:

[0356] Based on the current agent control strategy of the behavior type, when conducting adversarial training between the target traffic agent and the reference traffic agent in the traffic simulation environment, the current simulation environment information of the traffic simulation environment is obtained.

[0357] Based on the current simulation environment information, the interaction behavior between the target traffic agent and the reference traffic agent is evaluated and processed to obtain adversarial information.

[0358] (vi) Vehicle guidance unit 306.

[0359] The vehicle guidance unit 306 is used to provide driving guidance to vehicles in real road scenarios through updated intelligent agent control strategies for various behavior types.

[0360] In some embodiments, the vehicle guidance unit is configured to:

[0361] Collect real-time traffic information of the actual road scene where the target vehicle is located;

[0362] Based on real-time traffic information, select the target traffic agent control strategy from the updated agent control strategies for each behavior type.

[0363] The target traffic intelligent agent control strategy provides driving guidance to target vehicles in real road scenarios.

[0364] In some embodiments, the vehicle guidance unit is configured to:

[0365] Obtain strategy evaluation metrics;

[0366] Based on the policy evaluation index, the updated agent control policy for each behavior type is evaluated to obtain the policy evaluation information of the updated agent control policy for each behavior type.

[0367] Based on the policy evaluation information, the updated agent control policies for each behavior type are screened to obtain the screened agent control policies.

[0368] The selected intelligent agent control strategy provides driving guidance for vehicles in real road scenarios.

[0369] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0370] As described above, the intelligent driving implementation device in this embodiment acquires traffic data from a real road scenario by an acquisition unit, including traffic data from multiple traffic participants; a clustering unit performs cluster analysis on the traffic data of multiple traffic participants to obtain traffic intelligent agents of at least one behavior type; a construction and configuration unit constructs a traffic simulation environment for training traffic intelligent agents and configures multiple traffic intelligent agents of different behavior types in the traffic simulation environment; an intelligent agent selection unit selects a target traffic intelligent agent for each behavior type from the traffic simulation environment; a policy update unit performs adversarial training on the target traffic intelligent agent and a reference traffic intelligent agent in the traffic simulation environment based on the current intelligent agent control policy of the behavior type to obtain adversarial information, and updates the intelligent agent control policy of the behavior type according to the adversarial information, with the reference traffic intelligent agent being the traffic intelligent agent in the traffic simulation environment; and a vehicle guidance unit provides driving guidance to vehicles in a real road scenario through the updated intelligent agent control policies of each behavior type.

[0371] Therefore, the embodiments of this application can guide vehicles in real-world road scenarios to flexibly respond to various traffic situations. This improves the accuracy and timeliness of vehicles in dealing with complex traffic scenarios.

[0372] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0373] In some embodiments, the intelligent driving implementation device can also be integrated into multiple electronic devices. For example, the intelligent driving implementation device can be integrated into multiple servers, and the intelligent driving implementation method of this application can be implemented by multiple servers.

[0374] In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 4 As shown, it illustrates a schematic diagram of the server structure involved in an embodiment of this application. Specifically:

[0375] The server may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will understand that... Figure 4 The server architecture shown does not constitute a limitation on the server and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. Wherein:

[0376] The processor 401 is the control center of the server, connecting various parts of the server via various interfaces and lines. It performs various server functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 401.

[0377] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0378] The server also includes a power supply 403 that supplies power to the various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0379] The server may also include an input module 404, which can be used to receive input numeric or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0380] The server may also include a communication module 405. In some embodiments, the communication module 405 may include a wireless module, through which the server can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 405 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0381] Although not shown, the server may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the server loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402, thereby implementing the steps in the methods of the various embodiments of this application.

[0382] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0383] As shown above, traffic simulation environments can simulate various traffic scenarios that a target traffic agent under a given behavior type might face. This allows the target traffic agent to undergo adversarial training within these scenarios, continuously adjusting its current agent control strategy and enhancing its ability to cope with complex traffic scenarios. Since traffic agents are obtained through cluster analysis of traffic participant data, traffic agents of different behavior types can represent different traffic participants. Updated agent control strategies for each behavior type can guide vehicles in real-world road scenarios to flexibly respond to various traffic situations. This improves the accuracy and timeliness of vehicles in handling complex traffic scenarios.

[0384] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0385] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the intelligent driving implementation methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0386] Acquire traffic data from real-world road scenarios, including traffic data from multiple traffic participants;

[0387] Cluster analysis is performed on traffic data from multiple traffic participants to obtain traffic intelligent agents with at least one behavioral type;

[0388] Construct a traffic simulation environment for training traffic agents, and configure multiple traffic agents with different behavior types in the traffic simulation environment;

[0389] For each behavior type, select the target traffic agent of the behavior type from the traffic simulation environment;

[0390] Based on the current agent control strategy of the behavior type, conduct adversarial training on the target traffic agent and the reference traffic agent in the traffic simulation environment to obtain adversarial information, and update the agent control strategy of the behavior type according to the adversarial information. The reference traffic agent is a traffic agent in the traffic simulation environment;

[0391] Provide driving guidance for vehicles in the real road scenario through the updated agent control strategies of each behavior type.

[0392] Among them, the storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc.

[0393] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program includes computer programs / instructions, and the computer programs / instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer programs / instructions from the computer-readable storage medium, and the processor executes the computer programs / instructions, so that the electronic device executes the methods provided in various optional implementation manners of the intelligent driving implementation aspect provided in the above embodiments.

[0394] Since the instructions stored in the storage medium can execute the steps in any of the intelligent driving implementation methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any of the intelligent driving implementation methods provided in the embodiments of the present application can be realized. For details, please refer to the previous embodiments and will not be elaborated here.

[0395] The above has introduced in detail an intelligent driving implementation method and related devices provided in the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for implementing intelligent driving, characterized in that, The method comprises the following steps: acquiring traffic data in a real road scene, wherein the traffic data comprises traffic data of a plurality of traffic participants; performing cluster analysis on the traffic data of the plurality of traffic participants to obtain traffic agents of at least one behavior type; constructing a traffic simulation environment for training the traffic agents, and configuring a plurality of traffic agents of different behavior types in the traffic simulation environment; for each behavior type, selecting a target traffic agent of the behavior type from the traffic simulation environment; based on the current agent control strategy of the behavior type, performing adversarial training on the target traffic agent and a reference traffic agent in the traffic simulation environment to obtain adversarial information, updating the agent control strategy of the behavior type according to the adversarial information, and the reference traffic agent is a traffic agent in the traffic simulation environment; driving guidance is performed on vehicles in the real road scene through the updated agent control strategy of each behavior type. 2.The intelligent driving implementation method of claim 1, wherein, Before the driving guidance is performed on the vehicles in the real road scene through the updated agent control strategy of each behavior type, the method further comprises the following steps: selecting a cooperative traffic agent from the traffic agents in the traffic simulation environment through the adversarial relationship between the traffic agents in the traffic simulation environment; based on the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent, performing adversarial training on the cooperative traffic agent and the target traffic agent in the traffic simulation environment to obtain cooperative adversarial information; updating the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent according to the cooperative adversarial information. 3.The intelligent driving implementation method of claim 2, wherein, The method further comprises the following steps: triggering the cooperative traffic agent to identify the strategy vulnerability of the target traffic agent through the current agent control strategy to obtain to-be-attacked strategy vulnerability information; triggering the cooperative traffic agent to perform adversarial training with the target traffic agent in the traffic simulation environment based on the to-be-attacked strategy vulnerability information to obtain cooperative adversarial information. 4.The intelligent driving implementation method of claim 2, wherein, The method further comprises the following steps: determining a traffic agent alliance in the traffic simulation environment; obtaining alliance adversarial result information of the traffic agents in the traffic agent alliance through the adversarial relationship between the traffic agents in the traffic agent alliance; selecting a cooperative traffic agent from the traffic agents in the traffic agent alliance based on the alliance adversarial result information. 5.The intelligent driving implementation method of claim 4, wherein, Before the cooperative traffic agent and the target traffic agent are trained in the traffic simulation environment based on the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the target traffic agent to obtain cooperative confrontation information, the method comprises the following steps of: randomly selecting a sparring traffic agent of the cooperative traffic agent from the traffic agents in the traffic agent alliance; training the cooperative traffic agent and the sparring traffic agent in the traffic simulation environment based on the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the sparring traffic agent to obtain training confrontation information; updating the current agent control strategy of the cooperative traffic agent and the current agent control strategy of the sparring traffic agent according to the training confrontation information. 6.The intelligent driving implementation method of claim 1, wherein, Before the vehicle in the real road scene is guided to drive by the updated agent control strategy of each behavior type, the method further comprises the following steps of: determining a confrontation traffic agent of the target traffic agent; training the confrontation traffic agent and the target traffic agent in the traffic simulation environment based on the current agent control strategy of the confrontation traffic agent and the current agent control strategy of the target traffic agent to obtain defense confrontation information; updating the current agent control strategy of the confrontation traffic agent and the current agent control strategy of the target traffic agent according to the defense confrontation information. 7.The intelligent driving implementation method of claim 6, wherein, The method of determining the confrontation traffic agent of the target traffic agent comprises the following steps of: obtaining confrontation result information of each traffic agent through confrontation between the traffic agents in the traffic simulation environment; determining the confrontation traffic agent from the traffic agents based on the confrontation result information. 8.The intelligent driving implementation method of claim 6, wherein, The method of training the confrontation traffic agent and the target traffic agent in the traffic simulation environment based on the current agent control strategy of the confrontation traffic agent and the current agent control strategy of the target traffic agent to obtain defense confrontation information comprises the following steps of: triggering the confrontation traffic agent to identify strategy loopholes of the target traffic agent through the current agent control strategy to obtain to-be-referenced strategy loophole information; triggering the confrontation traffic agent to adjust the current agent control strategy of the confrontation traffic agent according to the to-be-referenced strategy loophole information to obtain the adjusted agent control strategy of the confrontation traffic agent; training the confrontation traffic agent and the target traffic agent in the traffic simulation environment based on the adjusted agent control strategy of the confrontation traffic agent and the current agent control strategy of the target traffic agent to obtain defense confrontation information. 9.The intelligent driving implementation method of claim 1, wherein, The method of guiding the vehicle in the real road scene to drive by the updated agent control strategy of each behavior type comprises the following steps of: collecting real-time road condition information of a real road scene where a target vehicle is located; selecting a target traffic agent control strategy from the updated agent control strategies of each behavior type according to the real-time road condition information; The target vehicle in the real road scene is guided to drive by the target traffic agent control strategy. 10.The intelligent driving implementation method of claim 1, wherein, The target traffic agent and a reference traffic agent are trained in the traffic simulation environment based on the current agent control strategy of the behavior type to obtain confrontation information, so as to update the agent control strategy of the behavior type according to the confrontation information, including: setting reward policy information of the traffic agent of each behavior type; The target traffic agent and a reference traffic agent are trained in the traffic simulation environment based on the current agent control strategy of the behavior type to obtain confrontation information; According to the confrontation information and the reward policy information, the reward value information of the target traffic agent is determined; According to the reward value information of the target traffic agent, the agent control strategy of the behavior type is updated. 11.The intelligent driving implementation method of claim 1, wherein, The target traffic agent and a reference traffic agent are trained in the traffic simulation environment based on the current agent control strategy of the behavior type to obtain confrontation information, including: When the target traffic agent and a reference traffic agent are trained in the traffic simulation environment based on the current agent control strategy of the behavior type, the current simulation environment information of the traffic simulation environment is obtained; According to the current simulation environment information, the interaction behavior between the target traffic agent and the reference traffic agent is evaluated and processed to obtain confrontation information. 12.The intelligent driving implementation method of claim 1, wherein, The vehicle in the real road scene is guided to drive by the updated agent control strategy of each behavior type, including: obtaining a policy evaluation index; According to the policy evaluation index, the updated agent control strategy of each behavior type is evaluated and processed to obtain policy evaluation information of the updated agent control strategy of each behavior type; According to the policy evaluation information, the updated agent control strategy of each behavior type is screened to obtain a screened agent control strategy; The vehicle in the real road scene is guided to drive by the screened agent control strategy.

13. An intelligent driving implementation apparatus, characterized by comprising: including: An acquisition unit is configured to acquire traffic data in a real road scene, the traffic data including traffic data of a plurality of traffic participants; A clustering unit is configured to perform clustering analysis on the traffic data of the plurality of traffic participants to obtain traffic agents of at least one behavior type; A configuration unit is configured to construct a traffic simulation environment for training the traffic agents and configure a plurality of traffic agents of different behavior types in the traffic simulation environment; An agent selection unit is configured to select a target traffic agent of each behavior type from the traffic simulation environment. a policy updating unit, configured to perform confrontation training on the target traffic agent and a reference traffic agent in the traffic simulation environment based on a current agent control policy of the behavior type, to obtain confrontation information, and to update the agent control policy of the behavior type according to the confrontation information, the reference traffic agent being a traffic agent in the traffic simulation environment; a vehicle guiding unit, configured to perform driving guidance on the vehicle in the real road scene by the updated agent control policy of each behavior type.

14. An electronic device, comprising: A processor and a memory are included, and the memory stores a plurality of instructions; the processor loads the instructions from the memory to execute the steps in the intelligent driving implementation method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions, and the instructions are adapted to be loaded by the processor to execute the steps in the intelligent driving implementation method according to any one of claims 1-12.

16. A computer program product, comprising a plurality of instructions, characterized in that, The instructions are executed by the processor to implement the steps in the intelligent driving implementation method according to any one of claims 1-12.

Images