Decision information generation method and apparatus, electronic device, and storage medium

Abstract

The disclosure provides a decision information generation method and device, electronic equipment and storage medium, and relates to the technical field of artificial intelligence and the technical field of automatic driving. The decision information generation method comprises: in response to receiving a driving service request of a target vehicle, obtaining real-time motion state information of the target vehicle, lane type information of a target area, and real-time relative motion state information of a traffic participant and the target vehicle. The real-time motion state information and the lane type information are processed by using a driving scene analysis model to obtain target driving scene information. According to the target driving scene information, a target driving behavior decision model is determined. The real-time motion state information and the real-time relative motion state information are processed by using the target driving behavior decision model to obtain driving action decision information of the target vehicle.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology and autonomous driving technology, and more specifically, to a decision information generation method, device, electronic device, storage medium, and program product. Background Art

[0002] With the continuous development of autonomous driving technology, autonomous vehicles, as intelligent devices that integrate multiple functions such as environmental perception, dynamic decision-making, route planning, behavior control, and command execution, are widely used in various fields. Related technologies typically make driving decisions based on static map data combined with pre-set rules to avoid static obstacles in the driving path.

[0003] In the process of realizing the concept of the present disclosure, the inventors discovered that there are at least the following problems in the related technologies: in actual driving environments, in addition to static obstacles, there are also dynamically changing dynamic obstacles, which leads to low accuracy of driving decisions obtained using related technologies, making the autonomous driving process safe. Summary of the Invention

[0004] In view of this, the present disclosure provides a decision information generation method, apparatus, electronic device, storage medium, and program product.

[0005] One aspect of the present disclosure provides a decision-making method, comprising:

[0006] In response to receiving a driving service request from a target vehicle, real-time motion state information of the target vehicle, lane type information of the target area, and real-time relative motion state information between traffic participants and the target vehicle are obtained. The real-time motion state information and lane type information are processed using a driving scenario analysis model to obtain target driving scenario information. Based on the target driving scenario information, a target driving behavior decision model is determined. The target driving behavior decision model is used to process the real-time motion state information and real-time relative motion state information to obtain driving action decision information for the target vehicle.

[0007] According to an embodiment of the present disclosure, real-time motion state information includes real-time driving direction information and real-time location information. The real-time motion state information and lane type information are processed using a driving scene analysis model to obtain target driving scene information, including:

[0008] The target task point's location information and the scene switching point's location information are determined from the target area. Based on the real-time location information and the scene switching point's location information, the distance between the real-time location and the scene switching point is obtained. Based on the target task point's location information, the lane type information of the target task point is retrieved from the lane type information in the target area. The driving scenario analysis model processes the distance information, real-time driving direction, and lane type information of the target task point to obtain the target driving scenario information.

[0009] According to an embodiment of the present disclosure, a driving scenario analysis model is used to process distance information, real-time driving direction, and lane type information of a target task point to obtain target driving scenario information, including:

[0010] Based on the real-time driving direction and the lane type information of the target task point, multiple candidate scene switching conditions are determined. Based on the distance information, the target driving scene information is determined from the multiple candidate scene switching conditions.

[0011] According to an embodiment of the present disclosure, the target driving behavior decision model is used to process the real-time motion state information and the real-time relative motion state information to obtain the driving action decision information of the target vehicle, including:

[0012] The target traffic participant is determined based on the target driving scenario information and the real-time relative motion state information. The target real-time relative motion state information of the target traffic participant and the target vehicle is filtered from the real-time relative motion state information. The target motion state attribute information is determined based on the target driving scenario information. The target real-time motion state information corresponding to the target motion state attribute information is filtered from the real-time motion state information of the target vehicle. The target real-time motion state information and the target real-time relative motion state information are analyzed using a target driving behavior decision model to obtain driving action decision information.

[0013] According to an embodiment of the present disclosure, determining a target traffic participant based on target driving scene information and real-time relative motion state information includes:

[0014] Based on real-time relative motion information, the relative positional relationship between the traffic participants and the target vehicle is determined. Based on this relative positional relationship, the risk level of the traffic participants to the target vehicle in the target driving scenario is determined. Based on this risk level, the target traffic participant is selected from the traffic participants.

[0015] According to an embodiment of the present disclosure, determining the risk level of traffic participants to a target vehicle in a target driving scenario based on relative position relationships includes:

[0016] Based on the relative position relationship, the correlation between the traffic participants and the driving trajectory of the target vehicle is obtained. Based on the correlation between the driving trajectories, the risk level is determined.

[0017] According to an embodiment of the present disclosure, a method for training a target driving behavior decision model includes:

[0018] In a simulated driving scenario, the system obtains information about the test vehicle's motion state and the relative motion state of traffic participants. Driving action information is randomly selected from the target vehicle's action probability distribution space. Using a proximal policy optimization algorithm, this information, along with the test vehicle's motion state and the relative motion state of traffic participants, is processed to train a pre-set model and generate a target driving behavior decision model.

[0019] According to an embodiment of the present disclosure, the driving action information includes S driving action information, where S is an integer greater than 1. The preset model includes a policy network and a value network. Based on the proximal policy optimization algorithm, the driving action information, the motion state information of the test vehicle, and the relative motion state information of the traffic participants and the test vehicle are processed to train the preset model to obtain a target driving behavior decision model, including:

[0020] For the sth driving action information, the policy network is used to process the sth driving action information, the motion state information of the test vehicle, and the relative motion state information between the traffic participants and the test vehicle to obtain the sth driving result information, where s is an integer greater than or equal to 1 and less than S. The value network is used to process the sth driving action information and the sth driving result information to obtain action value information. Based on the objective function, a policy advantage value is obtained based on the driving state information and the action value information. If it is determined that the policy advantage value does not meet a predetermined threshold, model parameters of the preset model are adjusted, and the processing operations using the policy network and the processing operations using the value network are returned to be executed, and s is incremented. If it is determined that the policy advantage value meets the predetermined threshold, the sth driving action information is determined as the target decision action information.

[0021] Another aspect of the present disclosure provides a decision information generation device, comprising: an acquisition module, an analysis module, a determination module, and a generation module. The acquisition module is used to obtain the real-time motion state information of the target vehicle, the lane type information of the target area, and the real-time relative motion state information between the traffic participants and the target vehicle in response to receiving a driving service request from the target vehicle. The analysis module is used to process the real-time motion state information and the lane type information using a driving scene analysis model to obtain target driving scene information. The determination module is used to determine a target driving behavior decision model based on the target driving scene information. The generation module is used to process the real-time motion state information and the real-time relative motion state information using a target driving behavior decision model to obtain driving action decision information of the target vehicle.

[0022] According to an embodiment of the present disclosure, real-time motion state information includes real-time driving direction information and real-time position information. The analysis module includes: a first determination unit, a first acquisition unit, a first query unit, and a second acquisition unit. Among them, the first determination unit is used to determine the position information of the target task point and the position information of the scene switching point from the target area. The first acquisition unit is used to obtain the distance information between the real-time position and the scene switching point based on the real-time position information and the position information of the scene switching point. The first query unit is used to query the lane type information of the target task point from the lane type information of the target area based on the position information of the target task point. The second acquisition unit is used to process the distance information, real-time driving direction and lane type information of the target task point using the driving scene analysis model to obtain the target driving scene information.

[0023] According to an embodiment of the present disclosure, the second obtaining unit includes a first determining subunit and a second determining subunit. The first determining subunit is configured to determine multiple candidate scene switching conditions based on the real-time driving direction and lane type information of the target task point. The second determining subunit is configured to determine target driving scene information from the multiple candidate scene switching conditions based on distance information.

[0024] According to an embodiment of the present disclosure, the generation module includes a second determination unit, a first screening unit, a third determination unit, a second screening unit, and a third acquisition unit. Among them, the second determination unit is used to determine the target traffic participant based on the target driving scene information and the real-time relative motion state information. The first screening unit is used to screen the target real-time relative motion state information of the target traffic participant and the target vehicle from the real-time relative motion state information. The third determination unit is used to determine the attribute information of the target motion state based on the target driving scene information; the second screening unit is used to screen the target real-time motion state information corresponding to the attribute information of the target motion state from the real-time motion state information of the target vehicle. The third determination unit is used to process the target real-time motion state information and the target real-time relative motion state information using the target driving behavior decision model to obtain driving action decision information.

[0025] According to an embodiment of the present disclosure, the second determination unit includes a third determination subunit, a fourth determination subunit, and a screening subunit. The third determination subunit is configured to determine the relative positional relationship between the traffic participant and the target vehicle based on real-time relative motion state information. The fourth determination subunit is configured to determine the risk level of the traffic participant to the target vehicle in the target driving scenario based on the relative positional relationship. The screening subunit is configured to screen the target traffic participant from the traffic participants based on the risk level.

[0026] According to an embodiment of the present disclosure, the fourth determination subunit is used to obtain an association relationship between the traffic participant object and the driving trajectory of the target vehicle based on the relative position relationship; and determine the risk level based on the association relationship of the driving trajectory.

[0027] According to an embodiment of the present disclosure, the generation module includes an acquisition unit and a training unit. The acquisition unit is used to obtain the test vehicle's motion state information and the relative motion state information between traffic participants and the test vehicle in a simulated driving scenario. The training unit is used to train a preset model based on the proximal policy optimization algorithm, using the driving action information, the test vehicle's motion state information, and the relative motion state information between traffic participants and the test vehicle to obtain a target driving behavior decision model.

[0028] According to an embodiment of the present disclosure, the driving maneuver information includes S driving maneuver information, where S is an integer greater than 1. The preset model includes a policy network and a value network. The training unit includes a processing subunit, a first obtaining subunit, a second obtaining subunit, and an adjustment subunit. The processing subunit is configured to process the sth driving maneuver information, the motion state information of the test vehicle, and the relative motion state information between the traffic participants and the test vehicle using the policy network to obtain sth driving result information, where s is an integer greater than or equal to 1 and less than S. The first obtaining subunit is configured to process the sth driving maneuver information and the sth driving result information using the value network to obtain action value information. The second obtaining subunit is configured to obtain a policy advantage value based on the driving state information and the action value information based on an objective function. The adjustment subunit is configured to, if it is determined that the policy advantage value does not meet a predetermined threshold, adjust model parameters of the preset model, return to executing processing operations using the policy network and processing operations using the value network, and increment s. If it is determined that the policy advantage value meets the predetermined threshold, the sth driving maneuver information is determined to be target decision action information.

[0029] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0030] Another aspect of the present disclosure provides a computer program product comprising computer executable instructions, which are used to implement the method described above when the instructions are executed.

[0031] According to the embodiments of the present disclosure, because a technical means is adopted in which a driving scene analysis model is used to determine target driving scene information from real-time motion state information and lane type information, and then a target driving behavior decision model corresponding to the target driving scene is used to process the real-time motion state information and the real-time relative motion state information to obtain driving action decision information of the target vehicle, the technical problem of low decision accuracy caused by making driving decisions based on information about static obstacles in the related art is at least partially overcome, thereby achieving the technical effect of dynamically making decisions on the driving actions of the target vehicle by combining the real-time motion state of the dynamic target vehicle, the relative motion state information of traffic participants and the target vehicle, and the lane type information of the target area to improve decision accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0033] Figure 1The following schematically illustrates an application scenario diagram to which the decision information generation method disclosed herein can be applied;

[0034] Figure 2 The following schematically shows a flow chart of a method for generating decision information according to an embodiment of the present disclosure;

[0035] Figure 3 The exemplary architecture of decision information generation in which the embodiments of the present disclosure can be applied is schematically implemented;

[0036] Figure 4 A schematic diagram schematically illustrates a driving scene switching according to an embodiment of the present disclosure;

[0037] Figure 5 The following schematically shows a flow chart of a method for training a target behavior decision model according to an embodiment of the present disclosure;

[0038] Figure 6 The following schematically shows a logic flow chart of a training method for a target behavior decision model according to an embodiment of the present disclosure;

[0039] Figure 7 A block diagram schematically illustrates a device for generating decision information according to an embodiment of the present disclosure; and

[0040] Figure 8 A block diagram of an electronic device suitable for implementing the decision information generation method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0041] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0042] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0043] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0044] When expressions such as "at least one of A, B and C, etc." are used, they should generally be interpreted in accordance with the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0045] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of the data involved (for example, including but not limited to user personal information) all comply with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken with respect to user personal information to prevent unauthorized access to user personal information data and to maintain the security of user personal information, network security, and national security.

[0046] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0047] In related technologies, autonomous vehicle behavior decision-making models typically determine the location of static obstacles, such as green belts, traffic signs, traffic lights, and roadblocks, based on map information along the planned route and images collected by the vehicle. Based on predetermined rules, driving decisions are made based on the distance between the vehicle and the static obstacle.

[0048] However, real-world driving scenarios are complex and diverse, including those at intersections, at lane merges, at lane separations, and within regular lanes. The traffic participants a target vehicle encounters in these scenarios can change dynamically, significantly impacting the target vehicle's driving decisions. Therefore, decision-making methods based on predetermined rules struggle to accurately determine driving maneuvers in complex driving scenarios, exposing the target vehicle to safety risks such as collisions with other traffic participants.

[0049] In view of this, embodiments of the present disclosure provide a method for generating decision information. The method includes, in response to receiving a driving service request from a target vehicle, obtaining real-time motion state information of the target vehicle, lane type information of a target area, and real-time relative motion state information between traffic participants and the target vehicle. The real-time motion state information and lane type information are processed using a driving scenario analysis model to obtain target driving scenario information. Based on the target driving scenario information, a target driving behavior decision model is determined. The target driving behavior decision model is used to process the real-time motion state information and real-time relative motion state information to obtain driving action decision information for the target vehicle.

[0050] Figure 1 The following schematically illustrates an application scenario diagram to which the decision information generation method disclosed herein can be applied.

[0051] It should be noted that Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0052] like Figure 1 As shown, the system architecture 100 according to this embodiment may include an onboard terminal 101 on a target vehicle, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the target vehicle 101 and the server 103. The network 102 may include various connection types, such as a wireless communication link, etc.

[0053] The user can use the vehicle-mounted terminal 101 on the target vehicle to interact with the server 103 via the network 102 to receive or send messages, etc. The vehicle-mounted terminal 101 on the target vehicle can be installed with various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software (only as examples).

[0054] The vehicle-mounted terminal 101 on the target vehicle may be any electronic device having a display screen and supporting web browsing, including but not limited to a tablet computer.

[0055] The server 103 may be a server that provides various services, such as a background management server that supports websites browsed by users using the vehicle-mounted terminal 101 in the target vehicle (for example only). The background management server may analyze and process received data such as user driving service requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the vehicle-mounted terminal of the terminal device.

[0056] It should be noted that the decision information generation method provided in the embodiment of the present disclosure can generally be executed by the server 103. Accordingly, the decision information generation device provided in the embodiment of the present disclosure can generally be set in the server 103. The decision information generation method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 103 and can communicate with the vehicle terminal 101 and / or the server 103. Accordingly, the decision information generation device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 103 and can communicate with the vehicle terminal 101 and / or the server 103. Alternatively, the decision information generation method provided in the embodiment of the present disclosure can also be executed by the vehicle terminal 101, or can also be executed by other terminal devices different from the vehicle terminal 101. Accordingly, the decision information generation device provided in the embodiment of the present disclosure can also be set in the vehicle terminal 101, or can be set in other terminal devices different from the vehicle terminal 101.

[0057] For example, the target vehicle can collect the real-time motion state of the target vehicle and the real-time relative motion state of the traffic participants and the target vehicle through the on-board sensors, and the lane information of the target area can be stored in the on-board terminal 101. Then, the on-board terminal 101 can locally execute the decision information generation method provided by the embodiment of the present disclosure, or send the real-time motion state of the target vehicle, the real-time relative motion state of the traffic participants and the target vehicle, and the lane information of the target area to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the real-time motion state of the target vehicle, the real-time relative motion state of the traffic participants and the target vehicle, and the lane information of the target area can execute the decision information generation method provided by the embodiment of the present disclosure.

[0058] It should be understood that Figure 1 The number of vehicle-mounted terminals, networks, and servers in the embodiment is merely illustrative. Any number of terminal devices, networks, and servers may be provided as required.

[0059] Figure 2 The flowchart of the decision information generation method according to the embodiment of the present disclosure is schematically shown.

[0060] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0061] In operation S210 , in response to receiving a driving service request of a target vehicle, real-time motion state information of the target vehicle, lane type information of a target area, and real-time relative motion state information between traffic participants and the target vehicle are acquired.

[0062] In operation S220 , the real-time motion state information and lane type information are processed using a driving scene analysis model to obtain target driving scene information.

[0063] In operation S230 , a target driving behavior decision model is determined based on the target driving scenario information.

[0064] In operation S240 , the real-time motion state information and the real-time relative motion state information are processed using the target driving behavior decision model to obtain driving action decision information of the target vehicle.

[0065] According to an embodiment of the present disclosure, the real-time motion state information of the target vehicle may include real-time driving direction information and real-time position information of the target vehicle. The real-time driving direction information may be the heading angle information of the driving direction of the target vehicle.

[0066] According to an embodiment of the present disclosure, the target area may be a map area between the real-time location of the target vehicle and the location of the destination. The lane type information may include lane merging and lane separation.

[0067] According to an embodiment of the present disclosure, traffic participants may be other vehicles, pedestrians, etc. in the target area except the target vehicle. Real-time relative motion state information with respect to the target vehicle may include: relative speed, relative heading angle, relative acceleration, etc. of other vehicles and the target vehicle.

[0068] According to an embodiment of the present disclosure, the driving scenario analysis model may be a finite state machine (FSM) driving scenario transition model, and the driving scenario may be determined based on a state transition trigger condition.

[0069] For example, the state transition trigger condition may be that the real-time movement direction of the target vehicle is toward an intersection, and the distance between the target vehicle and the next intersection is greater than a set threshold D1, then the driving scene may be determined to be a driving scene on an ordinary road.

[0070] In the disclosed embodiments, driving scenarios may include driving scenarios at lane separation, lane merging, intersections, and ordinary roads. Different behavioral decision models can be trained for each driving scenario. When a target vehicle triggers a specific driving scenario, the driving behavior decision model corresponding to that driving scenario can be switched to analyze real-time driving environment state information to obtain driving action decision information for the target vehicle.

[0071] Figure 3 An exemplary architecture for generating decision information to which the embodiments of the present disclosure can be applied is schematically illustrated.

[0072] like Figure 3As shown, this embodiment 300 may include a driving scenario analysis model 301, a driving behavior decision model set 302, and a target driving behavior decision model 303. The behavior decision model set may include driving behavior decision models corresponding to different driving scenarios. For example, a driving behavior decision model M1 (302-1) corresponding to driving scenario M1, a driving behavior decision model M2 (302-2) corresponding to driving scenario M2, and a driving behavior decision model M3 (302-3) corresponding to driving scenario M3.

[0073] According to an embodiment of the present disclosure, driving scenario analysis model 301 can obtain a target driving scenario based on the real-time operating status of the target vehicle and the lane type information of the target area. Based on the target driving scenario, target driving behavior decision model 303 is retrieved from behavior decision model set 302. For example, if the target driving scenario is M1, target driving behavior decision model 303 can be driving behavior decision model M1 (302-1) in behavior decision model set 302.

[0074] The target driving behavior model 303 is used to obtain the target vehicle's driving action decision information based on the target vehicle's real-time operating status and the real-time relative motion status information between the traffic participants and the target vehicle.

[0075] According to the embodiments of the present disclosure, because a technical means is adopted in which a driving scene analysis model is used to determine target driving scene information from real-time motion state information and lane type information, and then a target driving behavior decision model corresponding to the target driving scene is used to process the real-time motion state information and the real-time relative motion state information to obtain driving action decision information of the target vehicle, the technical problem of low decision accuracy caused by making driving decisions based on information about static obstacles in the related art is at least partially overcome, thereby achieving the technical effect of dynamically making decisions on the driving actions of the target vehicle by combining the real-time motion state of the dynamic target vehicle, the relative motion state information of traffic participants and the target vehicle, and the lane type information of the target area to improve decision accuracy.

[0076] According to an embodiment of the present disclosure, using a driving scene analysis model to process real-time motion state information and lane type information to obtain target driving scene information may include the following operations:

[0077] The target task point's location information and the scene switching point's location information are determined from the target area. Based on the real-time location information and the scene switching point's location information, the distance between the real-time location and the scene switching point is obtained. Based on the target task point's location information, the lane type information of the target task point is retrieved from the lane type information in the target area. The driving scenario analysis model processes the distance information, real-time driving direction, and lane type information of the target task point to obtain the target driving scenario information.

[0078] According to an embodiment of the present disclosure, the target mission point can be any position on the lane driving path within the target area. In order to improve the accuracy of the driving behavior decision of the target vehicle, multiple target mission points can be set.

[0079] According to embodiments of the present disclosure, a scene switching point can be a location point used for conditional determination in scene switching conditions. For example, in a scene switching condition where a normal road driving scene switches to an intersection driving scene, the scene switching point can be the next intersection closest to the current location of the target vehicle on the driving path. When the target vehicle is moving toward the next intersection and the distance between the target vehicle and the next intersection is less than a predetermined threshold, the driving scene can be determined to have switched to the intersection driving scene.

[0080] According to an embodiment of the present disclosure, the real-time motion state information may include real-time driving direction information and real-time location information. The lane type information of the target task point may include: lane merging, lane separation, etc.

[0081] According to an embodiment of the present disclosure, processing distance information, real-time driving direction, and lane type information of a target task point using a driving scenario analysis model to obtain target driving scenario information may include the following operations:

[0082] Based on the real-time driving direction and the lane type information of the target task point, multiple candidate scene switching conditions are determined. Based on the distance information, the target driving scene information is determined from the multiple candidate scene switching conditions.

[0083] Figure 4 A schematic diagram schematically illustrates a driving scene switching according to an embodiment of the present disclosure.

[0084] like Figure 4 As shown, in embodiment 400, when the real-time motion state of the target vehicle and the lane type information of the target area meet the scene switching condition A1, it indicates that the current driving scene is a normal road scene.

[0085] When the target vehicle is traveling on a normal road, and the target vehicle's real-time motion state and the target area's lane type information meet scene switching condition A2, the normal road scene switches to a lane separation scene. When the target vehicle's real-time motion state and the target area's lane type information meet scene switching condition A7, the normal road scene switches to a lane merging scene. When the target vehicle's real-time motion state and the target area's lane type information meet scene switching condition A5, the normal road scene switches to an intersection scene.

[0086] When the target vehicle is traveling in a lane separation scenario, and the real-time motion state of the target vehicle and the lane type information of the target area meet the scene switching condition A3, the lane separation scenario is switched to the normal road scenario.

[0087] When the target vehicle is traveling in the lane merging scene and the real-time motion state of the target vehicle and the lane type information of the target area meet the scene switching condition A6, the lane merging scene is switched to the normal road scene.

[0088] When the target vehicle is traveling in an intersection scene, and the real-time motion state of the target vehicle and the lane type information of the target area meet the scene switching condition A4, the intersection scene is switched to the ordinary road scene.

[0089] According to an embodiment of the present disclosure, the scene switching condition may be preset to the driving scene analysis model.

[0090] For example, the scene switching condition A1 may be: the real-time moving direction of the target vehicle is toward the next intersection, and the distance between the real-time position of the target vehicle and the next intersection is greater than a predetermined threshold D1.

[0091] The scene switching condition A2 can be: the lane type of the next mission point of the target vehicle is lane separation, the real-time movement direction of the target vehicle is toward the lane separation point, and the distance between the real-time position of the target vehicle and the lane separation point is less than the predetermined threshold D1.

[0092] The scene switching condition A3 can be: the lane type of the target vehicle's previous task point is lane separation, and the real-time movement direction of the target vehicle is away from the lane separation point, and the distance between the real-time position of the target vehicle and the lane separation point is greater than the predetermined threshold D2, and the angle between the real-time movement direction of the target vehicle and the direction of the next lane line is less than the predetermined angle.

[0093] The scene switching condition A4 may be: the real-time moving direction of the target vehicle is away from the intersection, and the real-time position of the target vehicle is greater than a predetermined threshold D2 from the exit point of the current intersection.

[0094] The scene switching condition A5 may be: the real-time moving direction of the target vehicle is toward the intersection, and the real-time position of the target vehicle is less than a predetermined threshold D1 from the merging point of the next intersection.

[0095] The scene switching condition A6 can be: the lane type of the target vehicle's previous task point is lane merging, the real-time movement direction of the target vehicle is away from the lane merging point, and the distance between the target vehicle's position and the lane merging point is greater than the predetermined threshold D2, and the angle between the target vehicle's real-time movement direction and the direction of the next lane line is less than the predetermined angle.

[0096] The scene switching condition A7 may be: the next mission point of the target vehicle is lane merging, and the distance between the real-time position of the target vehicle and the lane merging point is less than a predetermined threshold D1.

[0097] According to the actual example disclosed in the present invention, the driving scenario analysis model can be used to switch the driving scenario of the target vehicle in real time to make driving behavior decisions for different driving scenarios and improve the accuracy of decision-making.

[0098] According to an embodiment of the present disclosure, processing the real-time motion state information and the real-time relative motion state information using the target driving behavior decision model to obtain driving action decision information of the target vehicle may include the following operations:

[0099] The target traffic participant is determined based on the target driving scenario information and the real-time relative motion state information. The target real-time relative motion state information of the target traffic participant and the target vehicle is filtered from the real-time relative motion state information. Attribute information of the target motion state is determined based on the target driving scenario information. The target real-time motion state information corresponding to the target motion state attribute information is filtered from the real-time motion state information of the target vehicle. The target real-time motion state information and the target real-time relative motion state information are processed using a target driving behavior decision model to obtain driving action decision information.

[0100] According to the embodiments of the present disclosure, due to different driving scenarios, the traffic participants that need to be considered when making driving action decisions may also vary. For example, when entering a lane merging scenario, the traffic participants may include vehicles in lanes adjacent to the target vehicle, as well as vehicles in the lane currently occupied by the target vehicle. When entering an intersection scenario, the traffic participants may include vehicles in the intersection area that may be in a driving conflict with the target vehicle, such as vehicles entering the intersection from the opposite lane.

[0101] For example, in a lane merging or lane splitting scenario, the target traffic participant can be a vehicle in the same lane as the target vehicle or a vehicle in an adjacent lane. In this case, the target real-time relative motion state information between the target traffic participant and the target vehicle may include: relative speed, relative heading angle, relative acceleration, relative position, collision time, and relative distance from the merging point or exit point. The properties of the target motion state may include: the target vehicle's acceleration, steering wheel angle, heading angle, relative position to the destination, heading angle error to the preceding path point, distance to the current lane centerline, and distance to the merging point or exit point.

[0102] According to an embodiment of the present disclosure, the filtered target real-time relative motion state information and target real-time motion state information can be processed using a driving behavior decision model corresponding to a lane merging scenario to obtain target decision action information.

[0103] According to the embodiments of the present disclosure, based on different driving scenarios, the motion state attributes of the traffic participants and the target vehicles that need to be observed are screened out, and the target real-time relative motion state of the traffic participants and the target vehicles is obtained. The target real-time motion state can be used to obtain driving action decision information suitable for the driving scenario based on the characteristics of the real-time driving states of the traffic participants and the target vehicles in different driving scenarios.

[0104] According to an embodiment of the present disclosure, determining a target traffic participant based on target driving scene information and real-time relative motion state information may include the following operations:

[0105] Based on real-time relative motion information, the relative positional relationship between the traffic participants and the target vehicle is determined. Based on this relative positional relationship, the risk level of the traffic participants to the target vehicle in the target driving scenario is determined. Based on this risk level, the target traffic participant is selected from the traffic participants.

[0106] According to the embodiments of the present disclosure, intersections are relatively complex scenarios in real-world driving scenarios, requiring a large number of observed traffic participants and a diverse range of dynamic changes in the relative positions of these participants and the target vehicle. Therefore, when making driving decisions at intersections, the relative positions of these participants must be considered to determine whether these participants pose a risk to the target vehicle's driving process.

[0107] According to an embodiment of the present disclosure, the relative position relationship between the traffic participant object and the target vehicle may include: merging into the same lane position; the motion trajectory is expected to intersect, the traffic participant object is located in the adjacent lane of the target vehicle, the traffic participant object is located in the current lane of the target vehicle, etc.

[0108] For example, if the trajectory of a traffic participant and a target vehicle are expected to intersect, and the relative speed between the traffic participant and the target vehicle is high, it can be determined that the traffic participant poses a high degree of safety risk to the target vehicle in the intersection scenario. The degree of risk can be determined based on the relative distance between the traffic participant and the target vehicle according to predetermined judgment rules. If the risk level exceeds a predetermined risk threshold, the traffic participant is determined as a target traffic participant that requires special attention while the target vehicle is traveling in the intersection scenario.

[0109] According to an embodiment of the present disclosure, determining the risk level of traffic participants to a target vehicle in a target driving scenario based on relative positional relationships may include the following operations:

[0110] Based on the relative position relationship, the correlation between the traffic participants and the driving trajectory of the target vehicle is obtained. Based on the correlation between the driving trajectories, the risk level is determined.

[0111] According to an embodiment of the present disclosure, the association relationship of driving trajectories may include whether there is an association, for example: when the traffic participant and the target vehicle are in opposite lanes, the driving trajectories of the two may collide, indicating that the driving trajectories of the two are associated. The degree of risk can be determined based on the association of the driving trajectories between the traffic participant and the target vehicle. For example: the driving trajectories of the two overlap in opposite directions, indicating that the two may collide, and the corresponding risk level is higher. The driving trajectories of the two overlap in the same direction, indicating that the two may merge into the same lane. Although there is a risk, the risk level is lower than the risk level of the driving trajectories overlapping in opposite directions.

[0112] According to the embodiments of the present disclosure, the relative positional relationship between traffic participants and the target vehicle can be used to determine the risk level of each traffic participant to the target vehicle in the current driving scenario. This allows for the selection of higher-risk traffic participants as the traffic participants that the driving behavior decision model should focus on when making decisions. This eliminates redundant information and improves the accuracy of the model's decisions.

[0113] The target driving behavior decision model can be obtained by training based on a reinforcement learning algorithm. However, in traditional reinforcement learning algorithms, such as the Q-learning algorithm, the expected driving action can be output by inputting the driving action and the driving environment state. However, since the strategy to be updated and the strategy used to interact with the environment are the same strategy in traditional reinforcement learning algorithms, the data is relatively simple, which easily causes the algorithm to fall into an infinite loop of bad strategy-bad data-worse strategy, making the training process difficult to converge. Therefore, in the embodiment of the present disclosure, the proximal policy optimization algorithm (PPO: Proximal Policy Optimization) is used to solve the problem of slow convergence by limiting the policy update amplitude.

[0114] Figure 5 The training method of the target driving behavior decision model according to an embodiment of the present disclosure is schematically illustrated.

[0115] like Figure 5 As shown, the method 500 may include operations S510 to S530.

[0116] In operation S510 , in a simulated driving scenario, motion state information of a test vehicle and relative motion state information of traffic participants and the test vehicle are obtained.

[0117] In operation S520 , driving action information is randomly selected from the action probability distribution space of the target vehicle.

[0118] In operation S530, based on the proximal strategy optimization algorithm, the preset model is trained by processing the driving action information, the motion state information of the test vehicle, and the relative motion state information of the traffic participants and the test vehicle to obtain a target driving behavior decision model.

[0119] According to embodiments of the present disclosure, simulated driving scenarios can be scenarios identical to target driving scenarios, simulated in a simulated driving test environment, and used for model training. In embodiments of the present disclosure, actual driving scenarios are primarily categorized into the following four types: ordinary road scenarios, lane merging scenarios, lane separation scenarios, and intersection scenarios. For each driving scenario, a target behavior decision model is generated using the training methods provided in embodiments of the present disclosure.

[0120] According to an embodiment of the present disclosure, the driving action information includes S driving action information, where S is an integer greater than 1, and the preset model includes a strategy network and a value network.

[0121] Figure 6 The following schematically illustrates a logic flow chart of a method for training a target driving behavior decision model according to an embodiment of the present disclosure.

[0122] like Figure 6As shown, in embodiment 600, operations S610 to S650 may be included.

[0123] In operation S610, for the s-th driving action information, the policy network is used to process the s-th driving action information, the motion state information of the test vehicle, and the relative motion state information between the traffic participants and the test vehicle to obtain the s-th driving result information, where s is an integer greater than or equal to 1 and less than S.

[0124] In operation S620 , the s th driving action information and the s th driving result information are processed using a value network to obtain action value information.

[0125] In operation S630 , a policy advantage value is obtained according to the driving state information and the action value information based on the objective function.

[0126] In operation S640, it is determined whether the strategy advantage value meets a preset threshold. If so, operation S650 is executed. If not, the model parameters are adjusted, s is incremented, and the process returns to operation S610.

[0127] In operation S650 , if it is determined that the strategy advantage value satisfies a predetermined threshold, the s-th driving action information is determined as target decision action information.

[0128] According to an embodiment of the present disclosure, the objective function may be a function used in the PPO algorithm to limit the policy update range. The update range of the policy output space may be controlled by measuring the proportional term of the difference between the policy to be updated and the sampled policy.

[0129] According to the embodiments of the present disclosure, by processing the sth driving action information and the sth driving result information using a value network, action value information can be obtained based on the value function of the Q-learning algorithm. Reinforcement learning methods based on the Q-learning algorithm are relatively mature technologies and will not be described in detail here.

[0130] It should be noted that when applying the reinforcement learning method of the Q-learning algorithm, the embodiment of the present disclosure can design the reward function of the reinforcement learning from three aspects: safety, driving smoothness and compliance with traffic rules of the target vehicle.

[0131] For example, the reward function can be expressed as follows:

[0132] r=r c +r ttc +r o +r goal +r reach +r steering +r lane +r speed (1):

[0133] r c is the penalty term for a collision of the test vehicle. If a collision occurs, r c =-50, otherwise r c =0; r ttc =∑0.01*(now_ttc i -last_ttc i ) is a reward item to guide the test vehicle to increase the distance between other vehicles, now_ttc i is the TTC (time to collision) between the test vehicle and the i-th vehicle in the current frame, last_ttc i is the TTC between the test vehicle and the i-th vehicle in the previous frame. If the collision time increases, the test vehicle will be rewarded, otherwise it will be penalized; r o is the penalty item for the test vehicle going off the road. If the vehicle goes off the road r o =-50, otherwise r o = 0. These three items are designed to ensure the safety of the test vehicle.

[0134] r goal =-0.01*d goal , is inversely proportional to the distance between the test vehicle and the target point. The greater the distance, the smaller the reward; r reach Is the reward item for the test vehicle to reach the target point. If it reaches the target point r reach =50, otherwise r reach = 0. These two items are designed to ensure that the test vehicle can reach the target point.

[0135] r steering =-((speed-60)*steering*k) 2 This option only takes effect when the test vehicle's speed exceeds 60 km / h. It is inversely proportional to the steering wheel angle, with the larger the angle, the smaller the reward. This is designed to ensure a smooth ride.

[0136] r lane =-0.01*d lane , which is inversely proportional to the lateral distance between the test vehicle and the center line of the current lane. The larger the distance, the smaller the reward. This item guides the test vehicle to drive along the center line of the lane as much as possible; r speed =-0.01*(speed-speed_limit). This item will only take effect when the test vehicle's speed exceeds the road speed limit. The more the speed exceeds the road speed limit, the smaller the reward. These two items are designed to ensure compliance with traffic regulations.

[0137] The action space of the test vehicle in this disclosed embodiment is designed to be a continuous action space. That is, the output of the target driving behavior decision model is [steering wheel angle, brake force, throttle opening]. Accordingly, a continuous PPO algorithm should be selected. In this case, the output of the target driving behavior model is finally transformed to the range of [-1, 1] using a tanh function.

[0138] According to the embodiments of the present disclosure, by combining the proximal policy optimization algorithm with the traditional Q-learning algorithm, the policy update amplitude of the reinforcement learning training process is limited, so that the model reaches the convergence condition faster, and the model accuracy is also higher, which can improve the efficiency of model training.

[0139] Figure 7 The block diagram of the decision information generating device according to an embodiment of the present disclosure is schematically shown.

[0140] like Figure 7 As shown, the decision information generating device 700 of this embodiment may include: an acquisition module 710 , an analysis module 720 , a determination module 730 and a generation module 740 .

[0141] The acquisition module 710 is used to obtain the real-time motion status information of the target vehicle, the lane type information of the target area, and the real-time relative motion status information of the traffic participants and the target vehicle in response to receiving the driving service request of the target vehicle.

[0142] The analysis module 720 is configured to process the real-time motion state information and the lane type information using a driving scene analysis model to obtain target driving scene information.

[0143] The determination module 730 is configured to determine a target driving behavior decision model based on the target driving scenario information.

[0144] The generating module 740 is configured to process the real-time motion state information and the real-time relative motion state information using a target driving behavior decision model to obtain driving action decision information of the target vehicle.

[0145] According to an embodiment of the present disclosure, real-time motion state information includes real-time driving direction information and real-time position information. The analysis module includes: a first determination unit, a first acquisition unit, a first query unit, and a second acquisition unit. Among them, the first determination unit is used to determine the position information of the target task point and the position information of the scene switching point from the target area. The first acquisition unit is used to obtain the distance information between the real-time position and the scene switching point based on the real-time position information and the position information of the scene switching point. The first query unit is used to query the lane type information of the target task point from the lane type information of the target area based on the position information of the target task point. The second acquisition unit is used to process the distance information, real-time driving direction and lane type information of the target task point using the driving scene analysis model to obtain the target driving scene information.

[0146] According to an embodiment of the present disclosure, the second obtaining unit includes a first determining subunit and a second determining subunit. The first determining subunit is configured to determine multiple candidate scene switching conditions based on the real-time driving direction and lane type information of the target task point. The second determining subunit is configured to determine target driving scene information from the multiple candidate scene switching conditions based on distance information.

[0147] According to an embodiment of the present disclosure, the generation module includes a second determination unit, a first screening unit, a third determination unit, a second screening unit, and a third acquisition unit. Among them, the second determination unit is used to determine the target traffic participant based on the target driving scene information and the real-time relative motion state information. The first screening unit is used to screen the target real-time relative motion state information of the target traffic participant and the target vehicle from the real-time relative motion state information. The third determination unit is used to determine the attribute information of the target motion state based on the target driving scene information; the second screening unit is used to screen the target real-time motion state information corresponding to the attribute information of the target motion state from the real-time motion state information of the target vehicle. The third determination unit is used to process the target real-time motion state information and the target real-time relative motion state information using the target driving behavior decision model to obtain driving action decision information.

[0148] According to an embodiment of the present disclosure, the second determination unit includes a third determination subunit, a fourth determination subunit, and a screening subunit. The third determination subunit is configured to determine the relative positional relationship between the traffic participant and the target vehicle based on real-time relative motion state information. The fourth determination subunit is configured to determine the risk level of the traffic participant to the target vehicle in the target driving scenario based on the relative positional relationship. The screening subunit is configured to screen the target traffic participant from the traffic participants based on the risk level.

[0149] According to an embodiment of the present disclosure, the fourth determination subunit is used to obtain an association relationship between the traffic participant object and the driving trajectory of the target vehicle based on the relative position relationship; and determine the risk level based on the association relationship of the driving trajectory.

[0150] According to an embodiment of the present disclosure, the generation module includes an acquisition unit and a training unit. The acquisition unit is used to obtain the test vehicle's motion state information and the relative motion state information between traffic participants and the test vehicle in a simulated driving scenario. The training unit is used to train a preset model based on the proximal policy optimization algorithm, using the driving action information, the test vehicle's motion state information, and the relative motion state information between traffic participants and the test vehicle to obtain a target driving behavior decision model.

[0151] According to an embodiment of the present disclosure, the driving maneuver information includes S driving maneuver information, where S is an integer greater than 1. The preset model includes a policy network and a value network. The training unit includes a processing subunit, a first obtaining subunit, a second obtaining subunit, and an adjustment subunit. The processing subunit is configured to process the sth driving maneuver information, the motion state information of the test vehicle, and the relative motion state information between the traffic participants and the test vehicle using the policy network to obtain sth driving result information, where s is an integer greater than or equal to 1 and less than S. The first obtaining subunit is configured to process the sth driving maneuver information and the sth driving result information using the value network to obtain action value information. The second obtaining subunit is configured to obtain a policy advantage value based on the driving state information and the action value information based on an objective function. The adjustment subunit is configured to, if it is determined that the policy advantage value does not meet a predetermined threshold, adjust model parameters of the preset model, return to executing processing operations using the policy network and processing operations using the value network, and increment s. If it is determined that the policy advantage value meets the predetermined threshold, the sth driving maneuver information is determined to be target decision action information.

[0152] According to the modules, submodules, units, and subunits of the embodiments of the present invention, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be split into multiple modules for implementation. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in an appropriate combination of any of them. Alternatively, according to the modules, submodules, units, and subunits of the embodiments of the present invention, one or more thereof can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is run.

[0153] For example, any multiple of the acquisition module 710, the analysis module 720, the determination module 730, and the generation module 740 can be combined into one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more modules / units / sub-units in these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the acquisition module 710, the analysis module 720, the determination module 730, and the generation module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 710 , the analysis module 720 , the determination module 730 , and the generation module 740 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be performed.

[0154] It should be noted that the decision information generating device in the embodiment of the present disclosure corresponds to the decision information generating method in the embodiment of the present disclosure. The description of the decision information generating device part specifically refers to the data processing method part and will not be repeated here.

[0155] Figure 8A block diagram of an electronic device suitable for implementing the above-described method according to an embodiment of the present disclosure is schematically shown. Figure 8 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0156] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0157] Various programs and data required for the operation of the electronic device 800 are stored in the RAM 803. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. The processor 801 executes the programs in the ROM 802 and / or RAM 803 to perform various operations of the method flow according to the embodiment of the present disclosure. It should be noted that the programs may also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also execute the programs stored in the one or more memories to perform various operations of the method flow according to the embodiment of the present disclosure.

[0158] According to an embodiment of the present disclosure, electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to bus 804. System 800 may also include one or more of the following components connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 808 including a hard disk; and a communication section 809 including a network interface card such as a LAN card or modem. Communication section 809 performs communication processing via a network such as the Internet. Drive 810 is also connected to I / O interface 805 as needed. Removable media 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 810 as needed, so that computer programs read from the removable media can be installed into storage section 808 as needed.

[0159] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0160] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0161] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0162] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 802 and / or the RAM 803 described above and / or one or more memories other than the ROM 802 and the RAM 803 .

[0163] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiment of the present disclosure.

[0164] When the computer program is executed by the processor 801, the above functions defined in the system / device of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0165] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0166] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0167] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in various ways, and all such combinations and / or couplings fall within the scope of the present disclosure.

[0168] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for generating decision information, comprising: In response to receiving a driving service request from a target vehicle, obtaining real-time motion state information of the target vehicle, lane type information of a target area, and real-time relative motion state information between traffic participants and the target vehicle; Processing the real-time motion state information and the lane type information using a driving scene analysis model to obtain target driving scene information; as well as determining a target driving behavior decision model based on the target driving scenario information; The real-time motion state information and the real-time relative motion state information are processed using a target driving behavior decision model to obtain driving action decision information of the target vehicle.

2. The method according to claim 1, wherein The real-time motion state information includes real-time driving direction information and real-time location information. The real-time motion state information and lane type information are processed by the driving scene analysis model to obtain target driving scene information, including: Determine the location information of the target task point and the location information of the scene switching point from the target area; Obtaining distance information between the real-time position and the scene switching point according to the real-time position information and the position information of the scene switching point; According to the location information of the target task point, querying the lane type information of the target task point from the lane type information of the target area; and The distance information, the real-time driving direction, and the lane type information of the target task point are processed using a driving scene analysis model to obtain the target driving scene information.

3. The method according to claim 2, wherein: The using of the driving scene analysis model to process the distance information, the real-time driving direction, and the lane type information of the target task point to obtain the target driving scene information includes: Determining multiple candidate scene switching conditions based on the real-time driving direction and the lane type information of the target task point; and According to the distance information, target driving scene information is determined from the multiple candidate scene switching conditions.

4. The method according to claim 1, wherein The process of processing the real-time motion state information and the real-time relative motion state information using the target driving behavior decision model to obtain the driving action decision information of the target vehicle includes: determining a target traffic participant according to the target driving scene information and the real-time relative motion state information; Filtering the target real-time relative motion state information of the target traffic participant and the target vehicle from the real-time relative motion state information; Determine the target motion state attribute information based on the target driving scene information Filtering the target real-time motion state information corresponding to the attribute information of the target motion state from the real-time motion state information of the target vehicle; and The target driving behavior decision model is used to process the target real-time motion state information and the target real-time relative motion state information to obtain the driving action decision information.

5. The method according to claim 4, wherein The determining of a target traffic participant according to the target driving scene information and the real-time relative motion state information includes: Determining the relative position relationship between the traffic participant and the target vehicle based on the real-time relative motion state information; Determining, based on the relative position relationship, the risk level of the traffic participant to the target vehicle in the target driving scenario; and The target traffic participant is selected from the traffic participants according to the risk level.

6. The method according to claim 5, wherein: Determining the risk level of the traffic participant to the target vehicle in the target driving scenario based on the relative position relationship includes: Obtaining an association relationship between the traffic participant object and the driving trajectory of the target vehicle based on the relative position relationship; and The risk level is determined based on the correlation relationship of the driving trajectories.

7. The method according to claim 1, wherein The training method of the target driving behavior decision model includes: In a simulated driving scenario, obtaining motion state information of a test vehicle and relative motion state information of traffic participants and the test vehicle; Randomly select driving action information from the target vehicle's action probability distribution space; and Based on the proximal strategy optimization algorithm, the driving action information, the motion state information of the test vehicle, and the relative motion state information between the traffic participants and the test vehicle are processed to train the preset model to obtain the target driving behavior decision model.

8. The method according to claim 7, wherein: The driving action information includes S driving action information, where S is an integer greater than 1. The preset model includes a policy network and a value network. The proximal policy optimization algorithm uses the driving action information, the motion state information of the test vehicle, and the relative motion state information between the traffic participants and the test vehicle to process and train the preset model to obtain the target driving behavior decision model, including: For the s-th driving action information, the policy network is used to process the s-th driving action information, the motion state information of the test vehicle, and the relative motion state information between the traffic participants and the test vehicle to obtain the s-th driving result information, where s is an integer greater than or equal to 1 and less than S; Processing the s-th driving action information and the s-th driving result information using a value network to obtain action value information; Based on the objective function, obtaining a strategy advantage value according to the driving action information and the action value information; If it is determined that the strategy advantage value does not meet the predetermined threshold, adjust the model parameters of the preset model, return to executing the processing operation using the strategy network and the processing operation using the value network, and increment s; and When it is determined that the strategy advantage value meets a predetermined threshold, the s-th driving action information is determined as target decision action information.

9. A decision information generating device, comprising: an acquisition module, configured to, in response to receiving a driving service request from a target vehicle, acquire real-time motion state information of the target vehicle, lane type information of a target area, and real-time relative motion state information between traffic participants and the target vehicle; an analysis module, configured to process the real-time motion state information and the lane type information using a driving scene analysis model to obtain target driving scene information; a determination module, configured to determine a target driving behavior decision model based on the target driving scenario information; as well as The generating module is used to process the real-time motion state information and the real-time relative motion state information by using the target driving behavior decision model to obtain the driving action decision information of the target vehicle.

10. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images