Vehicle behavior generation device, vehicle behavior generation method, and vehicle behavior generation program

Abstract

The present invention provides a vehicle behavior generation device, a vehicle behavior generation method, and a vehicle behavior generation program. A plurality of actions that a vehicle can take are set (S240). Furthermore, for each action of the vehicle, a plurality of actions that other vehicles can take are set (S260). For each combination of the set actions of the vehicle and the actions of other vehicles, information related to the likelihood of contact between the vehicle and the other vehicle is output (S280). Furthermore, based on the output information, an action of the vehicle that is more likely to avoid contact between the vehicle and the other vehicle is selected (S310).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on patent application No. 2020-24653 filed in Japan on February 17, 2020, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to a vehicle behavior generation device, a vehicle behavior generation method, and a vehicle behavior generation program. Background Art

[0004] For example, Patent Document 1 describes a trajectory evaluation device for evaluating a vehicle's future trajectory. This trajectory evaluation device generates a future trajectory for the vehicle and predicts the future positions of objects around the vehicle. It then evaluates multiple locations on the vehicle's future trajectory based on their relative positional relationships with the predicted future positions of the objects in the longitudinal direction and the width direction of the road. The vehicle's future trajectory is then evaluated based on the evaluation results for each location.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-95149

[0006] In the trajectory evaluation device of Patent Document 1, when determining the future trajectory of the host vehicle and the future position of an object, it is assumed that the host vehicle and the object are moving at a constant speed or with a constant acceleration.

[0007] However, the vehicle and other objects, such as the onboard vehicle, do not always move at a constant speed or with a constant acceleration. For example, if the road width is narrowed by a parked vehicle in the onboard vehicle's lane, and the onboard vehicle initiates a maneuver around the parked vehicle, the oncoming vehicle may wait until the onboard vehicle returns to its original lane or slow down. Conversely, if the oncoming vehicle approaches the parked vehicle first, the oncoming vehicle may wait or slow down to avoid the other vehicle's passing.

[0008] As such, in real traffic environments, the actions of the host vehicle and those of other vehicles often interact with each other. Therefore, there is a concern that trajectories generated and evaluated based on the assumption that the host vehicle and objects are moving at a constant speed or with a constant acceleration, as in the trajectory evaluation device of Patent Document 1, may not be effective in real-world scenarios. Summary of the Invention

[0009] The present disclosure is completed in view of the above-mentioned points, and its purpose is to provide a vehicle behavior generation device, a vehicle behavior generation method and a vehicle behavior generation program that can generate the behavior of the vehicle that is effective in actual scenarios compared to situations where the mutual influence between the behavior of the vehicle and the behavior of other vehicles is not considered.

[0010] In order to achieve the above-mentioned object, the vehicle behavior generation device of the present disclosure is configured to include:

[0011] a first setting unit for setting a plurality of actions that the vehicle can take when traveling on a predetermined track;

[0012] a second setting unit for setting a plurality of actions that can be taken by other vehicles around the host vehicle for each of the plurality of actions of the host vehicle set by the first setting unit;

[0013] an output unit that outputs information related to the possibility of contact between the host vehicle and the other vehicle for each combination of the behavior of the host vehicle set by the first setting unit and the behavior of the other vehicle set by the second setting unit; and

[0014] The selection unit selects an action of the host vehicle based on the information output by the output unit.

[0015] In addition, the vehicle behavior generation method of the present disclosure is configured to include the following steps executed by at least one processor:

[0016] The first setting step is to set a plurality of actions that the vehicle can take when driving on a predetermined runway;

[0017] a second setting step of setting a plurality of actions that can be taken by other vehicles around the own vehicle for each of the plurality of actions of the own vehicle set in the first setting step;

[0018] an outputting step of outputting information related to the possibility of contact between the own vehicle and the other vehicle for each combination of the behavior of the own vehicle set in the first setting step and the behavior of the other vehicle set in the second setting step; and

[0019] The selecting step selects an action of the host vehicle based on the information outputted in the outputting step.

[0020] Furthermore, the vehicle behavior generation program of the present disclosure is a program stored in a computer-readable non-transitory storage medium and includes instructions. When executed by at least one processor, the instructions are configured to cause the at least one processor to perform the following processing:

[0021] Set multiple actions that the vehicle can take when driving on a predetermined track;

[0022] For each of the plurality of actions set for the own vehicle, a plurality of actions that other vehicles around the own vehicle can take are set;

[0023] outputting information related to the possibility of contact between the own vehicle and the other vehicle for each combination of the set behavior of the own vehicle and the set behavior of the other vehicle; and

[0024] The behavior of the host vehicle is selected based on the output information.

[0025] According to the above-mentioned vehicle action generation device, vehicle action generation method, and vehicle action generation program, multiple actions that can be taken by the vehicle are set, and for each action of the vehicle, multiple actions that can be taken by other vehicles are set. For each combination of the set actions of the vehicle and the actions of other vehicles, information related to the possibility of contact between the vehicle and the other vehicles is output. Moreover, based on the output information, the action of the vehicle that has a higher possibility of avoiding contact between the vehicle and the other vehicles is selected. Therefore, according to the vehicle action generation device, vehicle action generation method, and vehicle action generation program disclosed in the present invention, it is possible to generate actions of the vehicle that are effective in actual scenarios compared to a case where the mutual influence between the actions of the vehicle and the actions of other vehicles is not considered.

[0026] In order to facilitate understanding of the present disclosure, the reference numerals in the claims merely show an example of the correspondence relationship with specific configurations in the embodiments described later, and are not intended to limit the scope of the present disclosure.

[0027] In addition, the technical features described in each of the claims other than the above-mentioned features will become clear from the description of the embodiments and the accompanying drawings to be described later. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a diagram showing the overall configuration of an automatic driving system to which the vehicle behavior generation device according to the embodiment is applied.

[0029] Figure 2 This is a diagram showing an example of a spatiotemporal state-action diagram.

[0030] Figure 3 This is a diagram for explaining an example of a method for creating an approximate function.

[0031] Figure 4 It means in Figure 1 The flowchart of the processing performed to automatically drive a vehicle in the automatic driving system shown.

[0032] Figure 5 This is a flowchart showing a host vehicle behavior generation process for generating the behavior of the host vehicle.

[0033] Figure 6 This diagram explains actions that the host vehicle and other vehicles can take when there is a parked vehicle in the host vehicle's lane and the host vehicle must go around the parked vehicle and there are other vehicles traveling in the opposite lane.

[0034] Figure 7 This is a diagram showing an example of a spatiotemporal state behavior diagram that takes into account the driving characteristics of other vehicles.

[0035] Figure 8 As an example of preferably considering the driving characteristics of other vehicles, this diagram shows a situation in which another vehicle traveling on the main line exists near the vehicle when the vehicle attempts to merge from a side road onto the main line.

[0036] Figure 9 This is a diagram showing an example of a spatiotemporal state-action diagram taking into account the state of traffic signals.

[0037] Figure 10 As an example of preferably considering the state of a traffic signal, this diagram shows a situation in which another vehicle is traveling in the opposite lane while the vehicle is waiting to turn right at an intersection with a traffic light. DETAILED DESCRIPTION

[0038] (First embodiment)

[0039] The following describes an embodiment of the present disclosure in detail with reference to the accompanying drawings. Furthermore, this embodiment describes an example in which the vehicle behavior generation device is applied to an autonomous vehicle system. However, the application examples of the vehicle behavior generation device disclosed herein are not limited to autonomous vehicle systems. For example, the device can also be applied to a system that presents the driver of the vehicle with the behavior generated by the vehicle behavior generation device as a recommended behavior.

[0040] exist Figure 1 , the overall structure of the automatic driving system 100 to which the vehicle behavior generation device of this embodiment is applied is shown. Figure 1 As shown, the autonomous driving system 100 includes various sensors 10 , a long-term plan creating unit 20 , a medium-term plan creating unit 30 , an interactive prediction unit 40 , a short-term plan creating unit 60 , and a path following control unit 70 .

[0041] The various sensors 10 include, for example, at least one of sensors such as cameras, LiDAR, and millimeter-wave radar. The various sensors 10 detect the surrounding environment information of the vehicle, the vehicle information indicating the status of the vehicle, and other vehicle information indicating the status of other vehicles around the vehicle. The various sensors 10 detect obstacles such as fallen objects on the road, parked vehicles on the road, road surface displays such as guardrails, curbs, and driving dividing lines, and stationary objects such as trees as surrounding environment information. In addition, the various sensors 10 detect moving objects such as pedestrians, animals other than humans, and other vehicles as surrounding environment information. In addition, the various sensors 10 detect the position, direction, speed, and acceleration of the vehicle as vehicle information. Furthermore, the jerk (change in acceleration per unit time) of the vehicle can also be detected as vehicle information. The various sensors 10 detect the position, direction, speed, and acceleration of other vehicles as other vehicle information. Furthermore, the jerk of other vehicles can also be detected as other vehicle information. In addition, the autonomous driving system 100 can also obtain at least a portion of the vehicle information and other vehicle information from an external server such as a control center via communication.

[0042] The long-term plan creation unit 20 uses a road map database to create a route (driving path) for the vehicle to travel from the departure point (current location) to the destination, based on the departure point (current location) and destination specified by the vehicle's occupants. If the vehicle is equipped with a navigation device, the long-term plan creation unit 20 may be comprised of the navigation device. Alternatively, the long-term plan creation unit 20 may be located on an external server, such as a management center. Furthermore, the vehicle's occupants may transmit information related to the departure point (current location) and destination to the external server, which may then create a driving path to the destination.

[0043] The medium-term plan creation unit 30 sets the driving route to the destination in greater detail than the long-term plan creation unit 20. Specifically, if there are multiple lanes on the driving route, the medium-term plan creation unit 30 determines which lane the vehicle is traveling in. First, the medium-term plan creation unit 30 obtains the driving route to the destination created by the long-term plan creation unit 20. Furthermore, the medium-term plan creation unit 30 obtains road information (number of lanes, lane width, shape, etc.) surrounding the vehicle's current location from the long-term plan creation unit 20 (road map database). Furthermore, the medium-term plan creation unit 30 obtains information about the vehicle's surrounding environment, vehicle information, and other vehicle information detected by various sensors 10.

[0044] If the road on which the host vehicle is traveling has multiple lanes, the mid-term plan creation unit 30 determines the lane position of the host vehicle based on the acquired road information and the host vehicle information. Furthermore, if the mid-term plan creation unit 30 detects the presence of other vehicles around the host vehicle based on the acquired road information and surrounding environment information, it also determines the lane in which the other vehicle is traveling (whether it is a lane in the same direction of travel or an oncoming lane, and if there are multiple lanes, the lane position of the other vehicle).

[0045] The mid-term plan creation unit 30 then calculates the planned trajectory of the vehicle on the road until a predetermined time (e.g., 5 seconds) or a predetermined distance (e.g., 100 meters) in the future based on the acquired driving route, road information, surrounding environment information, and vehicle information. The predetermined time or distance may be fixed values ​​or variable values ​​that vary, for example, depending on the speed of the vehicle.

[0046] For example, if the mid-term plan creation unit 30 determines based on surrounding environment information that there are no obstacles in the lane the vehicle is traveling in and that there are no moving objects such as other vehicles around the vehicle, it may determine the vehicle's path, representing the vehicle's planned trajectory, without going through the interactive prediction unit 40. Because the likelihood of contact with obstacles or other vehicles is low, the mid-term plan creation unit 30 may calculate a planned trajectory along the lane the vehicle is traveling in. Furthermore, if the vehicle makes a left or right turn or a branching road, it may calculate a planned trajectory corresponding to the shape of the road. Even if an obstacle or other vehicle is detected, but the obstacle or other vehicle is not affected by the vehicle's movement, the mid-term plan creation unit 30 may linearly predict the movement of the other vehicle and determine the vehicle's path based on this prediction.

[0047] On the other hand, if the surrounding environment information indicates the presence of an obstacle such as a parked vehicle on the road, the mid-term plan creation unit 30 calculates a planned trajectory based on the road information to avoid the obstacle. Furthermore, if the driving path involves a right or left turn at the upcoming intersection, and if the vehicle is traveling in a right-turn lane or a lane different from the left-turn lane, the mid-term plan creation unit 30 calculates a planned trajectory for changing lanes to the right-turn lane or the left-turn lane.

[0048] Furthermore, when calculating the planned trajectory, the mid-term plan creation unit 30 does not consider the lane positions or positions of other vehicles, even when other vehicles are present. This is because the interactive prediction unit 40 determines the vehicle's behavior (the vehicle's path and target speed) to avoid contact with other vehicles based on the planned trajectory. Therefore, when other vehicles are present, the mid-term plan creation unit 30 outputs to the interactive prediction unit 40 the information necessary to determine the vehicle's path to avoid contact with other vehicles. Specifically, the mid-term plan creation unit 30 outputs to the interactive prediction unit 40 the vehicle's current (and past) vehicle information, lane position information, current (and past) other vehicle information, lane position information of other vehicles, the vehicle's planned trajectory, the location and size of obstacles, and road information.

[0049] The interactive prediction unit 40, functioning as a first setting unit, sets multiple actions that the vehicle can take. Examples of the set actions include "wait," "slow down," "move," and "turn signal on." Multiple actions may simply involve traveling at different speeds. Specifically, the interactive prediction unit 40 may set traveling at speed A and traveling at speed B, which is different from speed A, as multiple actions. Furthermore, the interactive prediction unit 40 determines a vehicle path representing the planned trajectory of the vehicle when executing each of the set actions. This vehicle path is determined based on the vehicle's behavior and its planned trajectory. For example, if the vehicle is waiting, the vehicle path remains at the waiting location, but if the vehicle is moving, the vehicle path is determined to depict the planned trajectory. Furthermore, the length of the vehicle path per unit time varies depending on whether the vehicle is traveling at a certain speed or at a speed relatively lower (or higher) than that speed. Furthermore, the trajectory of the vehicle path based on the planned trajectory may be altered depending on the vehicle's speed. For example, when the planned trajectory draws a curve, the vehicle path may be determined so that the curvature becomes smaller when the vehicle travels at a relatively high speed than when the vehicle travels at a relatively low speed.

[0050] The interactive prediction unit 40, functioning as a second setting unit, sets multiple actions that other vehicles can take for each action of the host vehicle. Furthermore, the interactive prediction unit 40 determines other vehicle paths representing the planned trajectory of the other vehicles when they execute each of the set actions. The other vehicle paths are determined in the same manner as the host vehicle path described above.

[0051] For example, the interactive prediction unit 40 determines the aforementioned own vehicle path and other vehicle paths until a predetermined time (e.g., 1 to 5 seconds) has elapsed. As described above, the length per unit time and trajectory of the own vehicle path and other vehicle paths vary depending on the actions of the own vehicle and other vehicles. Therefore, each combination of the own vehicle path corresponding to the multiple actions that the own vehicle can take and the other vehicle paths corresponding to the multiple actions that other vehicles can take in response to the own vehicle's actions not only includes distance (spatial) relationships between the own vehicle and other vehicles, but also includes temporal relationships. Therefore, the set of each combination of the own vehicle path corresponding to the multiple actions that the own vehicle can take and the other vehicle paths corresponding to the multiple actions that other vehicles can take in response to the own vehicle's actions constitutes a spatiotemporal state-action graph that represents the various states of the own vehicle and other vehicles in time and space. This spatiotemporal state-action graph may also include additional information such as the own vehicle's speed, acceleration, and direction at multiple locations on the own vehicle path and other vehicle paths, as well as additional information such as the speed, acceleration, and direction of other vehicles.

[0052] Figure 2 An example of a spatiotemporal state action diagram is shown. Figure 2 The example shown is a simple example in which the actions that can be taken by the own vehicle and other vehicles are set to two types: waiting and moving.

[0053] In the interactive prediction unit 40, an approximate function 50 is used to evaluate various combinations of multiple actions that the host vehicle can take and multiple actions that other vehicles can take in response to the actions of the host vehicle. Here, the approximate function 50 functions as an output unit that outputs information related to the possibility of contact between the host vehicle and other vehicles as the evaluation result. Figure 2While the example in which information related to the likelihood of contact between the host vehicle and another vehicle is used as a reward value is shown, the information related to the likelihood of contact between the host vehicle and another vehicle may also be letters, symbols, or the like, as long as it indicates the degree of likelihood of contact between the host vehicle and another vehicle. This evaluation is determined based on the position, speed, and path of the host vehicle, as well as the position, speed, and paths of other vehicles. Furthermore, the interactive prediction unit 40, functioning as a selection unit, selects an action for the host vehicle based on the evaluation results of the approximation function generator 50. Specifically, the interactive prediction unit 40 selects an action for the host vehicle that is most likely to avoid contact with another vehicle and outputs information representing this action to the medium-term plan creation unit 30. For example, the interactive prediction unit 40 outputs information representing the action of the host vehicle that maximizes the reward value to the medium-term plan creation unit 30. The information representing the action of the host vehicle may be one of a plurality of set actions. In addition to this single action, the path of the host vehicle when executing the action and the target speed within a specified time interval (e.g., one-second intervals) while traveling along the path may also be output to the medium-term plan creation unit 30. The latter is preferred because it reduces the computational burden on the short-term plan creation unit 60, which generates a more detailed driving trajectory. The evaluation method using the approximate function generator 50 will be described in detail later. The medium-term plan creation unit 30 and the interactive prediction unit 40 correspond to the vehicle behavior generation device in this disclosure.

[0054] The mid-term plan creation unit 30 and the interactive prediction unit 40 are formed by, for example, the vehicle-mounted ECU. The vehicle-mounted ECU is a computer mounted on the vehicle and is mainly composed of a microcomputer having at least one processor, a computer-readable non-transient storage medium such as RAM and ROM, an input / output interface, and a bus connecting these components. The non-transient storage medium of the vehicle-mounted ECU stores the data for executing the program. Figure 6 The vehicle behavior generation process shown in FIG. Furthermore, the processor included in the on-vehicle ECU executes the program. Furthermore, the mid-term plan creation unit 30 and the interactive prediction unit 40 may be comprised of a single on-vehicle ECU or separate on-vehicle ECUs. Furthermore, the functions of the mid-term plan creation unit 30 and the interactive prediction unit 40 may be located on an external server that is capable of communicating with the vehicle and is located outside the vehicle.

[0055] The medium-term plan creation unit 30 receives information representing the behavior of the vehicle output by the interactive prediction unit 40 and outputs the information to the short-term plan creation unit 60. Based on the received information representing the behavior of the vehicle (for example, the vehicle path and target speed), the short-term plan creation unit 60 creates a short-term plan (for example, a plan for 1 to 2 seconds later) for actually controlling the behavior of the vehicle and outputs it to the path following control unit 70. In this way, the detailed trajectory of the vehicle is determined. In addition, the short-term plan creation unit 60 is also configured to receive detection signals from various sensors 10. In the event of an unexpected situation (for example, the behavior of another vehicle different from that predicted, a moving object jumping out of a hiding place, etc.), the short-term plan creation unit 60 can also respond to unexpected situations by creating a short-term plan to make the vehicle stop suddenly or change the direction of travel of the vehicle.

[0056] The path following control unit 70, which includes a steering control unit, an engine control unit, and a brake control unit, controls the vehicle's steering direction, driving force, and braking force to control the vehicle's behavior according to the short-term plan created by the short-term plan creation unit 60. The short-term plan creation unit 60 and the path following control unit 70 are also implemented by, for example, the vehicle's onboard ECU.

[0057] Next, the evaluation method using the approximate function device 50 in the interactive prediction unit 40 will be described. Figure 3 , an example of a method for creating an approximate function 50 is described. Figure 3 As shown, the approximate function generator 50 is created by a simulator 120, a reward calculation unit 130, and an approximate function generator 140. The simulator 120, the reward calculation unit 130, and the approximate function generator 140 are installed outside the vehicle.

[0058] The simulator 120 generates driving trajectories for the vehicle and other vehicles in various traffic environments based on various scenarios representing the behavior of the vehicle and other vehicles (such as left or right turns, overtaking, passing around obstacles, merging, and branching), thereby simulating the driving conditions of the vehicle and other vehicles. The simulated driving conditions of the vehicle and other vehicles include changes in speed, including stopping. Furthermore, the simulated driving conditions include both driving conditions in which the vehicle and other vehicles can avoid contact and driving conditions in which contact has occurred.

[0059] The simulator 120 outputs driving data representing the driving conditions of the host vehicle and other vehicles in each of a plurality of stages, into which the driving conditions of the host vehicle and other vehicles are divided, to the reward calculation unit 130. Specifically, the simulator 120 outputs data such as the host vehicle's position, speed, acceleration, and other vehicle conditions; the host vehicle's planned path; the location of obstacles; the location of other vehicles, speed, acceleration, and other vehicle conditions; and elapsed time as driving data in each stage to the reward calculation unit 130.

[0060] The reward calculation unit 130 assigns rewards based on the driving conditions of the host vehicle and other vehicles based on the received driving data. Some examples of reward calculation methods are described below. The reward calculation unit 130 calculates a first cost based on the distance between the host vehicle and objects, including other vehicles, as determined from the driving data. The closer the distance, the larger the first cost. When multiple objects (e.g., obstacles and other vehicles) are located around the host vehicle, a first cost is calculated for each object. This is because the closer the distance between the host vehicle and an object, the higher the likelihood of contact. Furthermore, the reward calculation unit 130 calculates a second cost, which is assigned as a penalty if the host vehicle comes into contact with another vehicle. Furthermore, the reward calculation unit 130 calculates a third cost. If the host vehicle comes into contact with another vehicle, the shorter the time to contact, the larger the third cost. Furthermore, the reward calculation unit 130 calculates the reward as the inverse of the sum of the calculated first to third costs. As a result, a higher reward can be given to a driving situation in which the possibility of contact with another vehicle being avoidable is higher, and conversely, a lower reward can be given to a driving situation in which the possibility of contact with another vehicle is higher.

[0061] Furthermore, the reward calculation unit 130 may also calculate a fourth cost, setting the fourth cost higher as the time required to avoid contact between the host vehicle and another vehicle (elapsed time) increases. The reward calculation also takes this fourth cost into account (in addition to the sum of the first to third costs, the inverse of this sum is also calculated). This is because by taking elapsed time into account, a higher reward can be assigned to driving conditions (host vehicle behavior) that more quickly avoid contact with another vehicle.

[0062] The reward calculation unit 130 may also calculate a fifth cost, setting it to a higher value the longer the vehicle's duration and / or distance of departure from its intended lane, and also consider this fifth cost in calculating the reward. This is because, for example, if a portion of the vehicle's lane is blocked by a parked vehicle, and the vehicle needs to travel at least partially or entirely in the oncoming lane to bypass the parked vehicle, the longer the time and / or distance spent in the oncoming lane, the higher the likelihood of contact with the other vehicle. Furthermore, the reward calculation unit 130 may also calculate a sixth cost, setting it to a higher value the greater the vehicle's acceleration, and also consider this sixth cost in calculating the reward. This is because it is assumed that higher vehicle acceleration increases the likelihood of contact with other vehicles, etc.

[0063] In addition, the reward assigned to the action of the vehicle can be calculated based on other costs that reflect the possibility of contact between the vehicle and other vehicles, in addition to or instead of the first to sixth costs mentioned above. In addition, among the first to sixth costs mentioned above, the second cost that is a penalty when the vehicle comes into contact with other vehicles is set to be relatively larger than the other costs. And, the third cost assigned to the driving condition before the vehicle comes into contact with other vehicles is calculated by discounting the second cost at a discount rate corresponding to the time to contact. Specifically, the shorter the time to contact, the smaller the discount rate. Therefore, it is possible to assign very low rewards to the driving condition in which the vehicle comes into contact with other vehicles and the driving condition in which contact is reached.

[0064] In this manner, the reward calculation unit 130 creates driving condition-reward data by assigning rewards corresponding to the likelihood of contact (or the likelihood of avoiding contact) to various driving conditions of the host vehicle and other vehicles. The created driving condition-reward data is then provided to the approximation function generator 140. Alternatively, the reward calculation unit 130 may directly provide each driving condition and the reward data for each driving condition to the approximation function generator 140. Alternatively, the reward calculation unit 130 may aggregate the driving conditions and reward data for each planned trajectory of the host vehicle and provide the data to the approximation function generator 140.

[0065] Based on the driving condition-reward data created by the reward calculation unit 130, the approximating function generator 140 causes the approximating function generator 50 to learn a function that approximates the relationship between the various driving conditions (spatiotemporal states) of the host vehicle and other vehicles and rewards. A deep neural network can be used as the approximating function generator 50, and so-called deep Q-learning can be used as the learning method. However, the approximating function generator 50 is not limited to neural networks; for example, a support vector machine (SVM) can also be used. Such an approximating function generator 50 can assign a reward as an evaluation to each combination of the host vehicle's possible actions and the possible actions of other vehicles in response to the host vehicle's actions, as reflected in the spatiotemporal state-action graph. Furthermore, while the example in which a pre-created function is installed in the vehicle as the approximating function generator 50 is described, the function of the approximating function generator 50 can also be relearned based on past evaluation results or the vehicle's driving history.

[0066] Next, refer to Figure 4 The flowchart is Figure 1 The processing executed in the illustrated automatic driving system 100 for automatically driving the vehicle will be described.

[0067] exist Figure 4 In the flowchart of FIG. 1 , in step S100, the long-term plan creation unit 20 uses the road map database to set a route (driving route) for the vehicle to travel from the departure point (current location) to the destination. In step S110, the medium-term plan creation unit 30 obtains the driving route to the destination set by the long-term plan creation unit 20.

[0068] In step S120, the mid-term plan creation unit 30 and the interactive prediction unit 40 generate the behavior (vehicle path, target speed) that the vehicle should perform. Figure 5 The process of generating the host vehicle's behavior is described in detail with reference to a flowchart. In the following step S130, the short-term plan creation unit 60 creates a short-term plan for actually controlling the host vehicle's behavior based on the generated host vehicle's behavior. In step S140, the path following control unit 70 controls the host vehicle's behavior by controlling the steering direction, driving force, and braking force according to the created short-term plan.

[0069] In step S150, it is determined whether the vehicle has arrived at the designated destination. If not, the process from step S120 is repeated. If the destination has been changed, the process is executed again from the initial step S100.

[0070] Next, refer to Figure 5The vehicle behavior generation process executed in the mid-term plan creation unit 30 and the interactive prediction unit 40 will be described with reference to the flowchart of FIG. This vehicle behavior generation process corresponds to the step S120 described above.

[0071] In the first step S200, road information around the current position of the vehicle, surrounding environment information detected by various sensors 10, and vehicle information are input. In the next step S210, the shape, position, size, and whether they are moving or stationary objects of objects including other vehicles are identified based on the surrounding environment information.

[0072] In step S220, based on the recognition result in step S210, it is determined whether there are other vehicles around the host vehicle. If it is determined that there are other vehicles around the host vehicle, the process proceeds to step S230. On the other hand, if it is determined that there are no other vehicles around the host vehicle, the process proceeds to step S320.

[0073] In step S230, information necessary for setting the actions that the host vehicle can take and the actions that other vehicles can take in response to the actions of the host vehicle, as well as information necessary for evaluating the actions of the host vehicle in each combination of the actions of the host vehicle and the actions of other vehicles, is output to the interactive prediction unit 40. Specifically, information is output including the current and past vehicle state information such as the position, direction, speed, and acceleration of the host vehicle, lane position information of the host vehicle, current and past vehicle state information such as the position, direction, speed, and acceleration of other vehicles, lane position information of other vehicles, and road information including the planned trajectory of the host vehicle, the location and size of obstacles, the road shape, the number of lanes, and the like.

[0074] In step S240, the interactive prediction unit 40 sets a plurality of possible actions for the vehicle, including at least whether the vehicle is waiting or moving, based on the various acquired information. Furthermore, in step S250, the interactive prediction unit 40 determines a vehicle path representing the projected trajectory of the vehicle when executing each of the set actions.

[0075] In step S260, the interactive prediction unit 40 also sets multiple actions that other vehicles can take in response to each action of the host vehicle. These multiple actions include at least whether the other vehicle should wait or move. Furthermore, in step S270, the interactive prediction unit 40 determines another vehicle path representing the other vehicle's projected trajectory when the other vehicle executes each of the set actions.

[0076] Furthermore, in step S280, the interactive prediction unit 40 uses the approximating function 50 to assign rewards to the vehicle's actions based on the vehicle's position, speed, and path, as well as the positions, speeds, and paths of other vehicles, for each combination of the vehicle's possible actions and the actions of other vehicles in response to the vehicle's actions. For example, the approximating function 50 calculates rewards based on the learned content for each combination of the vehicle's position, direction, speed, and acceleration at multiple points within the vehicle's path, and the positions, directions, speeds, and accelerations of other vehicles at multiple points within the corresponding paths, under an environment determined by factors such as road shape, number of lanes, the vehicle's planned trajectory, and the location, direction, and size of obstacles. Furthermore, by summing the rewards at multiple points within the vehicle's path and the paths of other vehicles, the rewards are calculated for each combination of the vehicle's actions and the actions of other vehicles.

[0077] For example, Figure 6 As shown, a scenario in which a vehicle encounters another vehicle traveling in the opposite lane while a parked vehicle is in its lane and it must maneuver around it is shown. In this scenario, the vehicle approaches the parked vehicle first and, once it is able to begin maneuvering around it, the other vehicle in the opposite lane is likely to wait until the vehicle returns to its original lane or slows down. The driving condition-reward data also includes such driving conditions, in which a relatively high reward is assigned to the vehicle's maneuvering around the parked vehicle. Therefore, in such driving conditions, the vehicle's actions, combined with the actions of other vehicles, are rewarded more for maneuvering around the parked vehicle.

[0078] On the other hand, if another vehicle in the opposite lane approaches the parked vehicle first, the vehicle may need to wait near the parked vehicle or slow down to avoid the other vehicle passing. The driving condition-reward data also includes such driving conditions, in which a higher reward is assigned to the vehicle's actions of waiting for the other vehicle to pass or slowing down. Therefore, in such driving conditions, the vehicle's actions, when combined with the other vehicle's actions, are assigned a higher reward for waiting or slowing down.

[0079] In this embodiment, since an approximate function device 50 is used to assign rewards to the actions of the vehicle for each combination of multiple actions that the vehicle can take and multiple actions that other vehicles can take in response to the actions of the vehicle, a higher reward can be assigned to the actions of the vehicle that are suitable for the actual situation.

[0080] Again, return to Figure 5 The flowchart is further described. If, in step S280, a reward is assigned to the vehicle's action in each combination of the vehicle's action and the actions of other vehicles, the process proceeds to step S290, where a determination is made as to whether a difference of a predetermined value or greater exists between the highest and second-highest rewards assigned. If the difference is determined to be greater than the predetermined value, the process proceeds to step S310. On the other hand, if the difference is determined not to be greater than the predetermined value, the process proceeds to step S300.

[0081] If the difference between the highest and second-highest rewards does not exceed a predetermined value, it is considered impossible to fully determine which of the vehicle's actions is the best. Therefore, in this embodiment, the process loops back to step S240 to execute the following steps: setting multiple possible actions for the vehicle following each combination of the vehicle's actions and the actions of other vehicles; determining the vehicle's path when the vehicle executes each of the set actions; setting multiple possible actions for other vehicles in response to each of the vehicle's actions; and determining the paths of other vehicles when other vehicles execute each of the set actions. During this loop, the vehicle's actions and paths are considered to determine how the vehicle will behave and how their paths will be affected. Furthermore, when the process loops again, the effects of the actions and paths of other vehicles are considered based on the vehicle's actions and paths considered in the first loop. By looping this process, it is possible to accurately predict how the vehicle and / or other vehicles will behave in response to their actions. However, if this loop is allowed indefinitely, there is a risk that the loop will repeat, making it impossible to determine the vehicle's actions. Therefore, in step S310, it is determined whether the number of loops has reached a predetermined number of times. If the number of loops has not reached the predetermined number of times, the process loops to step S240, but if the number of loops has reached the predetermined number of times, the process proceeds to step S310.

[0082] In step S310, the vehicle's action with the highest reward is selected based on the assigned reward. The interactive prediction unit 40 then outputs information related to the selected vehicle's action (the vehicle's path and target speed) to the mid-term plan creation unit 30. Furthermore, in step S310, instead of selecting the vehicle's action based on the reward, the interactive prediction unit 40 may instruct the mid-term plan creation unit 30 on another action. For example, the mid-term plan creation unit 30 may be instructed to maintain the vehicle's current speed / acceleration, gradually reduce the speed, or stop.

[0083] On the other hand, if it is determined in step S220 that no other vehicles are around the vehicle, then in step S320, the mid-term plan creation unit 30 determines the vehicle's behavior based on the road information so that the vehicle follows the travel path. For example, if the vehicle is traveling along a road, the mid-term plan creation unit 30 calculates a predetermined trajectory along the lane in which the vehicle is traveling and calculates a target speed for traveling along this predetermined trajectory. Furthermore, if the vehicle makes a left or right turn or a branching road, the mid-term plan creation unit 30 calculates a predetermined trajectory and a target speed corresponding to the shape of the road or branching road.

[0084] As described above, according to the vehicle behavior generation device of this embodiment, since the behavior of this vehicle in each combination of the behavior of this vehicle and the behavior of other vehicles is evaluated, it is possible to consider the mutually influencing behavior of this vehicle and the behavior of other vehicles, and to generate the behavior of this vehicle that is effective in the actual scenario.

[0085] Furthermore, the processing performed by the vehicle behavior generation device of this embodiment to generate the behavior of the vehicle corresponds to the vehicle behavior generation method of this disclosure. Furthermore, the program used by the vehicle behavior generation device of this embodiment to execute the vehicle behavior generation method corresponds to the vehicle behavior generation program of this disclosure.

[0086] (Second embodiment)

[0087] Next, a second embodiment of the present disclosure will be described. However, since the vehicle behavior generation device of this embodiment has the same configuration as the vehicle behavior generation device of the first embodiment, description of the configuration will be omitted.

[0088] In the first embodiment described above, for example, based on Figure 6 This example illustrates a situation where another vehicle is traveling in the opposite lane, while the vehicle is in the vehicle's lane and must circumvent the parked vehicle. The example also describes the actions that the vehicle and the other vehicle can take. However, when the other vehicle is present and the driver of the other vehicle is in an emergency, the actions of the other vehicle often differ from those in the absence of such an emergency. This embodiment also takes the driving characteristics of these other vehicles into consideration, unlike the first embodiment.

[0089] In this embodiment, if Figure 7As shown, the spatiotemporal state-action diagram considers the driving characteristics of other vehicles. Specifically, based on the behavior of other vehicles detected by various sensors 10, it is determined whether the other vehicles exhibit aggressive or gentle driving characteristics. Because it is difficult to determine whether the other vehicles exhibit aggressive or gentle driving characteristics based solely on their behavior over a short period of time, it is preferable to calculate the probability of aggressive and gentle driving characteristics, for example. Furthermore, the driving characteristics of other vehicles can be categorized into three or more types, such as aggressive, average, and gentle.

[0090] If the driving characteristics of other vehicles can be determined, only the combination of the own vehicle's actions and the actions of other vehicles branching off from those driving characteristics can be considered. However, if the probabilities of each characteristic are calculated for the driving characteristics of other vehicles, the own vehicle's path and the paths of other vehicles can be determined by combining the own vehicle's actions and the actions of other vehicles branching off from those characteristics. In this case, the driving characteristics of other vehicles can be considered as the actions of other vehicles. For example, it can be estimated that the speed and acceleration of other vehicles with aggressive driving characteristics will be higher than those with gentle driving characteristics. Furthermore, the reward ultimately assigned to the own vehicle's actions can be increased or decreased based on the probabilities of each driving characteristic.

[0091] As an example of preferably considering the driving characteristics of other vehicles, for example, in addition to using Figure 6 In addition to the conditions described, Figure 8 The situation shown also meets the requirements. Figure 8 This shows that when the vehicle is about to merge onto the main line from the side road, there are other vehicles traveling on the main line near the vehicle. Figure 8 In the situation shown, if the other vehicle has aggressive driving characteristics, as indicated by the longer arrow, the vehicle is more likely to continue traveling at a higher speed. On the other hand, if the other vehicle has mild driving characteristics, the vehicle is more likely to reduce its speed to allow it to merge with the main line. The driving condition-reward data also includes driving conditions where the other vehicle's driving characteristics are aggressive and mild. Therefore, the approximation function 50 assigns a relatively high reward to the vehicle's action of reducing its speed when the other vehicle's driving characteristics are aggressive, while the likelihood of the vehicle's action of merging with the main line ahead of the other vehicle being assigned a relatively high reward when the other vehicle's driving characteristics are mild increases. In this way, by considering the driving characteristics of the other vehicle, the optimal action for the vehicle to take changes even if the relative positional relationship between the other vehicle and the vehicle remains the same.

[0092] And, as Figure 8 In similar situations, such as when the host vehicle needs to change lanes and another vehicle is traveling in the lane of the change destination, the driving characteristics of the other vehicle are preferably considered. Furthermore, in all situations where there are other vehicles around the host vehicle, the driving characteristics of the other vehicle may also be considered.

[0093] (Third embodiment)

[0094] Next, a third embodiment of the present disclosure will be described. It should be noted that since the vehicle behavior generation device of this embodiment is configured similarly to the vehicle behavior generation device of the first embodiment, description of the configuration will be omitted.

[0095] In the second embodiment described above, the spatiotemporal state-action diagram is created in consideration of the driving characteristics of other vehicles. However, this embodiment differs from the second embodiment in that the spatiotemporal state-action diagram is created in consideration of the state of traffic signals.

[0096] In this embodiment, if Figure 9 As shown in the spatiotemporal state-action diagram, consider the state of traffic signals. Specifically, Figure 10 As shown, when the vehicle intends to turn right at an intersection, and there are other vehicles approaching the intersection in the opposite lane, a spatiotemporal state-action graph is created corresponding to the state of the traffic signal. Furthermore, the state of the traffic signal can be determined based on the detection results of various sensors 10 to determine whether the traffic signal is illuminated green or yellow. Furthermore, if the traffic signal has a right-turn arrow, it can also be determined whether the right-turn arrow is illuminated.

[0097] If the traffic signal is detected to be green, Figure 9 In the spatiotemporal state action diagram shown, the combination of the vehicle's actions at a green light and the actions of other vehicles determines the vehicle's path and the paths of other vehicles. Here, when the light is green, the movement of other vehicles approaching the intersection in the opposite lane is prioritized over the vehicle's right turn. The driving condition-reward data also includes such driving conditions. Therefore, the approximation function 50 is more likely to assign a higher reward to the vehicle's actions in the combination of the vehicle waiting and the other vehicles moving. However, if it is assumed that there is sufficient time before other vehicles enter the intersection, the vehicle's actions signaling to turn right are more likely to be assigned a higher reward.

[0098] On the other hand, if the traffic signal is detected as yellow, the path of the vehicle and the paths of other vehicles are determined in the spatiotemporal state action diagram based on the combination of the vehicle's actions and the actions of other vehicles branching from the yellow light. Here, when the light is yellow, other vehicles approaching the intersection in the opposite lane are stopped at the intersection's stop line, and the probability that the vehicle, waiting to turn right, will proceed to turn right increases. The driving condition-reward data also includes such driving conditions. Therefore, the approximation function 50 increases the probability that the vehicle's actions will be assigned a higher reward in the combination of the vehicle's actions of proceeding to turn right and other vehicles stopping at the stop line. This probability is further increased when the right-turn arrow signal is illuminated. In this way, by considering the state of the traffic signal, the vehicle can select a more appropriate action for the scenario it encounters in the actual traffic environment.

[0099] As mentioned above, although the preferred embodiment of this disclosure was described, this disclosure is by no means limited to the above-mentioned embodiment, and various modifications can be implemented without departing from the scope of the gist of this disclosure.

[0100] For example, in the above embodiment, the autonomous driving system 100 is configured to include the long-term plan creation unit 20, the medium-term plan creation unit 30, and the short-term plan creation unit 60. However, these components may be arbitrarily integrated. Furthermore, the medium-term plan creation unit 30 and the interactive prediction unit 40, which serve as the vehicle behavior generation device, may also be integrated.

Claims

1. A vehicle behavior generation device comprising: a first setting unit for setting a plurality of actions that the vehicle can take when traveling on a predetermined track; a second setting unit for setting a plurality of actions that can be taken by other vehicles around the host vehicle for each of the plurality of actions of the host vehicle set by the first setting unit; an output unit that outputs information related to the possibility of contact between the host vehicle and the other vehicle for each combination of the behavior of the host vehicle set by the first setting unit and the behavior of the other vehicle set by the second setting unit; and A selection unit selects an action of the vehicle based on the information output by the output unit. The output unit includes a function device, and the function device simulates a plurality of environments with different combinations of the surrounding environment of the vehicle, the condition of the vehicle, and the condition of the other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to assign a higher reward to the condition of the vehicle that can avoid contact between the vehicle and the other vehicles, and to assign a reward lower than the reward to the condition of the vehicle in which the vehicle has already contacted the other vehicles. The output unit uses the function device to assign a reward to the action of the vehicle in each combination of the action of the vehicle and the action of the other vehicles, and outputs the reward as the information. The selection unit selects the behavior of the host vehicle to which the highest reward is given by the output unit.

2. The vehicle behavior generation device according to claim 1, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

3. The vehicle behavior generation device according to claim 1 or 2, wherein: The first setting unit and the second setting unit set actions that the host vehicle and the other vehicles can take based on a state of a traffic signal that controls traffic on a road.

4. The vehicle behavior generation device according to claim 1 or 2, wherein: The second setting unit sets an action that the other vehicle can take based on the driving characteristics of the other vehicle.

5. The vehicle behavior generation device according to claim 1 or 2, wherein: A trajectory generating unit is provided for generating a planned travel trajectory of the host vehicle according to the behavior of the host vehicle set by the first setting unit and a planned travel trajectory of the other vehicle according to the behavior of the other vehicle set by the second setting unit, The output unit outputs the information based on a combination of the planned travel trajectory of the host vehicle and the planned travel trajectory of the other vehicle.

6. A vehicle behavior generation method comprising the following processing executed by at least one processor: The first setting step is to set a plurality of actions that the vehicle can take when driving on a predetermined runway; a second setting step of setting a plurality of actions that can be taken by other vehicles around the host vehicle for each of the plurality of actions of the host vehicle set in the first setting step; an outputting step of outputting information related to the possibility of contact between the host vehicle and the other vehicle for each combination of the behavior of the host vehicle set in the first setting step and the behavior of the other vehicle set in the second setting step; and a selection step of selecting an action of the vehicle based on the information outputted in the output step; In the output step, a function is used to assign a reward to the action of the vehicle in each combination of the action of the vehicle and the action of the other vehicle, and the reward is output as the information, wherein: The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of the other vehicle, and learns based on driving condition-reward data. The driving condition-reward data is created to assign a higher reward to the condition of the own vehicle that can avoid contact between the own vehicle and the other vehicle, and assign a lower reward to the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. In the selecting step, the behavior of the host vehicle to which the highest reward was assigned in the outputting step is selected.

7. The vehicle behavior generation method according to claim 6, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

8. The vehicle behavior generation method according to claim 6 or 7, wherein: In the first setting step and the second setting step, actions that can be taken by the host vehicle and the other vehicles are set based on the state of a traffic signal that controls traffic on a road.

9. The vehicle behavior generation method according to claim 6 or 7, wherein: In the second setting step, actions that can be taken by the other vehicles are set in consideration of the driving characteristics of the other vehicles.

10. A computer-readable non-transitory storage medium storing a vehicle behavior generation program including instructions executed by at least one processor, wherein the instructions, when executed by the at least one processor, are configured to cause the at least one processor to perform the following processing: Set multiple actions that the vehicle can take when driving on a predetermined track; For each of the plurality of actions set for the host vehicle, a plurality of actions that other vehicles around the host vehicle may take are set; outputting information on the possibility of contact between the host vehicle and the other vehicle for each combination of the set behavior of the host vehicle and the set behavior of the other vehicle; and Based on the outputted information, the vehicle's actions are selected. When the command is executed by at least one of the processors, the at least one processor performs the following processing: When outputting the above information, a function is used to assign a reward to the action of the above vehicle in each combination of the action of the above vehicle and the action of the above other vehicles, and the reward is output as the above information, wherein, The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of the other vehicle, and learns based on driving condition-reward data. The driving condition-reward data is created to assign a higher reward to the condition of the own vehicle that can avoid contact between the own vehicle and the other vehicle, and assign a lower reward to the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. When selecting the action of the host vehicle, the action of the host vehicle that provides the highest reward is selected. The storage medium according to claim 10 , wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

12. The storage medium according to claim 10 or 11, wherein: When the command is executed by at least one of the processors, the at least one processor further performs the following processing: When setting the behavior of the host vehicle and when setting the behavior of the other vehicles, the behaviors that the host vehicle and the other vehicles can take are set based on the state of a traffic signal that controls traffic on a road.

13. The storage medium according to claim 10 or 11, wherein: When the command is executed by at least one of the processors, the at least one processor further performs the following processing: When setting the behavior of the other vehicle, the behavior that the other vehicle can take is set based on the driving characteristics of the other vehicle.

Images