Vehicle driving evaluation device, vehicle driving evaluation method, and vehicle driving evaluation program

Abstract

The present invention provides a vehicle behavior evaluation device, a vehicle behavior evaluation method, and a vehicle behavior evaluation program. An interactive prediction unit (40) of the vehicle behavior evaluation device uses an approximate function (50) to calculate a reward that takes into account the driving status of other vehicles for future actions that the vehicle can take. The behavior of the vehicle is evaluated based on the calculated reward. The approximate function (50) simulates a plurality of environments with different combinations of the surrounding environment of the vehicle, the condition of the vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward when the vehicle is in a condition where the vehicle can avoid contact with other vehicles, and to give a lower reward when the vehicle has already contacted other vehicles.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on Japanese Patent Application No. 2020-24654 filed in Japan on February 17, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a vehicle behavior evaluation device, a vehicle behavior evaluation method, and a vehicle behavior evaluation program. Background Art

[0004] For example, Patent Document 1 describes a trajectory evaluation device for evaluating a vehicle's future trajectory. This trajectory evaluation device generates a future trajectory for the vehicle and predicts the future positions of objects around the vehicle. It then evaluates multiple locations on the vehicle's future trajectory based on their relative positional relationships with the predicted future positions of the objects in the longitudinal direction and the width direction of the road. The future trajectory of the vehicle is then evaluated based on the evaluation results for each location.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-95149

[0006] The trajectory evaluation device of Patent Document 1, described above, calculates the target trajectory candidate for the vehicle and the future position of the object, assuming that the vehicle and the object are moving at a constant speed or with a constant acceleration. Furthermore, for each coordinate of the target trajectory candidate, an evaluation value is calculated based on the deviation distance from the centerline of the recommended lane, the magnitude of the lateral displacement (i.e., angular velocity), the amount of deviation from the target lane, and the distance from the object in the travel direction and lateral direction. The target trajectory candidate is evaluated by summing the evaluation results for each coordinate.

[0007] However, this vehicle and other objects such as vehicles do not always move at a constant speed or with a constant acceleration. Depending on the traffic conditions, the vehicle may slow down or even stop. In addition, this vehicle may encounter a situation where driving in the recommended lane is not necessarily the target track. For example, when the width of the drivable road is narrowed due to a parked vehicle in the lane in which this vehicle is traveling, this vehicle may need to temporarily move out to the opposite lane to travel. In this case, when there are other vehicles traveling in the opposite lane, the actions that this vehicle should take may vary depending on the actions of the other vehicles. For example, when this vehicle first starts to go around the parked vehicle, if the other vehicle traveling in the opposite lane slows down or stops, it is desirable for this vehicle to continue to go around the parked vehicle and return to the original driving lane. On the other hand, when the other vehicle approaches the position of the parked vehicle in the opposite lane, it is preferable for this vehicle to slow down or stop in order to allow the other vehicle to pass.

[0008] The trajectory evaluation device of Patent Document 1 calculates target trajectory candidates for the vehicle and the future position of the object, assuming that the vehicle and the object are moving at a constant speed or with a constant acceleration. Furthermore, the target trajectory candidate is evaluated solely based on the distance to the recommended lane and the distance to the object. Therefore, in situations such as the one described above, it is difficult to set a target trajectory candidate that allows for a desired action and assign a high evaluation to that target trajectory candidate. Summary of the Invention

[0009] The present disclosure is made in view of the above-mentioned points, and aims to provide a vehicle behavior evaluation device, a vehicle behavior evaluation method, and a vehicle behavior evaluation program that can appropriately evaluate the behavior of a vehicle suitable for various traffic conditions faced by the vehicle.

[0010] To achieve the above-mentioned object, the vehicle behavior evaluation device disclosed herein is a device for evaluating the future behavior of a vehicle. The vehicle behavior evaluation device is configured to include:

[0011] a setting unit for setting a plurality of actions that the vehicle can take when traveling on a predetermined course; and

[0012] An evaluation unit includes a function device, which simulates multiple environments with different combinations of the surrounding environment of the vehicle, the condition of the vehicle, and the conditions of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward when the vehicle is in a condition where the vehicle can avoid contact with other vehicles, and to give a reward lower than the above reward when the vehicle has already contacted other vehicles. The evaluation unit uses the function device to calculate a reward for the action of the vehicle set by the setting unit, taking into account the driving conditions of other vehicles around the vehicle, and evaluates the action of the vehicle based on the calculated reward.

[0013] Furthermore, the vehicle behavior evaluation method disclosed herein is a method for evaluating the future behavior of the vehicle.

[0014] executed by at least one processor,

[0015] The vehicle behavior evaluation method is configured to include:

[0016] A setting step for setting a plurality of actions that the vehicle can take when traveling on a predetermined runway; and

[0017] An evaluation step, using a function device, calculates a reward for the behavior of the vehicle set in the setting step taking into account the driving status of other vehicles around the vehicle, and evaluates the behavior of the vehicle based on the calculated reward, wherein the function device simulates a plurality of environments with different combinations of the surrounding environment of the vehicle, the condition of the vehicle, and the condition of other vehicles, and learns based on driving condition-reward data, and the driving condition-reward data is created as follows: a higher reward is given in the condition of the vehicle where the vehicle can avoid contact with other vehicles, and a reward lower than the above reward is given in the condition of the vehicle where the vehicle has already contacted other vehicles.

[0018] Furthermore, the vehicle behavior evaluation program of the present disclosure is a program stored in a computer-readable non-transitory storage medium and includes instructions. When the instructions are executed by at least one processor, the at least one processor is configured to perform the following steps:

[0019] Set multiple actions that the vehicle can take when driving on a predetermined track.

[0020] Using a function device, a reward for the set action of the vehicle is calculated taking into account the driving status of other vehicles around the vehicle, and the action of the vehicle is evaluated based on the calculated reward. The above-mentioned function device is simulated in multiple environments with different combinations of the surrounding environment of the vehicle, the condition of the vehicle and the condition of other vehicles, and is learned based on the driving condition-reward data. The above-mentioned driving condition-reward data gives a higher reward when the vehicle is able to avoid contact with other vehicles, and gives a reward lower than the above-mentioned reward when the vehicle has already contacted other vehicles.

[0021] According to the aforementioned vehicle behavior evaluation device, vehicle behavior evaluation method, and vehicle behavior evaluation program, a function unit calculates a reward for a set future action that the host vehicle can take, taking into account the driving conditions of other vehicles. The vehicle's behavior is evaluated based on this calculated reward. The function unit simulates multiple environments with varying combinations of the host vehicle's surroundings, the host vehicle's conditions, and the conditions of other vehicles, and learns based on driving condition-reward data. This driving condition-reward data is created to assign a higher reward when the host vehicle is in a condition where it can avoid contact with another vehicle, and a lower reward when it has already made contact with another vehicle. Therefore, the more the host vehicle's future action includes driving conditions similar to those in which contact with other vehicles can be avoided, the higher the reward is assigned. Therefore, based on the reward assigned to the host vehicle's future action, it is possible to appropriately evaluate whether the future action of the host vehicle is suitable for the traffic conditions facing the host vehicle.

[0022] In addition, in order to facilitate understanding of the present disclosure, the reference numerals in the claims merely show an example of the correspondence relationship with the specific configurations in the embodiments described later, and are not intended to limit the scope of the present disclosure.

[0023] In addition, the technical features described in each of the claims other than the above-mentioned features will become clear from the description of the embodiments and the accompanying drawings to be described later. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a structural diagram showing the overall structure of an automatic driving system to which the vehicle behavior evaluation device according to the embodiment is applied.

[0025] Figure 2 This is a diagram showing an example of a spatiotemporal state-action diagram.

[0026] Figure 3 This is a diagram for explaining an example of a method for creating an approximate function.

[0027] Figure 4 It means in Figure 1 The flowchart of the processing performed to automatically drive a vehicle in the automatic driving system shown.

[0028] Figure 5 This is a flowchart showing a process for generating a host vehicle behavior evaluation by evaluating the behavior of the host vehicle.

[0029] Figure 6 This diagram explains actions that the host vehicle and other vehicles can take when there is a parked vehicle in the host vehicle's lane and the host vehicle must go around the parked vehicle and there are other vehicles traveling in the opposite lane.

[0030] Figure 7 This is a diagram showing an example of a spatiotemporal state behavior diagram that takes into account the driving characteristics of other vehicles.

[0031] Figure 8 As an example of preferably considering the driving characteristics of other vehicles, this diagram shows a situation in which another vehicle traveling on the main line exists near the vehicle when the vehicle attempts to merge from a side road onto the main line.

[0032] Figure 9 This is a diagram showing an example of a spatiotemporal state-action diagram taking into account the state of traffic signals.

[0033] Figure 10 As an example of preferably considering the state of a traffic signal, this diagram shows a situation in which another vehicle is traveling in the opposite lane while the vehicle is waiting to turn right at an intersection with a traffic light. DETAILED DESCRIPTION

[0034] (First embodiment)

[0035] The following describes an embodiment of the present disclosure in detail with reference to the accompanying drawings. Furthermore, this embodiment describes an example in which a vehicle behavior evaluation device is applied to an autonomous driving system for a vehicle. It should be noted that the application example of the vehicle behavior evaluation device of the present disclosure is not limited to autonomous driving systems for vehicles. For example, it can also be applied to a system that presents the driver of the vehicle with recommended actions that have been evaluated as appropriate by the vehicle behavior evaluation device.

[0036] exist Figure 1 , the overall structure of the automatic driving system 100 to which the vehicle behavior evaluation device of this embodiment is applied is shown. Figure 1 As shown, the autonomous driving system 100 includes various sensors 10 , a long-term plan creating unit 20 , a medium-term plan creating unit 30 , an interactive prediction unit 40 , a short-term plan creating unit 60 , and a path following control unit 70 .

[0037] The various sensors 10 include, for example, at least one of sensors such as cameras, LiDAR, and millimeter-wave radar. The various sensors 10 detect the surrounding environment information of the vehicle, the vehicle information indicating the status of the vehicle, and other vehicle information indicating the status of other vehicles around the vehicle. The various sensors 10 detect obstacles such as fallen objects on the road, parked vehicles on the road, road surface displays such as guardrails, curbs, and driving dividing lines, and stationary objects such as trees as surrounding environment information. In addition, the various sensors 10 detect moving objects such as pedestrians, animals other than humans, and other vehicles as surrounding environment information. In addition, the various sensors 10 detect the position, direction, speed, and acceleration of the vehicle as vehicle information. Furthermore, the jerk (change in acceleration per unit time) of the vehicle can also be detected as vehicle information. The various sensors 10 detect the position, direction, speed, and acceleration of other vehicles as other vehicle information. Furthermore, the jerk of other vehicles can also be detected as other vehicle information. In addition, the autonomous driving system 100 can also obtain at least a portion of the vehicle information and other vehicle information from an external server such as a control center via communication.

[0038] The long-term plan creation unit 20 uses a road map database to create a route (driving path) for the vehicle to travel from the departure point (current location) to the destination, based on the departure point (current location) and destination specified by the vehicle's occupants. If the vehicle is equipped with a navigation device, the long-term plan creation unit 20 may be comprised of the navigation device. Alternatively, the long-term plan creation unit 20 may be located on an external server, such as a management center. Furthermore, the vehicle's occupants may transmit information related to the departure point (current location) and destination to the external server, which may then create a driving path to the destination.

[0039] The medium-term plan creation unit 30 sets the driving route to the destination in greater detail than the long-term plan creation unit 20. Specifically, for example, if there are multiple lanes on the driving route, the medium-term plan creation unit 30 determines which lane the vehicle is traveling in. First, the medium-term plan creation unit 30 obtains the driving route to the destination created by the long-term plan creation unit 20. Furthermore, the medium-term plan creation unit 30 obtains road information (number of lanes, lane width, shape, etc.) surrounding the vehicle's current location from the long-term plan creation unit 20 (road map database). Furthermore, the medium-term plan creation unit 30 obtains information about the vehicle's surrounding environment, vehicle information, and other vehicle information detected by various sensors 10.

[0040] If the road on which the host vehicle is traveling has multiple lanes, the mid-term plan creation unit 30 determines the lane position of the host vehicle based on the acquired road information and the host vehicle information. Furthermore, if the mid-term plan creation unit 30 detects the presence of other vehicles around the host vehicle based on the acquired road information and surrounding environment information, it also determines the lane in which the other vehicle is traveling (whether it is a lane in the same direction of travel or an oncoming lane, and if there are multiple lanes, the lane position of the other vehicle).

[0041] The mid-term plan creation unit 30 then calculates the planned trajectory of the vehicle on the road until a predetermined time (e.g., 5 seconds) or a predetermined distance (e.g., 100 meters) in the future based on the acquired driving route, road information, surrounding environment information, and vehicle information. The predetermined time or distance may be fixed values or variable values that vary, for example, depending on the speed of the vehicle.

[0042] For example, if the mid-term plan creation unit 30 determines based on surrounding environment information that there are no obstacles in the lane the vehicle is traveling in and that there are no moving objects such as other vehicles around the vehicle, it may determine the vehicle's path, representing the vehicle's planned trajectory, without going through the interactive prediction unit 40. Because the likelihood of contact with obstacles or other vehicles is low, the mid-term plan creation unit 30 may calculate a planned trajectory along the lane the vehicle is traveling in. Furthermore, if the vehicle makes a left or right turn or a branching road, it may calculate a planned trajectory corresponding to the shape of the road. Even if an obstacle or other vehicle is detected, but the obstacle or other vehicle is not affected by the vehicle's movement, the mid-term plan creation unit 30 may linearly predict the movement of the other vehicle and determine the vehicle's path based on this prediction.

[0043] On the other hand, if the surrounding environment information indicates the presence of an obstacle such as a parked vehicle on the road, the mid-term plan creation unit 30 calculates a planned trajectory based on the road information to avoid the obstacle. Furthermore, if the driving path involves a right or left turn at the upcoming intersection, and if the vehicle is traveling in a right-turn lane or a lane different from the left-turn lane, the mid-term plan creation unit 30 calculates a planned trajectory for changing lanes to the right-turn lane or the left-turn lane.

[0044] Furthermore, when calculating the planned trajectory, the mid-term plan creation unit 30 does not consider the lane positions or positions of other vehicles, even when other vehicles are present. This is because the interactive prediction unit 40 determines the vehicle's behavior (the vehicle's path and target speed) to avoid contact with other vehicles based on the planned trajectory. Therefore, when other vehicles are present, the mid-term plan creation unit 30 outputs to the interactive prediction unit 40 the information necessary to determine the vehicle's path to avoid contact with other vehicles. Specifically, the mid-term plan creation unit 30 outputs to the interactive prediction unit 40 the vehicle's current (and past) vehicle information, lane position information, current (and past) other vehicle information, lane position information of other vehicles, the vehicle's planned trajectory, the location and size of obstacles, and road information.

[0045] The interactive prediction unit 40, as a setting unit, sets multiple actions that the vehicle can take. Examples of the set actions include "waiting," "climbing slowly," "moving forward," and "turn signal on." Multiple actions may simply involve traveling at different speeds. That is, the interactive prediction unit 40 may set traveling at speed A and traveling at speed B, which is different from speed A, as multiple actions. Furthermore, the interactive prediction unit 40 determines a vehicle path representing the planned trajectory of the vehicle when executing each of the set actions. This vehicle path is determined based on the vehicle's behavior and its planned trajectory. For example, if the vehicle is waiting, the vehicle path remains at the waiting location, but if the vehicle is moving, the vehicle path is determined to depict the planned trajectory. Furthermore, the length of the vehicle path per unit time varies depending on whether the vehicle is traveling at a certain speed or at a speed relatively lower (or higher) than that speed. Furthermore, the trajectory of the vehicle path based on the planned trajectory may be altered depending on the vehicle's speed. For example, when the planned trajectory draws a curve, the vehicle path may be determined so that the curvature becomes smaller when the vehicle travels at a relatively high speed than when the vehicle travels at a relatively low speed.

[0046] The interactive prediction unit 40 sets multiple actions that other vehicles can take for each of the self-vehicle's actions. Furthermore, the interactive prediction unit 40 determines other vehicle paths, representing the planned paths of the other vehicles if they execute each of the set actions. The other vehicle paths are determined in the same manner as the self-vehicle path described above.

[0047] For example, the interactive prediction unit 40 determines the aforementioned own vehicle path and other vehicle paths until a predetermined time (e.g., 1 to 5 seconds) has elapsed. As described above, the length per unit time and trajectory of the own vehicle path and other vehicle paths vary depending on the actions of the own vehicle and other vehicles. Therefore, each combination of the own vehicle path corresponding to the multiple actions that the own vehicle can take and the other vehicle paths corresponding to the multiple actions that other vehicles can take in response to the own vehicle's actions not only includes distance (spatial) relationships between the own vehicle and other vehicles, but also includes temporal relationships. Therefore, the set of each combination of the own vehicle path corresponding to the multiple actions that the own vehicle can take and the other vehicle paths corresponding to the multiple actions that other vehicles can take in response to the own vehicle's actions constitutes a spatiotemporal state-action graph that represents the various states of the own vehicle and other vehicles in time and space. This spatiotemporal state-action graph may also include additional information such as the own vehicle's speed, acceleration, and direction at multiple locations on the own vehicle path and other vehicle paths, as well as additional information such as the speed, acceleration, and direction of other vehicles.

[0048] Figure 2 An example of a spatiotemporal state action diagram is shown. Figure 2 The example shown is a simple example in which the actions that can be taken by the own vehicle and other vehicles are set to two types: waiting and moving.

[0049] In the interactive prediction unit 40, an approximate function 50 is used to evaluate various combinations of multiple actions that the host vehicle can take and multiple actions that other vehicles can take in response to the actions of the host vehicle. Here, the approximate function 50 functions as an output unit that outputs information related to the possibility of contact between the host vehicle and other vehicles as the evaluation result. Figure 2While the example in which information related to the likelihood of contact between the host vehicle and another vehicle is used as a reward value is shown, the information related to the likelihood of contact between the host vehicle and another vehicle may also be letters, symbols, or the like, as long as it indicates the degree of likelihood of contact between the host vehicle and another vehicle. This evaluation is determined based on the position, speed, and path of the host vehicle, as well as the position, speed, and paths of other vehicles. Furthermore, the interactive prediction unit 40, functioning as a selection unit, selects an action for the host vehicle based on the evaluation results of the approximation function generator 50. Specifically, the interactive prediction unit 40 selects an action for the host vehicle that is most likely to avoid contact with another vehicle and outputs information representing this action to the medium-term plan creation unit 30. For example, the interactive prediction unit 40 outputs information representing the action of the host vehicle that maximizes the reward value to the medium-term plan creation unit 30. The information representing the action of the host vehicle may be one of a plurality of set actions. In addition to this single action, the path of the host vehicle when executing the action and the target speed within a specified time interval (e.g., one-second intervals) while traveling along the path may also be output to the medium-term plan creation unit 30. The latter is preferred because it can reduce the computational burden on the short-term plan creation unit 60, which generates a more detailed driving trajectory. The evaluation method using the approximate function generator 50 will be described in detail later. The interactive prediction unit 40 corresponds to the vehicle behavior evaluation device in this disclosure.

[0050] The mid-term plan creation unit 30 and the interactive prediction unit 40 are formed by, for example, the vehicle-mounted ECU. The vehicle-mounted ECU is a computer mounted on the vehicle and is mainly composed of a microcomputer having at least one processor, a computer-readable non-transient storage medium such as RAM and ROM, an input / output interface, and a bus connecting these components. The non-transient storage medium of the vehicle-mounted ECU stores the data for executing the program. Figure 6 The vehicle behavior generation process shown in FIG. Furthermore, the processor included in the on-vehicle ECU executes the program. Furthermore, the mid-term plan creation unit 30 and the interactive prediction unit 40 may be comprised of a single on-vehicle ECU or separate on-vehicle ECUs. Furthermore, the functions of the mid-term plan creation unit 30 and the interactive prediction unit 40 may be located on an external server that is capable of communicating with the vehicle and is located outside the vehicle.

[0051] The medium-term plan creation unit 30 receives information representing the behavior of the vehicle output by the interactive prediction unit 40 and outputs the information to the short-term plan creation unit 60. Based on the received information representing the behavior of the vehicle (for example, the vehicle path and target speed), the short-term plan creation unit 60 creates a short-term plan (for example, a plan for 1 to 2 seconds later) for actually controlling the behavior of the vehicle and outputs it to the path following control unit 70. In this way, the detailed trajectory of the vehicle is determined. In addition, the short-term plan creation unit 60 is also configured to receive detection signals from various sensors 10. In the event of an unexpected situation (for example, the behavior of another vehicle different from that predicted, a moving object jumping out of a hiding place, etc.), the short-term plan creation unit 60 can also respond to unexpected situations by creating a short-term plan to make the vehicle stop suddenly or change the direction of travel of the vehicle.

[0052] The path following control unit 70, which includes a steering control unit, an engine control unit, and a brake control unit, controls the vehicle's steering direction, driving force, and braking force to control the vehicle's behavior according to the short-term plan created by the short-term plan creation unit 60. The short-term plan creation unit 60 and the path following control unit 70 are also implemented by, for example, the vehicle's onboard ECU.

[0053] Next, the evaluation method using the approximate function device 50 in the interactive prediction unit 40 will be described. Figure 3 , an example of a method for creating an approximate function 50 is described. Figure 3 As shown, the approximate function generator 50 is created by a simulator 120, a reward calculation unit 130, and an approximate function generator 140. The simulator 120, the reward calculation unit 130, and the approximate function generator 140 are installed outside the vehicle.

[0054] The simulator 120 generates driving trajectories for the vehicle and other vehicles in various traffic environments based on various scenarios representing the behavior of the vehicle and other vehicles (such as left or right turns, overtaking, passing around obstacles, merging, and branching), thereby simulating the driving conditions of the vehicle and other vehicles. The simulated driving conditions of the vehicle and other vehicles include changes in speed, including stopping. Furthermore, the simulated driving conditions include both driving conditions in which the vehicle and other vehicles can avoid contact and driving conditions in which contact has occurred.

[0055] The simulator 120 outputs driving data representing the driving conditions of the host vehicle and other vehicles in each of a plurality of stages, into which the driving conditions of the host vehicle and other vehicles are divided, to the reward calculation unit 130. Specifically, the simulator 120 outputs data such as the host vehicle's position, speed, acceleration, and other vehicle conditions; the host vehicle's planned path; the location of obstacles; the location of other vehicles, speed, acceleration, and other vehicle conditions; and elapsed time as driving data in each stage to the reward calculation unit 130.

[0056] Based on the received driving data, the reward calculation unit 130 assigns a reward to the vehicle's condition in relation to the driving conditions of other vehicles. Several examples of reward calculation methods are described below. The reward calculation unit 130 calculates a first cost, which is assigned as a penalty if the vehicle comes into contact with another vehicle. Furthermore, the reward calculation unit 130 calculates a second cost, which is set to a higher value the shorter the time to contact, for the vehicle's driving conditions prior to contact. Furthermore, the reward calculation unit 130 calculates the reward as the inverse of the sum of the calculated first and second costs. Thus, driving conditions that are more likely to avoid contact with other vehicles are assigned a higher reward, while conversely, driving conditions that are more likely to result in contact with other vehicles are assigned a lower reward.

[0057] The reward calculation unit 130 may also calculate a third cost based on the distance between the host vehicle and an object, including other vehicles, as determined from driving data. The closer the distance, the larger the third cost. The reward calculation also considers this third cost (in addition to the sum of the first and second costs, and also the inverse of this sum). This is because the closer the distance between the host vehicle and an object, the higher the likelihood of contact. Furthermore, if there are multiple objects around the host vehicle (e.g., obstacles or other vehicles), the third cost is calculated for each object.

[0058] Furthermore, the reward calculation unit 130 may also calculate a fourth cost, setting the fourth cost higher as the time required to avoid contact between the host vehicle and another vehicle (elapsed time) increases. The reward calculation also takes this fourth cost into account (in addition to the sum of the first to third costs, the inverse of this sum is also calculated). This is because by taking elapsed time into account, a higher reward can be assigned to driving conditions (host vehicle behavior) that more quickly avoid contact with another vehicle.

[0059] The reward calculation unit 130 may also calculate a fifth cost, setting it to a higher value the longer the vehicle's duration and / or distance of departure from its intended lane, and also consider this fifth cost in calculating the reward. This is because, for example, if a portion of the vehicle's lane is blocked by a parked vehicle, and the vehicle needs to travel at least partially or entirely in the oncoming lane to bypass the parked vehicle, the longer the time and / or distance spent in the oncoming lane, the higher the likelihood of contact with the other vehicle. Furthermore, the reward calculation unit 130 may also calculate a sixth cost, setting it to a higher value the greater the vehicle's acceleration, and also consider this sixth cost in calculating the reward. This is because it is assumed that higher vehicle acceleration increases the likelihood of contact with other vehicles, etc.

[0060] In addition, the reward assigned to the action of the vehicle can be calculated based on other costs that reflect the possibility of contact between the vehicle and other vehicles, in addition to or instead of the first to sixth costs mentioned above. In addition, among the first to sixth costs mentioned above, the first cost, which is a penalty when the vehicle comes into contact with other vehicles, is set to be relatively larger than the other costs. And, the second cost assigned to the driving condition before the vehicle comes into contact with other vehicles is calculated by discounting the first cost at a discount rate corresponding to the time to contact. Specifically, the shorter the time to contact, the smaller the discount rate. Therefore, it is possible to assign very low rewards to the driving condition in which the vehicle comes into contact with other vehicles and the driving condition in which contact is reached.

[0061] In this manner, the reward calculation unit 130 creates driving condition-reward data by assigning rewards corresponding to the likelihood of contact (or the likelihood of avoiding contact) to various driving conditions of the host vehicle and other vehicles. The created driving condition-reward data is then provided to the approximation function generator 140. Alternatively, the reward calculation unit 130 may directly provide each driving condition and the reward data for each driving condition to the approximation function generator 140. Alternatively, the reward calculation unit 130 may aggregate the driving conditions and reward data for each planned trajectory of the host vehicle and provide the aggregated data to the approximation function generator 140.

[0062] Based on the driving condition-reward data created by the reward calculation unit 130, the approximating function generator 140 causes the approximating function generator 50 to learn a function that approximates the relationship between the various driving conditions (spatiotemporal states) of the host vehicle and other vehicles and rewards. A deep neural network can be used as the approximating function generator 50, and so-called deep Q-learning can be used as the learning method. However, the approximating function generator 50 is not limited to neural networks; for example, a support vector machine (SVM) can also be used. Such an approximating function generator 50 can assign a reward as an evaluation to each combination of the host vehicle's possible actions and the possible actions of other vehicles in response to the host vehicle's actions, as reflected in the spatiotemporal state-action graph. Furthermore, while the example in which a pre-created function is installed in the vehicle as the approximating function generator 50 is described, the function associated with the approximating function generator 50 can also be relearned based on past evaluation results or the vehicle's driving history.

[0063] Next, refer to Figure 4 The flowchart is Figure 1 The processing executed in the illustrated automatic driving system 100 for automatically driving the vehicle will be described.

[0064] exist Figure 4 In the flowchart of FIG. 1 , in step S100, the long-term plan creation unit 20 uses the road map database to set a route (driving route) for the vehicle to travel from the departure point (current location) to the destination. In step S110, the medium-term plan creation unit 30 obtains the driving route to the destination set by the long-term plan creation unit 20.

[0065] In step S120, the medium-term plan creation unit 30 and the interactive prediction unit 40 generate the behavior of the vehicle (the vehicle path, the target speed) that is evaluated as appropriate. Figure 5 The process of generating the host vehicle behavior evaluation is described in detail with reference to the flowchart of FIG. In the following step S130, the short-term plan creation unit 60 creates a short-term plan for actually controlling the host vehicle's behavior based on the generated host vehicle behavior. In step S140, the path following control unit 70 controls the host vehicle's behavior by controlling the steering direction, driving force, and braking force according to the created short-term plan.

[0066] In step S150, it is determined whether the vehicle has arrived at the designated destination. If not, the process from step S120 is repeated. If the destination has been changed, the process is executed again from the initial step S100.

[0067] Next, refer to Figure 5The vehicle behavior evaluation generation process executed in the mid-term plan creation unit 30 and the interactive prediction unit 40 will be described with reference to the flowchart of FIG. This vehicle behavior evaluation generation process corresponds to the step S120 described above.

[0068] In the first step S200, road information around the current position of the vehicle, surrounding environment information detected by various sensors 10, and vehicle information are input. In the next step S210, the shape, position, size, and whether they are moving or stationary objects of objects including other vehicles are identified based on the surrounding environment information.

[0069] In step S220, based on the recognition result in step S210, it is determined whether there are other vehicles around the host vehicle. If it is determined that there are other vehicles around the host vehicle, the process proceeds to step S230. On the other hand, if it is determined that there are no other vehicles around the host vehicle, the process proceeds to step S320.

[0070] In step S230, information necessary for setting the actions that the host vehicle can take and the actions that other vehicles can take in response to the actions of the host vehicle, as well as information necessary for evaluating the actions of the host vehicle in each combination of the actions of the host vehicle and the actions of other vehicles, is output to the interactive prediction unit 40. Specifically, information is output including the current and past vehicle state information such as the position, direction, speed, and acceleration of the host vehicle, lane position information of the host vehicle, current and past vehicle state information such as the position, direction, speed, and acceleration of other vehicles, lane position information of other vehicles, and road information including the planned trajectory of the host vehicle, the location and size of obstacles, the road shape, the number of lanes, and the like.

[0071] In step S240, the interactive prediction unit 40 sets a plurality of actions that the vehicle can take, including at least whether the vehicle is waiting or moving, based on the various information obtained. Furthermore, in step S250, the interactive prediction unit 40 defines a vehicle path representing the projected trajectory of the vehicle when the vehicle executes each of the set actions.

[0072] In step S260, the interactive prediction unit 40 also sets multiple actions that other vehicles can take in response to each action of the host vehicle. These multiple actions include at least whether the other vehicle should wait or move. Furthermore, in step S270, the interactive prediction unit 40 determines another vehicle path representing the other vehicle's projected trajectory when the other vehicle executes each of the set actions.

[0073] Furthermore, in step S280, the interactive prediction unit 40 uses the approximating function 50 to assign rewards to the vehicle's actions based on the vehicle's position, speed, and path, as well as the positions, speeds, and paths of other vehicles, for each combination of the vehicle's possible actions and the actions of other vehicles in response to the vehicle's actions. For example, the approximating function 50 calculates rewards based on the learned content for each combination of the vehicle's position, direction, speed, and acceleration at multiple points within the vehicle's path, and the positions, directions, speeds, and accelerations of other vehicles at multiple points within the corresponding paths, under an environment determined by factors such as road shape, number of lanes, the vehicle's planned trajectory, and the location, direction, and size of obstacles. Furthermore, by summing the rewards at multiple points within the vehicle's path and the paths of other vehicles, the rewards are calculated for each combination of the vehicle's actions and the actions of other vehicles.

[0074] For example, Figure 6 As shown, a scenario in which a vehicle encounters another vehicle traveling in the opposite lane while a parked vehicle is in its lane and it must drive around it is shown. In this scenario, the vehicle approaches the parked vehicle first and, once it is able to begin maneuvering around it, the other vehicle in the opposite lane is likely to wait until the vehicle returns to its original lane or slows down. The driving condition-reward data also includes such driving conditions, in which a relatively high reward is assigned to the vehicle's action of driving around the parked vehicle. Therefore, in such driving conditions, the vehicle's actions, combined with the actions of other vehicles, are rewarded more for driving around the parked vehicle.

[0075] On the other hand, if another vehicle in the opposite lane approaches the parked vehicle first, the vehicle may need to wait near the parked vehicle or slow down to avoid the other vehicle passing. The driving condition-reward data also includes such driving conditions. In such cases, a higher reward is assigned to the vehicle's actions of waiting for the other vehicle to pass or slowing down. Therefore, in such driving conditions, the vehicle's actions in the combination of its own actions and the actions of the other vehicle will be assigned a higher reward for waiting or slowing down.

[0076] In this embodiment, since an approximate function device 50 is used to assign rewards to the actions of the vehicle, for each combination of multiple actions that the vehicle can take and multiple actions that other vehicles can take in response to the actions of the vehicle, a higher reward can be assigned to the actions of the vehicle that are suitable for the actual situation.

[0077] Again, return to Figure 5 The flowchart is further described. If, in step S280, a reward is assigned to the vehicle's action in each combination of the vehicle's action and the actions of other vehicles, the process proceeds to step S290, where a determination is made as to whether a difference of a predetermined value or greater exists between the highest and second-highest rewards assigned. If the difference is determined to be greater than the predetermined value, the process proceeds to step S310. On the other hand, if the difference is determined not to be greater than the predetermined value, the process proceeds to step S300.

[0078] If the difference between the highest and second-highest rewards does not exceed a predetermined value, it is considered impossible to fully determine which of the vehicle's actions is the best. Therefore, in this embodiment, the process loops back to step S240 to execute the following steps: setting multiple possible actions for the vehicle following each combination of the vehicle's actions and the actions of other vehicles; determining the vehicle's path when the vehicle executes each of the set actions; setting multiple possible actions for other vehicles in response to each of the vehicle's actions; and determining the paths of other vehicles when other vehicles execute each of the set actions. During this loop, the vehicle's actions and paths are considered to determine how the vehicle will behave and how their paths will be affected. Furthermore, when the process loops again, the effects of the actions and paths of other vehicles are considered based on the vehicle's actions and paths considered in the first loop. By looping this process, it is possible to accurately predict how the vehicle and / or other vehicles will behave in response to their actions. However, if this loop is allowed indefinitely, there is a risk that the loop will repeat, making it impossible to determine the vehicle's actions. Therefore, in step S310, it is determined whether the number of loops has reached a predetermined number of times. If the number of loops has not reached the predetermined number of times, the process loops to step S240, but if the number of loops has reached the predetermined number of times, the process proceeds to step S310.

[0079] In step S310, the vehicle's action with the highest reward is selected based on the assigned reward. The interactive prediction unit 40 then outputs information related to the selected vehicle's action (the vehicle's path and target speed) to the mid-term plan creation unit 30. Furthermore, in step S310, instead of selecting the vehicle's action based on the reward, the interactive prediction unit 40 may instruct the mid-term plan creation unit 30 on another action. For example, the mid-term plan creation unit 30 may be instructed to maintain the vehicle's current speed / acceleration, gradually reduce the speed, or stop.

[0080] On the other hand, if it is determined in step S220 that no other vehicles are around the vehicle, then in step S320, the mid-term plan creation unit 30 determines the vehicle's behavior based on the road information so that the vehicle follows the travel path. For example, if the vehicle is traveling along a road, the mid-term plan creation unit 30 calculates a predetermined trajectory along the lane in which the vehicle is traveling and calculates a target speed for traveling along this predetermined trajectory. Furthermore, if the vehicle makes a left or right turn or a branching road, the mid-term plan creation unit 30 calculates a predetermined trajectory and a target speed corresponding to the shape of the road or branching road.

[0081] As described above, according to the vehicle behavior evaluation device of this embodiment, since the approximate function device 50 is used to evaluate the behavior of the vehicle in each combination of the behavior of the vehicle and the behavior of other vehicles, it is possible to appropriately evaluate whether the behavior of the vehicle is suitable for the traffic conditions faced by the vehicle based on the reward given to the (future) behavior of the vehicle.

[0082] Furthermore, in the vehicle behavior evaluation device of this embodiment, the processing performed by at least one processor to evaluate the behavior of the host vehicle corresponds to the vehicle behavior evaluation method of the present disclosure. Furthermore, in the vehicle behavior evaluation device of this embodiment, the program containing instructions for the at least one processor to execute the vehicle behavior evaluation method corresponds to the vehicle behavior evaluation program of the present disclosure.

[0083] (Second embodiment)

[0084] Next, a second embodiment of the present disclosure will be described. However, since the vehicle behavior evaluation device of this embodiment has the same configuration as the vehicle behavior evaluation device of the first embodiment, description of the configuration will be omitted.

[0085] In the first embodiment described above, for example, based on Figure 6 This example illustrates a situation where another vehicle is traveling in the opposite lane, while the vehicle is in the vehicle's lane and must circumvent the parked vehicle. The example also describes the actions that the vehicle and the other vehicle can take. However, when the other vehicle is present and the driver of the other vehicle is in an emergency, the actions of the other vehicle often differ from those in the absence of such an emergency. This embodiment also takes the driving characteristics of these other vehicles into consideration, unlike the first embodiment.

[0086] In this embodiment, if Figure 7As shown, the spatiotemporal state-action diagram considers the driving characteristics of other vehicles. Specifically, based on the behavior of other vehicles detected by various sensors 10, it is determined whether the other vehicles exhibit aggressive or gentle driving characteristics. Because it is difficult to determine whether the other vehicles exhibit aggressive or gentle driving characteristics based solely on their behavior over a short period of time, it is preferable to calculate the probability of aggressive and gentle driving characteristics, for example. Furthermore, the driving characteristics of other vehicles can be categorized into three or more types, such as aggressive, average, and gentle.

[0087] If the driving characteristics of other vehicles can be determined, only the combination of the own vehicle's actions and the actions of other vehicles branching off from those driving characteristics can be considered. However, if the probabilities of each characteristic are calculated for the driving characteristics of other vehicles, the own vehicle's path and the paths of other vehicles can be determined by combining the own vehicle's actions and the actions of other vehicles branching off from those characteristics. In this case, the driving characteristics of other vehicles can be considered as the actions of other vehicles. For example, it can be estimated that the speed and acceleration of other vehicles with aggressive driving characteristics will be higher than those with gentle driving characteristics. Furthermore, the reward ultimately assigned to the own vehicle's actions can be increased or decreased based on the probabilities of each driving characteristic.

[0088] As an example of preferably considering the driving characteristics of other vehicles, for example, in addition to using Figure 6 In addition to the conditions described, Figure 8 The situation shown also meets the requirements. Figure 8 This shows that when the vehicle is about to merge onto the main line from the side road, there are other vehicles traveling on the main line near the vehicle. Figure 8 In the situation shown, if the other vehicle has aggressive driving characteristics, as indicated by the longer arrow, the vehicle is more likely to continue traveling at a higher speed. On the other hand, if the other vehicle has mild driving characteristics, the vehicle is more likely to reduce its speed to allow it to merge with the main line. The driving condition-reward data also includes driving conditions where the other vehicle's driving characteristics are aggressive and mild. Therefore, the approximation function 50 assigns a relatively high reward to the vehicle's action of reducing its speed when the other vehicle's driving characteristics are aggressive, while the likelihood of the vehicle's action of merging with the main line ahead of the other vehicle being assigned a relatively high reward when the other vehicle's driving characteristics are mild increases. In this way, by considering the driving characteristics of the other vehicle, the optimal action for the vehicle to take changes even if the relative positional relationship between the other vehicle and the vehicle remains the same.

[0089] And, as Figure 8 In similar situations, such as when the host vehicle needs to change lanes and another vehicle is traveling in the lane of the change destination, the driving characteristics of the other vehicle are preferably considered. Furthermore, in all situations where there are other vehicles around the host vehicle, the driving characteristics of the other vehicle may also be considered.

[0090] (Third embodiment)

[0091] Next, a third embodiment of the present disclosure will be described. However, since the vehicle behavior evaluation device of this embodiment is configured similarly to the vehicle behavior evaluation device of the first embodiment, description of the configuration will be omitted.

[0092] In the second embodiment described above, the spatiotemporal state-action diagram is created in consideration of the driving characteristics of other vehicles. However, this embodiment differs from the second embodiment in that the spatiotemporal state-action diagram is created in consideration of the state of traffic signals.

[0093] In this embodiment, if Figure 9 As shown in the spatiotemporal state-action diagram, consider the state of traffic signals. Specifically, Figure 10 As shown in the figure, when the vehicle intends to turn right at an intersection and there are other vehicles approaching the intersection in the opposite lane, a spatiotemporal state-action graph corresponding to the state of the traffic signal is created. Furthermore, the state of the traffic signal can be determined based on the detection results of various sensors 10 to determine whether the traffic signal is illuminated green or yellow. Furthermore, if the traffic signal has a right-turn arrow, it can also be determined whether the right-turn arrow is illuminated.

[0094] If the traffic signal is detected to be green, Figure 9 In the spatiotemporal state action diagram shown, the combination of the vehicle's actions at a green light and the actions of other vehicles determines the vehicle's path and the paths of other vehicles. Here, when the light is green, the movement of other vehicles approaching the intersection in the opposite lane is prioritized over the vehicle's right turn. The driving condition-reward data also includes such driving conditions. Therefore, the approximation function 50 is more likely to assign a higher reward to the vehicle's actions in the combination of the vehicle waiting and the other vehicles moving. However, if it is assumed that there is sufficient time before other vehicles enter the intersection, the vehicle's actions signaling to turn right are more likely to be assigned a higher reward.

[0095] On the other hand, if the traffic signal is detected as yellow, the path of the vehicle and the paths of other vehicles are determined in the spatiotemporal state action diagram based on the combination of the vehicle's actions and the actions of other vehicles branching from the yellow light. Here, when the light is yellow, other vehicles approaching the intersection in the opposite lane are stopped at the intersection's stop line, and the probability that the vehicle, waiting to turn right, will proceed to turn right increases. The driving condition-reward data also includes such driving conditions. Therefore, the approximation function 50 increases the probability that the vehicle's actions will be assigned a higher reward in the combination of the vehicle's actions of proceeding to turn right and other vehicles stopping at the stop line. This probability is further increased when the right-turn arrow signal is illuminated. In this way, by considering the state of the traffic signal, the vehicle can select a more appropriate action for the scenario it encounters in the actual traffic environment.

[0096] As mentioned above, although the preferred embodiment of this disclosure was described, this disclosure is by no means limited to the above-mentioned embodiment, and various modifications can be implemented without departing from the scope of the gist of this disclosure.

[0097] For example, in the above-described embodiment, the autonomous driving system 100 is configured to include the long-term plan creation unit 20, the medium-term plan creation unit 30, and the short-term plan creation unit 60. However, these components may be arbitrarily integrated. Furthermore, the medium-term plan creation unit 30 and the interactive prediction unit 40 may also be integrated.

Claims

1. A vehicle behavior evaluation device for evaluating the future behavior of a vehicle, comprising: a setting unit for setting a plurality of actions that the vehicle can take when traveling on a predetermined course; and An evaluation unit includes a function device, the function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data, the driving condition-reward data being created to assign a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicles, and to assign a reward lower than the reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicles, the evaluation unit uses the function device to calculate a reward that takes into account the driving conditions of other vehicles around the own vehicle for the action of the own vehicle set by the setting unit, and evaluates the action of the own vehicle based on the calculated reward, The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A third cost is further calculated for the driving condition of the own vehicle in each of the multiple stages into which the driving conditions of the own vehicle and the other vehicles are subdivided, and the third cost is set to be larger as the distance to the object including the other vehicles is closer. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the third cost.

2. The vehicle behavior evaluation device according to claim 1, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

3. The vehicle behavior evaluation device according to claim 1, wherein: The first cost is set relatively large compared to other costs.

4. A vehicle behavior evaluation device for evaluating the future behavior of a vehicle, comprising: a setting unit for setting a plurality of actions that the vehicle can take when traveling on a predetermined course; and An evaluation unit includes a function device, the function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data, the driving condition-reward data being created to assign a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicles, and to assign a reward lower than the reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicles, the evaluation unit uses the function device to calculate a reward that takes into account the driving conditions of other vehicles around the own vehicle for the action of the own vehicle set by the setting unit, and evaluates the action of the own vehicle based on the calculated reward, The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A fourth cost is further calculated regarding the driving condition of the own vehicle in each of the multiple stages into which the driving conditions of the own vehicle and the other vehicle are subdivided. The fourth cost is set to be larger as the time required to avoid contact between the own vehicle and the other vehicle is longer. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the fourth cost.

5. The vehicle behavior evaluation device according to claim 4, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

6. The vehicle behavior evaluation device according to claim 4, wherein: The first cost is set relatively large compared to other costs.

7. A vehicle behavior evaluation device for evaluating the future behavior of a vehicle, comprising: a setting unit for setting a plurality of actions that the vehicle can take when traveling on a predetermined course; and An evaluation unit includes a function device, the function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data, the driving condition-reward data being created to assign a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicles, and to assign a reward lower than the reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicles, the evaluation unit uses the function device to calculate a reward that takes into account the driving conditions of other vehicles around the own vehicle for the action of the own vehicle set by the setting unit, and evaluates the action of the own vehicle based on the calculated reward, The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A fifth cost is further calculated with respect to the driving condition of the host vehicle in each of the multiple stages into which the driving conditions of the host vehicle and the other vehicles are subdivided. The longer the duration and / or distance of the deviation from the lane in which the host vehicle is to travel is, the greater the fifth cost is set. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the fifth cost.

8. The vehicle behavior evaluation device according to claim 7, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

9. The vehicle behavior evaluation device according to claim 7, wherein: The first cost is set relatively large compared to other costs.

10. A vehicle behavior evaluation device for evaluating the future behavior of a vehicle, comprising: a setting unit for setting a plurality of actions that the vehicle can take when traveling on a predetermined course; and An evaluation unit includes a function device, the function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data, the driving condition-reward data being created to assign a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicles, and to assign a reward lower than the reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicles, the evaluation unit uses the function device to calculate a reward that takes into account the driving conditions of other vehicles around the own vehicle for the action of the own vehicle set by the setting unit, and evaluates the action of the own vehicle based on the calculated reward, The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A sixth cost is further calculated regarding the driving condition of the own vehicle in each of the multiple stages into which the driving conditions of the own vehicle and the other vehicles are subdivided. The sixth cost is set to be larger as the acceleration of the own vehicle is greater. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the sixth cost.

11. The vehicle behavior evaluation device according to claim 10, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

12. The vehicle behavior evaluation device according to claim 10, wherein: The first cost is set relatively large compared to other costs.

13. A vehicle behavior evaluation method for evaluating the future behavior of a vehicle. The method is executed by at least one processor and comprises the following steps: A setting step for setting a plurality of actions that the vehicle can take when traveling on a predetermined runway; and An evaluation step of calculating a reward for the behavior of the host vehicle set in the setting step, taking into account the driving states of other vehicles around the host vehicle, using a function, and evaluating the behavior of the host vehicle based on the calculated reward, wherein: The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicle, and to give a lower reward than the above reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A third cost is further calculated for the driving condition of the own vehicle in each of the multiple stages into which the driving conditions of the own vehicle and the other vehicles are subdivided, and the third cost is set to be larger as the distance to the object including the other vehicles is closer. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the third cost.

14. The vehicle behavior evaluation method according to claim 13, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

15. The vehicle behavior evaluation method according to claim 13, wherein: The first cost is set relatively large compared to other costs.

16. A vehicle behavior evaluation method for evaluating the future behavior of a vehicle. The method is executed by at least one processor and comprises the following steps: A setting step for setting a plurality of actions that the vehicle can take when traveling on a predetermined runway; and An evaluation step of calculating a reward for the behavior of the host vehicle set in the setting step, taking into account the driving states of other vehicles around the host vehicle, using a function, and evaluating the behavior of the host vehicle based on the calculated reward, wherein: The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicle, and to give a lower reward than the above reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A fourth cost is further calculated regarding the driving condition of the own vehicle in each of the multiple stages into which the driving conditions of the own vehicle and the other vehicle are subdivided. The fourth cost is set to be larger as the time required to avoid contact between the own vehicle and the other vehicle is longer. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the fourth cost.

17. The vehicle behavior evaluation method according to claim 16, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

18. The vehicle behavior evaluation method according to claim 16, wherein: The first cost is set relatively large compared to other costs.

19. A vehicle behavior evaluation method for evaluating the future behavior of a vehicle. The method is executed by at least one processor and comprises the following steps: A setting step for setting a plurality of actions that the vehicle can take when traveling on a predetermined runway; and An evaluation step of calculating a reward for the behavior of the host vehicle set in the setting step, taking into account the driving states of other vehicles around the host vehicle, using a function, and evaluating the behavior of the host vehicle based on the calculated reward, wherein: The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicle, and to give a lower reward than the above reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A fifth cost is further calculated with respect to the driving condition of the host vehicle in each of the multiple stages into which the driving conditions of the host vehicle and the other vehicles are subdivided. The longer the duration and / or distance of the deviation from the lane in which the host vehicle is to travel is, the greater the fifth cost is set. The driving condition-remuneration data is created by calculating the remuneration in consideration of the fifth cost.

20. The vehicle behavior evaluation method according to claim 19, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

21. The vehicle behavior evaluation method according to claim 19, wherein: The first cost is set relatively large compared to other costs.

22. A vehicle behavior evaluation method for evaluating the future behavior of a vehicle. The method is executed by at least one processor and comprises the following steps: A setting step for setting a plurality of actions that the vehicle can take when traveling on a predetermined runway; and An evaluation step of calculating a reward for the behavior of the host vehicle set in the setting step, taking into account the driving states of other vehicles around the host vehicle, using a function, and evaluating the behavior of the host vehicle based on the calculated reward, wherein: The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicle, and to give a lower reward than the above reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A sixth cost is further calculated regarding the driving condition of the own vehicle in each of the multiple stages into which the driving conditions of the own vehicle and the other vehicles are subdivided. The sixth cost is set to be larger as the acceleration of the own vehicle is greater. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the sixth cost.

23. The vehicle behavior evaluation method according to claim 22, wherein: The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

24. The vehicle behavior evaluation method according to claim 22, wherein: The first cost is set relatively large compared to other costs.

25. A computer-readable non-transitory storage medium storing a vehicle behavior evaluation program, the program comprising instructions executed by at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform the following processing: Set multiple actions that the vehicle can take when driving on a predetermined track; By using a function, a reward is calculated for the set behavior of the above-mentioned vehicle taking into account the driving status of other vehicles around the above-mentioned vehicle, and the behavior of the above-mentioned vehicle is evaluated based on the calculated reward to evaluate the future behavior of the above-mentioned vehicle, wherein, The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicle, and to give a lower reward than the above reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. Regarding the driving conditions of the above-mentioned vehicle and the above-mentioned other vehicles in each stage divided into a plurality of stages, a third cost is further calculated, and the closer the distance to the object including the above-mentioned other vehicles, the larger the third cost is set. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the third cost.

26. The storage medium according to claim 25, wherein The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

27. The storage medium according to claim 25, wherein The first cost is set relatively large compared to other costs.

28. A computer-readable non-transitory storage medium storing a vehicle behavior evaluation program, the program comprising instructions executed by at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform the following processing: Set multiple actions that the vehicle can take when driving on a predetermined track; By using a function, a reward is calculated for the set behavior of the above-mentioned vehicle taking into account the driving status of other vehicles around the above-mentioned vehicle, and the behavior of the above-mentioned vehicle is evaluated based on the calculated reward to evaluate the future behavior of the above-mentioned vehicle, wherein, The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicle, and to give a lower reward than the above reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A fourth cost is further calculated regarding the driving condition of the own vehicle in each of the multiple stages into which the driving conditions of the own vehicle and the other vehicle are subdivided. The fourth cost is set to be larger as the time required to avoid contact between the own vehicle and the other vehicle is longer. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the fourth cost.

29. The storage medium according to claim 28, wherein The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

30. The storage medium according to claim 28, wherein The first cost is set relatively large compared to other costs.

31. A computer-readable non-transitory storage medium storing a vehicle behavior evaluation program, wherein the program includes instructions executed by at least one processor, wherein when executed by the at least one processor, the instructions cause the at least one processor to perform the following processing: Set multiple actions that the vehicle can take when driving on a predetermined track; By using a function, a reward is calculated for the set behavior of the above-mentioned vehicle taking into account the driving status of other vehicles around the above-mentioned vehicle, and the behavior of the above-mentioned vehicle is evaluated based on the calculated reward to evaluate the future behavior of the above-mentioned vehicle, wherein, The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicle, and to give a lower reward than the above reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A fifth cost is further calculated with respect to the driving condition of the host vehicle in each of the multiple stages into which the driving conditions of the host vehicle and the other vehicles are subdivided. The longer the duration and / or distance of the deviation from the lane in which the host vehicle is to travel is, the greater the fifth cost is set. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the fifth cost.

32. The storage medium according to claim 31, wherein The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

33. The storage medium according to claim 31, wherein The first cost is set relatively large compared to other costs.

34. A computer-readable non-transitory storage medium storing a vehicle behavior evaluation program, the program comprising instructions executed by at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform the following processing: Set multiple actions that the vehicle can take when driving on a predetermined track; By using a function, a reward is calculated for the set behavior of the above-mentioned vehicle taking into account the driving status of other vehicles around the above-mentioned vehicle, and the behavior of the above-mentioned vehicle is evaluated based on the calculated reward to evaluate the future behavior of the above-mentioned vehicle, wherein, The function device simulates a plurality of environments with different combinations of the surrounding environment of the own vehicle, the condition of the own vehicle, and the condition of other vehicles, and learns based on driving condition-reward data. The driving condition-reward data is created to give a higher reward in the condition of the own vehicle in which the own vehicle can avoid contact with the other vehicle, and to give a lower reward than the above reward in the condition of the own vehicle in which the own vehicle has already contacted the other vehicle. The driving condition-reward data is created as follows: a first cost and a second cost are calculated for the driving condition of the own vehicle in each stage into which the driving conditions of the own vehicle and the other vehicle are subdivided into a plurality of stages; the first cost is set as a penalty when the own vehicle comes into contact with the other vehicle; when the own vehicle comes into contact with the other vehicle, the second cost is set to a greater value the shorter the time until the contact is reached for the driving condition of the own vehicle in the stage before the contact; and the reward is calculated as the reciprocal of the sum of the first cost and the second cost. A sixth cost is further calculated regarding the driving condition of the own vehicle in each of the multiple stages into which the driving conditions of the own vehicle and the other vehicles are subdivided. The sixth cost is set to be larger as the acceleration of the own vehicle is greater. The driving condition-remuneration data is data created by calculating the remuneration in consideration of the sixth cost.

35. The storage medium according to claim 34, wherein The simulation includes different combinations of the behavior and speed of the host vehicle and the behavior and speed of the other vehicles in the plurality of environments.

36. The storage medium according to claim 34, wherein The first cost is set relatively large compared to other costs.

Images