AUTONOMOUS DRIVING EVALUATION DEVICE AND AUTONOMOUS DRIVING EVALUATION METHOD

Abstract

Autonomous driving evaluation device (100) for evaluating an autonomous driving algorithm by simulation, the device comprising an initial traffic scene setting unit (11) for setting an initial traffic scene involving an autonomous driving vehicle model and a moving object model by setting an initial state of the autonomous driving vehicle model in which autonomous driving is performed using the autonomous driving algorithm, an initial state of the moving object model, and a road environment in which the autonomous driving vehicle model and the moving object model are arranged, a past traffic scene calculation unit (12) for Calculating a past traffic scene candidate involving the autonomous driving vehicle model and the moving object model at a past time point traced back from a time point of the initial traffic scene based on the initial traffic scene, wherein the past traffic scene candidate is calculated by repeatedly calculating the states of the autonomous driving vehicle model and the states of the moving object model at the times that are times traced back from the time point of the initial traffic scene by a predetermined period of time, to determine whether both the state of the autonomous driving vehicle model and the state of the moving object model are a preset rule-compliant state, and to set the past traffic scene candidate as a past traffic scene based on both the state of the autonomous driving vehicle model and the state of the moving object model being a preset rule-compliant state, and a performance evaluation unit (15) for evaluating a performance of the autonomous driving algorithm based on the past traffic scene.

Description

TECHNICAL FIELDThe present disclosure relates to an autonomous driving evaluation device and an autonomous driving evaluation method that evaluate an autonomous driving algorithm.BACKGROUNDIn the related art, Japanese Unexamined Patent Publication JP 2017-105 453 A is known as a technical literature regarding evaluation of an autonomous driving function. In this literature, a method is disclosed in which a vehicle object for autonomous driving and another vehicle object are driven in an electronic game environment, and the function of autonomous driving is evaluated from a behavior of the vehicle object for autonomous driving according to the behavior of the other vehicle object.U.S. Pat. No. 9,463,797 B2 describes a method which assists the driving of an ego vehicle including a driver assistance system. A road user or an infrastructure element involved in the traffic situation is selected and taken into account for the analysis of the traffic scene. A hypothetical future trajectory for the ego vehicle is predicted by predicting and varying the current state of the ego vehicle to generate a plurality of ego trajectory alternatives, including the calculated hypothetical future ego trajectory. A hypothetical future trajectory of another road user, which is obtained by predicting the current state of the road user or calculating a hypothetical future state sequence of the infrastructure element, is determined. Based on at least one pair of ego trajectories plus another trajectory, risk functions are calculated over time or along the calculated hypothetical future ego trajectory alternatives. A risk function corresponds to an ego trajectory alternative. The risk functions are combined to form a risk map. A control signal is generated from an analysis result.DE 11 2007 002 946 B4 describes a vehicle control apparatus configured to calculate an input (x, δ) for changing a steering angle (δ) of a real vehicle based on a dynamic vehicle model modeling a moving state of a vehicle traveling according to a target path, judge whether the calculated input (x, δ) satisfies predetermined travel requirements of the real vehicle with respect to the target path, and calculate the steering angle (δ) based on the calculated input (x, δ) determined to satisfy the travel requirements, the travel requirements indicating a follow-up method of the real vehicle with respect to the target path.DE 10 2011 009 665 A1 describes a method for operating a vehicle during a traffic congestion condition, in which a collision risk assessment scheme uses information of the near future.DE 10 2010 013 402 A1 describes a graphical depiction on a front window in a motor vehicle. A prediction process module uses past object information and predicts current values.SUMMARYIncidentally, in the related art evaluation method, as mentioned above, an initial traffic scene in which the vehicle object for the autonomous driving is involved is set, and the function of the autonomous driving is evaluated from the behavior of the vehicle object by the autonomous driving algorithm after a certain time has elapsed since the initial traffic scene. As the initial traffic situation, for example, a situation in which the vehicle object for the autonomous driving and the other vehicle object may collide with each other when the two vehicle objects travel straight may be assumed. However, the autonomous driving algorithm is expected to control the vehicle object so that the initial traffic scene does not occur from the beginning, and therefore there is room for improvement in the evaluation of the autonomous driving algorithm.In the present technical field, therefore, it is desirable to provide an autonomous driving evaluation device or method that can appropriately evaluate the autonomous driving algorithm.In order to solve the above-described problems, according to an aspect of the present disclosure, there is provided an autonomous driving evaluation device for evaluating an autonomous driving algorithm through a simulation as specified in the appended claims.According to the autonomous driving evaluation device, the past traffic scene that is traced back from the initial traffic scene is calculated, and the performance of the autonomous driving algorithm is evaluated based on the past traffic scene.The autonomous driving evaluation device may further include: a past comparative traffic scene generation unit for generating a preset number of past comparative traffic scenes at the past time by minutely changing the past traffic scene; and an autonomous driving reflected scene calculation unit for respectively calculating an autonomous driving reflected scene after elapse of a preset time since the past traffic scene during a state in which the autonomous driving is performed on the autonomous driving vehicle model by the autonomous driving algorithm, and calculating an autonomous driving reflected comparative scene after elapse of a preset time since the past comparative traffic scene during a state, in which the autonomous driving is performed in the autonomous driving vehicle model by the autonomous driving algorithm. The performance evaluation unit may be configured to evaluate the performance of the autonomous driving algorithm based on the initial traffic scene, the past traffic scene, the past comparison traffic scene, the autonomous driving reflected scene, and the autonomous driving reflected comparison scene.According to the autonomous driving evaluation device described above, the autonomous driving reflected scene is calculated from the past traffic scene at the past time and the past comparison traffic scene is calculated from a plurality of past comparison traffic scenes at the past time, and then the autonomous driving algorithm is evaluated based on the autonomous driving reflected scene and the past comparison traffic scene. Therefore, according to the autonomous driving evaluation device described above, by reflecting the autonomous driving algorithm, it becomes possible to recognize the tendency of stable convergence or unstable divergence to the initial traffic scene from the past traffic scene and the past comparison traffic scene, and thus the performance of the autonomous driving algorithm against the initial traffic scene can be appropriately evaluated.In the autonomous driving evaluation device, the past traffic scene calculation unit is configured to repeatedly calculate the states of the autonomous driving traffic model and the states of the moving object model at the timings that are timings backward from the timing of the initial traffic scene by a predetermined period of time, and calculate the past traffic scene at a timing at which both the state of the autonomous driving vehicle model and the state of the moving object model are a rule-compliant state set in advance as the past timing.According to the above-described autonomous driving evaluation device, since the past traffic scene at the time is calculated as the past time point when both the state of the autonomous driving vehicle model and the state of the moving object model become the rule-compliant state by moving back in time from the time point of the initial traffic scene by a predetermined period of time, application of the past traffic scene of the rule-defective state that is unsuitable for the precondition of the evaluation of the autonomous driving algorithm can be avoided, and thus the autonomous driving algorithm can be appropriately evaluated.The autonomous driving evaluation device may be further configured to further include: a future state calculation unit for calculating a future state of the autonomous driving vehicle model when the autonomous driving is performed using the autonomous driving algorithm from the time point of the initial traffic scene and a future state of the moving object model that performs a preset movement from the time point of the initial traffic scene based on the initial state of the autonomous driving vehicle model, the initial state of the moving object model, and the road environment; and a ratio determination unit for determining whether the autonomous driving vehicle model and the moving object model have a low evaluation ratio based on the future state of the autonomous driving vehicle model and the future state of the moving object model. The performance evaluation unit may be configured to evaluate the performance of the autonomous driving algorithm based on the result of the determination performed by the ratio determination unit.According to the autonomous driving evaluation device described above, by calculating the future state of the autonomous driving vehicle model and the future state of the moving object model in performing the autonomous driving using the autonomous driving algorithm from the time of the initial traffic scene, it can be determined whether or not the two models have the low evaluation ratio when the time elapses since the initial traffic scene while the autonomous driving algorithm is reflected. Therefore, the autonomous driving algorithm can be appropriately evaluated.According to another aspect of the present disclosure, an autonomous driving evaluation method is provided in an autonomous driving evaluation device for evaluating an autonomous driving algorithm through a simulation as specified in the appended claims.According to the autonomous driving evaluation method, the past traffic scene is calculated by calculating the past state of the autonomous driving vehicle model at the past time back from the initial traffic scene and the past state of the moving object model at the past time, and the performance of the autonomous driving algorithm is evaluated based on the past traffic scene. Therefore, the autonomous driving algorithm can be appropriately evaluated compared with a case where the past traffic scene is not taken into account.The autonomous driving evaluation method may further include: generating a preset number of past comparative traffic scenes at the past time by minutely changing the past traffic scene; calculating an autonomous driving reflected scene after elapse of a preset time since the past traffic scene during a state in which the autonomous driving is performed on the autonomous driving vehicle model by the autonomous driving algorithm; calculating an autonomous driving reflected comparative scene after elapse of a preset time since the past comparative traffic scene during a state in which the autonomous driving is performed on the autonomous driving vehicle model by the autonomous driving algorithm; and evaluating the performance of the autonomous driving algorithm based on the initial traffic scene, the past traffic scene, the past comparison traffic scene, the autonomous driving reflected scene, and the autonomous driving reflected comparison scene.According to the above-described autonomous driving evaluation method, the autonomous driving reflected scene is calculated from the past traffic scene at the past time, and the past comparison traffic scene is calculated from a plurality of past comparison traffic scenes at the past time, and then the autonomous driving algorithm is evaluated based on the autonomous driving reflected scene and the past comparison traffic scene. Therefore, according to the autonomous driving evaluation method described above, by reflecting the autonomous driving algorithm, it becomes possible to recognize the tendency of stable convergence or unstable divergence to the initial traffic scene from the past traffic scene and the past comparative traffic scene, and thus the performance of the autonomous driving algorithm can be appropriately evaluated that the situation is not caused to reach the initial traffic scene.In the autonomous driving evaluation method, in the past traffic scene calculation step, the states of the autonomous driving vehicle model and the states of the moving object model at the timings that are the timings backward from a timing of the initial traffic scene by a predetermined period of time are repeatedly calculated, and the past traffic scene is calculated at a timing at which both the state of the autonomous driving vehicle model and the state of the moving object model are a rule-compliant state set in advance as the past timing.According to the above-described autonomous driving evaluation method, since the past traffic scene at the time when both the state of the autonomous driving vehicle model and the state of the moving object model become the rule-compliant state is calculated as the past time by being retarded from the time of the initial traffic scene by a predetermined time period, application of the past traffic scene of the rule-poor state that is unsuitable for the precondition of the evaluation of the autonomous driving algorithm can be avoided, and thus the autonomous driving algorithm can be appropriately evaluated.The autonomous driving evaluation method may further include: calculating a future state of the autonomous driving vehicle model when the autonomous driving is performed using the autonomous driving algorithm from the time of the initial traffic scene and a future state of the moving object model that performs a preset movement since the time of the initial traffic scene, based on the initial state of the autonomous driving vehicle model, the initial state of the moving object model, and the road environment, and determining whether the autonomous driving vehicle model and the moving object model have a low evaluation ratio set in advance, based on the future state of the autonomous driving vehicle model and the future state of the moving object model, wherein, in the performance evaluating step, the performance of the autonomous driving algorithm may be evaluated based on the result of the determination performed by the ratio determining step.According to the autonomous driving evaluation method as described above, by calculating the future state of the autonomous driving vehicle model and the future state of the moving object model in performing the autonomous driving using the autonomous driving algorithm, it can be determined from the time of the initial traffic scene whether or not the two models have the low evaluation ratio when the time has elapsed since the initial traffic scene while the autonomous driving algorithm is reflected. Therefore, the autonomous driving algorithm can be appropriately evaluated.As described above, according to the autonomous driving evaluation device of the embodiment of the present disclosure or the autonomous driving evaluation method of the other embodiment, the autonomous driving algorithm can be appropriately evaluated.BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a block diagram of an autonomous driving evaluation device according to a first embodiment. FIG. 2 is a diagram illustrating a hardware configuration of the autonomous driving evaluation device. FIG. 3 is a diagram showing an example of an initial traffic scene. FIG. 4A is a diagram showing an image when a timing to the initial traffic scene is stable. FIG. 4B is a diagram showing an image when a timing to the initial traffic scene is unstable. FIG. 5 is a diagram showing a specific example case when a timing of a past traffic scene and a past comparison traffic scene is unstable. FIG. 6 is a flowchart showing initial traffic scene setting processing and backward calculation processing. FIG. 7 is a flowchart of performance evaluation processing based on dynamic stability. FIG. 8 is a block diagram of an autonomous driving evaluation device according to a second embodiment. FIG. 9 is a table showing an example of performance evaluation according to the second embodiment FIG. 10 is a flowchart of feedforward processing.DETAILED DESCRIPTIONHereinafter, embodiments of the present disclosure will be described with reference to the drawings.First EmbodimentFIG. 1 is a block diagram of an autonomous driving evaluation device according to a first embodiment. The autonomous driving evaluation device 100 illustrated in FIG. 1 is a device for evaluating an autonomous driving algorithm through a simulation. The autonomous driving algorithm is an algorithm for performing the autonomous driving by controlling a vehicle capable of autonomous driving. The autonomous driving algorithm may be a stand-alone infracooperative algorithm. The autonomous driving is a vehicle controller that causes a vehicle to autonomously drive even when the driver of the vehicle does not perform a driving operation.The evaluation of the autonomous driving algorithm is performed by a simulation using an autonomous driving vehicle model in which the autonomous driving is reflected by the autonomous driving algorithm and a moving object model. The autonomous driving vehicle model is a model for mimicking a vehicle in the simulation, capable of autonomous driving to cause the autonomous driving to be reflected by the autonomous driving algorithm.The moving object model is a model for mimicking a moving object in the simulation for evaluating the autonomous driving algorithm. The moving objects include other vehicles, pedestrians, bicycles, animals, robots, mobility of persons, and the like. Behaviors of the models differ from each other depending on the types of the objects. The vehicle model has, for example, a higher upper speed limit and a lower upper direction angle speed limit (orientation change speed) than the pedestrian model. The behavior scope of the models is suitably set.In the related art, as evaluation of an autonomous driving algorithm, a method has been studied in which a time passes from a traffic scene in which the autonomous driving vehicle model and the moving object model interfere with each other, and then a function of an autonomous driving algorithm is evaluated based on the behavior of the autonomous driving vehicle model in which the autonomous driving algorithm is reflected. The traffic scene in which the autonomous driving vehicle model and the moving object model interfere with each other is, for example, a traffic scene in which the autonomous driving vehicle model and the moving object model may collide with each other when the two models approach each other.However, the autonomous driving algorithm is designed such that the autonomous driving vehicle model and the moving object model do not come into a problematic traffic scene (an extreme condition) like a scene in which the two models interfere with each other from the beginning. From the viewpoint of evaluating performance, therefore, it is desirable not to evaluate the behavior of the autonomous driving algorithm when the time passes since the problematic traffic scene, but to evaluate whether or not the autonomous driving algorithm generates a problematic traffic scene and whether or not the autonomous driving algorithm causes the situation to enter a problematic traffic scene.In the autonomous driving evaluation device 100, the performance of the autonomous driving algorithm is evaluated by checking whether or not the autonomous driving algorithm makes the situation enter a traffic scene to be evaluated (an initial traffic scene described below) through an appropriate simulation using the autonomous driving vehicle model and the moving object model.Configuration of Autonomous Driving Evaluation Device according to First EmbodimentA configuration of the autonomous driving evaluation device 100 according to the first embodiment will now be described. First, a hardware configuration of the autonomous driving evaluation device 100 will be described. FIG. 2 is a diagram showing the hardware configuration of the autonomous driving evaluation device 100.As shown in FIG. 2, the autonomous driving evaluation apparatus 100 may be physically configured as a computer including one or a plurality of CPUs ( 101), a random access memory (RAM) 102, and a read only memory (ROM) 103, input devices 104 such as a keyboard and a mouse, a storage device 105 such as a semiconductor memory, a communication module 106 representing a communication and reception device such as a network card, and an output device 107 such as a display device.The CPU 101 realizes various operations by loading various programs or the like required for executing the processing from the ROM 103 or the storage device 105 into the RAM 102 and executing the loaded programs. The program and data required for the processing may be input via the communication module 106. The autonomous driving evaluation device 100 may be configured with a plurality of computers.Next, a functional configuration of the autonomous driving evaluation device 100 will be described with reference to FIG. 1. As shown in FIG. 1, the autonomous driving evaluation device 100 includes an initial traffic scene setting unit 11, a past traffic scene calculation unit 12, a past comparative traffic scene generation unit 13, an autonomous driving reflected scene calculation unit 14, and a performance evaluation unit 15.The initial traffic scene setting unit 11 sets an initial traffic scene for evaluating the autonomous driving algorithm. The initial traffic scene is a traffic scene used as a reference for the evaluation of the autonomous driving algorithm by the simulation. The initial traffic scene includes an initial state of the autonomous driving vehicle model, an initial state of the moving object model, and a road environment in which the autonomous driving vehicle model and the moving object model are arranged.The initial state of the autonomous driving vehicle model is a state of the autonomous driving vehicle model in the initial traffic scene. The autonomous driving algorithm is assumed to execute the autonomous driving vehicle mode in accordance with rules because the algorithm has been generated in accordance with rules. The state of the autonomous driving vehicle model includes the position, orientation, and speed of the autonomous driving vehicle model. The state of the autonomous driving vehicle model subjected to the algorithm evaluation does not need to include yaw rate and acceleration (or deceleration). However, configuration including the yaw rate and the acceleration (or the deceleration) in the state of the autonomous driving vehicle model is not excluded.Similarly, the initial state of the moving object model is a state of the moving object model in the initial traffic scene. The state of the moving object model includes the position, orientation, and velocity of the moving object model. In the state of the moving object model, the yaw rate may be added, and the acceleration (or the deceleration) may be added. In the moving object model, a moving object algorithm that determines the behavior of the moving object model may be set in advance. Alternatively, dynamic motion may be directly present before and after the initial state of the moving object model. The moving object algorithm may be, for example, an algorithm in which the moving object maintains a movement at the current speed in the current orientation. The moving object algorithm may be set in advance for each model type (vehicle, pedestrian, and the like).The road environment in the simulation is a road environment in which the autonomous driving vehicle model and the moving object model are arranged. The road environment may be any environment in which the vehicle may travel and thus may be in a parking lot. The road environment may be, for example, the environment of a lane in which the autonomous driving vehicle model travels and the environment of the lane in which the moving object is moving. The vicinity of the lane includes a shape of the lane, curvature of the lane, and width of the lane. In the road environment, information on an allowed speed limit and information on a direction of the driving lane may be set in advance. The direction of the lane may be determined as either left-hand traffic or right-hand traffic, instead of being indicated for each lane. The road environment may also be the status of traffic signals on the lane.FIG. 3 is a diagram showing an example of an initial traffic scene. FIG. 3 illustrates an autonomous driving vehicle model M, a moving object model (another vehicle model) N, a lane R 1 on which the autonomous driving vehicle model M travels, an opposite lane R 2 adjacent to the lane R 1, a center line C between the lane R 1 and the opposite lane R 2. In the initial traffic scene illustrated in FIG. 3, the autonomous driving vehicle model M travels in the lane R 1. On the other hand, the moving object model N attempts to advance over the opposite lane R 2 and the center line C in front of the autonomous driving vehicle model M. The autonomous driving evaluation device 100 performs the evaluation of the autonomous driving algorithm using such an initial traffic scene.In a single initial traffic scene, there may be a plurality of moving object models, or there may be a plurality of autonomous driving vehicle models. The initial traffic scene may not be a problematic traffic scene as shown in FIG. 3, and the initial traffic scene may not be a traffic scene immediately before colliding the autonomous driving vehicle model and the moving object model with each other. The initial traffic scene may be a normal traffic scene in which both the autonomous driving vehicle model and the moving object model are traveling on the roadway at a speed within the speed limit.The past traffic scene calculation unit 12 calculates a past traffic scene that is a traffic scene that is temporally backward from the initial traffic scene. The past traffic scene calculation unit 12 extracts a time derivative component of the initial traffic scene (such as the speed of the autonomous driving vehicle model) and calculates the past traffic scene described as a dynamic system using backward calculation. A time point of the past traffic scene is referred to as a past time point.The past traffic scene calculation unit 12 calculates the past traffic scene including a state of the autonomous driving vehicle model at the past time and a state of the moving object model at the past time. The calculation of the past traffic scene is performed as a simulation of a dynamic system in which the time-back tracking is performed in an infinite time using the time derivative component of the initial traffic scene.An example of the backward calculation will be described below. First, assuming that the state (dynamic state) of the autonomous driving vehicle model is represented by ξa and a time-series operator is represented by ß, the temporal change of the state of the autonomous driving vehicle model is expressed as the following equation (1). Here, t represents time, and δt represents the infinite time. Formula 1In addition, the state (dynamic state) of the moving object model is represented by ξb, and the timing operator is represented as F^b. Here, it is assumed that the number of moving object models is a plurality, and therefore each moving object model is represented using b=1, 2, 3,... In this case, the temporal change of the states of the moving object models is expressed by the following equation (2). Formula 2The above equations (1) and (2) can be expressed as the following equations (3) and (4) in total. Formula 3Here, the value E(t) expressed as the equation (5) is referred to as E(t) as the traffic scene (corresponding to the initial traffic scene) at the time t, and T^(δt, A, F) is referred to as the time-out operator in the traffic scene. Formula 4Based on the assumptions described above, in order to obtain the past traffic scene E(t-δt), which is a result of a time-back by δt from the traffic scene E(t) at time t, it is necessary to obtain an inverse operator of a time inversion operator, i.e., a time-back operator T^ -1( δt, A, F). A recursive method for obtaining the past traffic scene E(t-δt), which is a result of a temporal traceback by δt, will now be described.First, a state (a dynamic state) of a vehicle model constituting a traffic scene is assumed. An example of the dynamic state ξ of the vehicle model includes a position x, a velocity v, an orientation φ, and an angular velocity ω of the orientation. These can be expressed as the following equation (6). Formula 5Here, ξ(t-δt) can be approximated by expanding the equation as expressed in the following equations (7) to (9). Formula 6Here, T̂-1(δt) is a linear temporal traceback operator.The dynamic temporal trace using the linear temporal trace operator that does not have an interaction between the autonomous driving vehicle model and the moving object model may be considered as Equation (10). In addition, the timing that takes into account the interaction between the autonomous driving vehicle model and the moving object model including the effects of the autonomous driving algorithm can be regarded as Equation (11). In Equation (11), e(t) represents a traffic scene (a restored traffic scene described below) when the time passes from the past traffic scene E(t-δt) to the time t of the initial traffic scene while considering the interaction between the autonomous driving vehicle model and the moving object model. Formula 7In this case, when the autonomous driving algorithm A to be evaluated does not attempt to cause the autonomous driving vehicle model to avoid the moving object model (when there is no interaction), it is expected that the following equation (12) will be obtained. Formula 8On the other hand, when the autonomous driving algorithm A tries to cause the autonomous driving vehicle model to avoid the moving object model (when there is an interaction), it is expected that the following equation (13) is obtained. Formula 9However, even in this case, E(t-δt) is expected to be present, which satisfies Equation (14) below. Formula 10E(t-δt) in this case can be recursively obtained using the following equations (16) to (18) when the initial state is determined as illustrated in the following equation (15). Here, α is a free parameter that determines the performance of convergence calculation. Formula 11Specifically, in Equations (19), when an error ΔE i+1( t) at time t is within a preset tolerance range (for example, less than or equal to a predetermined very small size), a past traffic scene candidate E i+1( t-δt) may be the past traffic scene E(t-δt) obtained by the past traffic scene calculation unit 12. Formula 12The past traffic scene calculation unit 12 may calculate the past traffic scene E(t-δt) by performing the above-described backward calculation.The past traffic scene calculation unit 12 may set the past timing as a timing when both the state of the autonomous driving vehicle model and the state of the moving object model are the states of a rule-compliant state or the normal driving state set in advance. Hereinafter, the rule-compliant state and the normal driving state are collectively referred to as a rule-compliant state when it is not necessary to distinguish between a rule-compliant state and the normal driving state. That is, the past traffic scene calculation unit 12 may repeat the calculation of the past traffic scene (the past traffic scene candidate) for each time trace back by a predetermined period of time from the time of the initial traffic scene, and may obtain a past traffic scene candidate in which both the state of the autonomous driving vehicle model and the state of the moving object model are the rule-compliant state within the allowable error range ΔE i+1( t). The predetermined time is a time set in advance.The proper state is a state in which each moving object model (including the autonomous driving vehicle model) is in accordance with the traffic rules and laws set in advance. Further, it is also conceivable to add a condition that each moving object model travels along the lane and in an area where the lanes are smoothly connected to each other at the intersection.The normal driving state is a state in which each moving object model rather complies with the general traffic rules and laws, and there is no possibility for a risky approach. A specific example of a normal traveling state is that each moving object model travels along the lane or in an area where the lanes are smoothly connected to each other at the intersection, many moving object models do not travel in parallel on the same lane or in the above-described area, and the speed of each moving object model is roughly equal to or less than the speed limit (limited speed). A speed limit roughly equal to or lower than the speed limit means a speed at which risk avoidance can be expected even when the speed of another moving object model exceeds the speed limit to ensure robustness of performance when performance evaluation of the autonomous driving algorithm is performed. The speed limit roughly less than or equal to the speed limit is, for example, a speed by 20% or by 15 km / h above the speed limit.The proper state and the normal driving state generally do not have a relationship mutually inclusive. In particular, if each moving object model exceeds the speed limit, the state is not a rule-compliant state. On the other hand, when a bicycle and a car are running on the same lane at a speed equal to or lower than the speed limit, the state when the car attempts to pass the bicycle running on the left side of the same lane while the car is running on the same lane is a regulation compliant state but is not a normal running state.The determination that the state is a rule-compliant state may include a determination as to whether or not the state is a rule-compliant state in a narrow sense, the determination as to whether or not the state is a normal driving state, a logical product of both, or a logical sum of both.As an example, the rule-compliant state of an autonomous driving vehicle model may be a state in which the autonomous driving vehicle model travels on the driving lane in the direction of the driving lane at a speed within the speed limit set for the driving lane on which the autonomous driving vehicle model travels. Similarly, when the moving object model is another vehicle model, the rule-conforming state may be a state in which the moving object travels on the lane along the traveling direction of the lane at a speed within the speed limit set in advance for the lane on which the moving object model travels. When the moving object model is a pedestrian, the rule-conforming state may be a state in which the moving object model moves over a pedestrian area set outside the lane or crosswalk. In addition, the past traffic scene calculation unit 12 may calculate the past traffic scene in consideration of the road environment. Another possible past time is a time when the state of the autonomous driving vehicle model deviates from the rule-compliant state (or a time a predetermined time before the above time). In this case, the autonomous driving algorithm operates such that the situation does not become the initial traffic scene by complying with the law.The past comparison traffic scene generation unit 13 generates a predetermined number of past comparison traffic scenes by minutely changing the past traffic scene. The minute change of the past traffic scene means, for example, a minute change of the state (parameter) of the moving object model at the past time or a minute change of the state (parameter) of the autonomous driving vehicle model at the past time. The past comparison traffic scene generation unit 13 may minutely change the state of each of the autonomous driving vehicle model and the moving object model, or may minutely change the state of only one of the autonomous driving vehicle model and the moving object model. The predetermined number may be a dimension number of the past traffic scene, which will be described below.The minute change of the state of the autonomous driving vehicle model at the past time is performed by adding a very small amount to the parameters such as the position, speed, and the like included in the state of the autonomous driving vehicle model. A preset value may be used as the very small amount. The very small amount may be a random value close to 0 (e.g., a random value less than 0.3). The minute change can be performed not only by adding the very small amount but also by subtracting the very small amount, by multiplying or dividing by / by a coefficient that can be a very small amount, or by using a calculation equation set in advance. The method of minute change is not limited to the above-described method.Here, an example of generating the past comparative traffic scene will be described. First, it is assumed that the past traffic scene E(t-kδt) is represented by an M-dimensional vector. Here, k is an arbitrary coefficient. In this case, following equations (20) to (22) can be considered using M-dimensional orthonormal vectors around (m=1, 2,..., M). Pm (m=1, 2,..., M) represents the past traffic scene, δP represents the very small amount near 0, and ΔPm represents a difference between the past traffic scene E(t-kδt) and the comparison past traffic scene Pm. Formula 13The past comparative traffic scene generation unit 13 may generate the M past comparative traffic scenes Pm set in advance using the above-described equations (20) to (22).The autonomous driving reflected scene calculation unit 14 calculates an autonomous driving reflected scene based on the past traffic scene, the road environment, and the autonomous driving algorithm. The autonomous driving reflected scene is a traffic scene in which a set time has elapsed since the past traffic scene in a state in which the autonomous driving is performed in the autonomous driving vehicle model by the autonomous driving algorithm. The set time is a preset time. As the set time, a very small time may be set. In addition, the set time may be a time for step calculation of the past traffic scene calculation unit 12 or a time equal to the time from the past time to the time of the initial traffic scene.The moving object model also moves with the moving object algorithm set in advance. The moving object model does not necessarily have to move, but may be in a stationary state. The moving object model may move such that the state changes from the state of the moving object model in the past traffic scene to the initial state of the moving object model in the initial traffic scene. The moving object algorithm may change the behavior of the moving object model in consideration of the interaction with the autonomous driving vehicle model. In this case, the autonomous driving algorithm performs the autonomous driving of the autonomous driving vehicle model based on the influence of the movement of the moving object model.In addition, the autonomous driving reflected scene calculation unit 14 calculates a comparative autonomous driving reflected scene based on the past comparative traffic scene, the road environment, and the autonomous driving algorithm. The comparison autonomous driving reflected scene is a traffic scene after elapse of a set time in a state where the autonomous driving is performed in the autonomous driving vehicle model by the autonomous driving algorithm from the past comparison traffic scene. The autonomous driving reflected scene calculation unit 14 calculates the comparison autonomous driving reflected scene for each comparison traffic scene. The autonomous driving reflected scene calculation unit 14 calculates the comparison autonomous driving reflected scene using a similar calculation as the calculation of the autonomous driving reflected scene.An example of calculation of the autonomous driving reflected scene and the comparison autonomous driving reflected scene will be described below. The autonomous driving reflected scene calculation unit 14 may express the autonomous driving reflected scene and the autonomous driving reflected comparative scene as Equations (23) and (24) below, which may be obtained from the past traffic scene E(t-kδt) and the past comparative traffic scene Pm for a set time δt after passage of time by the set time δt. Here, Qm (m=1, 2,..., M) is the autonomous driving reflected scene or the comparison autonomous driving reflected scene. Formula 14Here, ΔQm represents an amount of change of the past traffic scene E(t-kδt) described in Equations (21) and (22) per one step elapse of time. The change described as linear transformation using matrix H is expressed as the following equation. Formula 15The stability of the time reversal of the past traffic scene can be evaluated by analyzing the characteristics of this linear transformation as described below. Specifically, the evaluation may be performed based on whether or not the maximum eigenvalue of the matrix H is greater than 1. The time reversal can be evaluated as stable when the maximum eigenvalue is greater than 1 and unstable when it is less than or equal to 1.The performance evaluation unit 15 evaluates the performance of the autonomous driving algorithm based on the past traffic scene. Specifically, the performance evaluation unit 15 evaluates the performance of the autonomous driving algorithm by evaluating whether the autonomous driving algorithm stably converges the situation to the initial traffic scene from the past traffic scene and the past comparative traffic scene via reflecting the autonomous driving algorithm, or unstably diverges based on the past traffic scene, the autonomous driving reflected scene, the past comparative traffic scene, and the autonomous driving reflected comparative scene.Here, FIG. 4A illustrates an image when the timing to the initial traffic scene is stable. In FIG. 4A, arrows from the past indicate a situation of the past traffic scene and the past comparison traffic scene when the time is going toward the future. The arrows from the initial traffic scene to the future indicate the situation after the time passes from the initial traffic scene. The timing from the initial traffic scene will be described in a second embodiment.As shown in FIG. 4A, the arrows converge from the past to the initial traffic scene when the timing to the initial traffic scene is stable. That is, the past comparison traffic scene obtained by minutely changing the past traffic scene also reaches the initial traffic scene with the lapse of time. This shows that it is difficult to prevent the situation from reaching the initial traffic scene even when the autonomous driving is reflected by the autonomous driving algorithm in the autonomous driving vehicle model. In this case, it can be judged that the autonomous driving algorithm does not have performance for preventing the situation from reaching the initial traffic scene from the past traffic scene and the past comparison traffic scene. Here, the evaluation will be described with reference to an example case where the initial traffic scene is a problematic traffic scene.FIG. 4B illustrates an image when the timing to the initial traffic scene is unstable. As shown in FIG. 4B, if the timing to the initial traffic scene is unstable, arrows diverge from the past without converging at the initial traffic scene. That is, the past comparison traffic scene becomes different states when the time passes without reaching the initial state in the initial traffic scene. This shows that the situation of reaching the initial traffic scene from the past comparative traffic scenes can be likely prevented when the autonomous driving is reflected by the autonomous driving algorithm in the autonomous driving vehicle model. In this case, it may be judged that the autonomous driving algorithm has performance for preventing the situation from reaching the initial traffic scene. Even if a part of the arrows shown in FIG. 4B reach the initial traffic scene, the timing may be referred to as unstable when the remaining arrows do not reach the initial traffic scene.In order to make FIGS. 4A and 4B easily understood, the convergence of the arrows (stability) and the divergence of the arrows (unstable) to the initial traffic scene have been described, but the stability can be evaluated without elapse of time from the past time to the initial traffic scene. That is, when a part of the time period is viewed from the entire time period from the past time point to the time point of the initial traffic scene, when the arrows diverge without converging at the initial traffic scene, it can be considered that the divergence always occurs with the timings even when any part of the time period is selected.FIG. 5 is a diagram showing a specific example case in which the timing of the past traffic scene and the past comparison traffic scene is unstable. FIG. 5 illustrates past traffic scene E(t-kδt), a first past comparison traffic scene P 1( t-kδt), and a second past comparison traffic scene P 2( t-köt) at the past time.The first past comparison traffic scene P 1( t-kδt) is a traffic scene obtained by minutely changing the past traffic scene E(t-kδt) by δp·u 1. Similarly, the second past comparison traffic scene P 2( t-kδt) is a traffic scene obtained by minutely changing the past traffic scene E(t-kδt) by δp·u 2. The length of the arrows corresponds to the difference in the traffic scenes.In addition, FIG. 5 illustrates the autonomous driving reflected scene E(t-(k-1)δt), the first autonomous driving reflected comparison scene Q 1( t-(k-1)δt), and the second autonomous driving reflected comparison scene Q 2( t-(k-1)δt) at the time t-kδt, which is a time elapsed from the past time t-(k-1)δt by δt. The set time does not necessarily have to be δt.The first comparison autonomous driving reflected scene Q 1( t-(k-1)δt) is a traffic scene obtained from the first comparison past traffic scene P 1( t-kδt) after the time passes by δt during the state of reflecting the autonomous driving by the autonomous driving algorithm. Similarly, the second comparison autonomous driving reflected scene Q 2( t-(k-1)δt) is a traffic scene obtained from the second comparison past traffic scene P 2( t-kδt) after the lapse of time by δt during the state of reflecting the autonomous driving by the autonomous driving algorithm.In the situation in FIG. 5, a difference between the autonomous driving reflected scene E(t-(k-1)δt) and the first comparative autonomous driving reflected scene Q 1( t-(k-1)δt) after the lapse of time is larger than δp·u 1, which represents a difference between the first past comparative traffic scene E(t-kδt) and the first past comparative traffic scene P 1( t-kδt). In this case, since the difference between the autonomous driving reflected scene E(t-(k-1)δt) and the first comparative autonomous driving reflected scene Q 1( t-(k-1)δt) increases with the lapse of time, it can be judged to be instable diverging without converging at the initial traffic scene. That is, the autonomous driving algorithm may be evaluated to have a capability to prevent the situation from reaching the initial traffic scene.In this way, the performance evaluation unit 15 may evaluate the performance of the autonomous driving algorithm based on the past traffic scene, the autonomous driving reflected scene, the past comparative traffic scene, and the autonomous driving reflected comparative scene.Specifically, the performance evaluation unit 15 calculates the transformation matrix H(k) in the above-described equation (26), and can evaluate the stability of the autonomous driving algorithm using the transformation matrix H(k). When the maximum eigenvalue λmax(k) among the eigenvalues of the transformation matrix H(k) is larger than 1, even if the two traffic scenes (for example, the past traffic scene and the comparison past traffic scene) are very similar, it is known that the difference between traffic scenes becomes larger (unstable) with the lapse of time. It is also known as the main element of the time sequence.If backward temporal calculation is possible, the transformation matrix H(k) and the maximum eigenvalue λmax(k) of the time series along the time axis E(t-nδt)→E(t-(n-1)δt)→E(t-(n-2)δt)→...→E(t-δt)→E(t) can be calculated.The columns λmax(k) and k of the maximum eigenvalues obtained in this manner are expressed in the following equation (27). Formula 16Equation (27) described above represents the stability of the traffic scene along the passage of time. When λmax(k) is larger than 1, the difference between two traffic scenes (for example, the autonomous driving reflected scene and the autonomous driving reflected comparison scene) becomes large with the lapse of time, as can be seen from the calculations thus far. That is, the situation becomes unstable. In this case, it is seen that the product of λmax(k) expressed in the following equation (28) geometrically increases. Formula 17That is, even if there is a past traffic scene that can reach the initial traffic scene E(t), this means that if there is actually a very small change in the past traffic scene (if there is a small difference), the past traffic scene does not reach the initial traffic scene E(t). That is, it can be judged that in the autonomous driving algorithm, the situation does not enter the initial traffic scene E(t), and more specifically, the probability of entering the initial traffic scene E(t) asymptotically approaches zero at the infinite time δt=0.On the other hand, when λmax(k) is less than 1, the difference between the two traffic scenes after the lapse of time goes to zero extremely quickly with the lapse of time, and thus it can be seen that the situation becomes stable with the lapse of time. When λmax(k)=1, the situation can be considered to be stable.In this way, the performance evaluation unit 15 may evaluate whether the autonomous driving algorithm makes the situation stably converge to the initial traffic situation or diverge in an unstable manner. The performance evaluation unit 15 may evaluate that the autonomous driving algorithm has the performance of preventing the situation from reaching the initial traffic scene when the autonomous driving algorithm makes the situation to diverge toward the initial traffic scene in an unstable manner.Autonomous Driving Evaluation Method Using the Autonomous Driving Evaluation Device According to the First EmbodimentThe autonomous driving evaluation method using the autonomous driving evaluation device 100 according to the first embodiment will be described below. FIG. 6 is a flowchart showing initial traffic scene setting processing and backward calculation processing.As shown in FIG. 6, the autonomous driving evaluation device 100 sets an initial traffic scene using the initial traffic scene setting unit 11 as S 10 (initial traffic scene setting step). The initial traffic scene setting unit 11 sets the traffic scene as an initial traffic scene to be used for the performance evaluation of the autonomous driving algorithm.In S 12, the autonomous driving evaluation device 100 extracts the time derivative component of the initial traffic scene (such as the speed of the autonomous driving vehicle model and the speed of the moving object model) using the past traffic scene calculation unit 12.In S 14, the autonomous driving evaluation device 100 calculates a past traffic scene candidate before a predetermined time using the past traffic scene calculation unit 12. The past traffic scene calculation unit 12 calculates the past traffic scene candidate using, for example, the above-described equation (10).In S 16, the autonomous driving evaluation device 100 calculates the restored traffic scene using the past traffic scene calculation unit 12. the past traffic scene calculation unit 12 calculates the restored traffic scene from the past traffic scene candidate by considering the timing of the interaction between the autonomous driving vehicle model and the moving object model including the effects of the autonomous driving algorithm. The past traffic scene calculation unit 12 calculates the restored traffic scene using, for example, the above-described equation (11).In S 18, the autonomous driving evaluation device 100 calculates the difference between the initial traffic scene and the restored traffic scene using the past traffic scene calculation unit 12.In S 20, the autonomous driving evaluation device 100 determines whether or not the difference between the initial traffic scene and the restored traffic scene is within an allowable range using the past traffic scene calculation unit 12. The allowable range is a preset range. The allowable range includes, for example, an allowable speed threshold, an allowable distance threshold, and an allowable orientation threshold.The past traffic scene calculation unit 12 determines that the difference between the initial traffic scene and the restored traffic scene is within the allowable range, for example, when only the state of the autonomous driving vehicle model has the difference in the initial traffic scene candidate and the restored traffic scene, and when absolute values of the difference in speed, the difference in position (distance), and the difference in orientation of the autonomous driving vehicle model in the initial traffic scene candidate and the restored traffic scene are respectively equal to or smaller than the allowable speed threshold, the allowable distance threshold, and the allowable direction threshold. Even if only the moving object model has the difference, it is possible to similarly perform the determination using the threshold values. Components other than the speed, position, and orientation may be used for the above-described determination.If the timing of the past traffic scene is unstable, the difference between the initial traffic scene candidate and the restored traffic scene obtained from the past traffic scene by the simulation does not converge (diverges), and thus the above-described end condition may be unsuitable. In such a case, by using the fact that the error described in Equation (17) for each step is less than or equal to a predetermined value and the fact that the trace-back time kδt becomes a predetermined time, the end condition may be assumed to be satisfied.When the autonomous driving evaluation device 100 determines that the difference between the initial traffic scene and the restored traffic scene is within the allowable range (YES in S 20), the process proceeds to S 24. When it is determined that the difference between the initial traffic scene and the restored traffic scene is not within the allowable range (NO in S 20), the autonomous driving evaluation device 100 proceeds with the process to S 26.In S 22, the autonomous driving evaluation device 100 determines whether the autonomous driving vehicle model and the moving object model in the past traffic scene candidate are in the rule-compliant state or the end condition is satisfied, using the past traffic scene calculation unit 12. The fact that the autonomous driving vehicle model and the moving object model in the past traffic scene candidate are in the rule-compliant state means that they are in a state in which, for example, both the autonomous driving vehicle model and the moving object model in the lane in the traveling direction of the lane travel at the speed within the speed limit set for the respective lane on which the respective model travels.The end condition is a preset condition for determining whether or not to end the calculation of the past traffic scene. The past traffic scene calculation unit 12 may determine that the end condition is satisfied when the time traceback is performed for a certain time (for example, 10 seconds). The past traffic scene calculation unit 12 may determine that the end condition is satisfied when the autonomous driving vehicle model in the past traffic scene candidate is in an adverse state. The rule-aware state may include states that are different from the rule-compliant state.The non-regulatory state may include a state in which the speed of the autonomous driving vehicle model exceeds the speed limit of the driving lane. The non-regulatory state may include a state in which the autonomous driving vehicle model deviates from the travel lane without cause such as a lane change. The non-regulatory state may include a state in which the autonomous driving vehicle model travels in a direction substantially opposite to the traveling direction of the traveling lane. When the normal traveling state is used for the determination, a state in which a plurality of moving object models (autonomous driving vehicle models may be included) travel on the same lane may also be included.When it is not determined that the autonomous driving vehicle model and the moving object model are present in the past traffic scene candidate in the rule-compliant states and it is not determined that the end condition is satisfied (NO in S 22), the autonomous driving evaluation device 100 proceeds with the process to S 24. When it is determined that the autonomous driving vehicle model and the moving object model are in the rule-compliant state or the end condition is satisfied in the past traffic scene candidate (YES in S 22), the autonomous driving evaluation device 100 proceeds with the process to S 26.In S 24, the autonomous driving evaluation device 100 calculates, using the past traffic scene calculation unit 12, the traffic scene at a predetermined time before the current past traffic scene candidate as new past traffic scene candidates. Thereafter, the autonomous driving evaluation device 100 repeats the processing elements of S 16 for the new past traffic scene candidate.In S 26, the autonomous driving evaluation device 100 sets the past traffic scene candidate as the past traffic scene using the past traffic scene calculation unit 12. Thereafter, the autonomous driving evaluation device 100 starts the performance evaluation processing. S12 to S26 in FIG. 6 constitute a past traffic scene calculating step for calculating the past traffic scene.When the autonomous driving vehicle model is not in the rule-conforming state in the past traffic scene obtained as described above, since this is not consistent with the fact that the autonomous driving vehicle model is designed to satisfy the rule-conforming state, it can be judged that the autonomous driving vehicle model will not enter the initial traffic scene. Even if a deviation from the rule-compliant state is partially recognized to ensure the safety of the autonomous driving vehicle model, it is judged that a warning can be issued to a driver of the autonomous driving vehicle model before entering the initial traffic scene by detecting the partially rule-less state (or an abnormal driving state). On the other hand, when the autonomous driving vehicle model is in the rule-compliant state in the obtained past traffic scene, the evaluation is performed based on the mechanical stability.FIG. 7 is a flowchart of performance evaluation processing based on dynamic stability. As shown in FIG. 7, the autonomous driving evaluation device 100 reads the past traffic scene calculated by the past traffic scene calculation unit 12 in S 30.In S 32, the autonomous driving evaluation device 100 generates a past comparative traffic scene using the past comparative traffic scene generation unit 13 (a past comparative traffic scene generation step). The past comparison traffic scene generation unit 13 generates a preset number of past comparison traffic scenes by minutely changing the past traffic scene.In S 34, the autonomous driving evaluation device 100 calculates the autonomous driving reflected scene and the comparison autonomous driving reflected scene using the autonomous driving reflected scene calculation unit 14 (an autonomous driving reflected scene calculation step). The autonomous driving reflected scene calculation unit 14 calculates the autonomous driving reflected scene after elapse of a preset time in a state in which the autonomous driving is performed on the autonomous driving vehicle model by the autonomous driving algorithm from the past traffic scene, based on the past traffic scene and the autonomous driving algorithm. In addition, the autonomous driving reflected scene calculation unit 14 calculates the comparison autonomous driving reflected scene after elapse of a preset time in a state in which the autonomous driving is performed on the autonomous driving vehicle model by the autonomous driving algorithm from the comparison past traffic scene, based on the comparison past traffic scene and the autonomous driving algorithm.In S 36, the autonomous driving evaluation device 100 calculates the transformation matrix H using the performance evaluation unit 15. the performance evaluation unit 15 calculates the transformation matrix H illustrated in Equation (26) based on the past traffic scene, the past comparison traffic scene, the autonomous driving reflected scene, and the autonomous driving reflected comparison scene.In S 38, the autonomous driving evaluation device 100 obtains, using the performance evaluation unit 15, the maximum eigenvalue λmax of the transformation matrix H. The performance evaluation unit 15 can obtain the maximum eigenvalue λmax by the calculation along the time axis E(t-nδt)→E(t-(n-1)δt)→E(t-(n-2)δt)→...→E(t-δt)→E(t).In S 40, the autonomous driving evaluation device 100 evaluates the performance of the autonomous driving algorithm using the performance evaluation unit 15. The performance evaluation unit 15 evaluates the performance of the autonomous driving algorithm based on the maximum eigenvalue λmax. S 36, S 38 and S 40 in FIG. 7 constitute a performance evaluation step for evaluating the performance of the autonomous driving algorithm. As described above, in this result of the judgment, when the movement of the autonomous driving vehicle is dynamically unstable, it is judged that the vehicle does not enter the initial traffic scene, and when the movement of the autonomous driving vehicle is stable, it is judged that the vehicle can enter the initial traffic scene.Practical Effects of the Autonomous Driving Evaluation Device according to the First EmbodimentAccording to the autonomous driving evaluation device 100 of the first embodiment described above, the past traffic scene that is traced back from the initial traffic scene is calculated, and the performance of the autonomous driving algorithm is evaluated based on the past traffic scene, and therefore, the autonomous driving algorithm can be appropriately evaluated compared with a case where the past traffic scene is not considered.In addition, according to the autonomous driving evaluation device 100, the autonomous driving reflected scene is calculated from the past traffic scene, and the past comparative traffic scene is calculated from a plurality of past comparative traffic scenes at the past time, and then the autonomous driving algorithm is evaluated based on the autonomous driving reflected scene and the past comparative traffic scene. Therefore, according to the autonomous driving evaluation device 100, by reflecting the autonomous driving algorithm, the tendency of stable convergence or unstable divergence to the initial traffic scene from the past traffic scene and the past comparison traffic scene can be recognized, and therefore, the performance of the autonomous driving algorithm can be appropriately evaluated so that the situation does not reach the initial traffic scene.According to the autonomous driving evaluation device 100, since the past traffic scene at the time when both the state of the autonomous driving vehicle model and the state of the moving object model become the preset rule-compliant state is calculated as the past time point by time-tracing the time by a predetermined period from the time point of the initial traffic scene, application of the past traffic scene of the rule-poor state that is unsuitable for the precondition of the evaluation of the autonomous driving algorithm can be avoided, and thus the autonomous driving algorithm can be appropriately evaluated.Second EmbodimentFIG. 8 is a block diagram of an autonomous driving evaluation device according to a second embodiment. The autonomous driving evaluation device 200 shown in FIG. 8 is different from the autonomous driving evaluation device 100 of the first embodiment only in a point that the evaluation of the autonomous driving algorithm is performed by forward calculation.In the autonomous driving evaluation device 200, the autonomous driving algorithm is evaluated such that the situation stably converges to the initial traffic scene, and the performance evaluation of the autonomous driving algorithm is performed by evaluating the behavior of the autonomous driving algorithm when the time passes from the initial traffic scene. The autonomous driving evaluation device 200 may evaluate the behavior of the autonomous driving algorithm when the time passes from the initial traffic scene regardless of the stability evaluation described in the first embodiment.Configuration of Autonomous Driving Evaluation Device according to Second EmbodimentAs shown in FIG. 8, the autonomous driving evaluation device 200 of the second embodiment includes a future state calculation unit 20 and a ratio determination unit 21 compared to the first embodiment.The future state calculation unit 20 calculates a future state of the autonomous driving vehicle model when the autonomous driving is performed using the autonomous driving algorithm from the time of the initial traffic scene, and a future state of the moving object model that performs a preset movement from the time of the initial traffic scene, based on the initial state of the autonomous driving vehicle model, the initial state of the moving object model, and the road environment. The future state calculation unit 20 performs a forward calculation in the future, unlike the backward calculation.The future state of the autonomous driving vehicle model is a state of the autonomous driving vehicle model after the lapse of time from the time of the initial traffic scene. The future state of the moving object model is a state of the moving object model after the lapse of time from the time of the initial traffic scene. The moving object model moves with the moving object algorithm set in advance. The moving object model may be in a stationary state without movement when the initial state is a stationary state.The future state calculation unit 20 calculates the future state of the autonomous driving vehicle model and the future state of the moving object model from the initial traffic scene using a method similar to the method of calculating the autonomous driving reflected scene from the past traffic scene. The future state calculation unit 20 may calculate the future state of the autonomous driving vehicle model and the future state of the moving object model from the initial traffic scene using a known method.For example, the future state calculation unit 20 may repeat the calculation of the future state of the autonomous driving vehicle model and the future state of the moving object model for each time elapse by a predetermined period from the time of the initial traffic scene. The future state calculation unit 20 ends the calculation of the future state when the ratio determination unit 21 described below determines that the autonomous driving vehicle model and the moving object model are in a low evaluation ratio set in advance or when it is determined that the end condition is satisfied.The ratio determination unit 21 determines whether or not the autonomous driving vehicle model and the moving object model are in a low evaluation ratio based on the future state of the autonomous driving vehicle model and the future state of the moving object model. The low evaluation ratio is a ratio between the autonomous driving vehicle model and the moving object model set in advance. The low evaluation ratio includes, for example, a ratio at which the autonomous driving vehicle model and the moving object model will collide with each other when traveling straight, as illustrated in FIG. 3.The low evaluation ratio is not limited to a ratio at which the autonomous driving vehicle model and the moving object model will collide with each other. The low evaluation ratio may include a ratio at which the autonomous driving vehicle model becomes in an abnormal state to avoid the collision with the moving object model. The low evaluation ratio may include a ratio at which both the autonomous driving vehicle model and the moving object model are in the non-regulatory state.The ratio determination unit 21 determines whether or not a preset end condition is satisfied. The end condition is a preset condition for determining whether or not to end the calculation of the future state using the future state calculation unit 20. When the future state calculation unit 20 calculates the future state until a certain preset time (for example, after 30 seconds) from the time of the initial traffic scene, the ratio determination unit 21 determines that the end condition is satisfied. The ratio determination unit 21 may determine that the end condition is satisfied when the future state of the autonomous driving vehicle model and the future state of the moving object model start to stably increase the distance therebetween. The case where the future state of the autonomous driving vehicle model and the future state of the moving object model start to stably increase the distance therebetween means, for example, a case where the separation distance per a certain time is less than or equal to a threshold value.The performance evaluation unit 22 evaluates the performance of the autonomous driving algorithm based on the result of the determination performed by the ratio determination unit 21. When it is determined by the ratio determination unit 21 that the autonomous driving vehicle model and the moving object model have a low evaluation ratio, the performance evaluation unit 22 may evaluate that the autonomous driving algorithm does not have performance for avoiding the low evaluation ratio between the autonomous driving vehicle model and the moving object model when the two models reach the initial traffic scene. When it is determined by the ratio determination unit 21 that the autonomous driving vehicle model and the moving object model do not have a low evaluation ratio, the performance evaluation unit 22 may evaluate that the autonomous driving algorithm has a performance of avoiding the low evaluation ratio between the autonomous driving vehicle model and the moving object model even though the two models reach the initial traffic scene.FIG. 9 is a table illustrating an example of the performance evaluation according to the second embodiment. FIG. 9 shows an example of performance evaluation of the autonomous driving algorithm in a case where a problematic initial traffic scene (see FIG. 3 ) is set as a scene to be evaluated.As shown in FIG. 9, when it is judged by the backward calculation described in the first embodiment that the autonomous driving algorithm makes the situation to the initial traffic scene instable diverge, the performance judging unit 22 (OK judgment) judges that there is no problem with the performance of the autonomous driving algorithm, regardless of the agreement with rules and laws of the autonomous driving vehicle model and the moving object model in the past traffic scene and the result of the forward calculation from the initial traffic scene in the second embodiment. Even in the autonomous driving algorithm in which it is determined that the autonomous driving vehicle model and the moving object model have the low evaluation ratio when forward calculation is performed from the initial traffic scene, when the algorithm has the capability of preventing the situation from reaching the initial traffic scene from the beginning, the two models do not have a low evaluation ratio. Therefore, the performance evaluation unit 22 may evaluate that there is no problem with the performance of the autonomous driving algorithm.When the autonomous driving algorithm is evaluated by the backward calculation to stably converge the situation to the initial traffic scene, when it is determined that the autonomous driving vehicle model and the moving object model are in the rule-compliant states in the past traffic scene and it is determined by the forward calculation that the autonomous driving vehicle model and the moving object model do not have a low evaluation ratio, the performance evaluation unit 22 evaluates that there is no problem with the performance of the autonomous driving algorithm (OK evaluation).On the other hand, when the autonomous driving algorithm is evaluated to stably converge the situation to the initial traffic scene by the backward calculation, when it is determined that the autonomous driving vehicle model and the moving object model are in the rule-compliant states in the past traffic scene and it is determined by the forward calculation that the autonomous driving vehicle model and the moving object model have the low evaluation ratio, the performance evaluation unit 22 evaluates that there is a problem with the performance of the autonomous driving algorithm (NG evaluation). By reflecting the autonomous driving using the autonomous driving algorithm, the autonomous driving vehicle model and the moving object model are made to achieve the low evaluation ratio from the past traffic scene representing the rule-compliant state via the initial traffic scene. Therefore, the performance evaluation unit 22 may evaluate that there is a problem with the performance of the autonomous driving algorithm.When the autonomous driving algorithm is evaluated by the backward calculation to stably converge the situation to the initial traffic scene, when the autonomous driving vehicle model in the past traffic scene is in the unwanted state, the performance evaluation unit 22 evaluates that there is no problem with the performance of the autonomous driving algorithm (OK evaluation). In this case, since it can be considered that the autonomous driving algorithm does not make the situation reach the initial traffic scene when there is no abnormal situation that the autonomous driving vehicle model comes into an adverse state at the past time, the performance evaluation unit 22 evaluates that there is no problem in the autonomous driving algorithm.Autonomous Driving Evaluation Method Using the Autonomous Driving Evaluation Device According to the Second EmbodimentThe autonomous driving evaluation method using the autonomous driving evaluation device 200 of the second embodiment will be described below. Here, the feedforward processing will be described. The feedforward processing also includes the performance evaluation processing using the result of the feedforward calculation. The backward calculation processing is the same as in the first embodiment, and therefore, the description thereof is omitted.FIG. 10 is a flowchart of the feedforward processing. As shown in FIG. 10, the autonomous driving evaluation device 200 sets an initial traffic scene in S 50 (initial traffic scene setting step) using the initial traffic scene setting unit 11. The initial traffic scene setting unit 11 sets the traffic scene as an initial traffic scene to be used for the performance evaluation of the autonomous driving algorithm. S50 is the same processing as S10 in FIG. 5, and S50 may be omitted when the initial traffic scene has already been set. The backward calculation or the forward calculation may be performed first, and the forward calculation may be performed only when the situation is judged to be stable by the backward calculation.In S 52, the autonomous driving evaluation device 200 calculates the future state of the autonomous driving vehicle model and the future state of the moving object model when the autonomous driving is performed by the autonomous driving algorithm from a time point of the initial traffic scene to a predetermined past time, using the future state calculation unit 20 (future state calculation step). The future state calculation unit 20 calculates the future state of the autonomous driving vehicle model and the future state of the moving object model from the initial traffic scene using the same method as the method of calculating the autonomous driving reflected scene from the past traffic scene.In S 54, the autonomous driving evaluation device 200 determines whether the autonomous driving vehicle model and the moving object model have the low evaluation ratio or the end condition is satisfied using the ratio determination unit 21 (ratio determination step). The ratio determination unit 21 performs the above-described determination based on the future state of the autonomous driving vehicle model and the future state of the moving object model. When it is not determined that the autonomous driving vehicle model and the moving object model have the low evaluation ratio and it is not determined that the end condition is satisfied (NO in S 54), the autonomous driving evaluation device 200 proceeds with the process to S 56. On the other hand, when it is determined that the autonomous driving vehicle model and the moving object model have the low evaluation ratio or it is determined that the end condition is satisfied (YES in S 54), the autonomous driving evaluation device 200 proceeds with the process to S 58.In S 56, the autonomous driving evaluation device 200 calculates the future state of the autonomous driving vehicle model and the future state of the moving object model again by using the future state calculation unit 20 when the autonomous driving is performed using the autonomous driving algorithm until the predetermined past time. Thereafter, the autonomous driving evaluation device 200 returns the process to S 54 and repeats the process.In S 58, the autonomous driving evaluation device 200 evaluates the performance of the autonomous driving algorithm using the performance evaluation unit 22. the performance evaluation unit 22 evaluates the performance of the autonomous driving algorithm based on the result of the determination performed by the ratio determination unit 21 (result of the determination in S 54). The performance evaluation unit 22 determines whether the autonomous driving vehicle model and the moving object model have the low evaluation ratio when the forward calculation is performed from the initial traffic scene while the autonomous driving is reflected by the autonomous driving algorithm, based on the result of the determination performed by the ratio determination unit 21. When the backward calculation processing in the first embodiment is completed, the performance evaluation unit 22 may evaluate the performance of the autonomous driving algorithm based on the result of the backward calculation as shown in FIG. 9.Practical Effects of the Autonomous Driving Evaluation Device according to the Second EmbodimentAccording to the autonomous driving evaluation device 200 of the second embodiment described above, by calculating the future state of the autonomous driving vehicle model and the future state of the moving object model in performing the autonomous driving using the autonomous driving algorithm from the time of the initial traffic scene, it can be determined whether the two models have the low evaluation ratio set in advance when the time elapses from the initial traffic scene while the autonomous driving algorithm is reflected. Therefore, the autonomous driving algorithm can be appropriately evaluated.In the above, the preferred embodiments of the present disclosure have been described, but the present disclosure is not limited to the above-described embodiments. The present disclosure may be implemented in various forms including various modifications and improvements based on the knowledge of those skilled in the art in addition to the above-described embodiments.The initial traffic scene is not limited to the above-described traffic scene. The initial traffic scene may be a traffic scene in which another vehicle model (a moving object model) approaches an intersection of the driving lane of the autonomous driving vehicle model at the intersection. The initial traffic scene may be a traffic scene in which a pedestrian model (a moving object model) enters a parking space where an autonomous driving vehicle model attempts to park in a parking space. The initial traffic scene may be a traffic scene in which a pedestrian model (a moving object model) passes before the autonomous driving vehicle model when the autonomous driving vehicle model attempts to turn right or left at the intersection. The initial traffic scene may be a traffic scene in which a bicycle model that travels right behind the autonomous driving vehicle model travels straight when the autonomous driving vehicle model attempts to turn right.The elapsed time, which is the time of the past traffic scene, may be a fixed time a predetermined time before the initial traffic scene. That is, in calculation, the past traffic scene calculation unit 12 may calculate a traffic scene at a time a certain time before the initial traffic scene as a past traffic scene.In the evaluation of stability, the evaluation is performed while considering a part of the time period from the entire time period from the past time point to the time point of the initial traffic scene, but the entire time period and not only a part of the time period may be used. That is, the autonomous driving reflected scene calculation unit 14 may calculate the autonomous driving reflected scene in a state in which the time elapses from the past time point of the past traffic scene to the time point of the initial traffic scene, during a state in which the autonomous driving is performed in the autonomous driving vehicle model using the autonomous driving algorithm. Similarly, the autonomous driving reflected scene calculation unit 14 may calculate the comparison autonomous driving reflected scene during a state in which the time elapses from the past time point to the time point of the initial traffic scene.The number of past comparison traffic scenes may be a number greater than or equal to two less than a dimension of the past traffic scene.

Claims

An autonomous driving evaluation device (100) for evaluating an autonomous driving algorithm through a simulation, the device comprising an initial traffic scene setting unit (11) for setting an initial traffic scene in which an autonomous driving vehicle model and a moving object model are involved by setting an initial state of the autonomous driving vehicle model in which autonomous driving is performed using the autonomous driving algorithm, an initial state of the moving object model, and a road environment in which the autonomous driving vehicle model and the moving object model are arranged, a past traffic scene calculation unit (12) for calculating a past traffic scene candidate, wherein the autonomous driving vehicle model and the moving object model are involved at a past time point that is backward from a time point of the initial traffic scene based on the initial traffic scene, wherein the past traffic scene candidate is calculated by repeatedly calculating the states of the autonomous driving vehicle model and the states of the moving object model at the time points that are time points backward from the time point of the initial traffic scene by a predetermined time period, determining whether both the state of the autonomous driving vehicle model and the state of the moving object model are a preliminarily set rule-compliant state, and setting the past traffic scene candidate as a past traffic scene based thereon, wherein both the state of the autonomous driving vehicle model and the state of the moving object model are a rule-compliant state set in advance, and a performance evaluation unit (15) for evaluating a performance of the autonomous driving algorithm based on the past traffic scene.The autonomous driving evaluation device according to claim 1, further comprising a past comparative traffic scene generation unit (13) for generating a preset number of past comparative traffic scenes at the past time by minutely changing the past traffic scene and an autonomous driving reflected scene calculation unit (14) for respectively calculating an autonomous driving reflected scene after elapse of a preset time from the past traffic scene during a state in which the autonomous driving is performed in the autonomous driving vehicle model by the autonomous driving algorithm and calculating an autonomous driving reflected comparative scene after elapse of a preset time from the past comparative traffic scene during a state, in which the autonomous driving is performed in the autonomous driving vehicle model by the autonomous driving algorithm, wherein the performance evaluation unit (15) is configured to evaluate the performance of the autonomous driving algorithm based on the initial traffic scene, the past traffic scene, the past comparison traffic scene, the autonomous driving reflected scene, and the autonomous driving reflected comparison scene.The autonomous driving evaluation device according to claim 1 or 2, further comprising a future state calculation unit (20) for calculating a future state of the autonomous driving vehicle model when the autonomous driving is performed using the autonomous driving algorithm from the time point of the initial traffic scene and a future state of the moving object model that performs a preset movement from the time point of the initial traffic scene based on the initial state of the autonomous driving vehicle model, the initial state of the moving object model, and the road environment, and a ratio determination unit (21) for determining whether the autonomous driving vehicle model and the moving object model have a low evaluation ratio based on the future state of the autonomous driving vehicle model and the future state of the moving object model, wherein the performance evaluation unit (22) is configured to evaluate the performance of the autonomous driving algorithm based on the result of the determination performed by the ratio determination unit.An autonomous driving evaluation method in an autonomous driving evaluation device (100) for evaluating an autonomous driving algorithm through a simulation, the method comprising setting an initial traffic scene in which an autonomous driving vehicle model and a moving object model are involved by setting an initial state of the autonomous driving vehicle model in which the autonomous driving is performed using the autonomous driving algorithm, an initial state of the moving object model, and a road environment in which the autonomous driving vehicle model and the moving object model are arranged, calculating a past traffic scene candidate in which the autonomous driving vehicle model and the moving object model are involved at a past time, the state of the autonomous driving vehicle model and the state of the moving object model are repeatedly calculated at the time points that are the time points backward from the time point of the initial traffic scene by a predetermined time period in the calculation of the past traffic scene candidate, determining whether both the state of the autonomous driving vehicle model and the state of the moving object model are a preset rule-compliant state, setting the past traffic scene candidate as a past traffic scene based on both the state of the autonomous driving vehicle model and the state of the moving object model being a preset rule-compliant state, and evaluating performance of the autonomous driving algorithm based on the past traffic scene.The autonomous driving evaluation method according to claim 4, further comprising generating a preset number of past comparative traffic scenes at the past time by minutely changing the past traffic scene, and calculating an autonomous driving reflected scene after elapse of a preset time from the past traffic scene during a state in which the autonomous driving is performed on the autonomous driving vehicle model by the autonomous driving algorithm, and calculating an autonomous driving reflected comparative scene after elapse of a preset time from the past comparative traffic scene during a state in which the autonomous driving is performed on the autonomous driving vehicle model by the autonomous driving algorithm, wherein in the evaluation, the performance of the autonomous driving algorithm is evaluated based on the initial traffic scene, the past traffic scene, the past comparison traffic scene, the autonomous driving reflected scene, and the autonomous driving reflected comparison scene.The autonomous driving evaluation method according to claim 4 or 5, further comprising calculating a future state of the autonomous driving vehicle model when the autonomous driving is performed using the autonomous driving algorithm from the time point of the initial traffic scene and a future state of the moving object model that performs a preset motion from the time point of the initial traffic scene based on the initial state of the autonomous driving vehicle model, the initial state of the moving object model, and the road environment, and determining whether the autonomous driving vehicle model and the moving object model have a preset low evaluation ratio based on the future state of the autonomous driving vehicle model and the future state of the moving object model, wherein in the evaluation, the performance of the autonomous driving algorithm is evaluated based on the result of the determination performed in the determination.

Images