Action prediction device, vehicle, and action prediction method

Abstract

The purpose of the present invention is to provide an action prediction device and an action prediction method that reduce action prediction computation and are generically applicable regardless of the existence of change factors for object behavior. An action prediction device according to the present invention comprises an object detection unit that detects a prediction target on the basis of external environment information from an external environment sensor and an action prediction unit that predicts the actions of the prediction target. The action prediction unit includes an action pattern identification unit that identifies action patterns that could be taken by the prediction target on the basis of at least one of the state of the prediction target and the surrounding environment of the prediction target, an output condition setting unit that sets prediction output conditions for each of the identified action patterns, and a prediction processing unit that outputs prediction results for each of the action patterns on the basis of the output conditions set.

Description

Behavior prediction device, vehicle, and behavior prediction method

[0001] The present invention relates to a behavior prediction device, a vehicle, and a behavior prediction method for predicting the behavior of others in order to realize preventive safety functions and automatic driving of automobiles.

[0002] To realize preventive safety features and autonomous driving in automobiles, it is effective to predict the future behavior of surrounding objects detected by on-board sensors. When predicting the behavior of surrounding objects, it is sometimes necessary to take into account the interactions between objects and predict multiple possible behaviors. While these behavior prediction methods can achieve highly accurate predictions, they require complex calculations, which increases the computational load.

[0003] A behavior prediction method described in Patent Document 1 is known as a measure to reduce the computational load when predicting the behavior of a surrounding object. For example, the abstract of this document states that the problem is to "predict behavior accurately while reducing the computational load," and as a solution, it states that "based on objects around the host vehicle or the driving environment around the host vehicle, it is determined whether there is a change factor that will lead to a change in the behavior of the predicted object. The behavior of the predicted object is predicted based on the determination result of whether there is a change factor and information obtained from the objects around the host vehicle. If it is determined that there is no change factor, a short-term prediction is performed to predict the behavior of the predicted object based only on information obtained from the predicted object. If it is determined that there is a change factor, a long-term prediction is performed to predict the behavior of the predicted object based on information obtained from the predicted object and information obtained from surrounding objects around the predicted object."

[0004] As described above, Patent Document 1 discloses a method for reducing the computational load in an environment where there are no change factors by determining whether there are any change factors in object behavior (such as whether there are other objects intersecting), and performing long-term predictions with a high computational load when there are change factors, and performing short-term predictions with a low computational load when there are no change factors. This makes it possible to limit the situations in which high-accuracy behavior predictions are performed, and reduce the overall computational load, compared to behavior prediction methods that always perform predictions with a high computational load.

[0005] Japanese Patent Application Laid-Open No. 2020-166510

[0006] However, in the case of long-term prediction in Patent Document 1, paragraph 0081 of the same document states that "When calculating the candidate paths, the prediction processing unit 16 sets lane center lines La1 and Lb2 in the center of the lane. In the case of the candidate path Rb2 for straight travel, the prediction processing unit 16 calculates the candidate path Rb2 so that it is parallel to the lane center line La2, based on the current traveling position of the other vehicle Vb1. On the other hand, in the case of the candidate path Rb1 for lane change, the prediction processing unit 16 calculates the travel distance that the other vehicle Vb1 will travel until the lane change is completed, based on the time required to complete the lane change and the current traveling speed of the other vehicle Vb1." The time required for a lane change is a value, e.g., 3 seconds, that is preset based on the driving tendencies of typical drivers. The prediction processing unit 16 sets the position at which the lane change is completed on the lane center line La1 based on the calculated travel distance. The prediction processing unit 16 then connects the current driving position and the position at which the lane change is completed with an arbitrary curve, such as a clothoid curve, and sets this as the path candidate Rb1. This method, which requires a considerable computational load, predicts a large number of path candidates such as the path candidates Rb1 to Rb4 and Rc1 in Figure 6 of the same document. Therefore, the reduction in computational load achieved by the technology of Patent Document 1 could only be achieved in an environment where there were no change factors and short-term prediction was sufficient.

[0007] In view of the above, an object of the present invention is to provide a behavior prediction device and a behavior prediction method that can be generally applied regardless of whether there are factors that change object behavior, and that reduce the amount of behavior prediction calculations.

[0008] A behavior prediction device comprising: an object detection unit that detects a predicted object based on external information from an external sensor; and a behavior prediction unit that predicts the behavior of the predicted object, wherein the behavior prediction unit has: a behavior pattern identification unit that identifies a behavior pattern that the predicted object may perform based on at least one of the state of the predicted object or the surrounding environment of the predicted object; an output condition setting unit that sets prediction output conditions for each of the identified behavior patterns; and a prediction processing unit that outputs a prediction result for each of the behavior patterns based on the set output conditions.

[0009] According to the present invention, the amount of calculation can be reduced by dynamically changing the output conditions of the prediction in accordance with the possible behavioral patterns of the object.

[0010] 1 is a functional block diagram showing the configuration of a behavior prediction device according to an embodiment; FIG. 2 is a flowchart showing the procedure of a behavior prediction process according to an embodiment; FIG. 3 is an example of identifying a behavior pattern based on lane positions and connections; FIG. 4 is an example of identifying a behavior pattern based on traffic light displays; FIG. 5 is an example of identifying a behavior pattern based on interactions with other objects; FIG. 6 is an example of changing output conditions based on the probability of occurrence of a behavior pattern; FIG. 7 is an example of changing output conditions based on the risk of a behavior pattern; FIG. 8 is an example of predicting one trajectory by combining multiple prediction methods; and FIG. 9 is an example of predicting by allocating prediction points to each behavior pattern.

[0011] A behavior prediction device 1 according to an embodiment of the present invention will be described in detail below.

[0012] 1 is a functional block diagram showing the configuration of the behavior prediction device 1 of this embodiment. As shown in Fig. 1, the behavior prediction device 1 has an object detection unit 10, a surrounding information management unit 11, and a behavior prediction unit 12. The behavior prediction unit 12 also has a behavior pattern identification unit 12a, an output condition setting unit 12b, and a prediction processing unit 12c.

[0013] The behavior prediction device 1 may be applied to a vehicle having a preventive safety function or an autonomous driving function, or may be applied to a device other than a vehicle, such as a surveillance camera. In the following description, it is assumed that the behavior prediction device 1 is mounted on a vehicle. In the following description, the vehicle on which the behavior prediction device 1 is mounted may be referred to as the host vehicle.

[0014] 8 shows a hardware configuration diagram of a vehicle 9 equipped with the behavior prediction device 1 of this embodiment. The vehicle 9 includes the behavior prediction device 1, a vehicle control device 2, an external sensor 3, a display 4, a communication device 5, and a GPS receiver 6, which are interconnected by an in-vehicle network 7.

[0015] The behavior prediction device 1 includes a CPU 1A, which is a central processing unit; a ROM 1B, which is a read-only storage device; a RAM 1C, which is a volatile memory such as a DRAM; and a flash memory 1D, which is a non-volatile storage device. In the behavior prediction device 1, the CPU 1A deploys a program stored in the ROM 1B into the RAM 1C and executes it to realize various functional units. The vehicle control device 2, like the behavior prediction device 1, is an electronic control device equipped with a CPU and a storage device. The CPU executes a control program stored in the storage device to control various functions, such as the steering system, drive system, and braking system. The vehicle control device 2 can control the steering system, drive system, braking system, etc. based on the prediction results of the behavior prediction device 1. When the prediction results are used to control each system of the vehicle, safe autonomous driving with a low risk of collision can be achieved by planning the trajectory of the vehicle so as not to interfere with the predicted trajectory of other vehicles.

[0016] The external sensor 3 may be, for example, a LiDAR, a camera, a millimeter wave radar, etc. The detection result by the external sensor 3 is input to the object detection unit 10 of the behavior prediction device 1.

[0017] The display 4 is, for example, a display in a meter panel or a head-up display, but is not limited to these forms. The prediction results of the behavior prediction device 1 can be notified to the driver via the display 4. By notifying the driver of vehicles that are predicted to be at risk of contact with the driver's own vehicle, the driver can drive while paying particular attention to the status of the notified vehicles.

[0018] The communication device 5 is a communication module that performs wireless communication with the outside of the vehicle 9. The communication device 5 is configured to be able to communicate with a server, and acquires, for example, map information about the surroundings of the vehicle.

[0019] The GPS receiver 6 is a receiver for the GPS (Global Positioning System) that measures the position of the vehicle based on radio waves from satellites.

[0020] The details of each part of the behavior prediction device 1 will be described below in order. <<Object detection unit 10>> The object detection unit 10 is a functional unit that uses external world information from external sensors to detect moving objects including other vehicles such as cars and people, motorcycles, bicycles, and pedestrians, and stationary objects including parked vehicles and buildings. The object detection unit 10 also tracks moving objects and estimates state quantities such as the position, speed, and orientation of the objects.

[0021] <<Surrounding Information Management Unit 11>> The surrounding information management unit 11 is a functional unit that manages surrounding information around the vehicle. The surrounding information includes road structures, traffic light and road sign displays, lanes, etc. The surrounding information can be stored in advance as map information, acquired through external communication, or acquired through recognition by an external sensor mounted on the vehicle. Acquisition by an external sensor is performed, for example, by inputting a camera image and segmenting it into regions using a trained machine learning model.

[0022] <<Behavior Prediction Unit 12>> The behavior prediction unit 12 is a functional unit that predicts one or more future behaviors of an object. In this embodiment, the future behavior is predicted as a sequence of trajectory points that the object will pass through in the future. Each point in the trajectory point sequence has a position coordinate and a time of a future point as state quantities, and a trajectory is formed by connecting each point in time series. In order to predict a sequence of trajectory points, it is necessary to set a prediction period and a time resolution of the sequence of points. It is also necessary to set the number of trajectory points to be predicted. Therefore, the behavior prediction unit 12 sets the number of trajectory point sequences to be predicted, the prediction period, and the time resolution as output conditions for the prediction, and then performs the prediction. The behavior pattern identification unit 12a, the output condition setting unit 12b, and the prediction processing unit 12c, which sequentially execute these processes, will be described in detail below.

[0023] The behavior pattern identification unit 12a identifies one or more possible behavior patterns of an object based on at least one of the state of the object and surrounding information. A behavior pattern refers to a type of behavior expected of an object, such as turning right or left, going straight, stopping, or parking. The behavior pattern identification unit 12a can also calculate associated information such as the occurrence probability or risk level of each behavior pattern. The method for identifying a behavior pattern or the method for calculating the associated information may be conditional branching based on simple rules, a machine learning method, or a statistical method. Here, it is desirable to identify a behavior pattern using a method with a low computational load. This is because if the behavior pattern identification unit 12a uses a method with a high computational load, the overall computational load cannot be reduced even if the computational load is reduced in subsequent prediction processing.

[0024] The output condition setting unit 12b sets the output conditions for prediction based on the type of each behavior pattern, accompanying information, the computational resources of the computer, and the like.

[0025] The prediction processing unit 12c executes prediction processing in accordance with the output conditions set by the output condition setting unit 12b. The prediction method may be a machine learning method, a method for searching for an optimal trajectory, or a method using rule-based conditional branching. The prediction processing unit 12c can also store multiple prediction methods to accommodate dynamically set output conditions. For example, to perform predictions with different time resolutions, the prediction processing unit 12c can store a machine learning model suitable for predictions with low time resolution and a machine learning model suitable for predictions with high time resolution. Furthermore, multiple prediction methods can be combined to predict one trajectory point sequence.

[0026] <Example of Operation of Behavior Prediction Device 1> Next, an example of operation of the behavior prediction device 1 of this embodiment will be described in detail with reference to the flowchart in Fig. 2. Hereinafter, the operation of the behavior prediction device 1 will be described using an example in which an on-board camera is used as an external sensor and the object to be predicted is another vehicle in the vicinity of the host vehicle.

[0027] First, in step P1, the object detection unit 10 detects an object to be predicted (hereinafter, a predicted object) using a trained machine learning model that uses camera images. The object detection unit 10 also tracks the predicted object over time and applies a Kalman filter to estimate state quantities such as the position, speed, and orientation of the predicted object.

[0028] Next, in step P2, the surrounding information management unit 11 acquires information about the surroundings of the vehicle. Specifically, the surrounding information management unit 11 identifies the current vehicle position based on GPS information and acquires map information about the surroundings of the vehicle from a server. The surrounding information management unit 11 also acquires road shapes and traffic light indications using a trained machine learning model that uses camera images.

[0029] In step P3, the behavior pattern identification unit 12a identifies a behavior pattern that the predicted object may take. An example of a method for identifying a behavior pattern will be described below with reference to Figures 3A to 3C. Although not shown, it is assumed that the host vehicle is located in a position where each vehicle in the figure can be observed by an on-board camera (external sensor).

[0030] 3A shows an example of identifying a behavior pattern based on the position and connection of lanes. In this example, only a straight lane exists ahead of the prediction target vehicle V1, so going straight is identified as the behavior pattern. Furthermore, two lanes exist ahead of the prediction target vehicle V2, one for turning left and one for going straight, so these two lanes are identified as behavior patterns. Furthermore, three lanes exist ahead of the prediction target vehicle V3, one for turning left, one for going straight, and one for turning right, so these three lanes are identified as behavior patterns.

[0031] 3B shows an example of identifying a behavioral pattern based on the display of a traffic light ahead of the target vehicle V4. In this example, two behavioral patterns are identified: stopping when the traffic light is red, going straight when the traffic light is green, and stopping and going straight when the traffic light is yellow.

[0032] 3C shows an example of identifying a behavior pattern based on the relative positions of the target vehicle V5, an oncoming vehicle, and an obstacle. In the situation shown in the figure, the road is narrowed by an obstacle, allowing only one vehicle to pass through at a time. Therefore, in this example, two possible behavior patterns of the target vehicle V5 are identified: one in which the target vehicle V5 passes between the obstacles first, and one in which the oncoming vehicle passes between the obstacles first.

[0033] In step P4, the behavior pattern identification unit 12a calculates the probability of occurrence of each behavior pattern. The method for calculating the probability of occurrence of each behavior pattern of each vehicle in Figures 3A to 3C will be described below.

[0034] For the prediction target vehicle V1 in FIG. 3A, the behavior pattern is only straight driving, so the behavior pattern identification unit 12a calculates the occurrence probability of straight driving as 100%.

[0035] For the prediction target vehicle V2 and the prediction target vehicle V3 in FIG. 3A, the calculation method for the probability differs depending on whether the external sensor can recognize the blinking of the turn signal of each prediction target vehicle. First, consider the case where a blinking left turn signal is recognized. In this case, the probability of occurrence of a left turn is assumed to be equal to the recognition accuracy rate of the blinking left turn signal, and the remaining probabilities are evenly distributed among the remaining behavioral patterns to set the occurrence probability of each behavioral pattern. The calculation is performed in a similar manner when a blinking right turn signal is recognized.

[0036] If the blinking of the turn signal cannot be recognized, the probability of occurrence of each behavior pattern is calculated based on the direction of the vehicle. To this end, the behavior pattern identification unit 12a stores in advance a table indicating the relationship between the direction of the vehicle when entering an intersection and the probability of occurrence of each behavior pattern, such as turning left, going straight, or turning right. This table is created by collecting data on vehicles entering intersections in advance. As a result, the behavior pattern identification unit 12a can calculate the probability of occurrence of each behavior pattern based on the direction of the prediction target vehicle estimated by the object detection unit 10 and the table, even if the blinking of the turn signal cannot be recognized.

[0037] For the prediction target vehicle V4 in FIG. 3B , the behavior pattern identification unit 12a sets the probability of stopping to 100% when it recognizes a red light at a traffic light, and the probability of going straight to 100% when it recognizes a green light. On the other hand, when it recognizes a yellow light at a traffic light, the behavior pattern identification unit 12a calculates the probability based on the distance from the vehicle to the traffic light. For this reason, the behavior pattern identification unit 12a stores in advance a table showing the relationship between the distance from the vehicle to the traffic light and the probability of occurrence of each behavior pattern. This table is created by collecting data on vehicles entering intersections in advance. As a result, the behavior pattern identification unit 12a can calculate the distance from the vehicle to the traffic light using the vehicle position information estimated by the object detection unit 10 and the traffic light position information acquired by the surrounding information management unit 11, and use this information in conjunction with the table to determine the probability of occurrence of each behavior pattern.

[0038] For the prediction target vehicle V5 in Figure 3C, the behavior pattern identification unit 12a calculates the occurrence probability of each behavior pattern based on the distance between the vehicle and a line defined by connecting two obstacles. To this end, the behavior pattern identification unit 12a stores in advance a table indicating the relationship between the distance to the obstacle and the occurrence probability of each behavior pattern. This allows the behavior pattern identification unit 12a to calculate the distance using the vehicle position information estimated by the object detection unit 10 and the obstacle position information, and to use this in conjunction with the table to determine the occurrence probability of each behavior pattern.

[0039] Hereinafter, an example will be described in which prediction output conditions corresponding to each behavior pattern are set in steps P5, P6, and P7 for the prediction target vehicle V3 in Fig. 3A, and trajectories T1, T2, and T3 as shown in Fig. 4 are predicted in step P8. It is assumed that the occurrence probability of each behavior pattern is calculated as 80% for a left turn, 17% for a straight run, and 3% for a right turn in step P4.

[0040] In step P5, the output condition setting unit 12b sets the number of predicted trajectories for each behavior pattern. To this end, the output condition setting unit 12b stores in advance a table showing the correspondence between the occurrence probability and the number of predicted trajectories. For example, if the occurrence probability of a behavior pattern is less than 10%, zero trajectories are predicted; if it is 10% or more but less than 50%, one trajectory is predicted; and if it is 50% or more, two trajectories are predicted. In this example, taking into account the occurrence probability calculated in step P4, the output condition for the number of predicted trajectories is set so that two predicted trajectories are predicted for a left turn, one predicted trajectory for going straight, and zero predicted trajectories are predicted for a right turn.

[0041] In step P6, the output condition setting unit 12b sets the time resolution of each predicted trajectory. The output condition setting unit 12b holds a table in advance that indicates the correspondence between the type of behavior pattern and the time resolution. For example, if the behavior pattern is going straight, the point sequence is predicted at 1.0 second intervals, and if the behavior pattern is turning left or right, the point sequence is predicted at 0.3 second intervals. From the above, the left turn interval is set to 0.3 second intervals, and the straight forward interval is set to 1.0 second intervals.

[0042] In step P7, the output condition setting unit 12b sets the predicted length of each predicted trajectory. To this end, the output condition setting unit 12b stores in advance a table showing the correspondence between occurrence probability and predicted length. For example, if the occurrence probability of a behavior pattern is less than 50%, a sequence of points for 2 seconds is predicted, and if it is 50% or higher, a sequence of points for 3 seconds is predicted. Therefore, in this example, using this in conjunction with the occurrence probability calculated in step P4, the predicted trajectory for a left turn is set to 3 seconds, and the predicted trajectory for a straight run is set to 2 seconds.

[0043] In step P8, the prediction processing unit 12c predicts a trajectory based on the output conditions set above. Therefore, the prediction processing unit 12c has a prediction model with adjustable time resolution. The inputs of this prediction model are the state quantities of the vehicle to be predicted, the road shape, and the time resolution, and the output of the prediction model are candidate positions of the vehicle to be predicted one time step ahead and the movement probability corresponding to each position. Here, the resolution of one time step is determined by the time resolution of the input. The prediction model is an encoder-decoder model using the well-known LSTM (Long Short-Term Memory). After the input information is encoded by the encoder, the position coordinates are calculated by the decoder.

[0044] A method for predicting two trajectories corresponding to a left-turn behavior pattern using such a prediction model will be described below. First, a method for determining the first trajectory will be described. The state quantities of the predicted object determined in step P1, the road shape determined in step P2, and the time resolution determined in step P6 are input, and candidates for the position of the predicted object one step ahead and the movement probability are determined. Next, the position with the highest movement probability is used as input to the prediction model, and the position of the predicted object in the next step is determined. This process is repeated, and a sequence of points for the desired length of 3 seconds is determined as the first predicted trajectory.

[0045] Next, a method for determining the second predicted trajectory will be described. The current position of the object to be predicted determined in step P1 is used as an input, and candidates for the position of the object to be predicted one step ahead and the movement probability are determined. Here, the position with the second highest movement probability is used as an input for the prediction model, and the position of the object to be predicted in the next step is determined. Thereafter, the same process is repeated using the position with the highest movement probability as an input for the prediction model. Then, a sequence of points for three seconds, which is the desired length, is determined as the second predicted trajectory.

[0046] By these operations, two trajectories T1 and T2 with the desired output conditions can be obtained. In the case of going straight, the input time resolution is set to 1.0 second, and a trajectory T3 of 2 seconds is obtained in the same way as when obtaining the first trajectory for a left turn.

[0047] As described above, the behavior prediction device 1 of this embodiment identifies one or more possible behavior patterns of the prediction target, sets prediction output conditions for each behavior pattern, and outputs a prediction for each behavior pattern. This makes it possible to selectively use a prediction method with a low computational load and a prediction method with a high computational load. This reduces the computational load compared to when predictions are always made using a method with a high computational load.

[0048] <Functions and Effects of the Above Operational Example> The above operational example identifies one or more possible behavior patterns of the prediction target object, thereby enabling subsequent processing to be performed according to the behavior pattern.

[0049] The above operational example predicts the occurrence probability for each behavioral pattern, which makes it possible to set the output conditions for the prediction according to the occurrence probability of each behavioral pattern and the importance of the prediction.

[0050] The number of predictions to be output for each behavior pattern is set according to the above example of operation. This reduces the number of predictions for behavior patterns with low importance, or does not perform predictions at all, thereby reducing the computational load.

[0051] The above operational example sets the time resolution corresponding to each prediction result. By lowering the time resolution according to the behavioral pattern, the number of calculations of the prediction model can be reduced, thereby reducing the calculation load.

[0052] The above operational example sets a prediction period corresponding to each prediction result. By shortening the prediction period for predictions with low importance, the number of calculations for the prediction model can be reduced, thereby reducing the calculation load.

[0053] <Effect of Reduction of Computational Load> The computational load reduced by this embodiment will be roughly calculated. For simplicity, it is assumed that the number of prediction points is proportional to the computational load, and predictions made for three behavioral patterns (left turn, straight ahead, and right turn) are compared with predictions made by this embodiment, with two predictions made for each of the three patterns at 0.3-second intervals for three seconds. The former predicts six trajectories, each with 10 prediction points, for a total of 60 prediction points. In contrast, the latter predicts three trajectories (two left turns and one straight ahead), with 10 prediction points for each trajectory corresponding to a left turn and two prediction points for the trajectory corresponding to a straight ahead, for a total of 22 prediction points. Therefore, the operation of this embodiment described above can reduce the computational load to approximately one-third.

[0054] <When Determining Output Conditions, Considering Risk Instead of Occurrence Probability> In the above embodiment, the output conditions for the subsequent stage are set using the occurrence probability for each behavior pattern. However, risk can also be used instead of occurrence probability. Here, the risk of a behavior pattern is calculated using three levels: low, medium, and high. FIG. 5 shows prediction results when straight driving and right turning are identified as behavior patterns. As shown in the figure, an oncoming vehicle is approaching from in front of the prediction target vehicle V6, so if the prediction target vehicle V6 turns right, there is a possibility of a right-turn accident. Therefore, if there is an oncoming vehicle in the oncoming lane of the T-junction, the risk of turning right is determined to be high. On the other hand, if the prediction target vehicle V6 travels straight, the possibility of a collision with an oncoming vehicle is low, so the risk of traveling straight is determined to be low. Then, prediction is performed by setting the time resolution of the trajectory T4 of the low-risk behavior pattern lower than that of the trajectory T5 of the high-risk behavior pattern.

[0055] As a result, the risk level is predicted for each behavioral pattern. This makes it possible to set the output conditions for the prediction according to the risk level of each behavioral pattern, and also to set the output conditions according to the importance of the prediction.

[0056] <Combining Multiple Prediction Methods for Trajectory Prediction> In the present embodiment described so far, predictions are made using a single prediction model, but multiple prediction methods can be combined to predict one trajectory.

[0057] For example, consider trajectory prediction when the prediction target vehicle V7 in FIG. 6 turns right. Here, the output conditions are set as follows: the number of predictions is 1, the prediction period is 5 seconds, and the time resolution is set as 0.3 seconds for the first 3 seconds (section A1 in the figure) and 1.0 seconds for the remaining 2 seconds (section A2 in the figure). In this case, the trajectory prediction for the first 3 seconds is performed using a prediction model. The trajectory prediction for the remaining 2 seconds is performed using linear prediction, assuming that the vehicle moves linearly. This reduces the number of prediction points and the computational load compared to when a prediction model is used for all trajectory predictions for the 5 seconds. Furthermore, because linear prediction has a lower computational load than predictions using a machine learning model, the computational load associated with predicting a single point can also be reduced.

[0058] <When an Upper Limit is Set on the Prediction Score> In addition to the embodiment described above, the present invention allows for setting an upper limit on the prediction score and setting the prediction output conditions. Here, it is assumed that calculations can be performed sufficiently quickly if the total prediction score is up to 10 points. For simplicity, the prediction period is set to 3.0 seconds, and the number of predicted trajectories is fixed at one for each behavior pattern. Consider the case of the prediction target vehicle V8 in Figure 7 , where the calculated probability of a left turn is 50%, the probability of going straight is 30%, and the probability of a right turn is 20%. In this case, 10 points are multiplied by the occurrence probability to assign prediction scores of 5 points for a left turn, 3 points for a straight turn, and 2 points for a right turn. The prediction period is divided by each score, and the respective time resolutions are set to 0.6 seconds for a left turn, 1.0 seconds for a straight turn, and 1.5 seconds for a right turn. Finally, each time resolution is used as an input to the prediction model to predict trajectories T7, T8, and T9 for each behavior pattern. This allows trajectories to be predicted by appropriately allocating computational resources within a range that does not exceed the computational load tolerance.

[0059] <Effects of this embodiment> According to the behavior prediction device of this embodiment described above, the amount of calculation can be reduced by dynamically changing the prediction output conditions according to the possible behavior patterns of the prediction target. Note that the present invention is not limited to the above embodiment, and various modifications that can be understood by those skilled in the art can be made within the scope of the present invention.

[0060] 1 Behavior prediction device 10 Object detection unit 11 Surrounding information management unit 12 Behavior prediction unit 12a Behavior pattern identification unit 12b Output condition setting unit 12c Prediction processing unit V1 to V8 Prediction target vehicle T1 to T9 Predicted trajectory

Claims

1. A behavior prediction device comprising: an object detection unit that detects a predicted object based on external information from an external sensor; and a behavior prediction unit that predicts the behavior of the predicted object, wherein the behavior prediction unit has: a behavior pattern identification unit that identifies a behavior pattern that the predicted object may perform based on at least one of the state of the predicted object or the surrounding environment of the predicted object; an output condition setting unit that sets prediction output conditions for each of the identified behavior patterns; and a prediction processing unit that outputs a prediction result for each of the behavior patterns based on the set output conditions.

2. A behavior prediction device as described in claim 1, characterized in that the behavior pattern identification unit identifies the behavior pattern using at least one of the road structure, traffic rules, signs, and traffic light indications on the path of the predicted object.

3. A behavior prediction device according to claim 1, wherein the behavior pattern specification unit predicts an occurrence probability for each of the behavior patterns.

4. A behavior prediction device according to claim 1, wherein the behavior pattern identification unit predicts a degree of risk for each of the behavior patterns.

5. A behavior prediction device as described in claim 1, characterized in that the output condition setting unit sets, for each of the behavior patterns, the number of predicted trajectories generated that include prediction points that predict the behavior of the prediction target in a time series as the output condition.

6. A behavior prediction device as described in claim 1, characterized in that the output condition setting unit sets, for each of the behavior patterns, the time interval between prediction points of a prediction trajectory that includes prediction points that predict the behavior of the prediction target in a time series as the output condition.

7. A behavior prediction device as described in claim 1, characterized in that the output condition setting unit sets, for each of the behavior patterns, a prediction period of a predicted trajectory that includes prediction points that predict the behavior of the prediction object in a time series as the output condition.

8. The behavior prediction device according to claim 1, wherein the prediction processing unit combines a plurality of prediction methods to output a prediction result.

9. A vehicle equipped with the external sensor, a vehicle control device that controls a steering system, a drive system, and a braking system, and the behavior prediction device described in claim 1, characterized in that the vehicle control device controls the steering system, the drive system, and the braking system based on the prediction results of the behavior prediction device.

10. A behavior prediction method executed by a behavior prediction device equipped with a CPU, comprising: an object detection step of detecting a predicted object based on external information from an external sensor; a behavior pattern identification step of identifying a behavior pattern that the predicted object may perform based on the state of the predicted object or the surrounding environment of the predicted object; an output condition setting step of setting prediction output conditions for each of the identified behavior patterns; and a prediction processing step of outputting a prediction result for each of the behavior patterns based on the set output conditions, thereby predicting the behavior of the predicted object.

Images