Action prediction device, vehicle and action prediction method

Abstract

One object of the present invention is to provide an action prediction device and an action prediction method for reducing an action prediction calculation amount that are generally applicable regardless of whether a factor for a change in object behavior is present.An action prediction device comprises: an object detection unit that detects a prediction target object based on external information from an external sensor, and an action prediction unit that predicts an action of the prediction target object, wherein the action prediction unit comprises: an action pattern specification unit that specifies action patterns that can be assumed by the prediction target object based on at least one state of the prediction target object and an environment of the prediction target object, an output condition setting unit that sets a prediction output condition for each of the specified action patterns, and a prediction processing unit that outputs a prediction result for each of the action patterns based on the set output condition.

Description

Technical field

[0001] The present invention relates to an action prediction device, a vehicle and an action prediction method for predicting an action of another person in order to implement a preventive safety function or an automatic driving function of a motor vehicle. Technical background

[0002] To implement a preventive safety function or an automated driving function in a motor vehicle, it is effective to predict the future action of a surrounding object detected by an in-vehicle sensor. When predicting the action of the surrounding object, interactions between objects can be taken into account, or multiple possible actions can be predicted. These action prediction techniques can achieve highly accurate predictions but require complex calculations, which increases the computational load.

[0003] As a measure to reduce the computational load when predicting the action of the surrounding object, a behavior prediction method from PTL 1 is known. For example, PTL 1 describes the problem in its summary as: "The aim is to accurately predict behavior and reduce the computational load," and the solution as: "Based on objects around a host vehicle or a driving environment around the host vehicle, it is determined whether or not a change factor exists that leads to a change in the behavior of a predicted object. The behavior of the predicted object is predicted based on a determination of whether a change factor exists and on information obtained from the objects around the host vehicle."If no change factor is found, a short-term forecast is performed, in which the behavior of the predicted object is predicted solely based on information obtained from the predicted object itself. If a change factor is found, a long-term forecast is performed, in which the behavior of the predicted object is predicted based on information obtained from the predicted object and information obtained from surrounding objects around the predicted object.

[0004] As described above, PTL 1 discloses a method that reduces the computational load in an environment where no change factor is present by determining whether a change factor is present (such as whether another object is crossing or not) and by performing a long-term prediction with a high computational load when a change factor is present, and a short-term prediction with a low computational load when no change factor is present. This makes it possible to limit the scene in which behavior is predicted with high accuracy, thereby reducing the overall computational load compared to a behavior prediction method where prediction is always performed with a high computational load. List of patent literature

[0005] PTL 1: JP 2020-166510 A Summary of the invention: Technical problem

[0006] However, PTL 1 describes in paragraph 0081: “When calculating route candidates, the predictive processing unit 16 establishes lane centerlines La1 and Lb2 in the middle of the lanes. For a straight-ahead route candidate Rb2, the predictive processing unit 16 calculates route candidate Rb2 so that it is parallel to lane centerline La2, based on the current position of the other vehicle Vb1. On the other hand, for route candidate Rb1 to change lanes, the predictive processing unit 16 calculates a distance traveled by the other vehicle Vb1 until the lane change is complete, based on the time required to complete the lane change and the current speed of the other vehicle Vb1. The time required for the lane change is, for example, a value pre-determined from the driving tendencies of general drivers and is, for example, 3 seconds.”Prediction Processing Unit 16 establishes a position on the lane centerline La1, based on the calculated travel distance, when the lane change is complete. Then, Prediction Processing Unit 16 combines the current travel position and the position when the lane change is complete with a specific curve, for example, a clothoid curve, and designates this as the route candidate Rb1. Long-term prediction in PTL 1 is a process that requires a considerable computational load and involves a large number of route candidates, such as route candidates Rb1 to Rb4 and Rc1. Fig. 6 of PTL 1, can be predicted. Therefore, the computational load reduction effect of the technology of PTL 1 can only be obtained in an environment where short-term forecasting without a change factor is possible.

[0007] Therefore, with regard to such a problem, it is an object of the present invention to provide an action prediction device and an action prediction method for reducing an action prediction calculation amount that are generally applicable, regardless of whether a factor for a change in object behavior is present. Solution to the problem

[0008] An action prediction device comprises: an object detection unit that detects a prediction target object based on external information from an external sensor, and an action prediction unit that predicts an action of the prediction target object, wherein the action prediction unit comprises: an action pattern specification unit that specifies action patterns that can be assumed by the prediction target object based on at least one state of the prediction target object and an environment of the prediction target object, an output condition setting unit that sets a prediction output condition for each of the specified action patterns, and a prediction processing unit that outputs a prediction result for each of the action patterns based on the set output condition. Advantageous effects of the invention

[0009] According to the present invention, the computational effort can be reduced by dynamically changing the prediction output condition according to the action pattern that the object can adopt. Brief description of the drawings

[0010] They show: Fig. 1 a functional block diagram showing a configuration of an action prediction device according to an embodiment, Fig. 2 a flowchart showing procedures for an action prediction process according to one embodiment, Fig. 3A is an example in which a pattern of action is specified based on a position and a connection of a lane, Fig. 3B is an example in which a pattern of action is specified based on a traffic light display. Fig. 3C is an example where an action pattern is specified based on an interaction with another object. Fig. 4. An example where an output condition is changed based on the probability of an action pattern occurring. Fig. 5. An example where an output condition is changed based on the risk level of a pattern of action. Fig. 6. An example where a route is predicted by combining several prediction techniques. Fig. 7 an example in which a prediction is made by assigning the number of prediction points to each action pattern, and Fig. Figure 8 shows a hardware configuration diagram illustrating an action prediction device according to one embodiment. Description of embodiments

[0011] Below, an action prediction device 1 according to an embodiment of the present invention is described in detail. <Konfiguration der Handlungsvorhersagevorrichtung 1>

[0012] Fig. Figure 1 is a functional block diagram showing the configuration of the action prediction device 1 according to the present embodiment. As shown in Fig. As shown in Figure 1, the action prediction device 1 comprises an object detection unit 10, an environment information management unit 11, and an action prediction unit 12. Additionally, the action prediction unit 12 comprises an action pattern specification unit 12a, an output condition setting unit 12b, and a prediction processing unit 12c.

[0013] The action prediction device 1 can be applied to a vehicle with a preventive safety function or an automated driving function, or it can be applied to a device other than a vehicle, such as a surveillance camera. In the following, it is assumed that the action prediction device 1 is mounted on a vehicle. In the following description, the vehicle on which the action prediction device 1 is mounted can be referred to as the host vehicle.

[0014] Fig. Figure 8 is a hardware configuration diagram of a vehicle 9 equipped with the action prediction device 1 according to the present embodiment. The vehicle 9 has an action prediction device 1, a vehicle control device 2, an external sensor 3, a display 4, a communication device 5 and a GPS receiver 6, which are interconnected by an in-vehicle network 7.

[0015] The action prediction device 1 comprises a CPU 1A, which is a central processing unit, a ROM 1B, which is a read-only memory device, a RAM 1C, which is volatile memory similar to DRAM, and a flash memory 1D, which is a non-volatile memory device. The action prediction device 1 implements various functional units by having the CPU 1A load programs stored in the ROM 1B into the RAM 1C and execute the programs. The vehicle control device 2 is an electronic control device that, similar to the action prediction device 1, comprises a CPU and a memory device, and controls functions such as a steering system, a drive system, and a braking system by having the CPU execute a control program stored in the memory device.The vehicle control device 2 can control the steering system, the drive system, the braking system, and the like, based on the prediction result of the action prediction device 1. When the prediction result is used to control each system of the host vehicle, the host vehicle's route can be planned so that it does not interfere with a predicted route of another person, thereby achieving safe automated driving with a reduced risk of collision.

[0016] The external sensor 3 can be, for example, a LiDAR, a camera, a millimeter-wave radar, or the like. A detection result from the external sensor 3 is entered into the object detection unit 10 of the action prediction device 1.

[0017] Display 4 is, for example, a display in a gauge panel, a head-up display, or the like, but is not limited to these forms. A prediction result from the action prediction device 1 can be reported to a driver via display 4. By notifying the driver of a vehicle predicted to be at risk of coming into contact with the host vehicle, the driver can continue driving while paying particular attention to the condition of the reported vehicle.

[0018] The communication device 5 is a communication module that performs wireless communication with the area outside the vehicle 9. The communication device 5 is designed to communicate with a server and, for example, collects map information about the environment of the host vehicle.

[0019] The GPS receiver 6 is a receiver for a global positioning system (GPS) that measures the position of a vehicle based on radio waves from satellites.

[0020] The units of the action prediction device 1 are described in detail below, sequentially. <<Objekterfassungseinheit 10> >

[0021] The object detection unit 10 is a functional unit that uses external information from an external sensor to detect moving objects such as other vehicles and people, including other vehicles, motorcycles, bicycles, and pedestrians, as well as stationary objects, including parked vehicles and buildings. Furthermore, the object detection unit 10 tracks a moving object and estimates state variables such as the object's position, speed, and orientation. <<Umgebungsinformations-Managementeinheit 11> >

[0022] The Environmental Information Management Unit 11 is a functional unit that manages environmental information around the host vehicle. This environmental information includes road structures, traffic signals and signs, lanes, and similar data. Environmental information can be pre-stored as map data, acquired through communication with the outside, or detected by an external sensor mounted on the vehicle. For example, acquisition by the external sensor involves feeding a camera image into a trained machine learning model for semantic segmentation. <<Handlungsvorhersageeinheit 12> >

[0023] The action prediction unit 12 is a functional unit that predicts one or more future actions of an object. According to the present embodiment, a sequence of points along a route that an object will pass in the future is predicted as a future action. Each point in the route sequence has a position coordinate and a time at a future point as state variables, and a route is formed by connecting the points in time series. To predict the route sequence, it is necessary to define a prediction period and a time resolution of the point sequence. Additionally, it is necessary to define how many route sequences are to be predicted. Therefore, the action prediction unit 12 sets the number of route sequences to be predicted, the prediction period, and the time resolution as prediction output conditions and executes a prediction.The following section describes in detail the action pattern specification unit 12a, the output condition setting unit 12b and the prediction processing unit 12c, which execute these processes sequentially.

[0024] The action pattern specification unit 12a specifies one or more action patterns that the object can adopt, based on at least one piece of information about the object's state or its environment. The action pattern refers to an expected type of action for the object, such as turning right or left, driving straight ahead, stopping, or parking. Additionally, the action pattern specification unit 12a can also calculate accompanying information such as the probability of occurrence or the risk level of each action pattern. The action pattern specification procedure and the accompanying information calculation procedure can be conditional branching based on simple rules, machine learning techniques, or statistical methods. Here, it is desirable to specify the action pattern using a procedure with a low computational load.This is because if a procedure with a high computational load is used in the action pattern specification unit 12a, the overall computational load cannot be reduced, even if the computational load is reduced in a subsequent stage of the prediction process.

[0025] The output condition setting unit 12b sets prediction output conditions based on each type of action pattern, accompanying information, computer computing resources, and the like.

[0026] The prediction processing unit 12c executes a prediction process according to the output conditions defined by the output condition setting unit 12b. The prediction method can be a machine learning technique, an optimal route search technique, or a rule-based conditional branching procedure. Furthermore, the prediction processing unit 12c can contain multiple prediction methods to handle dynamically defined output conditions. For example, it is possible to maintain a machine learning model suitable for performing a prediction with low temporal resolution and a machine learning model suitable for performing a prediction with high temporal resolution to perform predictions with different temporal resolutions. Additionally, multiple prediction techniques can be used in combination to predict a route point sequence. <Beispiel des Betriebs der Handlungsvorhersagevorrichtung 1>

[0027] Next, an example of the operation of the action prediction device 1 according to the present embodiment will be given with reference to a flowchart of Fig. 2 described in detail. The operation of the action prediction device 1 is described below using a case in which an in-vehicle camera is used as an external sensor and the object for which a prediction is sought is, for example, another vehicle in the vicinity of the host vehicle.

[0028] First, in procedure P1, the object acquisition unit 10 detects an object for which a prediction is targeted (hereinafter referred to as the prediction target object) using a trained machine learning model that utilizes camera images. Additionally, state variables such as the position, velocity, and orientation of the prediction target object are estimated by tracking the prediction target object in time series and applying a Kalman filter.

[0029] Next, in procedure P2, the Environmental Information Management Unit 11 (EIM) gathers information about the host vehicle's surroundings. Specifically, EIM specifies a current host vehicle position based on GPS information and retrieves map information about the host vehicle's surroundings from the server. Additionally, EIM gathers road shapes and traffic light indicators using the trained machine learning model, which utilizes camera images.

[0030] In procedure P3, the action pattern specification unit 12a specifies an action pattern that can be adopted by the predicted target object. Examples of action pattern specification procedures are given below with reference to the Fig. 3A to 3C are described. It should be noted that, although not shown, it is assumed that the host vehicle is located in a position where each of the vehicles in the drawings can be observed by an in-vehicle camera (external sensor).

[0031] Fig. 3A is an example where an action pattern is specified based on the position and connection of a lane. In this example, there are only straight-ahead lanes in front of a predicted target vehicle V1, so going straight is specified as an action pattern. Additionally, there are two lanes in front of a predicted target vehicle V2 for turning left and going straight, so turning left and going straight are specified as action patterns. Furthermore, there are three lanes in front of a predicted target vehicle V3 for turning left, going straight, and turning right, so turning left, going straight, and turning right are specified as action patterns.

[0032] Fig. 3B is an example where an action pattern is specified based on a traffic light display in front of a predicted target vehicle V4. In this example, stopping is specified as the action pattern when the traffic light is red, going straight ahead is specified as the action pattern when the traffic light is green, and stopping and going straight ahead are specified as the action pattern when the traffic light is yellow.

[0033] Fig. 3C is an example where an action pattern is specified based on a positional relationship between a predictive target vehicle V5, an oncoming vehicle, and an obstacle. In the situation shown in the diagram, the road is assumed to be narrowed by obstacles, allowing only one vehicle to pass at a time. Therefore, this example specifies two possible action patterns for the predictive target vehicle V5: one in which the predictive target vehicle V5 passes between the obstacles first, and one in which the oncoming vehicle passes between the obstacles first.

[0034] In procedure P4, the action pattern specification unit 12a calculates a probability of occurrence for each action pattern. The following is a procedure for calculating a probability of occurrence for an action pattern for each of the vehicles in the Fig. 3A to 3C described.

[0035] For the predictive target vehicle V1 in Fig. 3A calculates the action pattern specification unit 12a, since the action pattern is only driving straight ahead, a probability of occurrence for driving straight ahead is 100%.

[0036] For the predictive target vehicle V2 and the predictive target vehicle V3 in Fig. 3A The probability calculation procedure differs depending on whether the external sensor can detect the flashing of the turn signal of each predicted target vehicle. First, a case is considered in which the flashing of the left-turn turn signal is detected. In this case, a probability of occurrence for each action pattern is determined by comparing a probability of occurrence for left turns with the accuracy rate of detecting the flashing of the left-turn turn signal and distributing the remaining probability equally among the other action patterns. If the flashing of the right-turn turn signal is detected, probabilities are calculated similarly.

[0037] If the flashing of the turn signal cannot be detected, the probability of occurrence of each action pattern is calculated based on the vehicle's orientation. Therefore, the action pattern specification unit 12a maintains a table that specifies a relationship between the distance of a vehicle to a traffic light and the probability of occurrence of each action pattern. This table is pre-generated by collecting data on vehicles entering intersections. As a result, even if the flashing of the turn signal cannot be detected, the action pattern specification unit 12a can obtain the probability of occurrence of each action pattern based on the orientation of the predicted target vehicle, estimated by the object detection unit 10, and the table.

[0038] For the V4 predictive target vehicle in Fig. 3B sets the action pattern specification unit 12a to a 100% probability of stopping when it detects that the traffic light is red, and a 100% probability of proceeding straight ahead when it detects that the traffic light is green. On the other hand, when it detects that the traffic light is yellow, the action pattern specification unit 12a calculates a probability based on the distance from the vehicle to the traffic light. Therefore, the action pattern specification unit 12a maintains a pre-existing table that specifies a relationship between the distance from a vehicle to a traffic light and the probability of each action pattern occurring. This table is pre-generated by collecting data on vehicles entering intersections.As a result, the action pattern specification unit 12a can calculate the distance from the vehicle to the traffic light using the vehicle position information estimated by the object detection unit 10 and the traffic light position information acquired by the environment information management unit 11, and obtain the probability of occurrence of each action pattern using the calculated distance together with the table.

[0039] For the V5 predictive target vehicle in Fig. 3C's action pattern specification unit 12a calculates the probability of each action pattern occurring based on the distance between the vehicle and a straight line defined by connecting the two obstacles with a straight line. Therefore, action pattern specification unit 12a maintains a table that specifies the relationship between the distance to an obstacle and the probability of each action pattern occurring. This allows action pattern specification unit 12a to calculate the distance using the vehicle position information estimated by object detection unit 10 and the obstacle position information, and to obtain the probability of each action pattern occurring using the calculated distance along with the table.

[0040] The following is described using the predictive target vehicle V3 in Fig. 3A describes an example in which prediction output conditions corresponding to each action pattern are defined in procedures P5, P6 and P7, and routes T1, T2 and T3, as in Fig. As shown in Figure 4, the outcomes are predicted in Procedure P8. In Procedure P4, it is assumed that the probability of each action pattern occurring is calculated as 80% for turning left, 17% for going straight, and 3% for turning right.

[0041] In procedure P5, output condition setting unit 12b determines the number of routes to be predicted for each action pattern. Therefore, output condition setting unit 12b maintains a table that specifies the correspondence between a probability of occurrence and the number of routes to be predicted. For example, 0 routes are predicted in a case where the probability of occurrence of the action pattern is less than 10%, 1 route is predicted in a case where the probability of occurrence of the action pattern is 10% or more and less than 50%, and 2 routes are predicted in a case where the probability of occurrence of the action pattern is 50% or more.Taking into account the probabilities calculated in procedure P4, in the present example the output conditions for the number of routes to be predicted are set such that 2 routes are to be predicted for left turns, 1 route is to be predicted for going straight ahead and 0 routes are to be predicted for right turns.

[0042] In procedure P6, output condition setting unit 12b defines a time resolution for each predicted route. Output condition setting unit 12b maintains a table that specifies the correspondence between a type of action pattern and a time resolution. For example, it is assumed that point sequences are predicted at intervals of 1.0 seconds if the action pattern is driving straight, and at intervals of 0.3 seconds if the action pattern is turning left or turning right. Based on the above, the interval for turning left is set to 0.3 seconds and the interval for driving straight is set to 1.0 seconds.

[0043] In procedure P7, output condition setting unit 12b determines the prediction length of each route to be predicted. Therefore, output condition setting unit 12b maintains a table that specifies the correspondence between a probability of occurrence and a prediction length. For example, if the probability of the action pattern occurring is less than 50%, a sequence of points is predicted for 2 seconds, and if the probability of the action pattern occurring is 50% or more, a sequence of points is predicted for 3 seconds. Therefore, in this example, using the probabilities calculated in procedure P4, the predicted route for turning left is set to 3 seconds, and the predicted route for going straight is set to 2 seconds.

[0044] In procedure P8, the prediction processing unit 12c predicts routes based on the output conditions defined above. Therefore, the prediction processing unit 12c has a prediction model that can change the time resolution. The state variables of the predicted target vehicle, the road shape, and the time resolution are input into the prediction model, and the prediction model outputs candidates for the position of the predicted target vehicle one time step ahead and a movement probability corresponding to each position. Here, the resolution of a time step is determined by the time resolution of the input. The prediction model is an encoder / decoder model that uses a well-known Long Short-Term Memory (LSTM). The encoder encodes the input information, and the decoder then receives position coordinates.

[0045] A method for predicting two routes corresponding to the left-turn action pattern is described using a prediction model. First, a procedure for obtaining an initial route is described. The state variables of the predicted target object obtained in procedure P1, the road shape obtained in procedure P2, and the time resolution determined in procedure P6 are input to obtain candidates for the position of the predicted target object one step ahead and their movement probabilities. Next, the position with the highest movement probability is input into the prediction model to obtain the position of the predicted target object in the next step. This operation is repeated to obtain a sequence of points for three seconds as the desired length as the first predicted route.

[0046] Next, a procedure for obtaining a second predicted route is described. The current position of the predicted target, obtained in procedure P1, is input to generate candidates for the target's position one step ahead and their movement probabilities. The position with the second-highest movement probability is then entered into the prediction model to obtain the target's position in the next step. The same operation is then repeated, this time entering the position with the highest movement probability into the prediction model. Finally, a sequence of points for three seconds, representing a desired length, is obtained as the second predicted route.

[0047] These operations make it possible to obtain two routes, T1 and T2, that meet the desired output conditions. For driving straight ahead, with an input time resolution of 1.0 second, a route T3 is obtained for 2 seconds using a similar procedure to the one used to obtain the first route for a left turn.

[0048] Based on the foregoing, the action prediction device 1, according to the present embodiment, specifies one or more action patterns that can be adopted by the object for which a prediction is sought, defines a prediction output condition for each action pattern, and outputs a prediction for each action pattern. This makes it possible to selectively use a prediction method with a low computational load and a prediction method with a high computational load. The computational load can be reduced compared to a case where a prediction is performed using a technique with a consistently high computational load. <Wirkung und Effekt des vorstehend beschriebenen Betriebsbeispiels>

[0049] One or more action patterns that the object for which a prediction is sought can adopt are specified by the preceding operational example. This allows a subsequent process to be carried out according to the action pattern.

[0050] The probability of occurrence is predicted for each action pattern using the preceding operational example. This makes it possible to define a prediction output condition based on the probability of occurrence of each action pattern, and an output condition based on the importance of the prediction.

[0051] The number of forecast outputs corresponding to each action pattern is determined by the preceding operational example. This makes it possible to reduce the computational load by decreasing the number of forecasts corresponding to the low-priority action pattern or by not making a forecast for the low-priority action pattern.

[0052] A time resolution corresponding to each prediction result is defined by the preceding operational example. By reducing the time resolution according to the action pattern, the number of calculations of the prediction model can be reduced, thus lowering the computational load.

[0053] A forecast period corresponding to each forecast result is defined by the preceding operational example. This allows the number of operating times of the forecast model and the computational load to be reduced by shortening the forecast period for a less important forecast. <rechenlastreduzierungseffekt>

[0054] A reduced computational load resulting from the present embodiment is estimated. For the sake of simplicity, and assuming that the number of prediction points and the computational load are proportional to each other, the predictions performed for 3 seconds at intervals of 0.3 seconds for the three action patterns—namely, turning left, driving straight ahead, and turning right, each involving two driving paths—are compared with the predictions according to the present embodiment. In the former case, 6 driving paths are predicted, and a total of 60 points are predicted, since each driving path has 10 prediction points.On the other hand, in the latter case, a total of 3 routes are predicted, including two routes for left turns and one for going straight ahead, resulting in a total of 22 predicted points, since each route corresponding to a left turn has 10 prediction points and the route corresponding to going straight ahead has 2 prediction points. Therefore, operation according to the embodiment described above can reduce the computational load to approximately 1 / 3. <Fall, in dem das Risikoniveau anstelle der Eintrittswahrscheinlichkeit beim Bestimmen der Ausgabebedingung berücksichtigt wird>

[0055] According to the foregoing embodiment, the output condition for the subsequent stage is determined using the probability of occurrence for each action pattern; however, the risk level can be used instead of the probability of occurrence. Here, the risk level of the action pattern is obtained in three levels: low, medium, and high. Fig. Figure 5 shows a prediction result in a case where going straight and turning right are specified as action patterns. As shown, an oncoming vehicle approaches from in front of a predicted target vehicle V6, and there is a possibility of a right-turn / straight-ahead collision if the predicted target vehicle V6 turns right. Therefore, if an oncoming vehicle is in the opposite lane of a T-junction, the risk level for turning right is determined to be high. On the other hand, if the predicted target vehicle V6 is going straight, the possibility of a collision with an oncoming vehicle is low, and the risk level for going straight is determined to be low. Then, a route T4 for an action pattern with a low risk level is set to have a lower time resolution than a route T5 for an action pattern with a high risk level, and a prediction is made.

[0056] Based on the above, a risk level is predicted for each action pattern. This makes it possible to define a forecast output condition according to the risk level of each action pattern, and an output condition according to the importance of the forecast. <Fall, in dem mehrere Vorhersagetechniken für die Fahrstreckenvorhersage kombiniert werden>

[0057] According to the embodiment described so far, a prediction is carried out using a single prediction model; however, a prediction can also be carried out by combining several prediction techniques when a journey route is predicted.

[0058] For example, a route prediction is considered when a predicted target vehicle V7 is in Fig. Turns right at point 6. The output conditions are assumed to be 1 for the number of predictions, 5 seconds for the prediction period, and 0.3 seconds for the first 3 seconds (section A1 in the diagram) and 1.0 second for the remaining 2 seconds (section A2 in the diagram). In this case, the route prediction for the first 3 seconds is performed using the predictive model. For the remaining 2 seconds, a linear prediction is performed, assuming the vehicle moves linearly. This reduces the number of prediction points and the computational load compared to using the predictive model for all route predictions over the 5-second period.Additionally, linear prediction has a lower computational load than prediction using the machine learning model, which makes it possible to reduce the computational load in terms of predicting a point. <Fall, in dem eine Obergrenze für die Anzahl der Vorhersagepunkte festgelegt wird>

[0059] In addition to the embodiment described above, the prediction output condition can be determined according to the present invention by setting the upper limit of the number of prediction points. Here, it is assumed that the calculation can be performed at a sufficiently high speed if the total number of prediction points is up to 10. For the sake of simplicity, the number of routes to be predicted is fixed at one for each action pattern, while the prediction period is 3.0 seconds. A case is considered in which, as with a prediction target vehicle V8 in Fig. In case 7, the probability of turning left is calculated as 50%, the probability of going straight ahead as 30%, and the probability of turning right as 20%. In this case, the probabilities are multiplied by 10 points, resulting in 5 points for turning left, 3 points for going straight ahead, and 2 points for turning right as the number of prediction points. The prediction period is divided by each number of points, and the time resolution is set to 0.6 seconds for turning left, 1.0 second for going straight ahead, and 1.5 seconds for turning right. Finally, the journey distances T7, T8, and T9 are predicted for each action pattern using each time resolution as an input to the prediction model.This makes it possible to predict travel routes by appropriately allocating computing resources within a range where the computing load does not exceed the permissible value. <Wirkungen der vorliegenden Ausführungsform>

[0060] The action prediction device according to the present embodiment described above is able to dynamically modify the prediction output condition according to the action pattern that the object for which a prediction is sought can adopt, thereby reducing the computational effort. It should be noted that the present invention is not limited to the above embodiment and various modifications, which can be understood by those skilled in the art, can be made within the scope of protection of the present invention. Reference symbol list 1 Action prediction device 10 object detection units 11 Environmental Information Management Unit 12 Action Prediction Unit 12a Action Pattern Specification Unit 12b Output condition setting unit 12c Predictive Processing Unit V1 to V8 Predictive Target Vehicle T1 to T9 predicted route QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2020-166510 A

[0005] < / rechenlastreduzierungseffekt>

Claims

[1] Action prediction device comprising: an object detection unit that detects a predictive target object based on external information from an external sensor, and an action prediction unit that predicts an action of the predicted target object, the action prediction unit has: an action pattern specification unit that specifies action patterns that can be assumed by the predicted target object based on at least one state of the predicted target object and an environment of the predicted target object, an output condition setting unit that defines a prediction output condition for each of the specified action patterns, and a prediction processing unit that outputs a prediction result for each of the action patterns based on the specified output condition. [2] Action prediction device according to claim 1, wherein the action pattern specification unit specifies the action patterns using at least one of a road structure, a traffic rule, a sign and a traffic light on a route of the prediction target object. [3] Action prediction device according to claim 1, wherein the action pattern specification unit predicts a probability of occurrence for each of the action patterns. [4] Action prediction device according to claim 1, wherein the action pattern specification unit predicts a risk level for each of the action patterns. [5] Action prediction device according to claim 1, wherein the output condition setting unit defines as the output condition the number of predicted routes to be generated, each having prediction points obtained by predicting the action of the predicted target object in time series for each of the action patterns. [6] Action prediction device according to claim 1, wherein the output condition setting unit defines as the output condition a time interval between prediction points of a predicted route, wherein the prediction points are obtained by predicting the action of the predicted target object in time series for each of the action patterns. [7] Action prediction device according to claim 1, wherein the output condition setting unit defines as the output condition a prediction period of a predicted route, which has prediction points obtained by predicting the action of the predicted target object in time series for each of the action patterns. [8] Action prediction device according to claim 1, wherein the prediction processing unit outputs a prediction result by combining several prediction techniques. [9] Vehicle, comprising: an external sensor a vehicle control device that controls a steering system, a drive system and a braking system, and the action prediction device according to claim 1, wherein the vehicle control device controls the steering system, the drive system and the braking system based on a prediction result from the action prediction device. [10] Action prediction method performed by an action prediction device which includes a CPU to predict an action of a prediction target object, wherein the action prediction method includes: an object detection step to detect a predictive target object based on external information from an external sensor, an action pattern specification step to specify action patterns that can be assumed by the predicted target, based on a state of the predicted target or an environment of the predicted target, an output condition setting step to define a prediction output condition for each of the specified action patterns, and a prediction processing step to output a prediction result for each of the action patterns based on the specified output condition.

Images