Autonomous driving system and autonomous driving method

Abstract

Provided is an autonomous driving system capable of appropriately planning the behavior of a host vehicle while reducing a calculation load as a whole, by predicting a future route with high accuracy even for a moving object that indirectly affects a driving plan of the host vehicle. The autonomous driving system for predicting a future route of a moving object around a host vehicle comprises: a periphery recognition part that recognizes an environment around the host vehicle on the basis of an output from a sensor; a simple prediction part that predicts the future route of a prediction-target object by using a simple AI model; an advanced prediction part that predicts the future route of the prediction-target object by using an advanced AI model; an object correlation acquisition part that acquires the correlation between the prediction-target object and an object other than the host vehicle; a prediction scheme selection part that selects the simple prediction part or the advanced prediction part on the basis of the correlation; and a vehicle control part that controls the host vehicle while taking into consideration the future route of the prediction-target object that has been predicted by the simple prediction part or the advanced prediction part.

Description

Autonomous driving system and autonomous driving method

[0001] The present invention relates to an autonomous driving system and an autonomous driving method that predict the future paths of moving objects around the vehicle and control the behavior of the vehicle according to the prediction results.

[0002] In recent years, vehicles equipped with advanced driver-assistance systems (ADAS) and autonomous driving (AD) functions have become increasingly common. Furthermore, as a prerequisite for realizing advanced ADAS and AD, vehicles equipped with artificial intelligence (AI) that predicts the future positions (coordinates) of vehicles and other traffic participants around the vehicle in a time series (in other words, predicts the future routes of traffic participants) are also becoming widespread.

[0003] However, because the computing power of in-vehicle AI is limited, a technology has been proposed that, when predicting the future paths of moving objects around the vehicle, select one of several prediction methods with different computational loads depending on the situation, thereby ensuring appropriate prediction accuracy while reducing the overall computational load.

[0004] For example, the abstract of Patent Document 1 states that the problem is "to reduce the computational load while ensuring accuracy in predicting the future position of moving objects," and describes the solution as "a moving object behavior prediction unit 202 predicts the future positions of pedestrians 603 and 604 around the vehicle 601. The moving object behavior prediction unit 202 is a moving object prediction device comprising a simplified prediction unit 401 that makes a simplified prediction of the future positions of pedestrians 603 and 604, an individual prediction unit 304 that makes a more accurate prediction of the future position of pedestrian 604 than the simplified prediction unit 401, and an object allocation unit 302 that allocates pedestrians 604 whose future positions are to be precisely predicted by the individual prediction unit 304 according to the results of the simplified prediction."

[0005] Thus, in Patent Document 1, the future position of moving objects with a high probability of interfering with the vehicle is predicted with high accuracy by individual prediction, while the future position of other moving objects is predicted simply by simplified prediction, thereby achieving both prediction accuracy and a reduction in computational load.

[0006] Japanese Patent Publication No. 2018-124663

[0007] However, the technology described in Patent Document 1 only provides a simplified prediction of the future position of a moving object that is unlikely to directly interfere with the vehicle. Therefore, it could not foresee interference between the vehicle and other moving objects caused by the behavior of such a moving object that cannot be predicted by the simplified prediction.

[0008] Therefore, the present invention aims to provide an automated driving system and an automated driving method that can appropriately plan the behavior of the vehicle while reducing the overall computational load by accurately predicting the future paths of moving objects that indirectly affect the driving plan of the vehicle.

[0009] To solve the above problems, the present invention provides an autonomous driving system that predicts the future path of a moving object around its own vehicle, comprising: an environment recognition unit that recognizes the environment around the vehicle based on the output of a sensor; a simple prediction unit that predicts the future path of a target object using a simple AI model; an advanced prediction unit that predicts the future path of a target object using an advanced AI model; an object correlation acquisition unit that acquires the correlation between the target object and an object other than the vehicle; a prediction method selection unit that selects the simple prediction unit or the advanced prediction unit based on the correlation; and a vehicle control unit that controls the vehicle taking into account the future path of the target object predicted by the simple prediction unit or the advanced prediction unit.

[0010] According to the autonomous driving system and autonomous driving method of the present invention, by accurately predicting the future paths of moving objects that indirectly affect the driving plan of the vehicle, the overall computational load can be reduced while appropriately planning the behavior of the vehicle.

[0011] Hardware configuration diagram of the autonomous driving system of Example 1. Functional block diagram of the autonomous driving system of Example 1. Conceptual diagram explaining the difference in predicted paths between the simplified prediction method and the advanced prediction method. Plan view showing the time-series changes in the correlation between moving objects and the selected prediction method in Example 1. Explanatory diagram of the collision margin time (TTC). Plan view showing the correlation between moving objects and the selected prediction method in Example 2. Plan view showing the correlation between moving objects and the selected prediction method in Example 3. Plan view showing the correlation between moving objects and the selected prediction method in Example 4. Functional block diagram of the autonomous driving system of Example 5. Timing chart of the prediction processing by the behavior prediction unit in Example 5. Functional block diagram of the autonomous driving system of Example 6.

[0012] Hereinafter, embodiments of the autonomous driving system of the present invention will be described with reference to the drawings.

[0013] First, an embodiment 1 of the autonomous driving system of the present invention will be described using Figures 1 to 4.

[0014] <Hardware configuration of the autonomous driving system> Figure 1 shows the vehicle Ob equipped with autonomous driving function. ０ This is a hardware configuration diagram of the autonomous driving system 1 of this embodiment, which is mounted on the vehicle. The autonomous driving system 1 shown here is an ECU (Electronic Control Unit) equipped with hardware such as a bus 101, CPU 102, ROM 103, RAM 104, timer 105, and accelerator 106. The CPU 102 and accelerator 106 execute a desired program obtained from the ROM 103 to realize the various functions described later. In the following explanation, such well-known technologies will be omitted as appropriate. The accelerator 106 is a computing device specialized for high-speed calculation of specific processes, such as a GPU (Graphics Processing Unit) specialized for image processing or AI processing.

[0015] Various sensors 2 are connected to the input side of the autonomous driving system 1. These sensors 2 are connected to the vehicle Ob ０This is an in-vehicle device for acquiring observational data of the surrounding environment, and includes, for example, a camera 21 for acquiring video data of the area around the vehicle, a radar 22 for acquiring distance data of the area around the vehicle, an in-vehicle Lidar 23 for acquiring point cloud data of the area around the vehicle, and a GNSS (Global Navigation Satellite System) and car navigation system 24 for acquiring the vehicle's position and map information.

[0016] Various actuators 3 are connected to the output side of the autonomous driving system 1. These actuators 3 control the vehicle Ob ０ To achieve autonomous driving, the autonomous driving system 1 controls the power source, specifically the vehicle Ob in response to steering commands. ０ Steering actuator 31 controls the direction of travel, and the vehicle Ob ０ These include a drive system actuator 32 and a braking system actuator 33, which control the speed and acceleration of the vehicle.

[0017] With this configuration, the autonomous driving system 1 of this embodiment controls various actuators 3 based on various information acquired by the sensor 2, and controls the vehicle Ob ０ This enables autonomous driving that adapts to the surrounding environment. The following describes an example of applying the technical concept of the present invention to an autonomous driving system 1, but the same concept may also be applied to a driver assistance system.

[0018] <Autonomous Driving System 1> Figure 2 is a functional block diagram of the autonomous driving system 1 of this embodiment. As shown here, the autonomous driving system 1 includes a surrounding area recognition unit 11, a behavior prediction unit 12, a planning unit 13, and a vehicle control unit 14.

[0019] <<Surroundings Recognition Unit 11>> The surroundings recognition unit 11 is a functional unit that sequentially recognizes traffic participants (vehicles, pedestrians, motorcycles, etc.) around the vehicle, as well as road lanes, shapes, signs, traffic lights, etc., based on data acquired from the sensor 2 (video data, distance data, point cloud data, vehicle position information, map information, etc.), and sequentially recognizes the position of each, as well as the speed and direction of movement of traffic participants.

[0020] <<Behavior Prediction Unit 12>> The behavior prediction unit 12 is a functional unit that predicts future paths at predetermined time intervals for a portion of traffic participants recognized by the surrounding recognition unit 11 (hereinafter referred to as "prediction target objects") based on the output of the surrounding recognition unit 11, and includes an object correlation acquisition unit 12a, a prediction method selection unit 12b, a simplified prediction unit 12c, and an altitude prediction unit 12d. The details of each unit will be described in order below. Note that the behavior prediction unit 12 does not need to predict only one future path for the prediction target objects; it may predict multiple future paths and the probability of each future path.

[0021] The object correlation acquisition unit 12a acquires the correlation between the objects to be predicted based on the output of the surrounding recognition unit 11. The correlation acquired by the object correlation acquisition unit 12a in this embodiment includes the relative position of the objects to be predicted, the distance between the objects to be predicted, the relative velocity of the objects to be predicted, and the position of the objects on the road.

[0022] The prediction method selection unit 12b is a functional unit that selects a prediction unit to be used to predict the future path of the target object based on the output of the surrounding recognition unit 11 and the output of the object correlation acquisition unit 12a. In this embodiment, the prediction method selection unit 12b selects a prediction unit to be used by focusing on the time-series change in the distance between moving objects, but the details of this selection method will be described later.

[0023] The simplified prediction unit 12c is a functional unit that predicts the future path of a target object using a simple AI model such as LSTM (Long Short-Term Memory). The AI ​​model used here has the advantage of relatively low computational load, but the disadvantage of relatively low prediction accuracy.

[0024] The advanced prediction unit 12d is a functional unit that predicts the future path of a target object using an advanced AI model such as a Transformer. The AI ​​model used here has the advantage of relatively high prediction accuracy, but also the disadvantage of relatively high computational load.

[0025] Figure 3 is a conceptual diagram illustrating the difference between the predicted route based on simplified prediction and altitude prediction. The figure shows the vehicle Ob recognized by the surrounding recognition unit 11. ０is an example of the surrounding environment, where the host vehicle Ob is traveling in the right lane of a two-lane road on one side ０ at a speed of V ０ and another vehicle Ob is traveling at a high speed of V in the left lane adjacent to it, and another vehicle Ob is traveling at a low speed of V in front of the other vehicle Ob (V < V < V). Hereinafter, it will be described on the premise that the prediction target object is a four-wheeled vehicle other than the host vehicle Ob, but the prediction target object may be a traffic participant other than a four-wheeled vehicle such as a pedestrian or a two-wheeled vehicle.)。なお、以下では、予測対象物体が自車両Ob ０ 以外の四輪車であるという前提で説明するが、予測対象物体は、歩行者や二輪車等の四輪車以外の交通参加者であっても良い。

[0026] In the simple prediction method (such as LSTM) shown in Fig. 3(a), only the past route of the other vehicle Ob is considered to predict the future route of the other vehicle Ob, so the predicted future route is like a straight-ahead route extending from the past route. Therefore, when using simple prediction, it is impossible to predict that the high-speed other vehicle Ob will change lanes to avoid colliding with the low-speed other vehicle Ob.

[0027] On the other hand, in the advanced prediction method (such as Transformer) shown in Fig. 3(b), not only the past route of the other vehicle Ob but also the past route of the other vehicle Ob and lane information included in the map information are considered to predict the future route of the other vehicle Ob. Therefore, it is possible to predict a future route in which the high-speed other vehicle Ob changes lanes to the right lane to avoid colliding with the low-speed other vehicle Ob.

[0028] [[ID=四十九]] However, in the advanced prediction method, the host vehicle Ob ０ ​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​Because it takes into account the correlation between moving objects, the number of phase functions that must be considered increases as the number of moving objects near the vehicle increases, resulting in an exponentially larger computational load. Therefore, it is not practical for an in-vehicle autonomous driving system 1, which has limited computing power, to predict the future paths of all numerous moving objects using advanced prediction.

[0029] Therefore, the prediction method selection unit 12b of this embodiment appropriately classifies moving objects into those for which a simplified prediction unit 12c with a low computational load is used and those for which an altitude prediction unit 12d with a high computational load is used, depending on the time change in the distance between objects.

[0030] Figure 4 is an example of a plan view showing the time-series changes in the correlation between moving objects acquired by the object correlation acquisition unit 12a and the prediction method selected by the prediction method selection unit 12b. In this figure, the "low importance" flag indicates an object whose future path is predicted by the simplified prediction unit 12c, and the "high importance" flag indicates an object whose future path is predicted by the altitude prediction unit 12d. Since the possibility of interference between vehicles traveling in different lanes is low, when assigning importance flags, only the correlation between vehicles traveling in the same lane is considered. Furthermore, below, vehicle Ob at time T Ａ and vehicle Ob Ｂ The distance between vehicles is L Ａ，Ｂ We will express it as (T).

[0031] Figure 4(a) shows time T ０ This is a plan view showing the position and speed of each vehicle, with the vehicle itself traveling in the left lane of the main line. ０ , other vehicles Ob １ , and other vehicles traveling in the right lane of the main line Ob ２ Ob ３ This indicates that.

[0032] Time T ０ So, the vehicle Ob is traveling in the left lane. ０ Velocity V ０ and other vehicles Ob １ Velocity V １ These are approximately equivalent, and the distance L between the two vehicles acquired by the object correlation acquisition unit 12a ０，１ (T ０ ) is the previous time T－１ Following distance L ０，１ (T －１ This is roughly equivalent to ( ). Therefore, the possibility of interference between the two vehicles traveling in the left lane is considered low, and other vehicles Ob in the same figure １ For this item, a flag indicating low importance is assigned, signifying the use of simplified predictions.

[0033] Meanwhile, other vehicles traveling in the right lane Ob ２ and other vehicles Ob ３ Now, the former speed V during steady-state driving ２ The latter speed V during braking ３ Because it is at a higher speed, the distance between the two vehicles L ２，３ (T ０ ) is the previous time T －１ Following distance L ２，３ (T －１ ) has decreased. Under these circumstances, it is considered highly likely that one or both vehicles will change their route or speed in order to avoid interference between the two vehicles, so other vehicles Ob ２ and other vehicles Ob ３ For this item, a "high" importance flag is assigned to indicate the use of altitude forecasting.

[0034] Figure 4(b) shows time T １ This is a plan view showing the position and speed of each vehicle, with the vehicle itself traveling in the left lane of the main line. ０ , other vehicles Ob １ Other vehicles changing lanes from the right lane to the left lane on the main road Ob ２ , and other vehicles stopped in the right lane of the main line Ob ３ This indicates that.

[0035] Time T ０ So, other vehicles Ob １ and other vehicles Ob ２ Although the possibility of interference between the two vehicles was low because they were traveling in different lanes, at time T １ So, other vehicles Ob １ Immediately before that, another vehicle Ob ２ As another vehicle cuts in, the possibility of interference between the two vehicles increases. Therefore, it is highly likely that one or both vehicles will change their route or speed to avoid interference, thus affecting other vehicles. １ and other vehicles Ob ２For both of them, a flag with a high importance level indicating the use of advanced prediction is assigned.

[0036] On the other hand, for another vehicle Ob parked in the right lane ３ regarding the other vehicle Ob ２ since the lane change of the other vehicle Ob to the left lane reduces the possibility of interference with the other vehicle Ob ２ a flag with a low importance level indicating the use of simple prediction is assigned.

[0037] Fig. 4(c) is a plan view showing the positions and speeds of each vehicle at time T ２ of the host vehicle Ob traveling in the left lane of the main line ０ another vehicle Ob １ Ob ２ and another vehicle Ob parked in the right lane of the main line ３ are shown.

[0038] At time T ２ due to the acceleration of the other vehicle Ob ２ the inter-vehicle distance L １ between the other vehicle Ob ２ and the other vehicle Ob １，２ (T ２ ) is larger than the inter-vehicle distance L １，２ (T １ ) at the previous time, and the possibility of interference between the two vehicles is low. Therefore, at time T ２ for the other vehicle Ob ２ a flag with a low importance level indicating the use of simple prediction is assigned.

[0039] On the other hand, when another vehicle Ob １ cuts in right in front of another vehicle Ob ２ and the other vehicle Ob １ suddenly brakes, the inter-vehicle distance L ０ between the host vehicle Ob １ and the other vehicle Ob ０，１ (T ２ ) rapidly decreases compared to the inter-vehicle distance L ０，１ (T １ ) at the previous time, and the possibility of interference between the host vehicle Ob ０ and the other vehicle Ob １ increases. Therefore, for the other vehicle Ob １ the assignment of a flag with a high importance level indicating the use of advanced prediction is maintained.

[0040] Thus, the behavior prediction unit 12 of this embodiment predicts the vehicle Ob ０ In addition to the correlation with other vehicles, the correlation between other vehicles is also considered, and altitude prediction is applied to other vehicles where a decrease in the distance between vehicles is observed. As a result, the behavior prediction unit 12, under the environment shown in Figure 4, will predict time T ０ At that point, time T １ Other vehicles Ob ２ Lane changes and time T ２ Other vehicles Ob １ It is possible to predict sudden braking.

[0041] If there is an upper limit to the number of objects that can be predicted by the altitude prediction unit 12d, altitude prediction processing may be assigned to moving objects in order of priority. This makes it possible to further reduce the processing load required for prediction processing while ensuring the safe driving of the vehicle. For example, the following methods can be used to set the priority: (1) If the object to be predicted is another vehicle traveling in the opposite lane, the priority should be lowered even if the distance between moving objects decreases. (2) If the object to be predicted is a pedestrian or cyclist moving on a sidewalk separated from the roadway, the priority should be lowered. (3) If the object to be predicted is a two-wheeled vehicle, the priority should be higher because it is more likely to behave unexpectedly than a four-wheeled vehicle. (4) Among the objects to be predicted, those that are close to the vehicle should have a higher priority, and those that are farther away should have a lower priority.

[0042] <<Planning Unit 13>> The Planning Unit 13 is a functional unit that plans an appropriate vehicle route and vehicle speed based on the future path of each moving object predicted by the Behavior Prediction Unit 12. For example, under the environment shown in Figure 4, the Behavior Prediction Unit 12 will determine the time T ０ At the time T ２ Other vehicles in Ob １ To predict sudden braking, the planning unit 13, at time T １ At that point, other vehicles Ob １ The vehicle Ob prepares for sudden braking. ０ Plan the behavior (specifically, a gradual deceleration).

[0043] <<Vehicle Control Unit 14>> The vehicle control unit 14 is a functional unit that generates steering commands to be output to the steering system actuator 31, acceleration commands to be output to the drive system actuator 32, and deceleration commands to be output to the braking system actuator 33, based on the vehicle's route and vehicle speed planned by the planning unit 13. For example, under the environment shown in Figure 4, the planning unit 13 generates steering commands to be output to the steering system actuator 31, acceleration commands to be output to the drive system actuator 32, and deceleration commands to be output to the braking system actuator 33. １ At that point, other vehicles Ob １ The vehicle Ob prepares for sudden braking. ０ Since the vehicle control unit 14 is planning the behavior (for example, braking), at time T １ At that point, my vehicle Ob ０ The braking can be initiated at time T ２ At that point, other vehicles Ob １ The vehicle detects the sudden braking and moves ０ Compared to conventional technology which also applies sudden braking, the vehicle Ob ０ This can improve safety and passenger comfort.

[0044] <Effects of this embodiment> According to the autonomous driving system of this embodiment described above, by predicting the future paths of moving objects that indirectly affect the driving plan of the vehicle with high accuracy, the overall computational load can be reduced while appropriately planning the behavior of the vehicle.

[0045] Next, using Figures 5 and 6, we will describe Embodiment 2 of the autonomous driving system of the present invention, which selects a prediction method based on the time to collision (TTC). Note that we will omit redundant explanations of points common to Embodiment 1.

[0046] First, let's explain the collision tolerance time (TTC) using Figure 5. As shown here, two vehicles are traveling in the same lane in the same direction, with a distance L between them, and the rear vehicle Ob Ａ Velocity V Ａ The vehicle in front Ob Ｂ Velocity V Ｂ If faster, the object correlation acquisition unit 12a of this embodiment will determine the vehicle Ob Ａ Vehicle Ob Ｂ The collision time (TTC), which is the time margin before a rear-end collision, is calculated using the following (Equation 1).

[0047] TTC = L / (V Ａ -V Ｂ ) ... (Equation 1) Subsequently, the prediction method selection unit 12b of this embodiment assigns a "high" importance flag to both vehicles indicating the use of altitude prediction if the calculated TTC is less than or equal to a predetermined threshold Tth, and assigns a "low" importance flag indicating the use of simplified prediction if it is otherwise.

[0048] Figure 6 is an example of a plan view showing the correlation between moving objects acquired by the object correlation acquisition unit 12a and the prediction method selected by the prediction method selection unit 12b.

[0049] First, your vehicle Ob ０ and other vehicles Ob １ , other vehicles Ob １ and other vehicles Ob ３ , and other vehicles Ob ２ and other vehicles Ob ３ We will examine the correlation between these two factors. In these combinations, the speed of the vehicle in front is faster than the speed of the vehicle behind, so the TTC calculated by (Equation 1) is negative. This means that there can be no interference between the two, so in this case, we assign a "low importance" flag to both to indicate the use of simplified prediction.

[0050] Next, my vehicle Ob ０ and other vehicles Ob ２ We will examine the correlation between these factors. In this combination, the speed V of the rear vehicle... ０ The speed of the vehicle in front is V ２ Because it is faster, the result of Equation 1 is positive, but the velocity V ０ and speed V ２ The difference is small, and the following distance L ０，２ Since is sufficiently large, the TTC calculated by (Equation 1) is greater than the threshold Tth. This means that even if the two interfere, there is sufficient preparation time to avoid a collision, so in this case as well, the other vehicle Ob ２ A flag indicating "low importance" is assigned to indicate the use of simplified predictions.

[0051] Furthermore, other vehicles Ob ３ and other vehicles Ob ４ We will examine the correlation between these factors. In this combination, the speed V of the rear vehicle... ３The speed of the vehicle in front is V ４ It is considerably faster, while maintaining a safe following distance L ３，４ Since this is insufficient, the TTC calculated by (Equation 1) is positive and less than or equal to the threshold Tth. This means that there is a high probability that the two vehicles will collide if neither takes evasive action, that is, there is a high probability that one or both vehicles will change their route or speed. In this case, both vehicles are given a "high" importance flag to indicate the use of altitude prediction.

[0052] Furthermore, if, as a result of multiple assessments, a vehicle is assigned both a "low" and a "high" importance flag, the "high" importance flag takes precedence, and the future route of that vehicle is predicted based on altitude prediction.

[0053] Next, using Figure 7, we will describe Embodiment 3 of the autonomous driving system of the present invention, in which a prediction method is selected based on the difference with the average speed Va. Note that we will omit redundant explanations of points common to the above embodiment.

[0054] Figure 7 is an example of a plan view showing the correlation between moving objects acquired by the object correlation acquisition unit 12a and the prediction method selected by the prediction method selection unit 12b.

[0055] In this environment, first, the object correlation acquisition unit 12a receives information from the surrounding recognition unit 11 about other vehicles Ob １ ~Ob ４ Velocity V １ ~V ４ Next, the object correlation acquisition unit 12a calculates the average speed of other vehicles other than the predicted object. For example, if the predicted object is other vehicle Ob １ If that is the case, other vehicles Ob １ Corresponding average speed Va １ This is calculated using (Equation 2).

[0056] Va １ = (V ２ +V ３ +V ４ ) / 3 ... (Equation 2) Similarly, other vehicles Ob ２ Ob ３ Ob ４ Corresponding average speed Va ２ Va ３ Va４ This is calculated using (Equation 3), (Equation 4), and (Equation 5).

[0057] Va ２ = (V １ +V ３ +V ４ ) / 3... (Formula 3) Va ３ = (V １ +V ２ +V ４ ) / 3... (Formula 4) Va ４ = (V １ +V ２ +V ３ ) / 3 ... (Equation 5) Subsequently, the prediction method selection unit 12b assigns a "high" importance flag to each target object if the value obtained by subtracting the average velocity Va calculated by the object correlation acquisition unit 12a from the velocity V of the target object is equal to or greater than a predetermined threshold Vth, indicating the use of altitude prediction; otherwise, it assigns a "low" importance flag indicating the use of simplified prediction.

[0058] Under the conditions shown in Figure 7, other vehicles Ob ３ Velocity V ３ From average speed Va ３ Since the value obtained by subtracting this is greater than or equal to Vth, it is assigned the "High Importance" flag. However, other vehicles do not meet this requirement, so they are assigned the "Low Importance" flag.

[0059] In the example shown in Figure 7, the prediction method was selected based on the average velocity. However, the prediction method could also be selected based on the average acceleration or average direction of movement of each target object. Furthermore, the prediction method could be selected using not only the current velocity, acceleration, and direction of movement, but also the fluctuations over N past samples.

[0060] Next, using Figure 8, we will describe Embodiment 4 of the autonomous driving system of the present invention, which selects a prediction method based on road information and the positions of other vehicles. Note that we will omit redundant explanations of points common to the above embodiment.

[0061] Figure 8 is an example of a plan view showing the correlation of moving objects acquired by the object correlation acquisition unit 12a and the prediction method selected by the prediction method selection unit 12b.

[0062] In this environment, first, the object correlation acquisition unit 12a receives map information and other vehicle information from the surrounding recognition unit 11. １ ~Ob ６ The location information is acquired. Next, the object correlation acquisition unit 12a confirms the positions of other vehicles on the road, taking into account the road shape near the vehicle read from the map information.

[0063] Subsequently, the prediction method selection unit 12b determines whether each target object is traveling in the overlapping section of the main line and the merging line, and determines whether the target object traveling in that overlapping section is likely to be affected by other vehicles (Ob １ Ob ３ Ob ５ Ob ６ A flag indicating "high" importance is given to objects that do not have a high probability of changing behavior (other vehicles Ob ２ Ob ４ ) is flagged with a "low importance" flag to indicate the use of simplified prediction.

[0064] In the example shown in Figure 8, the prediction method was selected based on whether or not the object to be predicted was traveling in an overlapping section of the main line and the merging line. However, in sections where branching lines, lane reductions, lane increases, intersections, or obstacles (including parked vehicles, construction, etc.) exist, a "high importance" flag may be assigned to nearby objects to be predicted.

[0065] Next, Embodiment 5 of the autonomous driving system of the present invention will be described using Figures 9 and 10. Note that common points with the above embodiments will not be explained again.

[0066] In the above embodiment, the future path of the target object was predicted by selecting either the simplified prediction unit 12c or the altitude prediction unit 12d. However, in this embodiment, the future path of the target object is predicted by selecting either the long-period prediction unit 12e or the short-period prediction unit 12f. In this embodiment, the future path of moving objects with a "low importance" flag is predicted by the long-period prediction unit 12e, and the future path of moving objects with a "high importance" flag is predicted by the short-period prediction unit 12f.

[0067] Figure 9 is a functional block diagram of the autonomous driving system 1 of this embodiment. As is obvious from a comparison with Figure 2, the autonomous driving system 1 of this embodiment replaces the simplified prediction unit 12c and altitude prediction unit 12d of Embodiment 1 with a long-period prediction unit 12e and a short-period prediction unit 12f. The long-period prediction unit 12e and the short-period prediction unit 12f are prediction units that execute the same altitude prediction method (e.g., Transformer) at different periods. For example, the short-period prediction unit 12f performs altitude prediction at half the period of the long-period prediction unit 12e.

[0068] Figure 10 is an example of a timing chart for prediction processing by the behavior prediction unit 12 in this embodiment. For example, the vehicle Ob ０ Other vehicles around it １ ~Ob ５ It is running, and other vehicles are ob １ Ob ２ The importance flag "Low" is assigned to it, and other vehicles Ob ３ Ob ４ Ob ５ When the "high importance" flag is assigned to a vehicle, the long-period prediction unit 12e and the short-period prediction unit 12f predict the future paths of other vehicles at the timings shown in the figure.

[0069] In other words, the long-period prediction unit 12e predicts other vehicles Ob １ Ob ２ The future paths of each vehicle are predicted with a long period of Δt × 8, and the short-period prediction unit 12f predicts the future paths of other vehicles Ob ３ Ob ４ Ob ５ The future paths of each vehicle are predicted in short periods of Δt × 4. Note that Δt in Figure 10 is a unit time longer than the processing time required to predict the future path using the desired prediction method (e.g., Transformer), and by using the timer 105, the prediction processing can be made to each prediction unit at the desired time interval. As a result, other vehicles Ob assigned the importance "high" flag are predicted. ３ Ob ４ Ob ５ While the future path of [vehicle name] is frequently predicted, other vehicles Ob [vehicle name] are flagged as "low importance". １ Ob ２The future path of this will be predicted with a relatively low frequency.

[0070] This embodiment also allows for the appropriate planning of the vehicle's behavior while reducing the overall computational load.

[0071] Next, an embodiment 6 of the autonomous driving system of the present invention will be described using Figure 11. Note that repetitive explanations of points common to embodiment 5 will be omitted.

[0072] Figure 11 is a functional block diagram of the autonomous driving system 1 of this embodiment. As is obvious from a comparison with Figure 9, the autonomous driving system 1 of this embodiment is the behavior prediction unit 12 of Embodiment 5 with the addition of a first error calculation unit 12g and a second error calculation unit 12h.

[0073] The first error calculation unit 12g compares the future path of the object to be predicted predicted by the long-period prediction unit 12e with the path that the object actually moved after the prediction, and calculates the error δ between the two. １ This is the function unit that calculates [the value].

[0074] error δ １ After the calculation, the prediction method selection unit 12b determines the error δ １ and a predetermined threshold Th １ Compare the error δ １ Th is the threshold １ If the above conditions are met, that is, if the prediction error of the long-period prediction unit 12e is large, the main predictor of the future path of the object to be predicted thereafter will be changed from the long-period prediction unit 12e to the short-period prediction unit 12f, thereby enabling the prediction of a more accurate future path.

[0075] Furthermore, the second error calculation unit 12h compares the future path of the predicted object predicted by the short-period prediction unit 12f with the path that the predicted object actually moved after the prediction, and calculates the error δ between the two. ２ This is the function unit that calculates [the value].

[0076] error δ ２ After the calculation, the prediction method selection unit 12b determines the error δ ２ and a predetermined threshold Th ２ Compare the error δ ２ Th is the threshold ２If the following conditions are met, that is, if the prediction error of the short-period prediction unit 12f is small, the calculation load required for predicting the future path of the object to be predicted can be reduced by changing the main predictor of the future path of the object from the short-period prediction unit 12f to the long-period prediction unit 12e.

[0077] Thus, for objects whose future paths can be correctly predicted even with long-period prediction, the use of the long-period prediction unit 12e is selected, and only for objects whose future paths cannot be correctly predicted without short-period prediction, the use of the short-period prediction unit 12f is selected. As a result, for most objects to be predicted, the use of the long-period prediction unit 12e is selected, and consequently, the amount of computation within the behavior prediction unit 12 can be reduced.

[0078] In this embodiment, the circumstances under which the error is calculated may be limited as follows: (1) The vehicle Ob ０ Only when the distance from the object to be predicted is shorter than a predetermined distance, the first error calculation unit 12g calculates an error δ １ This calculates the error δ even if the predicted object near the vehicle is assigned a "low importance" flag. １ When the value is large, it is desirable to predict future accounting by short-term forecasting. (2) Own vehicle Ob ０ Only when the distance from the object to be predicted is longer than a predetermined distance, the second error calculation unit 12h calculates an error δ. ２ This calculates the error δ even if the predicted object far from the vehicle is assigned a "high" importance flag. ２ This is because, when the period is small, it is considered sufficient to predict future accounting using long-period forecasting.

[0079] 1...Automated driving system, 11...Surroundings recognition unit, 12...Behavior prediction unit, 12a...Object correlation acquisition unit, 12b...Prediction method selection unit, 12c...Simplified prediction unit, 12d...Altitude prediction unit, 12e...Long-period prediction unit, 12f...Short-period prediction unit, 12g...First error calculation unit, 12h...Second error calculation unit, 13...Planning unit, 14...Vehicle control unit, 2...Sensor, 21...Camera, 22...Radar, 23...Lidar, 24...GNSS, 3...Actuator, 31...Steering system actuator, 32...Drive system actuator, 33...Braking system actuator

Claims

1. An automated driving system that predicts the future path of a moving object around its own vehicle, comprising: an environment recognition unit that recognizes the environment around the vehicle based on the output of a sensor; a simple prediction unit that predicts the future path of a target object using a simple AI model; an advanced prediction unit that predicts the future path of a target object using an advanced AI model; an object correlation acquisition unit that acquires the correlation between the target object and an object other than the vehicle; a prediction method selection unit that selects the simple prediction unit or the advanced prediction unit based on the correlation; and a vehicle control unit that controls the vehicle taking into account the future path of the target object predicted by the simple prediction unit or the advanced prediction unit.

2. An automated driving system according to claim 1, wherein the object correlation acquisition unit calculates the time change in the distance between a pair of predictable objects, and the prediction method selection unit causes the altitude prediction unit to predict the future path of the pair of predictable objects for which the distance between objects is decreasing, and causes the simplified prediction unit to predict the future path of the pair of predictable objects for which the distance between objects is not decreasing.

3. An automated driving system according to claim 1, wherein the object correlation acquisition unit calculates the collision margin time between a pair of predictable objects, and the prediction method selection unit causes the altitude prediction unit to predict the future path of the pair of predictable objects for which the collision margin time is less than or equal to a predetermined threshold, and causes the simplified prediction unit to predict the future path of the pair of predictable objects for which the collision margin time is less than or equal to a predetermined threshold.

4. An automated driving system according to claim 1, wherein the object correlation acquisition unit acquires the average value of the velocity, acceleration, or direction of movement of one target object and the velocity, acceleration, and direction of movement of other target objects; and the prediction method selection unit causes the altitude prediction unit to predict the future path of the target object if the value obtained by subtracting the average value from the velocity, acceleration, or direction of movement of the target object is greater than or equal to a predetermined threshold, and causes the simplified prediction unit to predict the future path of the target object if it is not such a value.

5. An automated driving system according to claim 1, wherein the object correlation acquisition unit acquires the position of a target object on the road, and the prediction method selection unit causes the high-altitude prediction unit to predict the future path of the target object in overlapping sections of the main line and merging lines, overlapping sections of the main line and branch lines, sections with fewer lanes, sections with more lanes, intersections, and near obstacles, and causes the simplified prediction unit to predict the future path of the target object that does not fall within these sections.

6. An automated driving system that predicts the future path of a moving object around its own vehicle, comprising: an environment recognition unit that recognizes the environment around the vehicle based on the output of a sensor; a long-period prediction unit that predicts the future path of a target object over a long period using an AI model; a short-period prediction unit that predicts the future path of a target object over a short period using the AI ​​model; an object correlation acquisition unit that acquires the correlation between the target object and an object other than the vehicle; a prediction method selection unit that selects the long-period prediction unit or the short-period prediction unit based on the correlation; and a vehicle control unit that controls the vehicle considering the future path of the target object predicted by the long-period prediction unit or the short-period prediction unit.

7. An automated driving system according to claim 6, further comprising: a first error calculation unit that calculates a first error which is the error between the future path predicted by the long-period prediction unit and the actual travel path; and a second error calculation unit that calculates a second error which is the error between the future path predicted by the short-period prediction unit and the actual travel path, wherein the prediction method selection unit changes the predicting unit for future paths of objects to be predicted whose first error is greater than or equal to a first threshold from the long-period prediction unit to the short-period prediction unit, and changes the predicting unit for future paths of objects to be predicted whose second error is less than or equal to a second threshold from the short-period prediction unit to the long-period prediction unit.

8. An automated driving method for predicting the future path of a moving object around a vehicle, comprising: an environment recognition step of recognizing the environment around the vehicle based on the output of a sensor; a simple prediction step of predicting the future path of a target object using a simple AI model; an advanced prediction step of predicting the future path of a target object using an advanced AI model; an object correlation acquisition step of acquiring the correlation between the target object and an object other than the vehicle; a prediction method selection step of selecting the simple prediction step or the advanced prediction step based on the correlation; and a vehicle control step of controlling the vehicle considering the future path of the target object predicted in the simple prediction step or the advanced prediction step.

9. An automated driving method for predicting the future path of a moving object around a vehicle, comprising: an environment recognition step of recognizing the environment around the vehicle based on the output of a sensor; a long-period prediction step of predicting the future path of a target object using an AI model over a long period; a short-period prediction step of predicting the future path of a target object using the AI ​​model over a short period; an object correlation acquisition step of acquiring the correlation between the target object and an object other than the vehicle; a prediction method selection step of selecting the long-period prediction step or the short-period prediction step based on the correlation; and a vehicle control step of controlling the vehicle considering the future path of the target object predicted in the long-period prediction step or the short-period prediction step.

Images