Autonomous driving system and autonomous driving method

Abstract

This system provides an autonomous driving system that can appropriately plan the vehicle's behavior while reducing the overall computational load by accurately predicting the future paths of moving objects that indirectly affect the vehicle's driving plan. [Solution] An autonomous driving system that predicts the future path of a moving object around its own vehicle, comprising: an environment recognition unit that recognizes the environment around the vehicle based on the output of a sensor; a simple prediction unit that predicts the future path of a target object using a simple AI model; an advanced prediction unit that predicts the future path of a target object using an advanced AI model; an object correlation acquisition unit that acquires the correlation between the target object and an object other than the vehicle; a prediction method selection unit that selects the simple prediction unit or the advanced prediction unit based on the correlation; and a vehicle control unit that controls the vehicle taking into account the future path of the target object predicted by the simple prediction unit or the advanced prediction unit.

Description

[Technical Field]

[0001] The present invention relates to an autonomous driving system and an autonomous driving method that predict the future paths of moving objects around the vehicle and control the behavior of the vehicle according to the prediction results. [Background technology]

[0002] In recent years, vehicles equipped with advanced driver-assistance systems (ADAS) and autonomous driving (AD) functions have become increasingly common. Furthermore, as a prerequisite for realizing advanced ADAS and AD, vehicles equipped with artificial intelligence (AI) that predicts the future positions (coordinates) of vehicles and other traffic participants around the vehicle in a time series (in other words, predicts the future paths of traffic participants) are also becoming widespread.

[0003] However, because the computing power of in-vehicle AI is limited, a technology has been proposed that, when predicting the future paths of moving objects around the vehicle, select one of several prediction methods with different computational loads depending on the situation, thereby ensuring appropriate prediction accuracy while reducing the overall computational load.

[0004] For example, the abstract of Patent Document 1 states that the problem is "to reduce the computational load while ensuring accuracy in predicting the future position of moving objects," and describes the solution as "a moving object behavior prediction unit 202 predicts the future positions of pedestrians 603 and 604 around the vehicle 601. The moving object behavior prediction unit 202 is a moving object prediction device comprising a simplified prediction unit 401 that makes a simplified prediction of the future positions of pedestrians 603 and 604, an individual prediction unit 304 that makes a more accurate prediction of the future position of pedestrian 604 than the simplified prediction unit 401, and an object allocation unit 302 that allocates pedestrians 604 whose future positions are to be accurately predicted by the individual prediction unit 304 according to the results of the simplified prediction."

[0005] Thus, Patent Document 1 achieved both accuracy in prediction and reduced computational load by predicting the future position with high precision using individual prediction for moving objects that have a high probability of interfering with the vehicle, and by predicting the future position of other moving objects simply using simplified prediction. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2018-124663 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, the technology described in Patent Document 1 only provides a simplified prediction of the future position of a moving object that is unlikely to directly interfere with the vehicle. Therefore, it could not foresee interference between the vehicle and other moving objects caused by the behavior of such a moving object that cannot be predicted by the simplified prediction.

[0008] Therefore, the present invention aims to provide an automated driving system and an automated driving method that can appropriately plan the behavior of the vehicle while reducing the overall computational load by accurately predicting the future paths of moving objects that indirectly affect the driving plan of the vehicle. [Means for solving the problem]

[0009] In order to solve the above problems, an automatic driving system of the present invention is an automatic driving system that predicts the future route of moving objects around the host vehicle, and includes a surrounding recognition unit that recognizes the environment around the host vehicle based on the output of sensors, a simple prediction unit that predicts the future route of the prediction target object using a simple AI model, an advanced prediction unit that predicts the future route of the prediction target object using an advanced AI model, an object correlation acquisition unit that acquires the correlation between the prediction target object and objects other than the host vehicle, a prediction method selection unit that selects the simple prediction unit or the advanced prediction unit based on the correlation, and a vehicle control unit that controls the host vehicle in consideration of the future route of the prediction target object predicted by the simple prediction unit or the advanced prediction unit.

Effect of the Invention

[0010] According to the automatic driving system and the automatic driving method of the present invention, by accurately predicting the future route of moving objects that indirectly affect the driving plan of the host vehicle, it is possible to appropriately plan the behavior of the host vehicle while reducing the overall computational load.

Brief Description of the Drawings

[0011] [Figure 1] Hardware configuration diagram of the automatic driving system of Example 1. [Figure 2] Functional block diagram of the automatic driving system of Example 1. [Figure 3] Conceptual diagram for explaining the difference in predicted routes between the simple prediction method and the advanced prediction method. [Figure 4] Plan view showing the time-series change of the correlation between moving objects and the selected prediction method in Example 1. [Figure 5] Explanation diagram of the collision margin time TTC. [Figure 6] Plan view showing the correlation between moving objects and the selected prediction method in Example 2. [Figure 7] Plan view showing the correlation between moving objects and the selected prediction method in Example 3. [Figure 8] Plan view showing the correlation between moving objects and the selected prediction method in Example 4. [Figure 9] Functional block diagram of the automatic driving system of Example 5. [Figure 10] Timing chart of prediction processing by the behavior prediction unit of Example 5. [Figure 11] Functional block diagram of the automatic driving system of Example 6.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments of the automatic driving system of the present invention will be described with reference to the drawings.

Examples

[0013] First, Example 1 of the automatic driving system of the present invention will be described with reference to FIGS. 1 to 4.

[0014] <Hardware Configuration of the Automatic Driving System> FIG. 1 is a hardware configuration diagram of the automatic driving system 1 of this embodiment mounted on the host vehicle Ob0 equipped with an automatic driving function. The automatic driving system 1 shown here is an ECU (Electronic Control Unit) equipped with hardware such as a bus 101, a CPU 102, a ROM 103, a RAM 104, a timer 105, and an accelerator 106. The CPU 102 and the accelerator 106 execute a desired program acquired from the ROM 103 to realize each function described later. However, hereinafter, such well-known technologies will be appropriately omitted in the description. The accelerator urchased from the ROM 103 to realize each function described later. However, hereinafter, such well-known technologies will be appropriately omitted in the description. The accelerator 106 is an arithmetic device specialized for high-speed arithmetic of specific processing, and for example, is a GPU (Graphics Processing Unit) specialized for image processing or AI processing.

[0015] Various sensors 2 are connected to the input side of the autonomous driving system 1. These sensors 2 are in-vehicle devices for acquiring observational data of the surrounding environment of the vehicle Ob0, and include, for example, a camera 21 for acquiring video data of the area around the vehicle, a radar 22 for acquiring distance data of the area around the vehicle, an in-vehicle Lidar 23 for acquiring point cloud data of the area around the vehicle, and a GNSS (Global Navigation Satellite System) and car navigation system 24 for acquiring the vehicle's position and map information.

[0016] Various actuators 3 are connected to the output side of the autonomous driving system 1. These actuators 3 are power sources controlled by the autonomous driving system 1 to realize autonomous driving of the vehicle Ob0. Specifically, they include a steering system actuator 31 that controls the direction of travel of the vehicle Ob0 in response to steering commands, and drive system actuators 32 and braking system actuators 33 that control the speed and acceleration of the vehicle Ob0 in response to speed commands.

[0017] With this configuration, the autonomous driving system 1 of this embodiment can control various actuators 3 based on various information acquired by the sensor 2, thereby realizing autonomous driving that is appropriate to the surrounding environment of the vehicle Ob0. While the following describes an example of applying the technical concept of the present invention to the autonomous driving system 1, the same concept may also be applied to a driver assistance system.

[0018] <Autonomous Driving System 1> Figure 2 is a functional block diagram of the autonomous driving system 1 of this embodiment. As shown here, the autonomous driving system 1 comprises a surrounding area recognition unit 11, a behavior prediction unit 12, a planning unit 13, and a vehicle control unit 14.

[0019] <<Peripheral recognition unit 11>> The surrounding recognition unit 11 is a functional unit that sequentially recognizes traffic participants (vehicles, pedestrians, motorcycles, etc.) around the vehicle, as well as road lanes, shapes, signs, traffic lights, etc., based on data acquired from the sensor 2 (video data, distance data, point cloud data, vehicle position information, map information, etc.), and sequentially recognizes the position of each, as well as the speed and direction of movement of traffic participants.

[0020] <<Behavior Prediction Unit 12>> The behavior prediction unit 12 is a functional unit that predicts future paths at predetermined time intervals for some of the traffic participants recognized by the surrounding recognition unit 11 (hereinafter referred to as "prediction target objects") based on the output of the surrounding recognition unit 11, and includes an object correlation acquisition unit 12a, a prediction method selection unit 12b, a simplified prediction unit 12c, and an altitude prediction unit 12d. The details of each unit will be described in order below. Note that the behavior prediction unit 12 does not need to predict only one future path for the prediction target objects; it may predict multiple future paths and the probability of each future path.

[0021] The object correlation acquisition unit 12a acquires correlations between predicted objects based on the output of the surrounding recognition unit 11. The correlations acquired by the object correlation acquisition unit 12a in this embodiment include the relative positions of the predicted objects, the distance between the predicted objects, the relative velocities of the predicted objects, and the positions of the predicted objects on the road.

[0022] The prediction method selection unit 12b is a functional unit that selects a prediction unit to be used to predict the future path of the target object based on the output of the surrounding recognition unit 11 and the output of the object correlation acquisition unit 12a. In this embodiment, the prediction method selection unit 12b selects a prediction unit to be used by focusing on the time-series change in the distance between moving objects, but the details of this selection method will be described later.

[0023] The simplified prediction unit 12c is a functional unit that predicts the future path of a target object using a simple AI model such as LSTM (Long Short-Term Memory). The AI ​​model used here has the advantage of relatively low computational load, but the disadvantage of relatively low prediction accuracy.

[0024] The advanced prediction unit 12d is a functional unit that predicts the future path of a target object using advanced AI models such as Transformer. The AI ​​models used here have the advantage of relatively high prediction accuracy, but the disadvantage of relatively high computational load.

[0025] Figure 3 is a conceptual diagram illustrating the difference between predicted routes based on simplified prediction and altitude prediction. This figure shows an example of the surrounding environment of the vehicle Ob0 as recognized by the surrounding recognition unit 11. The vehicle Ob0 is traveling at speed V0 in the right lane of a two-lane road, another vehicle Ob1 is traveling at high speed V1 in the left lane, and another vehicle Ob2 is traveling at low speed V2 in front of vehicle Ob1 (speed V2 < speed V0 < speed V1). In the following explanation, it is assumed that the object to be predicted is a four-wheeled vehicle other than the vehicle Ob0, but the object to be predicted may also be a pedestrian, a motorcyclist, or other traffic participant other than a four-wheeled vehicle.

[0026] In the simplified prediction method (LSTM, etc.) shown in Figure 3(a), only the past path of another vehicle Ob1 is considered to predict the future path of Ob1. Therefore, the predicted future path is like a straight path that is an extension of the past path. Consequently, when using this simplified prediction method, it is not possible to predict that the high-speed vehicle Ob1 will change lanes to avoid a collision with the low-speed vehicle Ob2.

[0027] On the other hand, the altitude prediction method (Transformer, etc.) shown in Figure 3(b) predicts the future path of other vehicle Ob1 by considering not only the past path of other vehicle Ob1 but also the past path of other vehicle Ob2 and lane information included in the map information. Therefore, it can predict a future path in which the high-speed other vehicle Ob1 changes lanes to the right lane to avoid a rear-end collision with the low-speed other vehicle Ob2.

[0028] However, because the advanced prediction method considers the correlation between moving objects, including the vehicle Ob0, the number of phase functions that must be considered increases as the number of moving objects near the vehicle increases, resulting in an exponentially larger computational load. Therefore, it is not practical for an in-vehicle autonomous driving system 1, which has limited computing power, to predict the future paths of all numerous moving objects using advanced prediction.

[0029] Therefore, the prediction method selection unit 12b of this embodiment appropriately classifies moving objects into those for which a simplified prediction unit 12c with a low computational load is used and those for which an altitude prediction unit 12d with a high computational load is used, depending on the time change in the distance between objects.

[0030] Figure 4 is an example of a plan view showing the time-series changes in the correlation between moving objects acquired by the object correlation acquisition unit 12a and the prediction method selected by the prediction method selection unit 12b. In this figure, the "low importance" flag indicates an object whose future path is predicted by the simplified prediction unit 12c, and the "high importance" flag indicates an object whose future path is predicted by the altitude prediction unit 12d. Since the possibility of interference between vehicles traveling in different lanes is low, when assigning importance flags, only the correlation between vehicles traveling in the same lane is considered. Furthermore, below, vehicle Ob at time T A and vehicle Ob B The distance between vehicles is L A,B We will express it as (T).

[0031] Figure 4(a) is a plan view showing the position and speed of each vehicle at time T0, showing the own vehicle Ob0 and other vehicles Ob1 traveling in the left lane of the main line, and other vehicles Ob2 and Ob3 traveling in the right lane of the main line.

[0032] At time T0, the speed V0 of the self-propelled vehicle Ob0 traveling in the left lane and the speed V1 of the other vehicle Ob1 are approximately the same, and the distance L between the two vehicles acquired by the object correlation acquisition unit 12a 0,1 (T0) is the previous time T -1 Following distance L 0,1 (T -1) is approximately equivalent. Therefore, since it is considered that the possibility of interference between two vehicles traveling in the left lane is low, a flag of "low" importance indicating the use of simple prediction is assigned to the other vehicle Ob1 in the figure.

[0033] On the other hand, between the other vehicle Ob2 and the other vehicle Ob3 traveling in the right lane, since the speed V2 of the former during steady driving is higher than the speed V3 of the latter during braking, the inter-vehicle distance L 2,3 (T0) is the inter-vehicle distance L at the previous time T -1 at 2,3 (T -1 ) is decreasing compared to that at time T

[0034] Figure 4(b) is a plan view showing the positions and speeds of each vehicle at time T1, and shows the host vehicle Ob0 traveling in the left lane of the main line, the other vehicle Ob1, the other vehicle Ob2 changing lanes from the right lane to the left lane of the main line, and the other vehicle Ob3 stopped in the right lane of the main line.

[0035] At time T0, since the driving lanes of the other vehicle Ob1 and the other vehicle Ob2 were different, the possibility of interference between the two vehicles was low. However, at time T1, since the other vehicle Ob2 cuts in immediately in front of the other vehicle Ob1, the possibility of interference between the two vehicles increases. Therefore, since it is considered that one or both vehicles are likely to change their routes or speeds to avoid interference between the two vehicles, a flag of "high" importance indicating the use of advanced prediction is assigned to both the other vehicle Ob1 and the other vehicle Ob2.

[0036] On the other hand, regarding the other vehicle Ob3 stopped in the right lane, since the possibility of interference with the other vehicle Ob2 is reduced due to the lane change of the other vehicle Ob2 to the left lane, a flag of "low" importance indicating the use of simple prediction is assigned.

[0037] Figure 4(c) is a plan view showing the position and speed of each vehicle at time T2, showing the own vehicle Ob0 traveling in the left lane of the main line, other vehicles Ob1 and Ob2, and another vehicle Ob3 stopped in the right lane of the main line.

[0038] At time T2, the distance L between vehicle Ob1 and vehicle Ob2 decreases due to the acceleration of vehicle Ob2. 1,2 (T2) is the distance L between vehicles at the previous time. 1,2 The field of view has expanded since (T1), and the possibility of interference between the two vehicles is low. Therefore, at time T2, a flag indicating low importance is given to the other vehicle Ob2, showing the use of simplified prediction.

[0039] On the other hand, if vehicle Ob2 cuts in directly in front of vehicle Ob1, and vehicle Ob1 brakes suddenly, the distance L between your vehicle Ob0 and vehicle Ob1 will be... 0,1 (T2) is the distance L between vehicles at the previous time. 0,1 (T1) decreases sharply, increasing the likelihood of interference between our vehicle Ob0 and another vehicle Ob1. Therefore, we maintain the "high importance" flag for the other vehicle Ob1, indicating the use of altitude prediction.

[0040] Thus, the behavior prediction unit 12 of this embodiment considers not only the correlation between its own vehicle Ob0 and other vehicles, but also the correlation between other vehicles themselves, and applies altitude prediction to other vehicles where a decrease in the distance between them is observed. As a result, under the environment shown in Figure 4, the behavior prediction unit 12 can predict, at time T0, that other vehicle Ob2 will change lanes at time T1, and that other vehicle Ob1 will brake suddenly at time T2.

[0041] Furthermore, if there is an upper limit to the number of objects that can be predicted by the altitude prediction unit 12d, altitude prediction processing may be assigned to moving objects in order of priority. This makes it possible to further reduce the processing load required for prediction processing while ensuring the safe driving of the vehicle. For example, the following methods can be used to set the priority. (1) If the object to be predicted is another vehicle traveling in the opposite lane, the priority will be lowered even if the distance between the moving objects decreases. (2) If the object to be predicted is a pedestrian or cyclist moving on a sidewalk separated from the roadway, the priority will be lowered. (3) When the object to be predicted is a two-wheeled vehicle, it is more likely to exhibit unexpected behavior compared to a four-wheeled vehicle, so it should be given a higher priority. (4) Among the objects to be predicted, those that are close to the vehicle will be given higher priority, and those that are farther away will be given lower priority.

[0042] <<Planning Department 13>> The planning unit 13 is a functional unit that plans an appropriate vehicle path and vehicle speed based on the future paths of each moving object predicted by the behavior prediction unit 12. For example, in the environment shown in Figure 4, the behavior prediction unit 12 predicts that another vehicle Ob1 will brake suddenly at time T2 at time T0, so the planning unit 13 plans the behavior of the vehicle Ob0 at time T1 in preparation for the sudden braking of the other vehicle Ob1 (specifically, a gradual deceleration).

[0043] <<Vehicle Control Unit 14>> The vehicle control unit 14 is a functional unit that generates steering commands to be output to the steering system actuator 31, acceleration commands to be output to the drive system actuator 32, and deceleration commands to be output to the braking system actuator 33, based on the vehicle's route and speed planned by the planning unit 13. For example, in the environment shown in Figure 4, the planning unit 13 plans the behavior of the vehicle Ob0 (e.g., braking) at time T1 in preparation for the sudden braking of another vehicle Ob1. Therefore, the vehicle control unit 14 can start braking the vehicle Ob0 at time T1, improving the safety of the vehicle Ob0 and the comfort of the occupants compared to conventional technology where the vehicle Ob0 also brakes suddenly after recognizing the sudden braking of the other vehicle Ob1 at time T2.

[0044] <Effects of this embodiment> According to the autonomous driving system of this embodiment described above, by accurately predicting the future paths of moving objects that indirectly affect the driving plan of the vehicle, it is possible to appropriately plan the behavior of the vehicle while reducing the overall computational load. [Examples]

[0045] Next, using Figures 5 and 6, we will describe Embodiment 2 of the autonomous driving system of the present invention, in which a prediction method is selected based on the collision margin time (TTC). Note that we will omit redundant explanations of points common to Embodiment 1.

[0046] First, let's explain the collision margin time (TTC) using Figure 5. As shown here, two vehicles are traveling in the same lane in the same direction, with a distance L between them, and the rear vehicle Ob A Velocity V A The vehicle in front Ob B Velocity V B If faster, the object correlation acquisition unit 12a of this embodiment will determine the vehicle Ob A Vehicle Ob B The collision time (TTC), which is the time margin before a rear-end collision, is calculated using the following (Equation 1).

[0047] TTC = L / (V A -V B )... (Formula 1) Subsequently, the prediction method selection unit 12b of this embodiment assigns a "high" importance flag to both vehicles if the calculated TTC is less than or equal to a predetermined threshold Tth, indicating the use of altitude prediction; otherwise, it assigns a "low" importance flag indicating the use of simplified prediction.

[0048] Figure 6 is an example of a plan view showing the correlation between moving objects acquired by the object correlation acquisition unit 12a and the prediction method selected by the prediction method selection unit 12b.

[0049] First, we examine the correlation between our own vehicle Ob0 and other vehicle Ob1, other vehicle Ob1 and other vehicle Ob3, and other vehicle Ob2 and other vehicle Ob3. In these combinations, the speed of the vehicle in front is faster than the speed of the vehicle behind, so the TTC calculated by (Equation 1) is negative. This means that there is no interference between the two, so in this case, we assign a "low importance" flag to both to indicate the use of simplified prediction.

[0050] Next, we examine the correlation between our own vehicle Ob0 and the other vehicle Ob2. In this combination, the speed V0 of the rear vehicle is faster than the speed V2 of the front vehicle, so the result of Equation 1 is positive. However, the difference between speed V0 and speed V2 is small, and the distance L between vehicles is small. 0,2 Since is sufficiently large, the TTC calculated by (Equation 1) is greater than the threshold Tth. This means that even if the two interfere, there is sufficient preparation time to avoid a collision, so in this case as well, the other vehicle Ob2 is given a low importance flag to indicate the use of simplified prediction.

[0051] Furthermore, we will examine the correlation between other vehicles Ob3 and Ob4. In this combination, the speed V3 of the trailing vehicle is considerably faster than the speed V4 of the leading vehicle, while the distance L between vehicles is significantly faster. 3,4 Since this is insufficient, the TTC calculated by (Equation 1) is positive and below the threshold Tth. This means that there is a high probability that the two vehicles will collide if neither takes evasive action, i.e., one or both vehicles are likely to change their route or speed. In this case, both vehicles are given a "high" importance flag indicating the use of altitude prediction.

[0052] Furthermore, if, as a result of multiple assessments, a vehicle is assigned both a "low" and a "high" importance flag, the "high" importance flag takes precedence, and the future route of that vehicle is predicted based on altitude prediction. [Examples]

[0053] Next, using Figure 7, we will describe Embodiment 3 of the autonomous driving system of the present invention, in which a prediction method is selected based on the difference with the average speed Va. Note that we will omit redundant explanations of points common to the above embodiment.

[0054] Figure 7 is an example of a plan view showing the correlation between moving objects acquired by the object correlation acquisition unit 12a and the prediction method selected by the prediction method selection unit 12b.

[0055] In this environment, first, the object correlation acquisition unit 12a acquires the speeds V1 to V4 of other vehicles Ob1 to Ob4 from the surrounding recognition unit 11. Next, the object correlation acquisition unit 12a calculates the average speed of other vehicles other than the object to be predicted. For example, if the object to be predicted is other vehicle Ob1, the average speed Va1 corresponding to other vehicle Ob1 is calculated using (Equation 2).

[0056] Va1= (V2+V3+V4) / 3 (Formula 2) Similarly, the average speeds Va2, Va3, and Va4 corresponding to the other vehicles Ob2, Ob3, and Ob4 are calculated using (Equation 3), (Equation 4), and (Equation 5).

[0057] Va2= (V1+V3+V4) / 3 (Formula 3) Va3= (V1+V2+V4) / 3 (Formula 4) Va4= (V1+V2+V3) / 3 (Formula 5) Subsequently, the prediction method selection unit 12b, for each target object to be predicted, assigns a "high" importance flag to the target object if the value obtained by subtracting the average velocity Va calculated by the object correlation acquisition unit 12a from the velocity V of the target object is equal to or greater than a predetermined threshold Vth, indicating the use of altitude prediction; otherwise, assigns a "low" importance flag to the target object to be predicted, indicating the use of simplified prediction.

[0058] In the environment shown in Figure 7, the value obtained by subtracting the average speed Va3 from the speed V3 of other vehicles Ob3 is greater than or equal to Vth, so it is assigned a "high importance" flag. However, other vehicles do not meet this condition, so they are assigned a "low importance" flag.

[0059] In the example shown in Figure 7, the prediction method was selected based on the average velocity. However, the prediction method could also be selected based on the average acceleration or average direction of movement of each target object. Furthermore, the prediction method could be selected using not only the current velocity, acceleration, and direction of movement, but also the fluctuations over the past N samples. [Examples]

[0060] Next, using Figure 8, we will describe Embodiment 4 of the autonomous driving system of the present invention, which selects a prediction method based on road information and the positions of other vehicles. Note that we will omit redundant explanations of points common to the above embodiment.

[0061] Figure 8 is an example of a plan view showing the correlation of moving objects acquired by the object correlation acquisition unit 12a and the prediction method selected by the prediction method selection unit 12b.

[0062] In this environment, first, the object correlation acquisition unit 12a acquires map information and location information of other vehicles Ob1 to Ob6 from the surrounding recognition unit 11. Next, the object correlation acquisition unit 12a confirms the position of each other vehicle on the road, taking into account the road shape near its own vehicle read from the map information.

[0063] Subsequently, the prediction method selection unit 12b determines whether each target object is traveling in the overlapping section of the main line and the merging line. It assigns a "high" importance flag to the target objects traveling in the overlapping section (other vehicles Ob1, Ob3, Ob5, Ob6, which are likely to experience behavioral changes) to indicate the use of altitude prediction, and assigns a "low" importance flag to the target objects that are not traveling in that section (other vehicles Ob2, Ob4, which are unlikely to experience behavioral changes) to indicate the use of simplified prediction.

[0064] In the example shown in Figure 8, the prediction method was selected based on whether or not the object to be predicted was traveling in an overlapping section of the main line and the merging line. However, in sections where branching lines, lane reductions, lane increases, intersections, or obstacles (including parked vehicles, construction, etc.) exist, a "high importance" flag may be assigned to nearby objects to be predicted. [Examples]

[0065] Next, Embodiment 5 of the autonomous driving system of the present invention will be described using Figures 9 and 10. Note that common points with the above embodiments will not be explained again.

[0066] In the above embodiment, the future path of the target object was predicted by selecting either the simplified prediction unit 12c or the altitude prediction unit 12d. However, in this embodiment, the future path of the target object is predicted by selecting either the long-period prediction unit 12e or the short-period prediction unit 12f. In this embodiment, the future path of moving objects with a "low importance" flag is predicted by the long-period prediction unit 12e, and the future path of moving objects with a "high importance" flag is predicted by the short-period prediction unit 12f.

[0067] Figure 9 is a functional block diagram of the autonomous driving system 1 of this embodiment. As is obvious from a comparison with Figure 2, the autonomous driving system 1 of this embodiment replaces the simplified prediction unit 12c and altitude prediction unit 12d of Embodiment 1 with a long-period prediction unit 12e and a short-period prediction unit 12f. The long-period prediction unit 12e and the short-period prediction unit 12f are prediction units that execute the same altitude prediction method (e.g., Transformer) at different periods. For example, the short-period prediction unit 12f performs altitude prediction at half the period of the long-period prediction unit 12e.

[0068] Figure 10 shows an example of a timing chart for prediction processing by the behavior prediction unit 12 in this embodiment. For example, when other vehicles Ob1 to Ob5 are traveling around the local vehicle Ob0, and other vehicles Ob1 and Ob2 are assigned a "low importance" flag, and other vehicles Ob3, Ob4, and Ob5 are assigned a "high importance" flag, the long-period prediction unit 12e and the short-period prediction unit 12f predict the future paths of each other vehicle at the timings shown in the figure.

[0069] In other words, the long-period prediction unit 12e predicts the future paths of other vehicles Ob1 and Ob2 with a long period of Δt × 8, while the short-period prediction unit 12f predicts the future paths of other vehicles Ob3, Ob4, and Ob5 with a short period of Δt × 4. Note that Δt in Figure 10 is a unit of time longer than the processing time required to predict the future path using the desired prediction method (e.g., Transformer), and by using the timer 105, prediction processing can be made to each prediction unit at the desired time interval. As a result, the future paths of other vehicles Ob3, Ob4, and Ob5, which are assigned the "high importance" flag, are predicted frequently, while the future paths of other vehicles Ob1 and Ob2, which are assigned the "low importance" flag, are predicted relatively less frequently.

[0070] This embodiment also allows for the appropriate planning of the vehicle's behavior while reducing the overall computational load. [Examples]

[0071] Next, we will describe Embodiment 6 of the autonomous driving system of the present invention using Figure 11. Note that we will omit redundant explanations of points common to Embodiment 5.

[0072] Figure 11 is a functional block diagram of the autonomous driving system 1 of this embodiment. As is obvious from a comparison with Figure 9, the autonomous driving system 1 of this embodiment is the behavior prediction unit 12 of Embodiment 5 with the addition of a first error calculation unit 12g and a second error calculation unit 12h.

[0073] The first error calculation unit 12g is a functional unit that compares the future path of the object to be predicted predicted by the long-period prediction unit 12e with the path that the object actually moved after the prediction, and calculates the error δ1 between the two.

[0074] After calculating the error δ1, the prediction method selection unit 12b compares the error δ1 with a predetermined threshold Th1. If the error δ1 is greater than or equal to the threshold Th1, that is, if the prediction error of the long-period prediction unit 12e is large, the unit that predicts the future path of the object to be predicted is changed from the long-period prediction unit 12e to the short-period prediction unit 12f, thereby enabling a more accurate prediction of the future path.

[0075] Furthermore, the second error calculation unit 12h is a functional unit that compares the future path of the object to be predicted predicted by the short-period prediction unit 12f with the path that the object actually moved after the prediction, and calculates the error δ2 between the two.

[0076] After calculating the error δ2, the prediction method selection unit 12b compares the error δ2 with a predetermined threshold Th2. If the error δ2 is less than or equal to the threshold Th2, that is, if the prediction error of the short-period prediction unit 12f is small, the unit that predicts the future path of the target object is changed from the short-period prediction unit 12f to the long-period prediction unit 12e, thereby reducing the computational load required for predicting the future path.

[0077] Thus, for objects whose future paths can be correctly predicted even with long-period prediction, the use of the long-period prediction unit 12e is selected, and only for objects whose future paths cannot be correctly predicted without short-period prediction, the use of the short-period prediction unit 12f is selected. As a result, for most objects to be predicted, the use of the long-period prediction unit 12e is selected, and consequently, the amount of computation within the behavior prediction unit 12 can be reduced.

[0078] In this embodiment, the circumstances under which the error is calculated may be limited as follows. (1) The first error calculation unit 12g is made to calculate an error δ1 only when the distance from the vehicle Ob0 to the object to be predicted is shorter than a predetermined distance. This is because, even if the object to be predicted near the vehicle is assigned a "low importance" flag, if the error δ1 is large, it is desirable to predict future financial results by short-period prediction. (2) The second error calculation unit 12h is made to calculate the error δ2 only when the distance from the vehicle Ob0 to the object to be predicted is longer than a predetermined distance. This is because, even if the object to be predicted that is far from the vehicle is assigned a "high" importance flag, if the error δ2 is small, it is considered sufficient to predict future accounting using long-period prediction. [Explanation of symbols]

[0079] 1. Autonomous driving system 11 Peripheral recognition unit 12 Behavior Prediction Units 12a Object correlation acquisition unit 12b Prediction Method Selection Section 12c Simplified Prediction Unit 12d Altitude prediction section 12e Long-period prediction section 12f Short-period prediction section 12g First error calculation section 12h Second error calculation section 13 Planning Department 14. Vehicle Control Unit 2 sensors 21 Cameras 22 Radar 23 Lidar 24 GNSS 3 Actuators 31 Steering system actuators 32 Drivetrain Actuator 33 Braking system actuator

Claims

1. An automated driving system that predicts the future paths of moving objects around the vehicle, A surrounding environment recognition unit that recognizes the environment around the vehicle based on the output of the sensor, A simplified prediction unit that predicts the future path of the target object using a simple AI model, An advanced prediction unit that uses a sophisticated AI model to predict the future path of the target object, An object correlation acquisition unit that acquires the correlation between the predicted target object and objects other than the vehicle, A prediction method selection unit that selects either the simplified prediction unit or the advanced prediction unit based on the correlation, A vehicle control unit that controls the vehicle taking into consideration the future path of the predicted object predicted by the simplified prediction unit or the altitude prediction unit, An autonomous driving system characterized by having the following features.

2. In the automated driving system according to claim 1, The object correlation acquisition unit calculates the time change in the distance between a pair of target objects, The automatic driving system is characterized in that the prediction method selection unit causes the altitude prediction unit to predict the future paths of the pair of objects to be predicted for which the distance between the objects is decreasing, and causes the simplified prediction unit to predict the future paths of the pair of objects to be predicted for which the distance between the objects is not decreasing.

3. In the automated driving system according to claim 1, The object correlation acquisition unit calculates the collision margin time between a pair of predicted target objects, The prediction method selection unit is characterized in that it causes the altitude prediction unit to predict the future paths of the pair of objects to be predicted whose collision margin time is less than or equal to a predetermined threshold, and causes the simplified prediction unit to predict the future paths of the pair of objects to be predicted whose collision margin time is less than or equal to a predetermined threshold.

4. In the automated driving system according to claim 1, The object correlation acquisition unit acquires the average value of the velocity, acceleration, or direction of movement of one target object and the velocity, acceleration, and direction of movement of other target objects. The prediction method selection unit is characterized in that it causes the altitude prediction unit to predict the future path of one of the objects to be predicted if the value obtained by subtracting the average value from the velocity, acceleration, or direction of movement of that object is greater than or equal to a predetermined threshold, and causes the simplified prediction unit to predict the future path of the other objects to be predicted if that value is not greater than or equal to a predetermined threshold.

5. In the automated driving system according to claim 1, The object correlation acquisition unit acquires the position of the predicted target object on the road, The automated driving system is characterized in that the prediction method selection unit causes the advanced prediction unit to predict the future paths of the predicted target objects in overlapping sections of the main line and merging line, overlapping sections of the main line and branch line, sections with fewer lanes, sections with more lanes, intersections, and near obstacles, and causes the simplified prediction unit to predict the future paths of the predicted target objects that are not included in the advanced prediction unit.

6. An automated driving system that predicts the future paths of moving objects around the vehicle, A surrounding environment recognition unit that recognizes the environment around the vehicle based on the output of the sensor, A long-period prediction unit that uses an AI model to predict the future path of the target object over a long period, A short-period prediction unit that uses the AI ​​model to predict the future path of the object to be predicted in a short period of time, An object correlation acquisition unit that acquires the correlation between the predicted target object and objects other than the vehicle, A prediction method selection unit that selects the long-period prediction unit or the short-period prediction unit based on the correlation, A vehicle control unit that controls the vehicle taking into consideration the future path of the object to be predicted predicted by the long-period prediction unit or the short-period prediction unit, An autonomous driving system characterized by having the following features.

7. In the automated driving system according to claim 6, A first error calculation unit calculates a first error, which is the difference between the future path predicted by the long-period prediction unit and the actual movement path. The system further includes a second error calculation unit that calculates a second error, which is the error between the future path predicted by the short-period prediction unit and the actual travel path. The aforementioned prediction method selection unit is: The predictor of the future path of an object whose first error is greater than or equal to the first threshold is changed from the long-period prediction unit to the short-period prediction unit. An automated driving system characterized by changing the predictor of the future path of an object to be predicted, such that the second error is less than or equal to the second threshold, from the short-period prediction unit to the long-period prediction unit.

8. An automated driving method for predicting the future paths of moving objects around the vehicle, A surrounding environment recognition step that recognizes the environment around the vehicle based on the output of the sensor, A simplified prediction step in which the future path of the target object is predicted using a simple AI model, An advanced prediction step in which the future path of the target object is predicted using an advanced AI model, The object correlation acquisition step involves acquiring the correlation between the predicted target object and objects other than the vehicle itself. A prediction method selection step that selects the simplified prediction step or the advanced prediction step based on the correlation, A vehicle control step that controls the vehicle itself, taking into account the future path of the object to be predicted, as predicted in the simplified prediction step or the advanced prediction step, An automated driving method characterized by comprising the following features.

9. An automated driving method for predicting the future paths of moving objects around the vehicle, A surrounding environment recognition step that recognizes the environment around the vehicle based on the output of the sensor, A long-period prediction step in which the future path of the target object is predicted over a long period using an AI model, A short-period prediction step in which the future path of the object to be predicted is predicted in a short period using the AI ​​model, The object correlation acquisition step involves acquiring the correlation between the predicted target object and objects other than the vehicle itself. A prediction method selection step that selects either the long-period prediction step or the short-period prediction step based on the correlation, A vehicle control step that controls the vehicle itself, taking into account the future path of the object to be predicted, as predicted in the long-period prediction step or the short-period prediction step, An automated driving method characterized by comprising the following features.

Images