Driving assistance method and driving assistance device

Abstract

A processor 10 of the present disclosure stores a trajectory of an own vehicle V1, estimated using peripheral information of the own vehicle V1 detected by an external sensor 2 during manual driving, as a stored route RT. After storing this trajectory, when subsequently traveling along the stored route RT, the processor 10 causes the own vehicle V1 to perform automatic driving using the stored route RT as a target route. The processor 10 determines an executable automatic driving level on the basis of data characteristics of the peripheral information detected when being manually driven along the stored route RT. The processor 10 presents the automatic driving level by using an HMI 6 when the current position of the own vehicle V1 is within a predetermined distance range from a start point DP of the stored route RT.

Description

Driving support method and driving support device

[0001] The present invention relates to a vehicle driving support method and a driving support device.

[0002] There is known a technology in which an autonomous driving level is determined based on current road conditions detected by an external sensor and detection accuracy information of a vehicle sensor that are sent to an autonomous driving support device on a server, and the autonomous driving level is switched.

[0003] Japanese Patent Application Laid-Open No.2022-161403

[0004] However, it is impossible to determine the autonomous driving level when causing the own vehicle to autonomously drive along a created route, based on detection information acquired during manual driving for route creation.

[0005] A problem to be solved by the present invention is to determine an autonomous driving level when causing the own vehicle to autonomously drive along a created route, based on detection information acquired during manual driving for route creation.

[0006] The present invention solves the above problem by: after storing a stored route based on a trajectory of an own vehicle estimated using peripheral information of the own vehicle detected by a sensor during manual driving, when the own vehicle travels along the stored route next time, causing the own vehicle to perform autonomous driving with the stored route set as a target route; determining an executable autonomous driving level based on data characteristics of the peripheral information detected when the stored route is manually driven; and presenting the autonomous driving level when a current position of the own vehicle belongs to a predetermined distance range from a start point of the stored route.

[0007] According to the present invention, it is possible to determine the autonomous driving level when causing the own vehicle to autonomously drive along the created route, based on detection information acquired during manual driving for route creation.

[0008] Figure 1 is a system configuration diagram of the driver assistance system. Figure 2 is a diagram showing one aspect of the HMI. Figure 3 is a diagram showing an example of a memorized path based on the driving trajectory by the driver's manual driving. Figure 4 is a diagram illustrating automatic driving along the memorized path. Figure 5(A) shows an example of a path with many features in the surrounding environment, and Figure 5(B) shows an example of a path with few features in the surrounding environment. Figure 6(A) shows the error in the variability of the estimated position when there are many features based on the surrounding environment of the memorized path, and Figure 6(B) shows the error in the variability of the estimated position when there are few features based on the surrounding environment of the memorized path. Figure 7(A) is a flowchart showing the control process of the memorized path creation mode, Figure 7(B) is a flowchart showing the process of determining the automatic driving level, and Figure 7(C) is a flowchart showing the control process of the automatic driving mode.

[0009] <First Embodiment> Figure 1 shows the hardware configuration of a driver assistance system 100 equipped with a driver assistance device 1 according to this embodiment. This driver assistance method is implemented by the processor 10 of the driver assistance device 1 using each piece of hardware of the driver assistance system 100. The driver assistance system 100 of this embodiment includes a driver assistance device 1, one or more external sensors 2 constituting a sensor, a position detection device 3, a storage device 4, a vehicle actuator 5, and an HMI (Human Machine Interface) 6. Each device exchanges information with each other via an on-board CAN (Controller Area Network).

[0010] Multiple external sensors 2 are installed on the vehicle and form a sensor group that works in cooperation with each other. The external sensors 2 detect the presence or absence of objects, including other vehicles, around the vehicle, the attributes of the objects (parked vehicles, pieces of wood, pallets, trash cans, pedestrians, puddles, other vehicles, etc.), the distance to the objects, the relative velocity of the objects, and the relative acceleration of the objects. The external sensors 2 provide the detection information to the processor 10. Based on the detection information from the external sensors 2, the processor 10 determines the behavior of the objects, including at least one of the following: position, direction of movement, amount of movement, velocity, and acceleration, and their changes, and uses the determination results to perform autonomous driving of the vehicle. The external sensors 2 include one or more cameras 21 placed on the vehicle. The cameras 21 include image sensors equipped with image sensors such as CCDs, ultrasonic cameras, and infrared cameras. The cameras 21 capture images of the vehicle's surroundings in all directions, and the detection results based on the captured images are used for autonomous driving control. The external sensors 2 include a radar device 22 that detects (measures distance) the presence of objects, the position of objects, and changes in their position around the vehicle. The radar system 22 measures the distance and direction from the vehicle to an object, as well as the distance between objects, by emitting radio waves and measuring the reflected waves from the object. The radar system 22 includes a laser radar, millimeter-wave radar (LRF), ultrasonic radar, and sonar. The external sensor 2 is equipped with a LiDAR (light detection and ranging) 23. The LiDAR 23 measures the distance and direction from the vehicle to an object, as well as the distance between objects, by irradiating laser light and measuring the reflected waves from the object.

[0011] The position detection device 3 receives signals from the GNSS (Global Navigation Satellite System) 31 and detects the position of the vehicle. The position detection device 3 is equipped with an IMU (Inertial Measurement Unit) 32. The IMU 32 is an inertial measurement device that detects three-dimensional inertial motion and detects the relative position information and attitude of the vehicle by measuring the tilt and acceleration of three axes. The position detection device 3 further detects the position of the vehicle using the detection information from the gyro sensor and / or the detection information from the vehicle speed sensor. The position detection device 3 detects the position of the vehicle while it is moving (current position) over time and calculates the trajectory of the vehicle based on the position at each point in time. The position detection device 3 provides the detection results to the processor 10.

[0012] The storage device 4 includes a map data storage unit 41. The map data storage unit 41 stores map data 41D in an absolute coordinate system. For example, based on changes in the LiDAR point cloud, a stored path is created using an XY point sequence with the point where the driving trajectory began to be stored as the coordinate origin. The map data 41D includes general map information that contains information on routes that the vehicle can travel. The map data storage unit 41 stores the stored path. The stored path is a path based on the trajectory of the vehicle estimated using surrounding information of the vehicle detected by the external sensor 2 during manual driving. In the stored path, each point on the stored path is associated with the characteristics of the surrounding information acquired when driving at each point. If the characteristics of the surrounding information are common, it can be determined that the vehicle is driving at the same location. Furthermore, based on the path point sequence and feature quantities included in the stored path information, at the automatic driving start point, the feature quantities of multiple stored path information can be matched with the feature quantities obtained from sensors such as LiDAR at the current location, and a matching stored path can be automatically selected. The memory path may be stored superimposed on the map information of the map data 41D. The storage device 4 allows the processor 10 to read and provides the requested information.

[0013] The vehicle actuator 5 includes a steering device 51, a drive device 52, and a braking device 53, and performs automatic driving so that the vehicle automatically travels along a target path in accordance with control commands generated by the processor 10. The vehicle actuator 5 also performs driving in accordance with control commands from the driver's manual operation.

[0014] The HMI6 is implemented, for example, in a touch panel display 61. The display 61 is equipped with a switch 62 that can be used for touch input or press input. In this embodiment, the HMI6 is provided with a GUI (Graphical User Interface) as shown in Figure 2, so that the driver can select between driving in memory path creation mode or driving in automatic driving mode. If the driver wants to create a memory path RT, they touch or press the switch 62a that selects "Memory Path Creation Mode" shown on screen Q of the display 61 in Figure 2. After creating and storing the memory path RT, if the driver wants to drive automatically, they touch or press the switch 62b that selects "Automatic Driving Mode". The mode selection command is output to the driving mode management unit 101 (see Figure 1), and the processor 10 executes driving support control according to each mode selected by the driver. The processor 10 outputs the memory path RT to be driven and the estimated position PV1 of the vehicle V1 (see Figure 2) from the peripheral information processing unit 102 to the HMI6 and displays them on screen Q of the display 61. Before starting autonomous driving in automatic driving mode, the driver manually moves the vehicle V1 onto the stored route RT while confirming the vehicle's current estimated position PV1. This enables autonomous driving of the vehicle V1 based on the stored route RT. Furthermore, when the vehicle V1 is traveling on the stored route RT in autonomous driving mode, the driver can know where it is currently traveling on the stored route RT. If there are multiple stored route RTs, they are displayed on the display 61, and the driver can select the stored route RT they wish to travel on using autonomous driving. In addition, the GUI of the display 61 can display the vehicle speed, the remaining distance to the target destination, and the current vehicle speed on screen Q.

[0015] In this embodiment, the driver is presented with an automated driving level before the automated driving mode is started. The processor 10 presents, via the HMI 6, which automated driving level the selected memory path can be driven at. When the driver approves of automated driving at the presented automated driving level and wishes to start automated driving, they touch or press the "Automated Driving Mode" switch 62b, and the driver assistance device 1 starts processing the automated driving mode. The HMI 6 may consist of a speaker 63 and a microphone 64. The HMI 6 can communicate the automated driving level to the driver via the speaker 63 and receive the driver's approval or instructions via the microphone 64.

[0016] The driver assistance device 1, as a system, creates a memory path during manual driving in accordance with the driver's input, and then performs automated driving to automatically drive the vehicle along the target path. The processor 10 of the driver assistance device 1 includes a ROM (Read Only Memory) 12 that stores a program for creating a memory path during manual driving, calculating the level of automated driving along the memory path, and then, the next time the vehicle travels along the memory path, executing driver assistance control to automatically drive the vehicle at the calculated level of automated driving; a CPU (Central Processing Unit) 11 that executes the program stored in the ROM 12; and a RAM (Random Access Memory) 13 that functions as an accessible storage device. The processor 10 may be composed of one or more integrated circuits.

[0017] As shown in Figure 1, the processor 10 includes a driving mode management unit 101, a peripheral information processing unit 102, an automated driving level determination unit 103, a route generation unit 104, and a route following control unit 105. The processor 10 implements this driving support method by coordinating the software for realizing each function of each unit (101, 102, 103, 104, and 105) with the hardware of the driving support system 100. Each unit (101, 102, 103, 104, and 105) will be described below.

[0018] [Driving Mode Management Unit] The driving mode management unit 101 manages the state of the vehicle in three modes: memorized route creation mode, automatic driving mode, and manual driving mode. When the driving mode management unit 101 receives a memorized route creation instruction from the driver via the switch 62a of the HMI 6, it starts the memorized route creation mode. In the memorized route creation mode, the driving mode management unit 101 outputs an instruction to the surrounding information processing unit 102 to store the trajectory, which is an accumulation of feature quantities extracted from the surrounding information detected by the external sensor 2 during manual driving and the vehicle's position estimated by the surrounding information matching process. Here, the feature quantities are stored in the map data storage unit 41 so that they can be used for estimating the vehicle's position during automatic driving, and the trajectory of the vehicle V1 is stored in the map data storage unit 41 so that it can be used as a target route during automatic driving. The memorized route creation mode is executed while the driver is driving manually, so it is not operated after the route generation unit 104 is started. The driving mode management unit 101 receives an instruction from the driver via switch 62b on the HMI 6 to execute automatic driving after the stored route RT has been stored. The driving mode management unit 101 activates the automatic driving mode when it receives an instruction from the driver via the HMI 6 to select the stored route RT that the driver wishes to drive automatically. Automatic driving of the vehicle is achieved by automatic driving control. The driving mode management unit 101 outputs an instruction to the surrounding information processing unit 102 to estimate the position of the vehicle by matching the feature quantities stored in the map data storage unit 41 of the storage device 4 with the surrounding information currently detected by the external sensor 2. At the same time, the driving mode management unit 101 outputs an instruction to the storage device 4 to output the selected stored route RT to the route generation unit 104. The manual driving mode is a mode in which the vehicle is driven by manual driving based on the driver's operation of the steering wheel and / or accelerator pedal or brake pedal, without intervention from automatic driving control. The automatic driving level determined by the automatic driving level determination unit 103 is presented to the driver using the HMI 6.

[0019] [Surrounding Information Processing Unit] The surrounding information processing unit 102 stores the estimated trajectory of the vehicle, which is determined using the surrounding information of the vehicle detected by the external sensor 2 during manual driving, as a stored path RT. In stored path creation mode, the trajectory traveled by the driver during manual driving is stored as a stored path RT in the map data storage unit 41. The surrounding information processing unit 102 extracts static feature quantities of features from the surrounding information (e.g., LiDAR point cloud) detected at each point by the external sensor 2, associates them with each point, and stores them as map data 41D in the map data storage unit 41 of the storage device 4. The surrounding information processing unit 102 estimates the position of the vehicle while storing the feature quantities extracted from the surrounding information acquired at each point during driving. The feature quantities stored in the map data storage unit 41 are used in a matching process with surrounding information acquired during automatic driving in order to estimate the position of the vehicle during automatic driving. The estimated position of the vehicle is stored as the vehicle's trajectory for creating a memory route, and after the memory route creation mode ends, it is saved as a memory route RT in the map data storage unit 41 of the memory device 4.

[0020] The peripheral information processing unit 102 acquires peripheral information based on the image captured by the camera 21 and peripheral information based on the point cloud of scan data from the radar device 22 and / or LiDAR 23. The peripheral information processing unit 102 applies a feature extraction algorithm according to the nature of the detected data and acquires features. The extracted "features" are defined as appropriate. For example, if the number or density of points in the scan data contained in a predefined grid cell or voxel is above a predetermined threshold, that point cloud is extracted as a feature. Alternatively, in the case of LiDAR, only voxels of point clouds with strong reflectivity may be stored as features. The predetermined threshold is defined as appropriate according to the performance of the external sensor 2 and the feature extraction algorithm. Incidentally, the peripheral information processing unit 102 can also store the point cloud acquired using LiDAR 23 as peripheral information as is. However, when using LiDAR 23, tens of thousands of points are acquired in one sample, which increases the amount of data. For this reason, in this embodiment, features are extracted from the peripheral information and stored. By saving the features extracted from the point cloud, the amount of data to be saved can be reduced, and the processing load for self-position estimation by matching can be reduced. If the environment is such that the GNSS31 signal can be received, the surrounding information processing unit 102 can perform self-position estimation using the GNSS31 and / or IMU32 data. Furthermore, the accuracy of self-position estimation may be improved by combining the judgment based on the feature matching process with self-position estimation based on the received signals of GNSS31 and / or IMU32. On the other hand, in automatic driving mode, the surrounding information processing unit 102 performs self-position estimation by matching the features stored in the map data storage unit 41 with the surrounding information currently detected by the external sensor 2, and outputs the estimated position and the stored path RT stored in the map data storage unit 41 to the path generation unit 104. As shown in Figure 1, the surrounding information processing unit 102 outputs the trajectory of the self-propelled vehicle and the stored path RT to the path generation unit 104.

[0021] [Automated Driving Level Determination Unit] The automated driving level determination unit 103 determines the executable automated driving level based on the data characteristics of the surrounding information detected when the stored route RT was manually driven. Specifically, after the storage route RT is completed, the automated driving level determination unit 103 calculates the error variance for the sequence of points of the vehicle's driving trajectory data, which was estimated from the surrounding information processing unit 102 during driving. The automated driving level determination unit 103 then determines the automated driving level required to execute the automated driving mode according to the magnitude of the error variance. The automated driving level determination unit 103 also determines the automated driving level required to execute the automated driving mode according to the number of features extracted from the surrounding information acquired from the surrounding information processing unit 102 during driving, after the storage route RT is completed. The number of features includes the number of extracted features and the data size of the features. Alternatively, the automated driving level may be determined based on the quantitative value in the error variance. The automated driving level determination unit 103 stores the determination result of the automated driving level in association with the corresponding storage route RT in the map data storage unit 41 and outputs it to the driving mode management unit 101. The processor 10 controls the vehicle actuator 5 according to the autonomous driving level. The determination of the autonomous driving level may be performed when the memory path creation mode ends, or before the start of the autonomous driving mode.

[0022] [Route Generation Unit] In automatic driving mode, the route generation unit 104 uses the position of the vehicle V1 output from the surrounding information processing unit 102 as a starting point, extracts a predetermined range of the vehicle's surroundings route from the stored route RT also output from the surrounding information processing unit 102, and sets it as the target route for route following control. Here, for example, the stored route RT may be smoothed to make the route smoother before being set as the target route. If an obstacle is detected in front of the vehicle, an avoidance route to avoid the obstacle may be generated and set as the target route. This target route is output to the route following control unit 105 as the target value for the vehicle's route following control. In addition, the route generation unit 104 may calculate the curvature of the route from the target route, and the route following control unit 105 may add a curve deceleration function according to the curvature of the route. As shown in Figure 1, the route generation unit 104 outputs the target route to the route following control unit 105. Note that the route generation process may be performed using a known navigation device.

[0023] [Route Following Control Unit] The route following control unit 105 automatically drives the vehicle along a target route including a stored route. After the trajectory is stored in the map data 41D of the map data storage unit 41, the route following control unit 105, when next traveling along the stored route RT, automatically drives the vehicle along the stored route RT at the determined automatic driving level. The route following control unit 105 outputs a steering command to the steering device 51 of the vehicle actuator 5 to control the lateral position of the vehicle so that the vehicle can follow the target route output from the route generation unit 104. The route following control unit 105 calculates the curvature based on the speed limit, set speed, or route shape, and calculates an acceleration / deceleration command to control the vehicle speed to a predetermined speed by curve deceleration, which applies more deceleration the greater the curvature, and outputs these commands to the drive device 52 and / or braking device 53 of the vehicle actuator 5, respectively. The path following control unit 105 outputs the necessary steering commands and / or acceleration / deceleration commands to the vehicle actuator 5 according to the autonomous driving level determined by the autonomous driving level determination unit 103.

[0024] The driving assistance method of this embodiment operates in the following two steps: [Step 1: Memory Route Creation Mode (Manual Driving)] The processor 10 of the driving assistance device 1 extracts static feature quantities of features from surrounding information (e.g., LiDAR point cloud) obtained from the external sensor 2 while the vehicle is driven along an arbitrary route by the driver's manual driving. At the same time, the processor 10 estimates the vehicle's position from the surrounding information that changes with the vehicle's movement as detected by the external sensor. For the vehicle position estimation process, GNSS 31 and IMU 32 can be used where GNSS can be acquired. Alternatively, the vehicle position can be estimated using both. As shown in Figure 3, when the processor 10 determines that the vehicle has reached the target point GL, it stores a trajectory TJ (driving trajectory) which is an accumulation of the vehicle's position estimated at a predetermined period from the starting point DP as a memory route RT. The memory route RT stores the vehicle speed at the time of driving associated with each vehicle position on the trajectory TJ. A stored route RT is the route from the starting point DP to the target point GL. To identify multiple stored route RTs, each stored route RT is assigned an identifier (such as home, work, or school) corresponding to the target location GL and stored accordingly. Although not particularly limited, the starting point DP of a stored route RT is identified by absolute coordinate values ​​based on GNSS 31 positioning information. In this embodiment, after the creation of a stored route RT is complete, the data characteristics of the surrounding information, including the variance error (error variance) of the estimated position point sequence constituting the stored trajectory TJ and the number of features, are determined, and the autonomous driving level is judged according to the magnitude of the error variance or the number of features. The determined autonomous driving level is stored in association with the stored route RT. When storing a stored route RT, the processor 10 notifies the driver of the determined autonomous driving level via the HMI 6. For example, the driver is presented with text or voice message stating, "With the stored route RT, you can drive at autonomous driving level X from next time." This not only informs the driver whether or not a stored route RT can be created, but also allows the driver to confirm the specific autonomous driving level that can be executed with the stored route RT during the processing of the stored route creation mode.

[0025] [Step 2: Automatic Driving Mode] As shown in Figure 4, the processor 10 of the driver assistance device 1 estimates the vehicle's position during automatic driving by a feature matching process that compares surrounding information detected by the external sensor 2 with features stored in the map data 41D. Based on the map data 41D, which is obtained from past manual driving data and stored in the map data storage unit 41, the processor estimates the vehicle's position using the current surrounding information detected by the external sensor 2. GNSS 31 positioning information can be used for the vehicle position estimation process. This allows the processor 10 to determine the estimated position PV1 of the vehicle V1 in motion and execute route-following control to drive automatically along the stored route RT. Although not particularly limited, the driver assistance of this embodiment is used in last-mile automatic driving, where the vehicle drives automatically along a route to a destination such as home in areas not included in high-precision maps, such as private roads / private land outside of public roads such as main roads. The estimated position PV1 of the vehicle V1 and the stored route RT are identified in an absolute coordinate system common to the map data 41D. The processor 10 outputs steering commands to control the lateral position of the vehicle and acceleration / deceleration commands to control the vehicle speed to a predetermined speed to the vehicle actuator 5 as needed. In this embodiment, the automated driving level stored in association with the memory route RT selected before the execution of the automated driving mode is presented to the driver via the HMI 6. If the driver agrees to drive along the memory route RT at the presented automated driving level, the execution of the automated driving mode is started.

[0026] The method for determining the autonomous driving level will be explained based on Figures 5 and 6. The processor 10 determines the feasible autonomous driving level based on the data characteristics of the surrounding information detected when the vehicle V1 manually drives the memorized path RT. The data characteristics of the surrounding information are the error variance of the estimated position of each point in the trajectory TJ of the memorized path RT calculated based on the surrounding information, or the quantity (number, amount of data) of features extracted from the surrounding information used to calculate the trajectory TJ of the memorized path RT. It is rare for the data characteristics of the surrounding information to be uniform over the entire length of the trajectory TJ of the memorized path RT. For this reason, the average, minimum, or maximum value of the error variance of the estimated position of each point in the trajectory TJ may be used as the data characteristics of the surrounding information, or the number of features extracted from the surrounding information used to calculate the trajectory TJ, or the size of the data amount of the features may be used as the data characteristics of the surrounding information. In addition, the memorized path RT can be divided into multiple sections, and based on the autonomous driving level determination result for each section, it is also possible to drive the first section at autonomous driving level 2, but the second section at autonomous driving level 3. The autonomous driving level determination unit 103 extracts feature quantities of local features from surrounding information detected by the external sensor 2 while driving along an arbitrary route, and estimates the position (driving position) of the vehicle V1 through feature quantity matching processing. When using this estimation method, the accuracy of the vehicle V1's position estimation is affected by the presence or absence of static local features such as buildings, signs, and road structures on the driving route, or the number of such features present.

[0027] As shown in Figure 5(A), when traveling along a route rt with a relatively large number of features OB (represented as boxes in Figure 5; the code OB is shown only for representative features, and others are omitted), many feature quantities FB (represented as diamonds in Figure 6; the code FB is shown only for representative feature quantities, and others are omitted) can be extracted from the surrounding information detected by the external sensor 2, as shown in Figure 6(A). On the other hand, as shown in Figure 5(B), when traveling along a route rt with few features OB, the number of feature quantities FB will also decrease, as shown in Figure 6(B). The travel route rt is the travel route rt from the starting point DP to the target point GL. Incidentally, once the travel along the travel route rt by manual operation is completed and the trajectory TJ of the travel route rt is estimated, the trajectory TJ and surrounding information are stored in the map data storage unit 41 as a stored route RT. As shown in the example in Figure 6(A), for a driving route rt with many features FB, the amount of information used in the matching process is large, and the error in estimating the position of the vehicle V1 on the trajectory TJ1 tends to be small, resulting in a smaller error in the variability of the estimated position. On the other hand, as shown in the example in Figure 6(B), for a driving route rt with few features FB, the amount of information used in the matching process is small, and the error in estimating the position of the vehicle V1 on the trajectory TJ2 tends to be large. Therefore, the variability of the estimated position becomes larger.

[0028] The degree of this variability error can be defined as error variance, and this degree is visualized in the error ellipses EE1 and EE2 shown in Figures 6(A) and (B). The signs EE1 and EE2 are shown only for the representative error ellipses, and are omitted for the others. The covariance matrix used to represent the error ellipses EE1 and EE2 is usually a symmetric matrix that includes the correlation between the x and y coordinates of the position, and the uncertainty and error of the position estimation for each coordinate axis. An error ellipse is an ellipse that illustrates the variance of a two-dimensional normal distribution. If the covariance matrix is ​​Σ, then for example in the case of two dimensions, it is expressed by the following equation (1). Here, σ x 2 and σ y 2 These represent the variance of the position estimation for the x and y coordinates, respectively, and σ xyThis represents the covariance between the x and y coordinates. If we want to evaluate the vehicle's orientation θ, which is the direction of travel, we can represent the covariance matrix Σ in three dimensions (three variables). Generally, the covariance matrix is ​​a symmetric matrix, with the variances of each coordinate in the position estimation placed on its diagonal, and the off-diagonal components containing correlations and covariances. As a result, by using the covariance matrix, the uncertainty and error of the position estimation can be represented by an error ellipse.

[0029] In Figures 6(A) and 6(B), the error variance of an arbitrary point Pn1 / Pn2 whose position is estimated for the vehicle V1 traveling along the trajectory TJ1 / TJ2 of the driving route rt1 / rt2 from the starting point DP1 / DP2 to the target point GL1 / GL2 is represented as an error ellipse. The symbols Pn1 / Pn2 are shown only for representative points, and others are omitted. If the size of the error ellipse EE1 / EE2 for point Pn1 / Pn2 is large, it is judged that the error variance of the position estimation of point Pn1 / Pn2 is large. The size of the error ellipse EE1 / EE2 is judged to be larger when the length of the major axis is long, the length of the minor axis is long, or the area is large. In Figure 6(A), where the variability error is small, the error ellipse EE1 for point Pn1 on the trajectory TJ1 of the driving route rt1 is small, while in Figure 6(B), where the variability error is large, the error ellipse EE2 for point Pn2 on the trajectory TJ2 of the driving route rt2 tends to be large. The length of the major or minor axis of error ellipse EE1 is shorter than that of error ellipse EE2, or the area of ​​error ellipse EE1 is smaller than that of error ellipse EE2.

[0030] From this relationship, it is predicted that when there are many feature quantities FV extracted from the surrounding information (see Figure 6(A)), the error variance of the estimated position Pn1 of the vehicle V1 will be small, and when there are few feature quantities FV (see Figure 6(B)), the error variance of the estimated position Pn2 of the vehicle V1 will be large. The variability error of the estimated position Pn1 / Pn2 when the automatic driving mode is executed depends on the number of feature quantities FV based on the surrounding information. In the trajectory TJ1 of the driving route rt1, where the number of feature quantities FV is large, the variability error of the estimated position is small, and stable automatic driving can be expected based on an accurate estimated position. On the other hand, in the trajectory TJ2 of the driving route rt2, where the number of feature quantities FV is small, the error of the estimated position is large, and it may be difficult to accurately follow the vehicle V1 along the memorized path RT2. Depending on the degree of the variability error of the estimated position, it may be possible to encounter situations where automatic driving cannot be continued, such as getting too close to the curb at the edge of the road or too close to a wall. To prepare for such situations, setting an autonomous driving level that allows driver intervention ensures that even if autonomous driving temporarily becomes impossible, it can continue without being canceled midway. Furthermore, depending on the degree of variability error, even if autonomous driving at autonomous driving level 3 is difficult, autonomous driving at autonomous driving level 2 may still be possible. In this embodiment, before autonomously driving along the stored route RT, the feasible autonomous driving level is determined in advance according to the degree of error variance. This allows the vehicle V1 to autonomously drive to the target points GL1 / GL2 of the stored route RT at a predetermined autonomous driving level without being canceled midway.

[0031] As an example, autonomous driving levels are defined as follows: (1) Level 0 (Manual Driving): The system does not have any functions to automatically control driving. The driver performs all driving tasks. (2) Level 1 (Driver Assistance): The system automatically supports one of the driving tasks, such as steering, acceleration, or deceleration, while the driver performs the other tasks. Examples include adaptive cruise control and lane keeping assist. (3) Level 2 (Partial Autonomous Driving): The system automatically assists multiple driving tasks simultaneously (e.g., steering and acceleration / deceleration). However, the driver must constantly monitor the driving environment and intervene as needed. (4) Level 3 (Conditional Autonomous Driving): Vehicles equipped with a Level 3 autonomous driving system can perform all driving tasks automatically under certain conditions. However, the system may request intervention from the driver, in which case the driver must take over driving. (5) Level 4 (Highly Autonomous Driving): Vehicles equipped with a Level 4 autonomous driving system can perform all driving tasks automatically without driver intervention under certain conditions or in certain areas. However, the autonomous driving function is limited to specific conditions or in certain areas. (6) Level 5 (Fully Autonomous Driving): A vehicle equipped with a Level 5 autonomous driving system can perform all driving tasks automatically without driver intervention under all conditions. In theory, a vehicle at this level can drive autonomously in any driving environment. Autonomous driving levels can be defined using definitions based on general standards. For example, autonomous driving levels may be defined based on SAE J3016 of the Society of Automotive Engineers (SAE), or based on ISO / TC204 of the International Organization for Standardization (ISO).

[0032] The criteria for determining the level of autonomous driving are defined using predetermined thresholds en (e1, e2, e3, e4, e5: e1>e2>e3>e4>e5). The processor 10 evaluates the error variance calculated from the trajectory data of the stored memory path RT based on the predetermined thresholds en. For example, the autonomous driving level is determined by comparing the range based on these thresholds en with the error variance. In this criterion, a lower error variance defines a higher level of autonomous driving. (1) If error variance ≥ e1, the autonomous driving level is determined to be 0. (2) If e1 > error variance ≥ e2, the autonomous driving level is determined to be 1. (3) If e2 > error variance ≥ e3, the autonomous driving level is determined to be 2. (4) If e3 > error variance ≥ e4, the autonomous driving level is determined to be 3. (5) If e4 > error variance ≥ e5, the autonomous driving level is determined to be 4. (6) If e5 > error variance, the autonomous driving level is determined to be 5.

[0033] The determined autonomous driving level is presented to the driver via the HMI6 in advance before autonomous driving control for autonomous driving along the memorized route RT is initiated. The processor 10 presents the autonomous driving level when the current position of the vehicle V1 is within a predetermined distance range from the starting point DP of the memorized route RT. In other words, when the vehicle V1 approaches the starting point DP of the memorized route RT (the upstream point along the direction of travel toward the destination) within a predetermined distance, the processor 10 notifies the driver of the autonomous driving level for autonomous driving along the memorized route RT before passing the starting point DP. The presentation of the autonomous driving level may be performed while driving toward the starting point DP of the memorized route RT, or it may be performed when the vehicle V1 stops before reaching the starting point DP. The driver recognizes the autonomous driving level for autonomous driving to be performed on the memorized route RT before the vehicle V1 enters the memorized route RT.

[0034] The processor 10 determines that the autonomous driving level is Level 3 (> Level 2). The processor 10 presents the autonomous driving level determination result, "You can drive at Level 2 (or 3) autonomous driving," using the HMI 6. The processor 10 also uses the HMI 6 to ask the driver whether they can accept autonomous driving at the presented autonomous driving level, asking, "Do you want to drive autonomously on memory route A (or B)?" If the driver approves autonomous driving at the announced autonomous driving level, the vehicle V1 can automatically drive on memory route RT at that autonomous driving level.

[0035] According to the driving assistance method of this embodiment, during manual driving in memorized route creation mode, the system determines the level of autonomous driving that can be performed on the driving route based on the data characteristics of the surrounding information acquired by the external sensor 2, stores a memorized route RT corresponding to the driving route, and presents the driver with the level of autonomous driving that can be performed on the memorized route RT before driving the memorized route RT. This allows the driver to know what level of autonomous driving is required to automatically drive the memorized route RT and reach the target destination before autonomous driving of the memorized route begins. In this embodiment, the system can inform the driver in advance not only whether autonomous driving can be performed, but also what level of autonomous driving will be performed. Furthermore, since the level of autonomous driving is determined in advance based on the data characteristics of the surrounding information when the memorized route RT is driven, it is possible to avoid the autonomous driving level being switched midway through driving the memorized route RT, or the autonomous driving being suddenly canceled.

[0036] In this embodiment, the processor 10 acquires the error variance of the sequence of points of estimated positions at each point in the trajectory TJ of the memory path RT calculated based on surrounding information as a data characteristic, and determines the autonomous driving level of the memory path RT such that the autonomous driving level of the memory path RT with a relatively small error variance is higher than that of the memory path RT with a relatively large error variance. Depending on the magnitude of the error variance of the estimated positions at each point in the memory path RT, the autonomous driving level is determined to be lower for memory path RTs with a large error variance and higher for memory path RTs with a small error variance. Specifically, if the error variance is above a predetermined threshold, the autonomous driving level is determined to be lower, and if the error variance is below the predetermined threshold, the autonomous driving level is determined to be higher. The autonomous driving level determined according to the error variance is presented to the driver via the HMI 6 before the memory path RT is automatically driven. According to this embodiment, if the error variance is above a predetermined threshold, the driver can be notified that the memory path RT can be driven at a low autonomous driving level. If the error variance is below the predetermined threshold, the driver can be notified that the memory path RT can be driven at a high autonomous driving level. For routes where autonomous driving cannot be maintained at a high level, a lower level of autonomous driving that allows for continued autonomous driving is determined, and the vehicle V1 can then move autonomously to the target point GL.

[0037] In this embodiment, the processor 10 acquires the number of features extracted from the surrounding information used to calculate the trajectory TJ of the memory path RT as a data characteristic, and determines the autonomous driving level such that the autonomous driving level of a memory path RT with a relatively large number of features is higher than the autonomous driving level of a memory path RT with a relatively small number of features. The features are a group of detection data caused by geographical features (OB) extracted from the surrounding information acquired during manual driving when the memory path RT is created. The amount of features is determined based on the number of features extracted in a predetermined section (predetermined distance), or the amount of data of the extracted features. The processor 10 determines a low autonomous driving level for memory paths RT with a small number of features extracted from the surrounding information, and a high autonomous driving level for memory paths RT with a large number of features in the surrounding information. Specifically, if the number of features is above a predetermined threshold, the autonomous driving level is determined to be high, and if the number of features is below a predetermined threshold, the autonomous driving level is determined to be low. The autonomous driving level, determined based on the quantity of features, is presented to the driver via the HMI6 before the vehicle travels along the memorized route RT. According to this embodiment, when a predetermined number or more of features are extracted from the surrounding information, it is determined that a higher level of autonomous driving is possible compared to the case where this is not the case, and the driver is informed that the vehicle can travel along the memorized route RT at that level of autonomous driving. Therefore, on routes where autonomous driving cannot be continued at a high level of autonomous driving, autonomous driving is continued at a lower level, and the vehicle V1 is able to reach the target destination. Furthermore, by determining the autonomous driving level using features, even when there is no trajectory TJ data or when the error variance of the estimated position cannot be calculated, the autonomous driving level can be determined using features extracted from the surrounding information acquired by the external sensor 2.

[0038] The processor 10 acquires the error variance of the estimated position of each point on the trajectory TJ of the memory path RT calculated based on surrounding information as a data characteristic, and determines the abundance of features based on the magnitude of the error variance of the estimated position of each point on the trajectory TJ of the memory path RT. The processor 10 then determines that the number of features is relatively large when the error variance is relatively large. As explained with reference to Figures 5 to 6, in this embodiment, when the error variance of the estimated position of each point on the trajectory TJ of the memory path RT is small, it is determined that there is a high probability that the environment is one in which many features are extracted from the surrounding information acquired when driving along the memory path RT. From this knowledge, the amount of features can be determined (estimated) based on the magnitude of the error variance of the estimated position of each point on the memory path RT, and the level of autonomous driving that can be performed on the memory path RT can be determined based on the result of this determination. The abundance of features extracted from surrounding information affects the accuracy of position estimation of the vehicle V1 when executing the autonomous driving mode. If the accuracy of position estimation is high, a high level of autonomous driving can be performed. In this embodiment, the amount of features extracted from surrounding information during the execution of the memory path creation mode is determined from the variance error (error variance) of the point sequence data of the estimated position of each point in the memory path RT's trajectory TJ. Therefore, if the data of the memory path RT's trajectory TJ (including error variance) is available, the amount of features can be determined, and the level of autonomous driving can be determined based on the amount of features.

[0039] The control procedure for the driving assistance method of this embodiment will be explained based on the flowcharts in Figures 7(A), 7(B), and 7(C). Figure 7(A) is a flowchart of the control procedure in memory path creation mode. When the driver inputs a command to start memory path creation mode via switch 62a of the HMI6, the processor 10 starts memory path creation mode (M1). After the memory path creation mode is started, manual driving is performed by the driver (M2). The processor 10 acquires surrounding information detected by the external sensor 2 while the vehicle V1 is traveling along an arbitrary route. The processor 10 accumulates the changes in the position of the vehicle V1 estimated based on the acquired surrounding information, generates a trajectory TJ, and creates a memory path RT including the trajectory TJ (M3). To end the memory path creation mode, for example, the "Memory Path Creation Mode" switch 62a on the GUI of the HMI6 can be pressed again to end the memory path creation mode. When the vehicle V1 reaches the target point GL set by the driver and stops the vehicle, the driver touches or presses switch 62a on the HMI6 again to input an instruction to terminate the memory path creation mode. Upon receiving the termination instruction, the processor 10 terminates the memory path creation process. Processes M2 and M3 continue in a loop until an instruction to terminate the memory path creation mode is input (NO in M4). Once the instruction to terminate the memory path creation mode is input and the memory path creation process is terminated (YES in M4), the created memory path RT is stored as map data 41D in the map data storage unit 41 of the memory device 4 (M5). The map data 41D includes the memory path RT which stores the estimated trajectory TJ of the vehicle, and feature quantities extracted from surrounding information detected by the external sensor 2. The feature quantities are stored in the map data storage unit 41 in correspondence to each position on the memory path RT. The memory path RT is used as the target path for the path-following control of the vehicle V1. The stored features are used for position estimation during autonomous driving. Surrounding information detected by the external sensor 2 is matched with the stored features, and the position of the vehicle V1 is estimated based on the results.

[0040] When the stored route RT is stored in the map data storage unit 41 and the stored route creation mode ends, the processor 10 calculates the error variance based on the estimated position data of each point on the vehicle V1's trajectory TJ of the stored stored route RT (M6). The processor 10 determines the autonomous driving level according to the error variance (M7). The determined autonomous driving level is associated with the stored route RT and stored as map data 41D (M8). The map data 41D includes stored route information that associates the identification information of the stored route RT, the coordinates of the estimated positions of each point on the stored route RT, the characteristics of the surrounding information detected at each estimated point, and the autonomous driving level that can be executed with the stored route RT. Note that in the processing of M6, instead of error variance, or based on error variance, the amount (size) of data of feature quantities extracted from the surrounding information may be calculated. Then, in M7, the autonomous driving level is determined according to the amount of data of the feature quantities, and in M8, the executable autonomous driving level is associated with the stored route RT and stored.

[0041] Fig. 7(B) shows specific details of the processing for determining the autonomous driving level in M7 of Fig. 7(A). In this processing, in order to evaluate the magnitude of error variance, one or more predetermined thresholds en are set for error variance, and the autonomous driving level determination processing is performed. In the present example, a plurality of thresholds e1, e2, e3, e4, e5 (e1> e2> e3> e4> e5) are set as the thresholds en. The processor 10 calculates the error variance. If error variance ≧ e1 (YES at M7-1), the processor 10 determines that the autonomous driving level is 0 (M7-2). If e1 > error variance (NO at M7-1) and error variance ≧ e2 (YES at M7-3), the processor 10 determines that the autonomous driving level is 1 (M7-4). If e2 > error variance (NO at M7-3) and error variance ≧ e3 (YES at M7-5), the processor 10 determines that the autonomous driving level is 2 (M7-6). If e3 > error variance (NO at M7-5) and error variance ≧ e4 (YES at M7-7), the processor 10 determines that the autonomous driving level is 3 (M7-8). If e4 > error variance (NO at M7-7) and error variance ≧ e5 (YES at M7-9), the processor 10 determines that the autonomous driving level is 4 (M7-10). If e5 > error variance (YES at M7-11), the processor 10 determines that the autonomous driving level is 5 (M7-12). If it cannot be determined at M7-11 that e5 > error variance (NO at M7-11), the processing is ended. After the autonomous driving level determination processing is completed, the processing proceeds to M8, and the determined autonomous driving level is associated with the storage route RT and stored in the map data storage unit 41 of Fig. 1 (M8). Note that, in the processing of M7-1, M7-3, M7-5, M7-7, M7-9, and M7-11, the error variance may be replaced with a feature amount extracted from surrounding information to execute the autonomous driving level determination processing. After the autonomous driving level is determined and stored in association with the storage route RT, when the vehicle travels autonomously along the storage route RT from the next time onward, it becomes possible to read the travelable autonomous driving level from the map data storage unit 41 and present the same to the driver.

[0042] Figure 7(C) is a flowchart showing the control procedure in automatic driving mode. When the processor 10 has multiple memory route RTs stored, it selects one memory route RT (A1). For example, the processor 10 can display one or more candidate memory route RTs on the touch panel display 61 of the HMI 6, receive selection information from the driver regarding the memory route RT they wish to drive automatically on, and select one memory route RT. The processor 10 may select memory route RTs as candidates if the distance between the starting point DP of each memory route RT and the current position of the vehicle V1 is less than or equal to a predetermined distance. The processor 10 may also select memory route RTs as candidates if the distance between the starting point DP of each memory route RT and the current position of the vehicle V1 is less than or equal to a predetermined distance and the memory route RT is located on the side of the vehicle V1's direction of travel. The processor 10 may also determine the memory route RT that has the shortest distance between the starting point DP and the vehicle, where the distance between the starting point DP and the vehicle is less than or equal to a predetermined distance, as the memory route RT for which automatic driving will be performed. When a memory route RT is selected, the processor 10 presents the driver with the available autonomous driving levels for that memory route RT via the HMI 6 (A2). If the driver inputs via the switch 62b of the HMI 6 that they wish to start the autonomous driving mode (YES in A3), the driver assistance device 1 becomes capable of autonomous driving. The processor 10 waits for input until the driver inputs that they wish to start the autonomous driving mode, such as by touching or pressing the autonomous driving mode switch 62b (NO in A3). Once the driver inputs that they wish to start the autonomous driving mode via the autonomous driving mode switch 62b, the processor 10 confirms that the vehicle V1 is on the memory route RT and starts route following control. If the current position of the vehicle V1 is far from the memory route RT, the driver manually moves the vehicle V1 to the starting point DP or a point on the memory route RT. If the vehicle V1 is not present on the memory path RT, the processor 10 guides the driver to move to a point on the memory path RT, such as the starting point DP, by manual driving (A4).Although not particularly limited, the processor 10 displays the current position of the vehicle V1 and the starting point DP on the stored route RT on the display 61 to prompt the driver to move by manual driving. The display of the starting point DP may be performed when the vehicle V1 is automatically driving on a main road leading to the starting point DP, or when the vehicle V1 comes to a temporary stop before the starting point DP. The processor 10 recognizes the starting point DP (map coordinate value) on the map data 41D (A5). If the processor 10 determines that the vehicle V1 is on the stored route RT (YES in A6), it starts automatic driving with route following control to make the vehicle V1 travel along the stored route RT (A7). The processor 10 continues automatic driving along the target route until the vehicle reaches the target point GL, and when the vehicle reaches the target point GL (YES in A8), it ends the automatic driving mode.

[0043] 100...Driving support system, 1...Driving support device, 10...Processor, 11...CPU, 12...ROM, 13...RAM, 101...Driving mode management unit, 102...Peripheral information processing unit, 103...Automated driving level determination unit, 104...Route generation unit, 105...Route following control unit, 2...External sensor, 21...Camera, 22...Radar device, 23...LiDAR, 3...Position detection device, 31...GNSS, 32...IMU, 4...Storage device, 41...Map data storage unit, 41D...Map data, 5...Vehicle actuator, 51...Steering device, 52...Drive device, 53...Braking device, 6...HMI, 61...Display, 62...Switch, 63...Speaker, 64...Microphone

Claims

1. A driving assistance method used in a processor to control manual or automated driving of a vehicle, wherein the processor stores the trajectory of the vehicle estimated using surrounding information of the vehicle detected by a sensor during manual driving as a stored path, and after storage, when the vehicle next travels along the stored path, it is instructed to drive the vehicle automatically using the stored path as the target path, the processor determines an executable level of automated driving based on the data characteristics of the surrounding information detected when the stored path was driven manually, and presents the level of automated driving when the current position of the vehicle falls within a predetermined distance range from the starting point of the stored path.

2. The driving assistance method according to claim 1, wherein the processor obtains the error variance of the estimated position of each point on the trajectory of the memory path calculated based on the surrounding information as the data characteristic, and determines the autonomous driving level such that the autonomous driving level of the memory path with a relatively small error variance is higher than the autonomous driving level of the memory path with a relatively large error variance.

3. The driving assistance method according to claim 1, wherein the processor obtains the quantity of feature quantities extracted from the surrounding information used to calculate the trajectory of the memory path as the data characteristics, and determines the autonomous driving level such that the autonomous driving level of the memory path with a relatively large quantity of feature quantities is higher than the autonomous driving level of the memory path with a relatively small quantity of feature quantities.

4. The driving support method according to claim 3, wherein the processor obtains the error variance of the estimated position of each point on the trajectory of the memory path calculated based on the surrounding information as the data characteristic, determines the quantity of the feature based on the magnitude of the error variance of the estimated position of each point on the trajectory of the memory path, and determines that the quantity of the feature is relatively large if the error variance is relatively large.

5. A driving assistance device comprising a processor that controls manual or automated driving of the vehicle, wherein the processor stores the trajectory of the vehicle estimated using surrounding information of the vehicle detected by sensors during manual driving as a stored path, and after storage, when the vehicle next travels along the stored path, it is instructed to drive the vehicle automatically using the stored path as the target path, the device determines an executable level of automated driving based on the data characteristics of the surrounding information detected when the stored path was driven manually, and presents the level of automated driving when the current position of the vehicle falls within a predetermined distance range from the starting point of the stored path.

Images