Driving assistance device and driving assistance method

Abstract

To provide a driving assistance device which enables more suitable parking or exiting assistance.SOLUTION: A driving assistance device comprises: a teacher data generation part which stores a movement path when a vehicle travels by a user's driving operation in a teacher travel mode, and generates teacher data concerning a target path for automatically traveling the vehicle in an automatic travel mode on the basis of the movement path; a position estimation part which estimates a current position of the vehicle on the basis of sensor information from a surrounding sensor monitoring the environment around the vehicle; and a vehicle control part which, on the basis of the initial position of the vehicle at the start of the automatic travel mode, moves the vehicle toward an entry start target position among target positions between a start point and an end point of the target path, and then causes the vehicle to travel along the target path from the entry start target position. The vehicle control part terminates automatic travel control of the vehicle if an accelerator operation, brake operation, or steering operation by the user is detected while the vehicle is traveling in the automatic travel mode.SELECTED DRAWING: Figure 2B

Description

Technical Field

[0001] The present disclosure relates to a driving support device and a driving support method.

Background Art

[0002] Generally, there are many narrow parking spaces in houses, and it may be difficult to park. Therefore, there is a high need for automatic parking and leaving the warehouse.

[0003] To realize this type of automatic driving, various driving support devices are known. As this type of driving support device, for example, when a driver stops a vehicle at a predetermined initial stop position, it performs automatic driving that automatically supports parking travel from the initial stop position to a target parking position. Then, such a driving support device automatically recognizes a parking space and the position of the vehicle by various sensors mounted on the vehicle, and automatically performs steering operations, accelerator operations, brake operations, and the like.

[0004] In Patent Document 1, in order to realize such automatic driving, by the driving operation of a user (hereinafter abbreviated as "user") riding in a vehicle, the vehicle is driven from a predetermined position outside a parking lot to a target parking position, and the movement path at that time is stored as teaching data. Then, in a subsequent parking scene, a technique for automatically driving the vehicle along the movement path is disclosed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

[0007] According to the prior art driving assistance device described in Patent Document 1, by performing a single training drive (meaning a drive performed by the user to generate training data; the same applies hereinafter) through the user's driving operations, it becomes possible to automatically perform parking or exit operations thereafter. Therefore, such technology can be said to be highly convenient.

[0008] A challenge with such technologies is improving safety when vehicles are driven autonomously.

[0009] This disclosure has been made in view of the above-mentioned problems and aims to provide a driver assistance device and a driver assistance method that enable more suitable parking assistance or vehicle exit assistance. [Means for solving the problem]

[0010] The main disclosure that addresses the aforementioned issues is: A driver assistance device that assists in the operation of a vehicle, In the training driving mode, the training data generation unit stores the travel path when the vehicle is driven by the user's driving operations, and generates training data related to the target path when the vehicle is driven automatically in the automatic driving mode based on the travel path. A position estimation unit estimates the current position of the vehicle based on sensor information from surrounding sensors that monitor the surrounding environment of the vehicle, A vehicle control unit moves the vehicle towards the entry start target position among the target positions between the start and end points of the target route, based on the initial position of the vehicle when the automatic driving mode is started, and then drives the vehicle along the target route from the entry start target position of the target route. Equipped with, The vehicle control unit terminates the automatic driving control of the vehicle when the vehicle is being driven in the automatic driving mode and the user performs an accelerator, brake, or steering operation. It is a driver assistance system.

[0011] Also, in other situations, A driver assistance method that assists in the operation of a vehicle, In the training driving mode, the process involves storing the travel path when the vehicle is driven by the user's driving operations, and generating training data related to the target path when the vehicle is driven automatically in the automatic driving mode based on the travel path. A process to estimate the current position of the vehicle based on sensor information from surrounding sensors that monitor the environment around the vehicle, The system includes a process of moving the vehicle towards an entry start target position among the target positions between the start and end points of the target route, based on the initial position of the vehicle when the automatic driving mode is started, and then driving along the target route from the entry start target position of the target route. When the vehicle is being driven in the aforementioned automatic driving mode, if the user operates the accelerator, brake, or steering wheel, the automatic driving control of the vehicle is terminated. This is a driver assistance method. [Effects of the Invention]

[0012] The driver assistance device described herein enables more suitable parking assistance or vehicle exit assistance. [Brief explanation of the drawing]

[0013] [Figure 1] Block diagram showing an example of the overall configuration of a vehicle [Figure 2A] Diagram showing an example of the functional blocks of the driving support device (in the instructor driving mode) [Figure 2B] Diagram showing an example of the functional blocks of the driving support device (in the automatic driving mode) [Figure 3A] Diagram showing an example of the movement path when the vehicle travels in the instructor driving mode [Figure 3B] Diagram showing an example of the movement path along which the vehicle is driving-controlled in the automatic driving mode [Figure 4A] Diagram showing the driving mode in the instructor driving mode [Figure 4B] Diagram showing the driving mode in the automatic driving mode [Figure 5] Diagram showing an example of the teacher data generated by the teacher data generation unit [Figure 6] Diagram showing an example of the map data generated by the teacher data generation unit [Figure 7] Diagram for explaining an example of the process of the position estimation unit [Figure 8] Diagram showing an example of the movement path of the vehicle when feedback control based on the current position of the vehicle is not performed (here, when control is performed only by dead reckoning) [Figure 9] Diagram for explaining a more preferable aspect of the vehicle control unit [Figure 10] Flowchart showing an example of the operation executed by the driving support device in the instructor driving mode [Figure 11] Flowchart showing an example of the operation executed by the driving support device in the automatic driving mode [Figure 12] Flowchart showing an example of the operation executed by the driving support device according to the modification in the automatic driving mode

Mode for Carrying Out the Invention

[0014] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same function are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0015] [Overall vehicle configuration] Hereinafter, an example of the vehicle configuration according to one embodiment will be described with reference to Figure 1.

[0016] Figure 1 is a block diagram showing an example of the overall configuration of Vehicle 1.

[0017] Vehicle 1 is equipped with a driver assistance system 10, a vehicle drive system 20, an on-board sensor 30, an on-board camera 40, an external storage device 50, and an HMI (Human Machine Interface) 60.

[0018] The vehicle drive system 20 is a means for realizing the driving, braking, and turning movements necessary for the vehicle 1 to travel, and is composed of, for example, a drive motor, a power transmission mechanism, a brake device, a steering device, etc., and an electronic driving support device that controls them. The vehicle drive system 20 generates power with a drive motor, transmits this power to the wheels via a power transmission mechanism (propeller shaft, differential gear, drive shaft, etc.), and drives the vehicle 1. In this embodiment, the operation of the vehicle drive system 20 is controlled by the driving support device 10 when in automatic driving mode.

[0019] The on-board sensors 30 are various sensors mounted on the vehicle 1 that detect the driving state of the vehicle 1. The on-board sensors 30 include, for example, an accelerator opening sensor that detects the accelerator opening, a steering angle sensor that detects the steering angle of the steering device, an acceleration sensor that detects the acceleration acting on the vehicle 1 in the longitudinal direction, a torque sensor that detects the torque acting on the power transmission mechanism between the wheels of the vehicle 1 and the drive motor, and a vehicle speed sensor, wheel speed sensor, etc. that detect the vehicle speed of the vehicle 1. The on-board sensors 30 output the sensor information obtained through their own detection to the driver assistance device 10.

[0020] The on-board camera 40 (corresponding to the "surrounding sensor" of the present invention) is mounted on the vehicle 1 and is a surrounding sensor that monitors the environment around the vehicle 1. In this embodiment, the on-board camera 40 is used, for example, to detect objects present around the vehicle 1 (typically objects fixed to the ground) and to estimate the location of the vehicle 1 from the positional relationship between the vehicle 1 and the objects present around it. The on-board camera 40 is composed of four cameras arranged to capture images in four directions: the front, rear, left, and right of the vehicle 1. The on-board camera 40 outputs the camera images it generates to the driver assistance device 10.

[0021] Furthermore, instead of the on-board camera 40, LiDAR, radar, or ultrasonic sensors may be used as surrounding sensors to estimate the vehicle's own position.

[0022] The external storage device 50 is, for example, an auxiliary storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or flash memory. The external storage device 50 stores, for example, training data D1 and map data D2 that are generated when the driver assistance device 10 is driving in training mode.

[0023] HMI60 is a user interface that accepts input operations from a user riding in vehicle 1, such as a touch panel, commander, buttons, or operation keys. HMI60 is configured to accept input operations such as execution commands for executing teacher driving mode or execution commands for executing automatic driving mode. HMI60 outputs information related to the input operations received from the user to the driver assistance device 10.

[0024] The driver assistance system 10 is an electronic control unit that provides comprehensive control over various parts of the vehicle 1. The driver assistance system 10 controls the vehicle drive system 20 to optimize the driving state of the vehicle 1, while referring to sensor information from the on-board sensors 30. The driver assistance system 10 is also configured to control the vehicle drive system 20 so that the vehicle 1 can drive automatically.

[0025] The driver assistance device 10 is composed of, for example, a CPU (Central Processing Unit) 10a, a ROM (Read Only Memory) 10b, a RAM (Random Access Memory) 10c, an input port (not shown), and an output port (not shown). The various functions of the driver assistance device 10, which will be described later, are realized, for example, by the CPU 10a referring to control programs and various data stored in the RAM 10c and ROM 10b.

[0026] Furthermore, the driver assistance system 10 is interconnected with the vehicle drive system 20, on-board sensors 30, on-board cameras 40, external storage devices 50, and HMI 60 via an on-board network (for example, a communication network compliant with the CAN communication protocol), enabling the mutual transmission and reception of necessary data and control signals.

[0027] [Configuration of driver assistance systems] Next, an example of the configuration of the driver assistance device 10 will be described with reference to Figures 2A to 9. Note that the driver assistance device 10 according to this embodiment is applied, for example, to the automatic parking of a vehicle 1 in a parking lot.

[0028] The driver assistance device 10 is configured to switch between a teacher driving mode and an automatic driving mode based on user input. The teacher driving mode is a mode for registering a target route to be used when the vehicle 1 is driven automatically in automatic driving mode. In teacher driving mode, the vehicle 1 is controlled by the user's driving operations. On the other hand, the automatic driving mode is a mode in which the vehicle 1 is driven automatically according to the target route registered in teacher driving mode (however, the vehicle 1 is driven from the end point to the starting point of the travel route in teacher driving mode. Details will be described later). In automatic driving mode, the vehicle 1 is automatically controlled by the driver assistance device 10 without requiring any user driving operations.

[0029] Figures 2A and 2B show an example of the functional blocks of the driver assistance device 10 according to this embodiment. Figure 2A shows only the functional parts that function when the vehicle 1 is in teacher driving mode, and Figure 2B shows only the functional parts that function when the vehicle 1 is in automatic driving mode.

[0030] Figure 3A shows an example of the movement path (path indicated by the solid arrow) when Vehicle 1 is driven in teacher driving mode, and Figure 3B shows an example of the movement path (path indicated by the solid arrow) when Vehicle 1 is controlled to move in automatic driving mode. In Figure 3A, in teacher driving mode, the user's driving operation is performed to exit the parking lot from the target parking position (hereinafter also referred to as "first position") P1 to a desired position outside the parking lot (hereinafter also referred to as "second position") P2, and in Figure 3B, in automatic driving mode, the entry operation from second position P2 to first position P1 is performed by automatic driving.

[0031] Figures 4A and 4B show the relationship between the teacher driving mode and the automated driving mode. Figure 4A shows the driving behavior in the teacher driving mode, and Figure 4B shows the driving behavior in the automated driving mode.

[0032] <When teacher-led mode is running> First, we will explain the functional configuration of the driver assistance device 10 that operates when the teacher driving mode is executed.

[0033] When the teacher driving mode is executed, the vehicle information acquisition unit 110, the dead reckoning unit 120, and the teacher data generation unit 130 of the driver assistance device 10 are activated (see Figure 2A).

[0034] The vehicle information acquisition unit 110 acquires sensor information indicating the driving status of vehicle 1 from the on-board sensor 30. The vehicle information acquisition unit 110 then sends the acquired sensor information to the dead reckoning unit 120.

[0035] The dead reckoning unit 120 estimates the current position of the vehicle 1 based on the temporal changes in sensor values ​​of the on-board sensor 30 that detects the driving state of the vehicle 1. For example, the dead reckoning unit 120 calculates the amount of movement of the vehicle 1 from a reference position (for example, the starting position when the teacher driving mode is started) based on the temporal changes in the vehicle speed and yaw rate indicated by the on-board sensor 30, and estimates the current position of the vehicle 1 based on this amount of movement.

[0036] The current position of vehicle 1 estimated by the dead reckoning unit 120 is an approximate current position of vehicle 1, and the position estimation by the dead reckoning unit 120 has low estimation accuracy. Therefore, in the driver assistance device 10 according to this embodiment, the actual current position of vehicle 1 is corrected based on the camera image generated by the on-board camera 40. Reasons for the low estimation accuracy of the position estimation by the dead reckoning unit 120 include the difficulty in constructing a complete kinematics model (e.g., tire diameter and tread width), the fact that the observed values ​​from the on-board sensor 30 always contain noise, and the fact that external factors that cannot be observed by the on-board sensor 30 (e.g., tire slip) greatly affect the position estimation. Furthermore, errors caused by these factors accumulate in the position estimation by the dead reckoning unit 120.

[0037] The training data generation unit 130 stores the travel path during training driving and generates training data D1 related to the target path for when the vehicle 1 is automatically driven in automatic driving mode from the said travel path. For example, the training data generation unit 130 starts storing the travel path when a training driving mode start command is received from the user, and stops storing the travel path when a training driving mode end command is received from the user. Typically, the training data generation unit 130 starts storing the travel path when the vehicle 1 is stopped at the parking target position (first position) P1, and stops storing the travel path when the vehicle 1 is stopped at a desired position outside the parking lot (second position) P2. The training data D1 generated by the training data generation unit 130 is stored in the external storage device 50.

[0038] Figure 5 shows an example of training data D1 generated by the training data generation unit 130. The data D1a shown in Figure 5 is data that sequentially stores the travel route actually traveled by vehicle 1 and the driving manner at each position along the travel route, and training data D1 is generated from this training data D1a.

[0039] The training data D1 includes, for example, the target route when driving in automatic driving mode, the orientation (i.e., direction) of vehicle 1 at each target position along the target route, the direction of vehicle 1's movement (i.e., forward or reverse) at each target position along the target route, and reference driving information at each target position along the target route.

[0040] The target path in training data D1 is generated by reversing the movement path during the training run so that it proceeds from the end point to the starting point. In other words, each target position in the target path of training data D1 is generated by reversing the positions of vehicle 1 estimated when vehicle 1 traveled from the first position P1 to the second position P2 during the training run, so that it proceeds from the second position P2 side to the first position P1 side.

[0041] The orientation of Vehicle 1 at each target position in the target route, as stored in training data D1, is the same as the orientation of Vehicle 1 at each position during training driving. Furthermore, the direction of travel of Vehicle 1 at each target position in the target route, as stored in training data D1, is the opposite of the direction of travel in training driving mode.

[0042] Furthermore, the reference driving information for each target position in the target path stored in the training data D1 is, for example, driving information (e.g., steering angle and vehicle speed) detected at each position in the travel path when vehicle 1 travels in training driving mode. This reference driving information serves as a reference for the vehicle control unit 160 when controlling the vehicle drive unit 20 (e.g., steering angle of the steering device) at each position in the travel path when executing automatic driving. However, when the vehicle control unit 160 makes vehicle 1 travel automatically, it uses each target position in the target path as a target value and provides feedback control to the vehicle drive unit 20 (e.g., steering angle of the steering device) to ensure that vehicle 1 does not deviate from the target path indicated by the training data D1 (details will be described later).

[0043] Furthermore, the training data generation unit 130 may also store in the external storage device 50 the data D1a relating to the actual travel route taken by vehicle 1, which was stored during the training run, so that the training run can be reproduced (see the modified example described later).

[0044] Here, the training data generation unit 130 generates training data D1 while creating map data D2 for estimating the current position of vehicle 1 from camera images of the in-vehicle camera 40, for example. This technique is also known as the SLAM (Simultaneous Localization and Mapping) method.

[0045] Figure 6 shows an example of map data D2 generated by the training data generation unit 130. Figure 6 also shows the real-space locations of feature points Q in the actual scenery stored in the map data D2, as a bird's-eye view.

[0046] Map data D2 is data that stores, for each of multiple feature points in a real-world scene, associates the three-dimensional position of that feature point in real space with the feature quantity of that feature point obtained from a camera image taken when the map data D2 was created. Feature points stored as map data D2 are, for example, parts (e.g., corners) from camera images of objects that can serve as landmarks in the real-world scene (e.g., trees, walls, or pillars). Furthermore, multiple feature points in map data D2 are stored in a way that allows for individual identification, for example, by identification numbers.

[0047] The three-dimensional positions of feature points stored in map data D2 are represented in a three-dimensional Cartesian coordinate system (X, Y, Z).

[0048] The feature quantities of feature points stored in map data D2 include brightness and density on the camera image, as well as SIFT (Scale Invariant Feature Transform) features or SURF (Speeded Up Robust Features) features. Furthermore, even for feature points at the same three-dimensional location, the feature quantity data for each feature point may be stored separately for each camera's shooting position and direction when the feature point was captured. Additionally, the feature quantity data for feature points stored in map data D2 may be stored in association with images of objects possessing those feature points.

[0049] The training data generation unit 130 identifies the coordinates of feature points in the actual scene, for example, based on stereo photogrammetry. Specifically, the training data generation unit 130 reads multiple camera images generated at different times and associates the same feature points that are commonly captured in these multiple camera images. Then, the training data generation unit 130 obtains information related to the provisional position of vehicle 1 at the time these multiple camera images were generated from the dead reckoning unit 120, and identifies the provisional coordinates of the feature points in the actual scene using the principle of triangulation. Then, the training data generation unit 130 performs bundle adjustment using the provisional position of vehicle 1 and the provisional coordinates of the feature points in the actual scene as reference information, and calculates the official position of vehicle 1 and the official coordinates of the feature points in the actual scene so as to minimize the reprojection error when each feature point in the actual scene is projected onto all camera images. The training data generation unit 130 then stores the official position of vehicle 1 as data D1a indicating the position of vehicle 1 during training runs. Furthermore, the training data generation unit 130 stores the official coordinates of feature points in the actual scenery as map data D2 in the external storage device 50.

[0050] Furthermore, the map data D2 may be measured in advance using LIDAR (Light Detection and Ranging) or a stereo camera, rather than using the SLAM method. However, since distortion is often present in the map data D2 depending on the measurement method used, it is preferable to use the SLAM method from the viewpoint of suppressing a decrease in position estimation accuracy caused by distortion in the map data D2 itself.

[0051] <When automatic driving mode is activated> Next, we will explain the functional configuration of the driver assistance system 10 that operates when the automatic driving mode is activated.

[0052] When the automatic driving mode is activated, the following functions of the driver assistance system 10 are activated: the vehicle information acquisition unit 110, the dead reckoning unit 120, the position estimation unit 140, the target route reading unit 150, and the vehicle control unit 160 (see Figure 2B).

[0053] The vehicle information acquisition unit 110 and the dead reckoning unit 120 have the same configuration as described above, so their explanation is omitted here.

[0054] The position estimation unit 140 estimates the current position of the vehicle 1 based on the map data D2 and the camera image from the in-vehicle camera 40.

[0055] Figure 7 illustrates an example of the processing performed by the position estimation unit 140. In Figure 7, points R1, R2, and R3 represent three feature points extracted from the camera image of the in-vehicle camera 40, and points Q1, Q2, and Q3 represent the three-dimensional positions in real space of the feature points R1, R2, and R3 stored in the map data D2. RP1 represents the imaging plane of the in-vehicle camera 40. Point P1 represents the position of the in-vehicle camera 40 (i.e., the position of vehicle 1) obtained from the three feature points R1, R2, and R3 extracted from the camera image of the in-vehicle camera 40 and points Q1, Q2, and Q3 stored in the map data D2.

[0056] For example, the position estimation unit 140 first compares feature points extracted from the camera image of the in-vehicle camera 40 with feature points stored in the map data D2, using pattern matching, feature search, etc. Then, the position estimation unit 140 randomly selects several feature points (for example, 3 to 6) from the feature points extracted from the camera image of the in-vehicle camera 40 that can be matched with feature points stored in the map data D2, and estimates the position of the vehicle 1 in real space based on the positions of these feature points in the camera image and the three-dimensional positions of these feature points in real space stored in the map data D2. At this time, the position estimation unit 140 calculates the position of the vehicle 1 by solving a PnP problem using a known method such as Lambda Twist (see, for example, Non-Patent Document 1).

[0057] Furthermore, when the position estimation unit 140 compares feature points extracted from the camera image of the in-vehicle camera 40 with feature points stored in the map data D2, it may, for example, use the current position of the vehicle 1 estimated by the dead reckoning unit 120 as a reference to narrow down the feature points to be compared with the feature points extracted from the camera image of the in-vehicle camera 40 from among the feature points stored in the map data D2.

[0058] The information regarding the current position of vehicle 1 estimated by the position estimation unit 140 is sent to the vehicle control unit 160. The information regarding the current position of vehicle 1 estimated by the position estimation unit 140 includes information regarding the two-dimensional position of vehicle 1 in real space (X coordinate, Y coordinate) and information regarding the orientation of vehicle 1.

[0059] The target path reading unit 150 reads the training data D1 for moving the vehicle 1 from the external storage device 50 and sends it to the vehicle control unit 160. Furthermore, when the driver assistance device 10 is configured to execute both a forward automatic driving mode and a reverse automatic driving mode as automatic driving modes, the target path reading unit 150 selects whether the target path to be read from the external storage device 50 is related to the forward automatic driving mode or the reverse automatic driving mode, based on the user's command signal from the HMI 60, and sends it to the vehicle control unit 160 (see the modified example described later).

[0060] When the vehicle control unit 160 receives a command from the user to execute the automatic driving mode, it recognizes the current position of the vehicle 1 and moves the vehicle 1 along the target path indicated by the training data D1 received from the target path reading unit 150. That is, when the vehicle control unit 160 receives a command from the user to execute the automatic driving mode, it automatically drives the vehicle 1 from the second position P2 to the first position P1 along the target path indicated by the training data D1, in the opposite direction to the driving direction in the training driving mode. Then, the vehicle control unit 160 stops the vehicle 1 at the first position P1, which corresponds to the starting point during the training driving mode.

[0061] Here, the vehicle control unit 160 performs feedback control of the vehicle 1 based on the current position of the vehicle 1 estimated by the position estimation unit 140 and each target position in the target path indicated by the training data D1. The reason for performing such control is that the initial position of the vehicle 1 when the automatic driving mode is started may not exactly match the initial position of the target path indicated by the training data D1 (in this case, the second position P2). Furthermore, the method of reproducing the travel path using only dead reckoning tends to result in larger errors, especially in the low-speed range and for travel paths with multiple turns. In addition, if the driving speed of the vehicle 1 is different during training driving and automatic driving, the steering angle of the vehicle 1 required to drive the vehicle 1 along the same travel path will also change between training driving and automatic driving.

[0062] Figure 8 shows an example of the movement path of vehicle 1 when feedback control based on the current position of vehicle 1 is not performed (in this case, when control is performed only by dead reckoning). Figure 8 shows the movement path of vehicle 1 when the actual initial position of vehicle 1 (in this case, the orientation of vehicle 1) P2' when the automatic driving mode is started is deviated from the initial position P2 of the target path indicated by the training data D1. In such a case, as shown in path L' of Figure 8, vehicle 1 moves significantly deviating from the target path indicated by the training data D1. Furthermore, even if the error in the orientation of vehicle 1 is only 1 degree, after traveling 50m, the goal position P1' will be 0.87m off from the goal position P1 of the target path.

[0063] In this regard, as in the control method of the vehicle control unit 160 according to this embodiment, by feedback controlling the vehicle 1 based on the current position of the vehicle 1 estimated by the position estimation unit 140 and each target position in the target path indicated by the training data D1, it becomes possible to accurately drive the vehicle 1 along the target path. In this case, the vehicle control unit 160 typically suppresses such position deviation by feedback controlling the steering angle of the steering device of the vehicle 1.

[0064] Furthermore, when the vehicle control unit 160 makes the vehicle 1 automatically drive, it is not necessarily required to start the automatic driving from the starting point of the target route indicated by the training data D1, but may start the automatic driving from a position between the starting point and the ending point of the target route indicated by the training data D1.

[0065] Figure 9 illustrates a more preferred embodiment of the vehicle control unit 160. Preferably, as shown in Figure 9, the vehicle control unit 160 determines the entry start target position Pt from among the target positions between the start and end points of the target path L indicated by the training data D1, based on the initial position P2a of the vehicle 1 when the automatic driving mode starts. After moving the vehicle 1 from the initial position P2a toward the entry start target position Pt, it drives along the target path L from the entry start target position Pt. At this time, the vehicle control unit 160 can, for example, use a predetermined evaluation function to determine the target position that is easiest to enter from among the target positions between the start and end points of the target path L indicated by the training data D1, based on the initial position P2a and orientation of the vehicle 1 when the automatic driving mode starts. This allows the vehicle 1 to reach the target path more smoothly, even if the initial position P2a of the vehicle 1 when the automatic driving mode starts is offset from the starting position P2 of the target path L.

[0066] In another embodiment, if the initial position P2a of the vehicle 1 at the start of the automatic driving mode is deviated from the starting position P2 of the target path L, the vehicle control unit 160 may calculate an additional target path to bring the vehicle 1 closer to one of the target positions in the target path L indicated by the training data D1, and then have the vehicle 1 drive along the additional target path before driving along the target path L. The additional target path may be calculated based, for example, the initial position P2a of the vehicle 1, each target position in the target path L, vehicle body information of the vehicle 1 (size of the vehicle 1, minimum turning radius when turning, etc.), and obstacle information around the vehicle 1 (i.e., surrounding information obtained from the camera image of the on-board camera 40).

[0067] Preferably, when the vehicle control unit 160 automatically drives vehicle 1 in automatic driving mode, it drives vehicle 1 at a lower speed than the vehicle speed at which vehicle 1 was driven during the training run. This makes it possible to improve safety when vehicle 1 is driven automatically. Typically, during the training run, the user drives vehicle 1 forward at a relatively fast speed. If vehicle 1 is driven at the same speed as during the training run during automatic driving, it will be driving in reverse at a relatively fast speed. Vehicle 1 driving in reverse at a relatively fast speed may cause fear among pedestrians around vehicle 1 or may induce a collision with pedestrians around vehicle 1.

[0068] Preferably, when the vehicle control unit 160 is automatically driving the vehicle 1 in automatic driving mode, if a stop command is received from the user, the control unit 160 terminates the control of the vehicle 1 to drive automatically. This makes it possible to respond to the user's desire to switch to manual driving earlier. The stop command from the user may be, for example, an accelerator operation, brake operation, or steering operation performed by the user on the vehicle 1. For example, if the user performs an accelerator operation, the automatic driving mode may be canceled and the vehicle 1 may be switched to manual driving without stopping the vehicle 1.

[0069] [Operation Flow of Driver Assistance Systems] Figure 10 is a flowchart illustrating an example of the operations performed by the driver assistance device 10 when it is in teacher driving mode. The flowchart shown in Figure 10 represents a process that the driver assistance device 10 repeatedly performs at predetermined intervals (for example, 100ms intervals) according to a computer program.

[0070] In step S1, the driver assistance device 10 determines whether or not the user has issued a command to start the teacher driving mode. If there is no command to start the teacher driving mode (S1: NO), the driver assistance device 10 terminates the operation flow shown in Figure 10 without performing any special processing. If there is a command to start the teacher driving mode (S1: YES), the device proceeds to step S2.

[0071] In step S2, the driver assistance device 10 acquires camera images from the on-board camera 40, creates map data D2 based on the camera images, and estimates the position of vehicle 1. The driver assistance device 10 then stores the map data D2 and the position of vehicle 1 in RAM 10c. At this time, the position of vehicle 1 is stored in chronological order, and this chronologically ordered position data becomes data indicating the travel path during the training run.

[0072] In step S2, the driver assistance device 10 determines the provisional position of vehicle 1 by dead reckoning, and uses this provisional position as reference information to perform the process of creating map data D2 and estimating the position of vehicle 1.

[0073] In step S3, the driver assistance device 10 determines whether or not the user has issued a command to end the teacher driving mode. If there is no command to end the teacher driving mode (S3: NO), the driver assistance device 10 returns to step S2 and again executes the process of creating map data D2 and estimating the position of vehicle 1. If there is a command to end the teacher driving mode (S3: YES), the process proceeds to step S4.

[0074] In step S4, the driver assistance device 10 reverses the start and end points of the travel path during the training run to generate a target path to be referenced in automatic driving mode, and generates training data D1 for replaying the training run in reverse. The driver assistance device 10 then stores the map data D2 and training data D1 in the external storage device 50.

[0075] Figure 11 is a flowchart illustrating an example of the operations performed by the driver assistance system 10 when it is in automatic driving mode. The flowchart shown in Figure 11 represents a process that the driver assistance system 10 repeatedly performs at predetermined intervals (for example, every 100ms) according to a computer program.

[0076] In step S11, the driver assistance device 10 determines whether or not the user has issued a command to start the automatic driving mode. If there is no command to start the automatic driving mode (S11: NO), the driver assistance device 10 terminates the operation flow shown in Figure 11 without performing any special processing. If there is a command to start the automatic driving mode (S11: YES), the device proceeds to step S12.

[0077] In step S12, the driving support device 10 reads the training data D1 from the external storage device 50.

[0078] In step S13, the driver assistance device 10 acquires a camera image from the in-vehicle camera 40 and estimates the position of the vehicle 1 based on the camera image and the map data D2 stored in the external storage device 50. In step S13, the driver assistance device 10 also determines the provisional position of the vehicle 1 by dead reckoning and uses this provisional position as reference information to perform the position estimation process for the vehicle 1.

[0079] In step S14, the driver assistance device 10 provides feedback control to the steering angle of the vehicle 1's steering system, based on the deviation between each target position in the target path and the vehicle 1's current position, so that the vehicle 1 travels along the target path indicated by the training data D1.

[0080] In step S15, the driver assistance device 10 determines whether vehicle 1 has reached the goal position of the target path indicated by the training data D1. If vehicle 1 has not reached the goal position (S15: NO), the driver assistance device 10 returns to step S13 and performs the same process again. If vehicle 1 has reached the goal position (S15: YES), the driver assistance device 10 stops vehicle 1 and terminates the operation flow shown in Figure 11.

[0081] [effect] As described above, the driver assistance device 10 according to this embodiment makes it possible to automatically drive the vehicle 1 in reverse of the training drive and park it in an appropriate position. This allows the user to perform the training drive in forward driving, which is relatively easy to operate. This prevents unnecessary reversals from being inserted into the target path stored as training data D1, and prevents the final target parking position stored as training data D1 from being in an inappropriate position.

[0082] In particular, the driver assistance device 10 according to this embodiment, when in automatic driving mode, estimates the current position of the vehicle 1 using surrounding sensors (in this embodiment, an on-board camera 40), and provides feedback control of the vehicle 1 based on the current position of the vehicle 1 and each target position of the target path stored as training data D1. This makes it possible to smoothly drive the vehicle 1 along the target path even if the initial position of the vehicle 1 does not perfectly match the starting position of the target path (i.e., the ending position during training driving).

[0083] (modified version) The driver assistance device 10 may be configured to automatically drive vehicle 1 along the travel path that vehicle 1 traveled in the teacher driving mode when it was driven in the teacher driving mode. Here, the automatic driving mode in which vehicle 1 is automatically driven along the travel path that vehicle 1 traveled in the teacher driving mode is referred to as the "forward automatic driving mode," and the automatic driving mode in which vehicle 1 is automatically driven along a travel path that is the reverse of the travel path that vehicle 1 traveled in the teacher driving mode, moving from the end point to the starting point (the automatic driving mode described in the above embodiment) is referred to as the "reverse automatic driving mode."

[0084] In addition, in the forward automatic driving mode, the driver assistance device 10 typically drives the vehicle 1 in the same direction as when it was driven in the teacher driving mode. Then, in the forward automatic driving mode, the driver assistance device 10 drives the vehicle 1 along the path taken when it was driven in the teacher driving mode and stops it at a position corresponding to the end of that path.

[0085] If the reverse-direction automatic driving mode is considered the automatic parking mode, then the forward-direction automatic driving mode corresponds to the automatic exit mode. In other words, this makes it possible to perform both automatic parking and automatic exit with a single training run.

[0086] In order to enable the forward automatic driving mode, the driver assistance device 10 stores the data D1a (see Figure 5) related to the travel path generated during the training mode as training data for forward automatic driving in the external storage device 50 after the training mode has ended. When the automatic driving mode is executed, the driver assistance device 10 decides whether to execute the forward automatic driving mode or the reverse automatic driving mode, for example, according to the user's selection. The driver assistance device 10 may also decide whether to execute the forward automatic driving mode or the reverse automatic driving mode depending on the current position of the vehicle 1 when the automatic driving mode is executed.

[0087] Figure 12 is a flowchart showing an example of the operation performed by the driver assistance device 10 according to this modified example when it is in automatic driving mode. The processes in steps S11 and S13 to S15 are the same as those described in the flowchart of Figure 11, but this flowchart differs from the flowchart of Figure 11 in that the process in step S12 has been changed to steps S12a to S12d.

[0088] Step S12a is a process performed when the user issues a command to execute the automatic driving mode. In this process, the driver assistance device 10 waits for the user to perform a mode selection operation (S12a: NO), and if a mode selection operation is performed (S12a: YES), the process proceeds to step S12b.

[0089] In step S12b, the driver assistance device 10 determines whether the user has selected the forward automatic driving mode. If the user has selected the forward automatic driving mode (S12b: YES), the driver assistance device 10 reads the training data D1a for forward automatic driving from the external storage device 50 (step S12c). If the user has not selected the forward automatic driving mode (S12b: NO), the driver assistance device 10 reads the training data D1 for reverse automatic driving from the external storage device 50 (step S12d).

[0090] After performing the above processing, the driver assistance device 10 controls the vehicle 1 so that it travels along the path indicated by the read-out training data D1 or D1a, similar to the processing described in the flowchart of Figure 11.

[0091] As described above, the driver assistance device 10 according to this embodiment makes it possible to perform both parking and exiting automated driving with a single training run.

[0092] (Other embodiments) The present invention is not limited to the embodiments described above, and various modifications are possible.

[0093] For example, in the above embodiment, the driver assistance device 10 was shown to be applied to the use of automatically parking a vehicle 1 in a parking lot. However, the driver assistance device 10 of the present invention may also be applied to the use of automatically exiting a parking lot.

[0094] Furthermore, although the above embodiment shows that each function of the driver assistance device 10 is realized by processing by the CPU 10a, some or all of the functions of the driver assistance device 10 may be realized by processing by a DSP (Digital Signal Processor) or dedicated hardware circuitry (for example, ASIC or FPGA) instead of, or in conjunction with, processing by the CPU 10a.

[0095] Although specific examples of the present invention have been described in detail above, these are merely illustrative and do not limit the scope of the claims. The technologies described in the claims include various modifications and changes to the specific examples illustrated above. [Industrial applicability]

[0096] The driver assistance device described herein enables more suitable parking assistance or vehicle exit assistance. [Explanation of Symbols]

[0097] 1 vehicle 10. Driving support systems 20 Vehicle drive system 30 Automotive Sensors 40 In-car cameras 50 External storage device 60 HMI 110 Vehicle Information Acquisition Unit 120 Dead Reckoning Section 130 Training Data Generation Unit 140 Position estimation part 150 Target path reading unit 160 Vehicle Control Unit D1 Training Data D2 Map Data P1 1st position P2 2nd position

Claims

1. A driver assistance device that assists in the operation of a vehicle, In the training driving mode, the training data generation unit stores the travel path when the vehicle is driven by the user's driving operations, and generates training data related to the target path when the vehicle is driven automatically in the automatic driving mode based on the travel path. A position estimation unit estimates the current position of the vehicle based on sensor information from surrounding sensors that monitor the surrounding environment of the vehicle, A vehicle control unit moves the vehicle towards the entry start target position among the target positions between the start and end points of the target route, based on the initial position of the vehicle when the automatic driving mode is started, and then drives the vehicle along the target route from the entry start target position of the target route. Equipped with, The vehicle control unit terminates the automatic driving control of the vehicle when the vehicle is being driven in the automatic driving mode and the user performs an accelerator, brake, or steering operation. Driving assistance system.

2. The surrounding sensor is a camera that takes pictures of the area around the vehicle. The driving support device according to claim 1.

3. The training data generation unit generates the training data while creating map data for estimating the current position of the vehicle from the camera image of the camera. The driving support device according to claim 2.

4. The system further includes a dead reckoning unit that estimates the current position of the vehicle based on the temporal changes in sensor values ​​of an on-board sensor that detects the vehicle's driving state. The position estimation unit operates to correct the current position of the vehicle estimated by the dead reckoning unit. The driving support device according to claim 2 or 3.

5. When the vehicle control unit receives a command to execute the reverse automatic driving mode in the automatic driving mode, it causes the vehicle to drive along the target path and stops the vehicle at a position corresponding to the starting point of the travel path. The driving support device according to any one of claims 1 to 4.

6. In the automatic driving mode in the reverse direction, when the vehicle control unit is automatically driving the vehicle, if the user issues a command to stop automatic driving, the control unit terminates the control of automatically driving the vehicle. The driving support device according to any one of claims 1 to 5.

7. When the vehicle control unit receives a command to execute the forward automatic driving mode in the automatic driving mode, it provides feedback control to the vehicle so that the vehicle travels along the travel path, based on the current position of the vehicle estimated by the position estimation unit and each position along the travel path. The driving support device according to any one of claims 1 to 6.

8. When the vehicle control unit receives a command to execute the forward automatic driving mode in the automatic driving mode, it drives the vehicle along the travel path and stops the vehicle at a position corresponding to the end of the travel path. The driving support device according to any one of claims 1 to 7.

9. When the vehicle control unit receives a command to execute the forward automatic driving mode, it causes the vehicle to travel along the travel path in the same direction as the driving direction in the teacher driving mode. The driving support device according to claim 7 or 8.

10. In the forward automatic driving mode, if the vehicle control unit receives a command from the user to stop automatic driving while the vehicle is being driven automatically, it terminates the control of the vehicle to drive automatically. The driving support device according to any one of claims 7 to 9.

11. The vehicle control unit automatically drives the vehicle from a position between the start and end points of the target route indicated by the training data. The driving support device according to any one of claims 1 to 10.

12. The driving assistance device determines, based on the user's selection, whether to execute a forward automatic driving mode in which the vehicle is automatically driven along the travel path, or a reverse automatic driving mode in which the vehicle is automatically driven along a travel path that is the reverse of the travel path, moving from the end point towards the starting point. The driving support device according to any one of claims 1 to 11.

13. The driving assistance device determines, based on the vehicle's current position, whether to execute a forward automatic driving mode in which the vehicle is automatically driven along the travel path, or a reverse automatic driving mode in which the vehicle is automatically driven along a travel path that is the reverse of the travel path, moving from the end point towards the starting point. The driving support device according to any one of claims 1 to 12.

14. A driver assistance method that assists in the operation of a vehicle, In the training driving mode, the process involves storing the travel path when the vehicle is driven by the user's driving operations, and generating training data related to the target path when the vehicle is driven automatically in the automatic driving mode based on the travel path. A process to estimate the current position of the vehicle based on sensor information from surrounding sensors that monitor the environment around the vehicle, The system includes a process of moving the vehicle towards an entry start target position among the target positions between the start and end points of the target route, based on the initial position of the vehicle when the automatic driving mode is started, and then driving along the target route from the entry start target position of the target route. When the vehicle is being driven in the aforementioned automatic driving mode, if the user operates the accelerator, brake, or steering wheel, the automatic driving control of the vehicle is terminated. Driving assistance methods.

15. The surrounding sensor is a camera that takes pictures of the area around the vehicle. The driving assistance method according to claim 14.

16. While creating map data for estimating the vehicle's current position from the camera image of the aforementioned camera, the training data is generated. The driving assistance method according to claim 15.

17. The system further includes a process for estimating the current position of the vehicle based on the temporal change in sensor values ​​of an on-board sensor that detects the vehicle's driving state. It operates to correct the estimated current position of the vehicle, The driving assistance method according to claim 15 or 16.

18. When a command to execute the reverse automatic driving mode in the aforementioned automatic driving mode is received, the vehicle is made to travel along the target path and stop at a position corresponding to the starting point of the travel path. The driving assistance method according to any one of claims 14 to 17.

19. In the reverse automatic driving mode of the aforementioned automatic driving mode, if the user issues a command to stop automatic driving while the vehicle is being driven automatically, the control to automatically drive the vehicle is terminated. The driving assistance method according to any one of claims 14 to 18.

20. When an execution command for the forward automatic driving mode in the aforementioned automatic driving mode is received, the vehicle is controlled by feedback based on the vehicle's current position and each position along the travel path, so that the vehicle travels along the travel path. The driving assistance method according to any one of claims 14 to 19.

21. When a command to execute the forward automatic driving mode in the aforementioned automatic driving mode is received, the vehicle is made to travel along the aforementioned travel path and to stop at a position corresponding to the end of the aforementioned travel path. The driving assistance method according to any one of claims 14 to 20.

22. When the command to execute the forward automatic driving mode is received, the vehicle is driven along the travel path in the same direction as the driving direction in the teacher driving mode. The driving assistance method according to claim 20 or 21.

23. In the forward automatic driving mode, if the user issues a command to stop automatic driving while the vehicle is being driven automatically, the control to automatically drive the vehicle is terminated. The driving assistance method according to any one of claims 20 to 22.

24. The system will automatically drive from a position between the start and end points of the target route indicated by the aforementioned training data. The driving assistance method according to any one of claims 14 to 23.

25. Based on the user's selection, the system determines whether to execute a forward automatic driving mode in which the vehicle automatically drives along the aforementioned travel path, or a reverse automatic driving mode in which the vehicle automatically drives along a travel path that is the reverse of the aforementioned travel path, moving from the end point towards the starting point. The driving assistance method according to any one of claims 14 to 24.

26. Based on the vehicle's current position, the system determines whether to execute a forward automatic driving mode, in which the vehicle is automatically driven along the aforementioned travel path, or a reverse automatic driving mode, in which the vehicle is automatically driven along a travel path that is the reverse of the aforementioned travel path, moving from the end point towards the starting point. The driving assistance method according to any one of claims 14 to 25.

Images