Driving assistance device and driving assistance method

Abstract

To provide a driving assistance device which enables more suitable parking or exiting assistance.SOLUTION: A driving assistance device comprises: a teacher data generation part which stores a movement path when a vehicle travels by a user's driving operation in a teacher travel mode, and generates teacher data concerning a target path for automatically traveling the vehicle in an automatic travel mode on the basis of the movement path; a position estimation part which estimates a current position of the vehicle on the basis of sensor information from a surrounding sensor monitoring the environment around the vehicle; and a vehicle control part which, on the basis of the initial position of the vehicle at the start of the automatic travel mode, moves the vehicle toward an entry start target position among target positions between a start point and an end point of the target path, and then causes the vehicle to travel along the target path from the entry start target position. The vehicle control part terminates automatic travel control of the vehicle if an accelerator operation, brake operation, or steering operation by the user is detected while the vehicle is traveling in the automatic travel mode.SELECTED DRAWING: Figure 2B

Description

[Technical field]

[0001] The present disclosure relates to a driving assistance device and a driving assistance method. [Background technology]

[0002] Generally, parking spaces in homes are often narrow and can make parking difficult, so there is a high demand for autonomous driving to park and exit the vehicle.

[0003] To realize this type of automatic driving, various driving assistance devices are known. For example, this type of driving assistance device performs automatic driving to automatically assist the vehicle in parking from the initial stopping position to the target parking position when the driver stops the vehicle at a predetermined initial stopping position. The driving assistance device automatically recognizes the parking space and the position of the vehicle using various sensors mounted on the vehicle, and automatically performs steering, accelerator, brake, etc.

[0004] Patent Document 1 discloses a technology for achieving such automated driving in which the vehicle is driven by a user (hereinafter referred to as "user") in the vehicle from a predetermined position outside a parking lot to a target parking position, and the travel route taken during this process is stored as training data, thereby allowing the vehicle to automatically travel along that travel route in subsequent parking situations. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2017-138664 A [Non-patent literature]

[0006] [Non-Patent Document 1] Mikael Persson et al. “Lambda Twist: An Accurate Fast Robust Perspective Three Point (P3P) Solver.”, ECCV 2018, pp 334-349, published in 2018, http: / / openaccess.thecvf.com / content_ECCV_2018 / papers / Mikael_Persson_Lambda_Twist_An_ECCV_2018_paper.pdf Summary of the Invention [Problem to be solved by the invention]

[0007] According to the conventional driving assistance device described in Patent Document 1, a single teacher run (meaning a run performed by the user to generate teacher data; the same applies below) can be performed by the user's driving operation, and thereafter the vehicle can be automatically parked or pulled out, making this technology highly convenient.

[0008] One issue facing such technology is how to improve safety when autonomously driving a vehicle.

[0009] The present disclosure has been made in consideration of the above problems, and aims to provide a driving assistance device and a driving assistance method that are capable of more suitable parking assistance or exit assistance. [Means for solving the problem]

[0010] The present disclosure, which solves the above-mentioned problems, is A driving assistance device that assists driving of a vehicle, a teacher data generating unit that stores a travel route when the vehicle is traveled by a driving operation of a user in a teacher travel mode, and generates teacher data related to a target route when the vehicle is automatically traveled in an automatic travel mode based on the travel route; a position estimation unit that estimates a current position of the vehicle based on sensor information from a surrounding sensor that monitors a surrounding environment of the vehicle; a vehicle control unit that moves the vehicle toward an entry start target position among each target position between the start point and the end point of the target route based on an initial position of the vehicle at the start of the autonomous driving mode, and then causes the vehicle to travel along the target route from the entry start target position of the target route; Equipped with The vehicle control unit terminates the automatic driving control of the vehicle when a user performs an accelerator operation, a brake operation, or a steering operation while the vehicle is traveling in the automatic driving mode. It is a driving assistance device.

[0011] In other respects, A driving assistance method for assisting driving of a vehicle, comprising: A process of storing a travel route when the vehicle is traveled by a driving operation of a user in a teacher travel mode, and generating teacher data related to a target route when the vehicle is automatically traveled in an automatic travel mode based on the travel route; A process of estimating a current position of the vehicle based on sensor information of a surrounding sensor that monitors a surrounding environment of the vehicle; a process of moving the vehicle toward an entry start target position among the target positions between the start point and the end point of the target route based on an initial position of the vehicle at the start of the autonomous driving mode, and then causing the vehicle to travel along the target route from the entry start target position of the target route, When the user performs an accelerator operation, a brake operation, or a steering operation while the vehicle is traveling in the autonomous driving mode, the autonomous driving control of the vehicle is terminated. A driving assistance method. Effect of the Invention

[0012] According to the driving assistance device according to the present disclosure, more suitable parking assistance or exit assistance is possible. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram showing an example of an overall configuration of a vehicle. [Figure 2A] FIG. 1 is a diagram showing an example of a functional block diagram of a driving support device (in a teacher driving mode); [Figure 2B] FIG. 1 is a diagram showing an example of a functional block of a driving assistance device (in an autonomous driving mode); [Figure 3A] FIG. 13 is a diagram showing an example of a route traveled by a vehicle in teacher driving mode. [Figure 3B] FIG. 1 is a diagram showing an example of a travel route in which a vehicle is controlled to travel in an automatic driving mode; [Figure 4A] FIG. 13 is a diagram showing a traveling mode in a teacher traveling mode; [Figure 4B] FIG. 2 is a diagram showing a driving manner in an automatic driving mode. [Diagram 5] FIG. 13 is a diagram showing an example of teacher data generated by a teacher data generating unit. [Figure 6] FIG. 13 is a diagram showing an example of map data generated by a training data generating unit; [Figure 7] FIG. 13 is a diagram for explaining an example of processing by a position estimation unit; [Figure 8] FIG. 1 is a diagram showing an example of a vehicle's movement path when feedback control based on the current position of the vehicle is not performed (here, when control is performed only by dead reckoning). [Figure 9] FIG. 1 is a diagram for explaining a more preferred embodiment of a vehicle control unit. [Figure 10] A flowchart showing an example of an operation performed by the driving support device in a teacher driving mode. [Figure 11] A flowchart showing an example of an operation performed by a driving assistance device in an automatic driving mode. [Figure 12] A flowchart showing an example of an operation performed by a driving assistance device according to a modified example in an automatic driving mode. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functions are denoted by the same reference numerals, and redundant description will be omitted.

[0015] [Overall vehicle configuration] Hereinafter, an example of the configuration of a vehicle according to an embodiment will be described with reference to FIG.

[0016] FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle 1.

[0017] The vehicle 1 includes a driving assistance device 10, a vehicle drive device 20, an on-board sensor 30, an on-board camera 40, an external storage device 50, and an HMI (Human Machine Interface) 60.

[0018] The vehicle drive device 20 is a means for realizing the drive, braking, and turning motion required for driving the vehicle 1, and is configured to include, for example, a drive motor, a power transmission mechanism, a brake device, a steering device, etc., and an electronic driving support device that controls them. The vehicle drive device 20 generates power, for example, with a drive motor, and transmits the power to the wheels via a power transmission mechanism (such as a propeller shaft, a differential gear, and a drive shaft), to drive the vehicle 1. Note that the operation of the vehicle drive device 20 according to this embodiment is controlled by the driving support device 10 in the automatic driving mode.

[0019] The on-vehicle sensors 30 are various sensors mounted on the vehicle 1 and detect the driving state of the vehicle 1. The on-vehicle sensors 30 include, for example, an accelerator opening sensor that detects an accelerator opening, a steering angle sensor that detects a steering angle of a steering device, an acceleration sensor that detects acceleration acting in the longitudinal direction of the vehicle 1, a torque sensor that detects torque acting on a power transmission mechanism between the wheels of the vehicle 1 and a drive motor, a vehicle speed sensor that detects the vehicle speed of the vehicle 1, a wheel speed sensor, etc. The on-vehicle sensors 30 output sensor information obtained by their own detection to the driving assistance device 10.

[0020] The vehicle-mounted camera 40 (corresponding to the "surrounding sensor" of the present invention) is a surrounding sensor mounted on the vehicle 1 and monitors the surrounding environment of the vehicle 1. In this embodiment, the vehicle-mounted camera 40 is applied to an application in which, for example, an object (typically an object fixed on the ground) existing around the vehicle 1 is detected, and the position of the vehicle 1 is estimated from the positional relationship between the vehicle 1 and the object existing around the vehicle 1. The vehicle-mounted camera 40 is composed of, for example, four cameras arranged so as to capture images in four directions, namely, the front direction, the rear direction, the left direction, and the right direction of the vehicle 1. The vehicle-mounted camera 40 outputs the camera image generated by itself to the driving assistance device 10.

[0021] As a surrounding sensor for estimating the self-position of the vehicle 1, instead of the in-vehicle camera 40, a LiDAR, a radar, an ultrasonic sensor, or the like may be used.

[0022] The external storage device 50 is, for example, an auxiliary storage device such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The external storage device 50 stores, for example, teacher data D1 and map data D2 generated when the driving assistance device 10 runs in a teacher running mode.

[0023] The HMI 60 is a user interface that accepts input operations from a user who is in the vehicle 1, such as a touch panel, a commander, a button, or an operation key. The HMI 60 is configured to be able to accept input operations such as an execution command when executing the teacher driving mode or an execution command when executing the automatic driving mode. The HMI 60 outputs information related to the input operations accepted from the user to the driving assistance device 10.

[0024] The driving support device 10 is an electronic control unit that controls each part of the vehicle 1. The driving support device 10 controls the vehicle drive device 20 so as to optimize the driving condition of the vehicle 1 while referring to sensor information from the on-board sensor 30. The driving support device 10 is also configured to control the vehicle drive device 20 so as to automatically drive the vehicle 1.

[0025] The driving support device 10 includes, for example, a central processing unit (CPU) 10a, a read only memory (ROM) 10b, a random access memory (RAM) 10c, an input port (not shown), and an output port (not shown), etc. Each function of the driving support device 10, which will be described later, is realized, for example, by the CPU 10a referring to a control program and various data stored in the RAM 10c and the ROM 10b.

[0026] In addition, the driving assistance device 10 is interconnected with the vehicle drive device 20, the on-board sensor 30, the on-board camera 40, the external storage device 50, and the HMI 60 via an on-board network (e.g., a communication network conforming to the CAN communication protocol), and is capable of transmitting and receiving necessary data and control signals to and from each other.

[0027] [Configuration of driving support device] Next, an example of the configuration of the driving support device 10 will be described with reference to Fig. 2A to Fig. 9. The driving support device 10 according to this embodiment is applied to, for example, automatic parking of the vehicle 1 in a parking lot.

[0028] The driving support device 10 is configured to be able to switch between a teacher driving mode and an automatic driving mode based on an input operation by a user. The teacher driving mode is a mode for registering a target route for automatic driving of the vehicle 1 in the automatic driving mode. In the teacher driving mode, the vehicle 1 is controlled to travel by the driving operation of the user. On the other hand, the automatic driving mode is a mode in which the vehicle 1 is automatically driven according to the target route registered in the teacher driving mode (however, the vehicle 1 is driven from the end point side to the start point side of the travel route in the teacher driving mode. Details will be described later). In the automatic driving mode, the vehicle 1 is automatically controlled to travel by the driving support device 10 without the need for driving operation by the user.

[0029] 2A and 2B are diagrams showing an example of functional blocks of the driving support device 10 according to this embodiment. Note that Fig. 2A shows only functional units that function when the vehicle 1 is in the teacher driving mode, and Fig. 2B shows only functional units that function when the vehicle 1 is in the automatic driving mode.

[0030] Fig. 3A is a diagram showing an example of a travel route (route indicated by solid arrows) when the vehicle 1 travels in the teacher driving mode, and Fig. 3B is a diagram showing an example of a travel route (route indicated by solid arrows) when the vehicle 1 is controlled to travel in the automatic driving mode. Fig. 3A shows an example of an exit operation from a target parking position (hereinafter also referred to as "first position") P1 to a desired position (hereinafter also referred to as "second position") P2 outside the parking lot by a driving operation by a user in the teacher driving mode, and Fig. 3B shows an example of an entrance operation from the second position P2 to the first position P1 by automatic driving in the automatic driving mode.

[0031] 4A and 4B show the relationship between the teacher driving mode and the automatic driving mode, with FIG. 4A showing the driving manner in the teacher driving mode and FIG. 4B showing the driving manner in the automatic driving mode.

[0032] <When teacher driving mode is running> First, the functional configuration of the driving support device 10 that functions when the teacher driving mode is executed will be described.

[0033] When the teacher driving mode is executed, the vehicle information acquisition unit 110, the dead reckoning unit 120, and the teacher data generation unit 130 function in the driving assistance device 10 (see FIG. 2A).

[0034] The vehicle information acquisition unit 110 acquires sensor information indicating the traveling state of the vehicle 1 from the on-board sensor 30. Then, the vehicle information acquisition unit 110 transmits the acquired sensor information to the dead reckoning unit 120.

[0035] The dead reckoning unit 120 estimates the current position of the vehicle 1 based on the temporal change in the sensor value of the on-board sensor 30 that detects the driving state of the vehicle 1. For example, the dead reckoning unit 120 calculates the amount of movement of the vehicle 1 from a reference position (for example, the driving start position when the teacher driving mode is started) based on the temporal change in the vehicle speed and yaw rate indicated by the on-board sensor 30, and estimates the current position of the vehicle 1 based on the amount of movement.

[0036] The current position of the vehicle 1 estimated by the dead reckoning unit 120 is a rough current position of the vehicle 1, and since the position estimation by the dead reckoning unit 120 has low estimation accuracy, in the driving assistance device 10 according to this embodiment, the actual current position of the vehicle 1 is corrected based on a camera image generated by the in-vehicle camera 40. The reasons for the low estimation accuracy of the position estimation by the dead reckoning unit 120 include the difficulty in constructing a perfect kinematics model (for example, tire diameter and tread width), the fact that the observation value of the in-vehicle sensor 30 necessarily contains noise, and the fact that external factors (for example, tire slippage) that cannot be observed by the in-vehicle sensor 30 greatly affect the position estimation. Furthermore, in the position estimation by the dead reckoning unit 120, errors due to these factors accumulate.

[0037] The teacher data generating unit 130 stores the travel route during teacher driving, and generates teacher data D1 related to the target route when the vehicle 1 is automatically driven in the automatic driving mode from the travel route. The teacher data generating unit 130 starts storing the travel route, for example, when a teacher driving mode start command is received from the user, and ends storing the travel route when a teacher driving mode end command is received from the user. The teacher data generating unit 130 typically starts storing the travel route when the vehicle 1 is stopped at the parking target position (first position) P1, and ends storing the travel route when the vehicle 1 is stopped at a desired position (second position) P2 outside the parking lot. The teacher data D1 generated by the teacher data generating unit 130 is stored in the external storage device 50.

[0038] Fig. 5 is a diagram showing an example of teacher data D1 generated by the teacher data generating unit 130. Data D1a shown in Fig. 5 is data showing the route actually traveled by the vehicle 1 and the driving behavior at each position on the route, which is sequentially stored during teacher driving, and the teacher data D1 is generated from this data D1a during teacher driving.

[0039] The teacher data D1 includes, for example, a target route when traveling in automatic driving mode, the orientation (i.e., direction) of the vehicle 1 at each target position on the target route, the driving direction (i.e., forward or reverse) of the vehicle 1 at each target position on the target route, and reference driving information at each target position on the target route.

[0040] The target route of the teacher data D1 is generated by reversing the movement route during teacher driving from the end point side to the start point side. In other words, each target position of the target route of the teacher data D1 is generated by reversing each position of the vehicle 1 estimated when the vehicle 1 traveled from the first position P1 to the second position P2 during teacher driving from the second position P2 side to the first position P1 side.

[0041] The orientation of the vehicle 1 at each target position on the target route stored in the teacher data D1 is the same as the orientation of the vehicle 1 at each position during teacher driving. Also, the driving direction of the vehicle 1 at each target position on the target route stored in the teacher data D1 is opposite to the driving direction in teacher driving mode.

[0042] Furthermore, the reference driving information at each target position on the target route stored in the teacher data D1 is, for example, driving information (e.g., steering angle and vehicle speed) detected at each position on the moving route when the vehicle 1 is traveling in the teacher driving mode. This reference driving information serves as a reference when the vehicle control unit 160 controls the vehicle driving device 20 (e.g., steering angle of the steering device) at each position on the moving route when performing automatic driving. However, when causing the vehicle 1 to travel automatically, the vehicle control unit 160 feedback controls the vehicle driving device 20 (e.g., steering angle of the steering device) using each target position on the target route as a target value so that the vehicle 1 does not deviate from the target route indicated by the teacher data D1 (details will be described later).

[0043] Furthermore, the teacher data generation unit 130 may also store data D1a relating to the route actually traveled by the vehicle 1 that was stored during the teacher driving in the external storage device 50 so that the teacher driving can be reproduced (see the modified example described later).

[0044] Here, the teacher data generating unit 130 generates the teacher data D1 while creating map data D2 for estimating the current position of the vehicle 1 from a camera image of the in-vehicle camera 40. This technology is also called a SLAM (Simultaneous Localization and Mapping) method.

[0045] Fig. 6 is a diagram showing an example of map data D2 generated by the teacher data generating unit 130. Note that Fig. 6 shows a bird's-eye view of the position in real space of a feature point Q in a real scene stored in the map data D2.

[0046] The map data D2 is data that stores, for each of a plurality of feature points in a real scene, the three-dimensional position of the feature point in real space and the feature amount of the feature point obtained from a camera image taken when the map data D2 was created, in association with each other. The feature points stored as the map data D2 are, for example, parts (e.g., corners) from which a characteristic image pattern can be obtained from a camera image of an object (e.g., a tree, a wall, a pillar, etc.) that can serve as a landmark in the real scene. The plurality of feature points in the map data D2 are stored so as to be identifiable, for example, by an identification number.

[0047] The three-dimensional positions in real space of the feature points stored in the map data D2 are represented in a three-dimensional Cartesian coordinate system (X, Y, Z).

[0048] The feature amounts of the feature points stored in the map data D2 may be luminance or density on the camera image, as well as SIFT (Scale Invariant Feature Transform) feature amounts or SURF (Speed ​​Up Robust Features) feature amounts. The feature amount data of the feature points stored in the map data D2 may be of feature points at the same three-dimensional position, or may be stored separately for each camera shooting position or shooting direction when the feature points were photographed. The feature amount data of the feature points stored in the map data D2 may be stored in association with an image of an object having the feature points.

[0049] The teacher data generating unit 130 specifies the coordinates of the feature points in the real scene based on, for example, stereo photogrammetry. Specifically, the teacher data generating unit 130 reads out a plurality of camera images generated at different times, and associates the same feature points that are commonly captured in the plurality of camera images. Then, the teacher data generating unit 130 acquires, for example, information related to the provisional position of the vehicle 1 when the plurality of camera images are generated from the dead reckoning unit 120, and specifies provisional coordinates of the feature points in the real scene based on the principle of triangulation. Then, the teacher data generating unit 130 performs bundle adjustment using, for example, the provisional position of the vehicle 1 and the provisional coordinates of the feature points in the real scene as reference information, and calculates the formal position of the vehicle 1 and the formal coordinates of the feature points in the real scene so that the reprojection error when each feature point in the real scene is projected onto all camera images is minimized. Then, the teacher data generating unit 130 stores the formal position of the vehicle 1 as data D1a indicating the position of the vehicle 1 during teacher driving. Moreover, the teacher data generating unit 130 stores the formal coordinates of the feature points in the actual scene in the external storage device 50 as map data D2.

[0050] The map data D2 may be measured in advance using a LIDAR (Light Detection and Ranging) or a stereo camera instead of using the SLAM method. However, since the map data D2 often contains distortion depending on the measurement method used when generating the map data D2, it is preferable to use the SLAM method from the viewpoint of suppressing a decrease in the accuracy of position estimation caused by distortion of the map data D2 itself.

[0051] <When in automatic driving mode> Next, the functional configuration of the driving assistance device 10 that functions when the autonomous driving mode is executed will be described.

[0052] When the autonomous driving mode is executed, the vehicle information acquisition unit 110, the dead reckoning unit 120, the position estimation unit 140, the target route reading unit 150, and the vehicle control unit 160 function in the driving assistance device 10 (see FIG. 2B).

[0053] The vehicle information acquisition unit 110 and the dead reckoning unit 120 have the same configuration as described above, and therefore a description thereof will be omitted here.

[0054] The position estimation unit 140 estimates the current position of the vehicle 1 based on the map data D2 and the camera image of the vehicle-mounted camera 40.

[0055] Fig. 7 is a diagram for explaining an example of the processing of the position estimation unit 140. In Fig. 7, points R1, R2, and R3 represent three feature points extracted from the camera image of the vehicle-mounted camera 40, and points Q1, Q2, and Q3 represent the three-dimensional positions in real space of the feature points R1, R2, and R3 stored in the map data D2. RP1 represents the imaging surface of the vehicle-mounted camera 40. Point P1 represents the position of the vehicle-mounted camera 40 (i.e., the position of the vehicle 1) determined from the three feature points R1, R2, and R3 extracted from the camera image of the vehicle-mounted camera 40 and the points Q1, Q2, and Q3 stored in the map data D2.

[0056] For example, the position estimation unit 140 first compares feature points extracted from the camera image of the vehicle-mounted camera 40 with feature points stored in the map data D2 using pattern matching, feature amount search, etc. Then, the position estimation unit 140 randomly selects several (e.g., 3 to 6) feature points among the feature points extracted from the camera image of the vehicle-mounted camera 40 that can be compared with the feature points stored in the map data D2, and estimates the position of the vehicle 1 in the real space based on the positions of these several feature points in the camera image and the three-dimensional positions of the several feature points in the real space stored in the map data D2. At this time, the position estimation unit 140 calculates the position of the vehicle 1 by solving the PnP problem using a known method such as Lambda Twist (see, for example, Non-Patent Document 1).

[0057] In addition, when comparing the feature points extracted from the camera image of the in-vehicle camera 40 with the feature points stored in the map data D2, the position estimation unit 140 may, for example, use the current position of the vehicle 1 estimated by the dead reckoning unit 120 as a reference to narrow down the feature points stored in the map data D2 to be compared with the feature points extracted from the camera image of the in-vehicle camera 40.

[0058] The information on the current position of the vehicle 1 estimated by the position estimation unit 140 is sent to the vehicle control unit 160. The information on the current position of the vehicle 1 estimated by the position estimation unit 140 includes information on the two-dimensional position of the vehicle 1 in the real space (X coordinate, Y coordinate) and information on the orientation of the vehicle 1.

[0059] The target route reading unit 150 reads out the teacher data D1 for moving the vehicle 1 from the external storage device 50, and sends it to the vehicle control unit 160. When the driving assistance device 10 is configured to be able to execute a forward direction automatic driving mode and a reverse direction automatic driving mode as the automatic driving mode, the target route reading unit 150 selects, based on a user's command signal from the HMI 60, the target route to be read out from the external storage device 50 from either the target route related to the forward direction automatic driving mode or the target route related to the reverse direction automatic driving mode, and sends it to the vehicle control unit 160 (see the modified example described later).

[0060] When the vehicle control unit 160 receives an execution command for the automatic driving mode from the user, the vehicle control unit 160 moves the vehicle 1 along the target route indicated by the teacher data D1 received from the target route reading unit 150 while recognizing the current position of the vehicle 1. That is, when the vehicle control unit 160 receives an execution command for the automatic driving mode from the user, the vehicle control unit 160 automatically drives the vehicle 1 from the second position P2 to the first position P1 along the target route indicated by the teacher data D1 in the opposite direction to the driving direction in the teacher driving mode. Then, the vehicle control unit 160 stops the vehicle 1 at the first position P1, which corresponds to the starting point of the teacher driving mode.

[0061] Here, the vehicle control unit 160 performs feedback control of the vehicle 1 based on the current position of the vehicle 1 estimated by the position estimation unit 140 and each target position in the target route indicated by the teacher data D1. The reason for performing such control is that the initial position of the vehicle 1 when the automatic driving mode is started may not exactly match the initial position of the target route indicated by the teacher data D1 (here, the second position P2). In addition, in the method of reproducing the moving route using only dead reckoning, errors are likely to be large, especially in low-speed areas and moving routes with turns. In addition, when the traveling speed of the vehicle 1 is made different between teacher driving and automatic driving, the steering angle of the vehicle 1 itself for making the vehicle 1 travel along the same moving route also changes between teacher driving and automatic driving.

[0062] FIG. 8 is a diagram showing an example of a moving route of the vehicle 1 when feedback control based on the current position of the vehicle 1 is not performed (here, control is performed only by dead reckoning). FIG. 8 shows a moving route of the vehicle 1 when the actual initial position (here, the orientation of the vehicle 1) P2' of the vehicle 1 when the automatic driving mode is started is deviated from the initial position P2 of the target route indicated by the teacher data D1. In such a case, as shown by the route L' in FIG. 8, the vehicle 1 moves significantly deviating from the target route indicated by the teacher data D1. Even if the error in the orientation of the vehicle 1 is only 1 degree, after traveling 50 m, the goal position P1' will be deviated by 0.87 m from the goal position P1 of the target route.

[0063] In this regard, as in the control method of the vehicle control unit 160 according to this embodiment, feedback control of the vehicle 1 based on the current position of the vehicle 1 estimated by the position estimation unit 140 and each target position on the target route indicated by the teacher data D1 makes it possible to accurately drive the vehicle 1 along the target route. Note that at this time, the vehicle control unit 160 typically suppresses such position deviation by feedback control of the steering angle of the steering device of the vehicle 1.

[0064] Furthermore, when the vehicle control unit 160 causes the vehicle 1 to automatically drive, it is not necessary for the vehicle control unit 160 to start automatic driving from the starting point of the target route indicated by the teacher data D1, but the vehicle control unit 160 may cause the vehicle 1 to automatically drive from a position between the starting point and the end point of the target route indicated by the teacher data D1.

[0065] FIG. 9 is a diagram for explaining a more preferred embodiment of the vehicle control unit 160. As shown in FIG. 9, the vehicle control unit 160 preferably determines an entry start target position Pt among the target positions between the start point and the end point of the target route L indicated by the teacher data D1 based on the initial position P2a of the vehicle 1 at the start of the automatic driving mode, moves the vehicle 1 from the initial position P2a toward the entry start target position Pt, and then drives the vehicle 1 along the target route L from the entry start target position Pt. At this time, the vehicle control unit 160 may determine the most easily enterable target position among the target positions between the start point and the end point of the target route L indicated by the teacher data D1, for example, using a predetermined evaluation function, from the initial position P2a and orientation of the vehicle 1 at the start of the automatic driving mode. This allows the vehicle 1 to reach the target route more smoothly, even if the initial position P2a of the vehicle 1 at the start of the automatic driving mode is deviated from the position P2 of the start point of the target route L.

[0066] As yet another aspect, when the initial position P2a of the vehicle 1 at the start of the autonomous driving mode is deviated from the position P2 of the start point of the target route L, the vehicle control unit 160 may calculate an additional target route for moving the vehicle 1 closer to any of the target positions in the target route L indicated by the teacher data D1, and may cause the vehicle 1 to travel along the additional target route, and then to travel along the target route L. The additional target route may be calculated based on, for example, the initial position P2a of the vehicle 1, each target position in the target route L, vehicle body information of the vehicle 1 (such as the size of the vehicle 1 and the minimum turning radius when turning), and obstacle information around the vehicle 1 (i.e., surrounding information obtained from a camera image of the vehicle-mounted camera 40).

[0067] Preferably, when the vehicle control unit 160 automatically drives the vehicle 1 in the automatic driving mode, the vehicle control unit 160 drives the vehicle 1 at a lower speed than the speed at which the vehicle 1 drives in the teacher driving mode. This can improve safety when the vehicle 1 is automatically driven. Typically, during teacher driving, the user drives the vehicle 1 forward at a relatively high speed. If the vehicle 1 is driven at the same speed as during teacher driving during automatic driving, the vehicle 1 will drive backward at a relatively high speed. The vehicle 1 driving backward at a relatively high speed may cause fear among pedestrians around the vehicle 1 or may induce a collision with pedestrians around the vehicle 1.

[0068] Preferably, when the vehicle control unit 160 is automatically driving the vehicle 1 in the automatic driving mode, if a stop command is received from the user, the vehicle control unit 160 ends the control of automatically driving the vehicle 1. This makes it possible to respond to a user's desire to switch to manual driving earlier. Note that the stop command from the user may be, for example, an accelerator operation, brake operation, steering operation, or the like, performed by the user on the vehicle 1. For example, when the accelerator operation is performed by the user, the automatic driving mode may be cancelled and the vehicle may be shifted to manual driving without stopping the vehicle 1.

[0069] [Operation flow of driving support device] Fig. 10 is a flowchart showing an example of an operation performed in the teacher driving mode by the driving assistance device 10. The flowchart shown in Fig. 10 is a process that is repeatedly performed by the driving assistance device 10 at a predetermined interval (for example, 100 ms interval) according to a computer program, for example.

[0070] In step S1, the driving support device 10 judges whether or not a command to start the teacher driving mode has been issued from the user. If there is no command to start the teacher driving mode (S1: NO), the driving support device 10 ends the operation flow in FIG. 10 without performing any particular process, and if there is a command to start the teacher driving mode (S1: YES), the driving support device 10 proceeds to step S2.

[0071] In step S2, the driving support device 10 acquires a camera image from the vehicle-mounted camera 40, and creates map data D2 based on the camera image, and estimates the position of the vehicle 1. Then, the driving support device 10 stores the map data D2 and the position of the vehicle 1 in the RAM 10c. At this time, the position of the vehicle 1 is stored in chronological order, and this chronological position data becomes data indicating the travel route during teacher driving.

[0072] In step S2, the driving support device 10 grasps the provisional position of the vehicle 1 by dead reckoning, and performs the process of creating the map data D2 and the process of estimating the position of the vehicle 1 by using this provisional position as reference information.

[0073] In step S3, the driving support device 10 judges whether or not there is a command from the user to end the teacher driving mode. If there is no command to end the teacher driving mode (S3: NO), the driving support device 10 returns to step S2 and executes the process of creating the map data D2 and the process of estimating the position of the vehicle 1 again. If there is a command to end the teacher driving mode (S3: YES), the driving support device 10 proceeds to step S4.

[0074] In step S4, the driving support device 10 inverts the start point side and the end point side of the movement route during teacher driving to generate a target route to be referred to in the automatic driving mode, and generates teacher data D1 for replaying the teacher driving in reverse. Then, the driving support device 10 stores the map data D2 and the teacher data D1 in the external storage device 50.

[0075] Fig. 11 is a flowchart showing an example of an operation performed in the automatic driving mode by the driving assistance device 10. The flowchart shown in Fig. 11 is a process that is repeatedly performed by the driving assistance device 10 at a predetermined interval (for example, at 100 ms intervals) according to a computer program, for example.

[0076] In step S11, the driving support device 10 judges whether or not there is a command from the user to start the automatic driving mode. At this time, if there is no command to start the automatic driving mode (S11: NO), the driving support device 10 ends the operation flow of FIG. 11 without performing any particular process, and if there is a command to start the automatic driving mode (S11: YES), the driving support device 10 proceeds to step S12.

[0077] In step S12, the driving assistance device 10 reads out the teacher data D1 from the external storage device 50.

[0078] In step S13, the driving support device 10 acquires a camera image from the in-vehicle camera 40, and estimates the position of the vehicle 1 based on the camera image and the map data D2 stored in the external storage device 50. In addition, in this step S13, the driving support device 10 grasps a tentative position of the vehicle 1 by dead reckoning, and performs a process of estimating the position of the vehicle 1 using this tentative position as reference information.

[0079] In step S14, the driving assistance device 10 feedback controls the steering angle of the steering device of the vehicle 1 based on the deviation between each target position on the target route and the current position of the vehicle 1 so that the vehicle 1 travels along the target route indicated by the teacher data D1.

[0080] In step S15, the driving support device 10 judges whether the vehicle 1 has reached the goal position of the target route indicated by the teacher data D1. If the vehicle 1 has not reached the goal position (S15: NO), the driving support device 10 returns to step S13 and executes the same process again. If the vehicle 1 has reached the goal position (S15: YES), the driving support device 10 stops the vehicle 1 and ends the operation flow of FIG.

[0081] [effect] As described above, the driving support device 10 according to this embodiment makes it possible to automatically drive the vehicle 1 in a manner that plays back the teacher driving in reverse, and park the vehicle in an appropriate position. This allows the user to perform teacher driving in forward driving, which is relatively easy to drive. This makes it possible to prevent unnecessary turns from being inserted in the target route stored as the teacher data D1, and to prevent the final target parking position stored as the teacher data D1 from being an inappropriate position.

[0082] In particular, in the autonomous driving mode, the driving support device 10 according to this embodiment estimates the current position of the vehicle 1 using a surrounding sensor (in-vehicle camera 40 in this embodiment) and performs feedback control of the vehicle 1 based on the current position of the vehicle 1 and each target position of the target route stored as teacher data D1. This makes it possible to make the vehicle 1 travel smoothly along the target route even if the initial position of the vehicle 1 does not completely match the start position of the target route (i.e., the end position during teacher driving).

[0083] (Modification) In the automatic parking mode, the driving assistance device 10 may be configured to automatically drive the vehicle 1 along the movement route taken when the vehicle 1 traveled in the teacher driving mode. Here, the automatic driving mode in which the vehicle 1 is automatically driven along the movement route taken when the vehicle 1 traveled in the teacher driving mode is referred to as a "forward direction automatic driving mode", and the automatic driving mode in which the vehicle 1 is automatically driven along a movement route that is a reverse of the movement route taken when the vehicle 1 traveled in the teacher driving mode from the end point side to the start point side (the automatic driving mode described in the above embodiment) is referred to as a "reverse direction automatic driving mode".

[0084] In the forward automatic driving mode, the driving assistance device 10 typically drives the vehicle 1 in the same driving direction as the driving direction when the vehicle 1 is driven in the teacher driving mode. In the forward automatic driving mode, the driving assistance device 10 drives the vehicle 1 along the travel path when the vehicle 1 is driven in the teacher driving mode, and stops the vehicle 1 at a position corresponding to the end point of the travel path.

[0085] If the reverse direction automatic driving mode is the automatic parking mode, the forward direction automatic driving mode corresponds to the automatic leaving mode. In other words, this makes it possible to perform both automatic driving for parking and leaving with a single supervised driving run.

[0086] In order to enable the forward direction automatic driving mode, after the teacher driving mode ends, the driving assistance device 10 also stores data D1a (see FIG. 5) relating to the travel route generated when traveling in the teacher driving mode in the external storage device 50 as teacher data for forward direction automatic driving. Then, when the automatic driving mode is executed, the driving assistance device 10 determines whether to execute the forward direction automatic driving mode or the reverse direction automatic driving mode in response to, for example, a selection operation by the user. Note that, when the automatic driving mode is executed, the driving assistance device 10 may determine whether to execute the forward direction automatic driving mode or the reverse direction automatic driving mode in response to the current position of the vehicle 1.

[0087] Fig. 12 is a flowchart showing an example of the operation executed by the driving assistance device 10 according to this modification in the autonomous driving mode. The processes of steps S11 and S13 to S15 are the same as those described in the flowchart of Fig. 11, but this flowchart differs from the flowchart of Fig. 11 in that the process of step S12 is changed to steps S12a to S12d.

[0088] Step S12a is a process that is performed when a command to execute the automatic driving mode is received from the user. In this process, the driving assistance device 10 waits for the user to perform a mode selection operation (S12a: NO), and if a mode selection operation is performed (S12a: YES), the process proceeds to step S12b.

[0089] In step S12b, the driving support device 10 judges whether or not the user has selected the forward direction automatic driving mode. If the user has selected the forward direction automatic driving mode (S12b: YES), the driving support device 10 reads the teacher data D1a for the forward direction automatic driving from the external storage device 50 (step S12c), and if the user has not selected the forward direction automatic driving mode (S12b: NO), the driving support device 10 reads the teacher data D1 for the reverse direction automatic driving from the external storage device 50 (step S12d).

[0090] After performing this process, the driving assistance device 10 controls the vehicle 1 so that the vehicle 1 travels along the route indicated by the read teacher data D1 or D1a, in the same manner as the process described in the flowchart of FIG.

[0091] As described above, according to the driving assistance device 10 of this embodiment, it is possible to execute both automatic driving for parking and automatic driving for leaving a parking lot by performing a single teacher driving run.

[0092] (Other embodiments) The present invention is not limited to the above-described embodiment, and various modifications are possible.

[0093] For example, in the above embodiment, the driving assistance device 10 is applied to automatically parking the vehicle 1 in a parking lot, but the driving assistance device 10 of the present invention may also be applied to automatically withdrawing the vehicle 1 from a parking lot.

[0094] In addition, in the above embodiment, the functions of the driving assistance device 10 are realized by processing by the CPU 10a. However, some or all of the functions of the driving assistance device 10 may be realized by processing by a DSP (Digital Signal Processor) or a dedicated hardware circuit (e.g., ASIC or FPGA) instead of or in addition to processing by the CPU 10a.

[0095] Although the specific examples of the present invention have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples exemplified above. [Industrial Applicability]

[0096] According to the driving assistance device according to the present disclosure, more suitable parking assistance or exit assistance is possible. [Explanation of symbols]

[0097] 1 vehicle 10 Driving assistance devices 20 Vehicle drive unit 30 In-vehicle sensors 40 In-vehicle camera 50 External storage device 60 HMI 110 Vehicle information acquisition unit 120 Dead Reckoning Department 130 Teacher Data Generation Unit 140 Position estimation part 150 Target route readout unit 160 Vehicle control unit D1 Teacher data D2 Map Data P1 1st position P2 2nd position

Claims

1. A driving assistance device that assists driving of a vehicle, a teacher data generating unit that stores a travel route when the vehicle is traveled by a driving operation of a user in a teacher travel mode, and generates teacher data related to a target route when the vehicle is automatically traveled in an automatic travel mode based on the travel route; a position estimation unit that estimates a current position of the vehicle based on sensor information from a surrounding sensor that monitors a surrounding environment of the vehicle; a vehicle control unit that moves the vehicle toward an entry start target position among each target position between the start point and the end point of the target route based on an initial position of the vehicle at the start of the autonomous driving mode, and then causes the vehicle to travel along the target route from the entry start target position of the target route; Equipped with The vehicle control unit terminates the automatic driving control of the vehicle when a user performs an accelerator operation, a brake operation, or a steering operation while the vehicle is traveling in the automatic driving mode. Driving assistance device.

2. The surrounding sensor is a camera that captures the surroundings of the vehicle. The driving assistance device according to claim 1 .

3. The teacher data generation unit generates the teacher data while creating map data for estimating a current position of the vehicle from a camera image of the camera. The driving assistance device according to claim 2 .

4. a dead reckoning unit that estimates a current position of the vehicle based on a change over time in a sensor value of an on-board sensor that detects a traveling state of the vehicle; The position estimation unit operates to correct the current position of the vehicle estimated by the dead reckoning unit. The driving assistance device according to claim 2 or 3.

5. When the vehicle control unit receives an execution command for a reverse direction automatic driving mode in the automatic driving mode, the vehicle control unit causes the vehicle to travel along the target route and stops the vehicle at a position corresponding to a start point of the movement route. A driving assistance device according to any one of claims 1 to 4.

6. the vehicle control unit terminates control of automatically driving the vehicle when an automatic driving stop command is received from the user while the vehicle is automatically driving in a reverse direction automatic driving mode in the automatic driving mode. A driving assistance device according to any one of claims 1 to 5.

7. When the vehicle control unit receives an execution command of a forward direction automatic driving mode in the automatic driving mode, the vehicle control unit performs feedback control of the vehicle based on a current position of the vehicle estimated by the position estimation unit and each position of the movement route so that the vehicle travels along the movement route. A driving assistance device according to any one of claims 1 to 6.

8. When the vehicle control unit receives an execution command of a forward direction automatic driving mode in the automatic driving mode, the vehicle control unit causes the vehicle to travel along the travel route and stops the vehicle at a position corresponding to an end point of the travel route. A driving assistance device according to any one of claims 1 to 7.

9. when receiving a command to execute the forward direction automatic driving mode, the vehicle control unit causes the vehicle to travel along the travel route in the same direction as the traveling direction in the teacher driving mode. A driving assistance device according to claim 7 or 8.

10. the vehicle control unit terminates control of automatically driving the vehicle when an automatic driving stop command is received from the user while the vehicle is automatically driving in the forward direction automatic driving mode. A driving assistance device according to any one of claims 7 to 9.

11. The vehicle control unit causes the vehicle to automatically travel from a position between a start point and an end point of the target route indicated by the teaching data. A driving assistance device according to any one of claims 1 to 10.

12. The driving assistance device determines whether to execute a forward automatic driving mode in which the vehicle is automatically driven along the travel route, or a reverse automatic driving mode in which the vehicle is automatically driven along a travel route obtained by reversing the travel route from the end point side to the start point side, based on a selection operation by a user. A driving assistance device according to any one of claims 1 to 11.

13. The driving assistance device determines whether to execute a forward automatic driving mode in which the vehicle is automatically driven along the travel route, or a reverse automatic driving mode in which the vehicle is automatically driven along a travel route obtained by reversing the travel route from the end point side to the start point side, based on the current position of the vehicle. A driving assistance device according to any one of claims 1 to 12.

14. A driving assistance method for assisting driving of a vehicle, comprising: A process of storing a travel route when the vehicle is traveled by a driving operation of a user in a teacher travel mode, and generating teacher data related to a target route when the vehicle is automatically traveled in an automatic travel mode based on the travel route; A process of estimating a current position of the vehicle based on sensor information of a surrounding sensor that monitors a surrounding environment of the vehicle; a process of moving the vehicle toward an entry start target position among the target positions between the start point and the end point of the target route based on an initial position of the vehicle at the start of the autonomous driving mode, and then causing the vehicle to travel along the target route from the entry start target position of the target route, When the user performs an accelerator operation, a brake operation, or a steering operation while the vehicle is traveling in the autonomous driving mode, the autonomous driving control of the vehicle is terminated. Driving assistance methods.

15. The surrounding sensor is a camera that captures the surroundings of the vehicle. The driving assistance method according to claim 14.

16. generating the teacher data while creating map data for estimating the current position of the vehicle from the camera image; The driving assistance method according to claim 15.

17. The method further includes estimating a current position of the vehicle based on a change over time in a sensor value of an on-board sensor that detects a traveling state of the vehicle, operative to correct an estimated current position of the vehicle; A driving assistance method according to claim 15 or 16.

18. When a command to execute a reverse direction automatic driving mode in the automatic driving mode is received, the vehicle is caused to travel along the target route and the vehicle is stopped at a position corresponding to a start point of the travel route. A driving assistance method according to any one of claims 14 to 17.

19. When the user issues an automatic driving stop command while the vehicle is automatically traveling in a reverse direction automatic driving mode of the automatic driving mode, the control for automatically traveling the vehicle is terminated. A driving assistance method according to any one of claims 14 to 18.

20. When an execution command of a forward direction automatic driving mode in the automatic driving mode is received, feedback control is performed on the vehicle based on a current position of the vehicle and each position on the movement route so that the vehicle travels along the movement route. A driving assistance method according to any one of claims 14 to 19.

21. When an execution command for a forward direction automatic driving mode in the automatic driving mode is received, the vehicle is caused to travel along the travel route, and the vehicle is stopped at a position corresponding to an end point of the travel route. A driving assistance method according to any one of claims 14 to 20.

22. when a command to execute the forward direction automatic driving mode is received, the vehicle is caused to travel along the travel route in the same direction as the driving direction in the teacher driving mode.

22. A driving assistance method according to claim 20 or 21.

23. when an automatic driving stop command is received from the user while the vehicle is being automatically driven in the forward automatic driving mode, the control for automatically driving the vehicle is terminated. A driving assistance method according to any one of claims 20 to 22.

24. and automatically traveling from a position between the start point and the end point of the target route indicated by the teaching data. A driving assistance method according to any one of claims 14 to 23.

25. Based on a user's selection operation, determining whether to execute a forward automatic driving mode in which the vehicle is automatically driven along the travel route, or a reverse automatic driving mode in which the vehicle is automatically driven along a travel route obtained by reversing the travel route from the end point side to the start point side. A driving assistance method according to any one of claims 14 to 24.

26. determining whether to execute a forward automatic driving mode in which the vehicle is automatically driven along the travel route based on the current position of the vehicle, or a reverse automatic driving mode in which the vehicle is automatically driven along a travel route obtained by reversing the travel route from the end point side to the start point side; A driving assistance method according to any one of claims 14 to 25.

Images