Vehicle control method and vehicle control device

Abstract

A vehicle control device (10) is mounted on a vehicle (1) equipped with: a camera (16) that acquires video of the surroundings; a display device (22) visible to an occupant of the vehicle (1); and a movement control device (12) that controls at least steering. The vehicle control device (10) causes the movement control device (12) to control at least steering along a training route obtained according to training travel from a predetermined position (P1) to a parking target position and causes the vehicle (1) to travel autonomously to the parking target position. The vehicle control device (10) causes the display device (22) to display the video of the surroundings acquired by the camera (16). In a case where the vehicle (1) deviates from the training route by a first predetermined distance or more while traveling along the training route from a predetermined position to the parking target position, the vehicle control device (10) causes the display device (22) to superimpose and display, on the video of the surroundings, a return position at which the vehicle can return to the training route.

Description

Vehicle control method and vehicle control device

[0001] The present disclosure relates to a vehicle control method and a vehicle control device.

[0002] Generally, parking spaces in homes are often narrow and can be difficult to park in, so there is a high demand for automated parking and exit.

[0003] Vehicle control devices that realize this type of autonomous driving are known. For example, a technology has been disclosed in which a driver drives a vehicle from a predetermined position outside a parking lot to a target parking position, the route traveled during this process is stored in advance as a training route, and the vehicle autonomously drives along the training route in subsequent parking situations (see Patent Documents 1 to 4, etc.).

[0004] JP 2017-138664 A JP 2022-133181 A U.S. Patent Application Publication No. 2014 / 0200802 JP 2022-133230 A

[0005] However, with conventional technology, if a vehicle deviates from the teacher path due to manual operation such as obstacle avoidance while autonomously driving along the teacher path, the pre-registered feature points around the vehicle necessary for driving along the teacher path may not match the feature points that can actually be identified, making it difficult to resume or continue autonomous driving. In other words, with conventional technology, it may be difficult to provide suitable parking assistance.

[0006] An object of the present disclosure is to provide a vehicle control method and a vehicle control device that can provide more suitable parking assistance.

[0007] The vehicle control method disclosed herein is a vehicle control method that is mounted on a vehicle equipped with a camera that acquires surrounding images, a display device that is visible to the occupants, and a mobile control device that controls at least steering, and that causes the mobile control device to control at least steering along a teacher path obtained by teacher driving from a predetermined position to a parking target position, and is executed by a vehicle control device that causes the vehicle to autonomously drive to the parking target position, wherein the surrounding image acquired by the camera is displayed on the display device, and when the vehicle moves away from the teacher path by more than a first predetermined distance while moving along the teacher path from the predetermined position to the parking target position, the display device displays a returnable position where the vehicle can return to the teacher path, superimposed on the surrounding image.

[0008] According to the vehicle control method of the present disclosure, more suitable parking assistance can be provided.

[0009] FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle. FIG. 2 is an explanatory diagram of an example of the arrangement of sensors and cameras. FIG. 3 is a schematic diagram showing an example of the configuration of the exterior of a vehicle. FIG. 4 is a diagram showing an example of the configuration near the driver's seat of a vehicle according to an embodiment. FIG. 5 is an explanatory diagram of an example of a teacher route. FIG. 6 is a diagram showing an example of the data configuration of teacher route data. FIG. 7A is a schematic diagram of an example of map data. FIG. 7B is a schematic diagram of an example of map data. FIG. 8 is an explanatory diagram of an example of a process for estimating the current position of a vehicle executed by a control unit in autonomous driving mode. FIG. 9 is a schematic diagram of an example of a display screen. FIG. 10 is a flowchart showing an example of the flow of information processing executed by a control unit in teacher driving mode. FIG. 11 is a flowchart showing an example of the flow of information processing executed by a control unit in autonomous driving mode. FIG. 12 is a block diagram showing an example of the hardware configuration of a vehicle control device.

[0010] Hereinafter, embodiments of a vehicle control method and a vehicle control device according to the present disclosure will be described with reference to the drawings.

[0011] [Overall Configuration of Vehicle] FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle 1. As shown in FIG.

[0012] The vehicle 1 includes a vehicle control device 10 , a movement control device 12 , a sensor 14 , a camera 16 , a storage device 18 , an operation device 20 , and a display device 22 .

[0013] The vehicle control device 10 is connected to the movement control device 12, the sensor 14, the camera 16, the storage device 18, the operation device 20, and the display device 22 so as to be able to exchange data or signals. In other words, the vehicle control device 10 is configured to be communicatively connected to at least the display device 22 and the movement control device 12.

[0014] The mobility control device 12 controls at least the steering of the vehicle 1. The mobility control device 12 is a means for realizing the driving, braking, and turning movements required for the vehicle 1 to travel. For example, the mobility control device 12 is configured to include a drive motor, a power transmission mechanism, a brake device, a steering device, etc., and an electronic vehicle control device that controls them. The mobility control device 12 travels the vehicle 1, for example, by generating power using the drive motor and transmitting the power to the wheels via the power transmission mechanism. The power transmission mechanism is, for example, a propeller shaft, a differential gear, a drive shaft, etc.

[0015] In detail, the movement control device 12 includes an auxiliary control device 12A, a brake control device 12B, an engine control device 12C, and a power steering control device 12D. The brake control device 12B, the engine control device 12C, and the power steering control device 12D can be collectively referred to as actuator control units that control the operation of the vehicle 1B.

[0016] The auxiliary control device 12A is a control device that monitors the transmission status of the vehicle control device 10 and operates as a backup to execute appropriate degeneration control in the event of a failure of the vehicle control device 10. Note that if safety can be ensured by providing a degeneration control function within the vehicle control device 10 even in the event of a failure of the vehicle control device 10, degeneration control is not necessary.

[0017] The brake control device 12B is a control device that performs brake control (braking force control) of the vehicle 1. For example, the brake control device 12B performs brake control of the vehicle 1 in response to the intensification (depression) and relaxation of the operation of the brake pedal (brake operation unit) by the passenger. Furthermore, the brake control device 12B performs brake control in response to surrounding images during autonomous driving.

[0018] The engine control device 12C is a control device that controls an engine that generates a driving force for the vehicle 1. The power steering control device 12D is a control device that controls the power steering of the vehicle 1.

[0019] The sensors 14 are various sensors mounted on the vehicle 1 and detect the driving state of the vehicle 1 and the state around the vehicle 1. The sensors 14 include, for example, an accelerator opening sensor that detects an accelerator opening, a steering angle sensor that detects a steering angle of a steering device, an acceleration sensor that detects acceleration acting in the longitudinal direction of the vehicle 1, a torque sensor that detects torque acting on a power transmission mechanism between the wheels of the vehicle 1 and the drive motor, a vehicle speed sensor that detects the vehicle speed of the vehicle 1, a wheel speed sensor, etc. The sensors 14 also include LiDAR (Light detection and ranging), radar, ultrasonic sensors, etc. The sensors 14 output sensor information obtained by detection to the vehicle control device 10.

[0020] The camera 16 is a surroundings sensor mounted on the vehicle 1 and monitors the environment surrounding the vehicle 1. In this embodiment, the camera 16 captures images of the surroundings of the vehicle 1 and sequentially outputs the captured images to the vehicle control device 10. In this embodiment, the camera 16 outputs a surroundings video made up of a plurality of captured images in chronological order obtained by capturing images of the surroundings of the vehicle 1 in chronological order to the vehicle control device 10. In this embodiment, the camera 16 is also used to detect objects present around the vehicle 1 and estimate the current position of the vehicle 1 from the positional relationship between the vehicle 1 and the objects present around the vehicle 1.

[0021] The positions, number of cameras 16, and shooting directions are adjusted in advance so that the cameras 16 can capture images of the surroundings of the vehicle 1. For example, the vehicle 1 is provided with four cameras 16 arranged so as to capture images in four directions, namely, the front, rear, left, and right directions of the vehicle 1. Note that the number of cameras 16 provided on the vehicle 1 is not limited to four.

[0022] The storage device 18 stores various types of data. In this embodiment, the storage device 18 stores data such as teacher route data D1 and map data D2. The teacher route data D1 and the map data D2 will be described in detail later. The storage device 18 is, for example, an auxiliary storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory.

[0023] The operation device 20 acquires input operations from a user. The operation device 20 is, for example, an input device such as a keyboard, a touch panel, a switch, etc. The operation device 20 is an example of an HMI (Human Machine Interface).

[0024] The display device 22 is a display that displays various images. The display device 22 is installed in a position where it can be seen by a user who is a passenger in the vehicle 1. Examples of the display include a liquid crystal display (LCD), an organic electroluminescence (EL) display, and a projector. The display device 22 may be a touch panel display that is an integral configuration of the display device 22 and the operation device 20. The display device 22 is an example of an HMI.

[0025] The display device 22 is not limited to having only one display area. For example, the display device 22 may have multiple display areas. Furthermore, the vehicle 1 may be equipped with multiple display devices 22.

[0026] FIG. 2 is an explanatory diagram of an example of the arrangement of the sensor 14 and the camera 16.

[0027] Vehicle 1 is provided with, for example, four cameras 16 (camera 16A to camera 16D) so that the external situation of vehicle 1B can be acquired in at least four directions, for example, in front, behind, to the right, and to the left of vehicle 1B.

[0028] Specifically, for example, the cameras 16 include cameras 16A, 16B, 16C, and 16D. Camera 16A is disposed at the front of the vehicle 1 and captures the area in front of the vehicle 1. Camera 16B is disposed at the right side of the vehicle 1 and captures the area to the right of the vehicle 1. Camera 16C is disposed at the left side of the vehicle 1 and captures the area to the left of the vehicle 1. Camera 16D is disposed at the rear of the vehicle 1 and captures the area to the rear of the vehicle 1. The number of cameras 16 provided on the vehicle 1 is not limited to four. It is also preferable that the positions and numbers of object detection sensors included in the sensor 14, such as lidar, radar, sonar, and ultrasonic sensors, are adjusted in advance so that the external conditions on the right side, left side, front, and rear of the vehicle 1 can be acquired. For example, as shown in FIG. 2, the sensor 14 includes sensors 14A to 14F. These sensors 14A to 14F are arranged on the vehicle 1 so as to be able to acquire external conditions on the right side, left side, front, and rear of the vehicle 1B. Note that the sensors 14 that detect objects, such as lidar, radar, sonar, and ultrasonic sensors, may be arranged only in the rear of the vehicle 1.

[0029] Next, the configuration of the vehicle 1 will be described.

[0030] FIG. 3 is a schematic diagram showing an example of the external configuration of the vehicle 1.

[0031] The vehicle 1 includes a vehicle body 2 and two pairs of wheels 23 arranged along a predetermined direction on the vehicle body 2. The two pairs of wheels 23 include a pair of front tires 23F and a pair of rear tires 23R (see also FIG. 6 ). Note that FIGS. 2 and 3 show an example in which the vehicle 1 includes four wheels 23. However, the number of wheels 23 provided on the vehicle 1 is not limited to this.

[0032] Next, the configuration of the vicinity of the driver's seat of the vehicle 1 of this embodiment will be described.

[0033] FIG. 4 is a diagram showing an example of the configuration of the vicinity of the driver's seat 24A of the vehicle 1 according to this embodiment.

[0034] The vehicle 1 includes a driver's seat 24A and a passenger seat 24B. In front of the driver's seat 24A are provided a windshield 25, a dashboard 26, a steering wheel (steering operation unit, handle) 20A, operation buttons 20B, and a display device 22. In addition, a shift lever 20C, which is a lever for changing gears in the transmission, is provided near the driver's seat 24A.

[0035] The steering wheel 20A, the operation buttons 20B, and the shift lever 20C are examples of the operation device 20.

[0036] The steering wheel 20A is provided in front of the driver's seat 24A and can be operated by the passenger. The rotation angle of the steering wheel 20A, i.e., the steering angle, is electrically or mechanically linked to the change in the direction of the front tires 23F, which are the steered wheels. Note that the steered wheels may be the rear tires 23R, or both the front tires 23F and the rear tires 23R may be steered wheels.

[0037] The operation button 20B is a button that can accept operation by the passenger. The operation button 20B may include a direction indicator. The position of the operation button 20B is not limited to the example shown in FIG. 4 , and the operation button 20B may be provided on the steering wheel 20A, for example. Although one operation button 20B is illustrated in FIG. 4 , a plurality of operation buttons 20B may be provided. When the display device 22 also functions as a touch panel, the display device 22 may be an example of the operation device 20.

[0038] Returning to FIG. 1, the explanation will be continued.

[0039] The vehicle control device 10 is an electronic control unit that controls each part of the vehicle 1 .

[0040] The vehicle control device 10 uses sensor information and surrounding images received from the sensors 14 and the cameras 16, respectively, to control the movement control device 12 so as to optimize the driving conditions of the vehicle 1. The vehicle control device 10 also controls the movement control device 12 to cause the vehicle 1 to drive autonomously.

[0041] The vehicle control device 10 includes a control unit 11. Part or all of the control unit 11 may be a software configuration implemented by cooperation between a processor and various programs stored in memory, or part or all of the control unit 11 may be a hardware configuration implemented by a dedicated circuit or the like.

[0042] The control unit 11 controls each part of the vehicle 1 in an integrated manner.

[0043] In this embodiment, the control unit 11 is configured to be able to switch the driving mode between the supervised driving mode and the autonomous driving mode based on an input operation by the user on the operation device 20. Note that the driving modes that can be executed by the vehicle 1 may include various driving modes other than the supervised driving mode and the autonomous driving mode.

[0044] The supervised driving mode is a mode for registering a supervised route for autonomous driving of the vehicle 1. The supervised route is a route obtained by supervised driving from a predetermined position to a target parking position. In the supervised driving mode, the vehicle 1 is controlled to travel by the driving operation of the user. That is, in the supervised driving mode, the control unit 11 controls the mobility control device 12 to travel in accordance with the driving operation of the user.

[0045] The autonomous driving mode is a mode in which the vehicle 1 drives autonomously. In this embodiment, the autonomous driving mode refers to a mode in which the vehicle 1 drives autonomously along a teacher route. In the autonomous driving mode, the control unit 11 controls at least steering and controls the mobile control device 12 so that the vehicle 1 drives along the teacher route. In the autonomous driving mode, the vehicle 1 is automatically controlled to drive by the vehicle control device 10 without any driving operation by the user.

[0046] FIG. 5 is an explanatory diagram of an example of the teacher route R1.

[0047] In the supervised driving mode, the user drives the vehicle to perform supervised driving from a predetermined position P1 to a target parking position P2. The target parking position P2 may be, for example, a parking lot, but is not limited to this. The predetermined position P1 may be any position of the user in real space.

[0048] The travel route R traveled during teacher driving is treated as a teacher route R1, and teacher route data D1 for the teacher route R1 is stored in the storage device 18. During teacher driving, the user may drive the vehicle 1 so as to travel from the parking target position P2 toward the predetermined position P1, or from the predetermined position P1 toward the parking target position P2. When the vehicle 1 travels from the parking target position P2 toward the predetermined position P1 in teacher driving mode, the control unit 11 may create teacher route data D1 for the travel route R in the opposite direction to the direction of travel during teacher driving. When the vehicle 1 travels from the predetermined position P1 toward the parking target position P2 during teacher driving, the control unit 11 may create teacher route data D1 for the travel route R in the same direction as the direction of travel during teacher driving. The creation of the teacher route data D1 will be described in detail later.

[0049] In the autonomous driving mode, the control unit 11 controls at least the steering along the driving route R obtained by the supervised driving, and causes the vehicle 1 to autonomously drive to the parking target position P2. Note that in the autonomous driving mode, the control unit 11 controls the steering and the front and rear acceleration / deceleration of the vehicle 1, but at least a part of the front and rear acceleration / deceleration control may be performed by the driver's operation.

[0050] Next, the control by the control unit 11 in the teacher driving mode and the autonomous driving mode will be described in detail.

[0051] [Instructor Travel Mode] First, the control of the control unit 11 in the instructor travel mode will be described in detail.

[0052] When the control unit 11 receives a signal indicating an instruction to start the teacher-driven driving mode by a user operating the operation device 20, the control unit 11 switches the driving mode to the teacher-driven driving mode. Then, when the control unit 11 is in the teacher-driven driving mode, the control unit 11 executes the following processing.

[0053] The control unit 11 acquires sensor information indicating the traveling state of the vehicle 1 from the sensor 14. Then, the control unit 11 estimates the current position of the vehicle 1 based on the temporal changes in the sensor values ​​represented by the sensor information. For example, the control unit 11 calculates the amount of movement of the vehicle 1 from a reference position, such as the traveling start position when the teacher traveling mode was started, based on the temporal changes in the vehicle speed and yaw rate represented by the sensor values, and estimates the current position of the vehicle 1 based on the amount of movement.

[0054] Note that the accuracy of estimating the current position based on the amount of movement may be low in some cases. Therefore, the control unit 11 may correct the estimated current position based on the surrounding image of the vehicle 1 acquired by the camera 16, and use the corrected result as the current position.

[0055] The control unit 11 sequentially stores the current position of the vehicle 1 estimated sequentially as the vehicle 1 travels in the storage device 18. In detail, the control unit 11 defines the travel route R during teacher travel represented by a group of current positions sequentially estimated from the time when an instruction to start the teacher travel mode is received until the time when an instruction to end the teacher mode is received as a teacher route R1, and stores teacher route data D1 representing the teacher route R1 in the storage device 18.

[0056] FIG. 6 is a diagram showing an example of the data structure of the teacher route data D1.

[0057] The teacher route data D1 is composed of a group of driving information for each position, which is the current position that is sequentially estimated during teacher driving. The driving information includes an INDEX, a driving position, a bearing, a driving direction, and reference driving information. The INDEX is identification information for the driving information. The driving position is the estimated position of the vehicle 1. The bearing indicates the orientation of the vehicle 1 at that position. The driving direction indicates the direction in which the vehicle 1 is driving at that position, and is expressed, for example, as forward or reverse. The reference driving information is information that indicates the driving state at that position. The reference driving information is, for example, information such as the steering angle and vehicle speed detected at each position during teacher driving.

[0058] Furthermore, when the vehicle 1 is traveling on a teacher's course, the control unit 11 creates map data D2 for estimating the current position of the vehicle 1 from the surrounding image captured by the camera 16. The method for estimating the current position of the vehicle 1 from the surrounding image may be the SLAM (Simultaneous Localization and Mapping) method or the like.

[0059] Fig. 7A is a schematic diagram of an example of the map data D2, which shows the position in real space of a feature point Q in a real scene stored in the map data D2 in a bird's-eye view.

[0060] The map data D2 is map data in which a plurality of characteristic points Q around the vehicle 1 when traveling along the teacher route R1 are registered.

[0061] The feature points Q are characteristic points obtained by image analysis of the images captured by the camera 16 during teacher driving. For example, the feature points Q are portions of objects (e.g., trees, walls, pillars, etc.) that can serve as landmarks in the actual scene, from which characteristic image patterns are obtained by analyzing the captured images. Such portions are, for example, the edge portions of the objects. The map data D2 includes a plurality of feature points Q, and each feature point Q is assigned an identification number and registered so that it can be identified.

[0062] The feature point Q is represented by feature point data including a three-dimensional position and a feature amount.

[0063] The three-dimensional position of the feature point Q is the three-dimensional position of the feature point Q in real space, and is expressed, for example, in a three-dimensional Cartesian coordinate system (X, Y, Z).

[0064] The feature amount of the feature point Q is a characteristic amount represented by image analysis of the captured image of the feature point Q. The feature amount of the feature point Q is, for example, the brightness or density on the captured image, a SIFT (Scale Invariant Feature Transform) feature amount, a SURF (Speeded Up Robust Features) feature amount, etc.

[0065] In the map data D2, one feature point Q is registered for each identical three-dimensional position. Note that, for the same three-dimensional position, the map data D2 may register multiple feature points Q for each photographing position and photographing direction by the camera 16 at that three-dimensional position. Furthermore, the feature point data of the feature point Q registered in the map data D2 may further include image data of an object having the feature point Q.

[0066] During teacher driving, the control unit 11 identifies the coordinates of the feature points Q in the actual scene, for example, based on stereo photogrammetry. Specifically, the control unit 11 reads multiple captured images taken at different times and associates the same feature points Q that appear in the multiple captured images. The control unit 11 then estimates the virtual position of the vehicle 1 at the time the multiple captured images were taken and identifies the virtual coordinates of the feature points Q in the actual scene using the principle of triangulation. The control unit 11 then performs bundle adjustment, for example, using the virtual position of the vehicle 1 and the virtual coordinates of the feature points Q in the actual scene as reference information, to calculate the formal position of the vehicle 1 and the formal coordinates of the feature points Q in the actual scene so as to minimize the reprojection error when each feature point Q in the actual scene is projected onto all captured images. The control unit 11 then stores, in the storage device 18, map data D2 in which the feature points Q are registered, represented by feature point data including the formal coordinates of the feature points Q in the actual scene as three-dimensional positions.

[0067] The three-dimensional positions of the feature points Q registered in the map data D2 may be positions measured in advance using LiDAR (Light Detection and Ranging) or a stereo camera instead of using the SLAM method. However, from the viewpoint of suppressing a decrease in the accuracy of position estimation, it is preferable to use the SLAM method.

[0068] The control unit 11 may generate one piece of map data D2 that includes the entire teacher route R1, as shown in Figure 7A, or may create multiple pieces of partial map data that are at least partially non-overlapping along the teacher route R1 as the map data D2.

[0069] 7B is a schematic diagram of an example of the map data D2, which shows the position in real space of a feature point Q in the real scene stored in the map data D2 in a bird's-eye view.

[0070] 7B, the map data D2 may be composed of a plurality of partial map data D2P that are at least partially non-overlapping along the teacher route R1. Each partial map data D2P may have registered therein feature point data for the feature point Q in the same manner as described above.

[0071] By configuring the map data D2 from a plurality of partial map data D2P, the data volume of the map data D2 can be reduced compared to when one piece of map data D2 including the entire teacher route R1 is created.

[0072] As described above, the control unit 11 executes the above process in the teacher driving mode. Therefore, in the teacher driving mode, the control unit 11 generates teacher route data D1 of the teacher route R1 obtained by teacher driving from a predetermined position P1 to a target parking position P2, and map data D2 in which the three-dimensional positions of each of a plurality of feature points Q around the vehicle 1 while traveling along the teacher route R1 and the feature quantities of the feature points Q are registered, and stores these in the storage device 18.

[0073] [In Autonomous Traveling Mode] Next, the control of the control unit 11 in the autonomous traveling mode will be described in detail.

[0074] When the control unit 11 receives a signal indicating an instruction to start the autonomous driving mode by a user's operation of the operation device 20, the control unit 11 switches the driving mode to the autonomous driving mode. Then, when in the autonomous driving mode, the control unit 11 executes the following processing.

[0075] The control unit 11 reads the teacher route data D1 and the map data D2 from the storage device 18, and controls the mobile control device 12 to perform autonomous traveling along the teacher route R1 represented by the teacher route data D1.

[0076] The control unit 11 estimates the current position of the vehicle 1 based on the map data D2 and an image of the surroundings of the vehicle 1 captured by at least one camera 16 .

[0077] FIG. 8 is an explanatory diagram of an example of the process of estimating the current position of the vehicle 1 executed by the control unit 11 in the autonomous driving mode.

[0078] 8, S1, S2, and S3 represent three characteristic points extracted from the image captured by the camera 16, and points Q1, Q2, and Q3 are characteristic points Q stored in the map data D2 and represent the three-dimensional positions of the characteristic points S1, S2, and S3 in real space. RP1 represents the imaging plane of the camera 16. Point P' represents the position of the camera 16 (i.e., the position of the vehicle 1) determined from the three characteristic points S1, S2, and S3 extracted from the image captured by the camera 16 and the characteristic points Q (Q1, Q2, and Q3) stored in the map data D2.

[0079] For example, the control unit 11 first compares characteristic points extracted from the image captured by the camera 16 with characteristic points Q stored in the map data D2 using pattern matching, feature search, etc. Then, the control unit 11 randomly selects several (e.g., 3 to 6) characteristic points from the characteristic points extracted from the image captured by the camera 16 that can be compared with the characteristic points Q stored in the map data D2.

[0080] The control unit 11 then estimates the current position of the vehicle 1 in real space based on the positions of these several characteristic points in the captured image and the three-dimensional position in real space of a characteristic point Q registered in the map data D2 that corresponds to the several characteristic points. In this case, the control unit 11 estimates the current position of the vehicle 1 by solving the PnP problem using a known method such as Lambda Twist (for example, see the literature: Mikael Persson et al., “Lambda Twist: An Accurate Fast Robust Perspective Three Point (P3P) Solver.”, ECCV 2018, pp. 334-349, published in 2018, http: / / openaccess.thecvf.com / content_ECCV_2018 / papers / Mikael_Persson_Lambda_Twist_An_ECCV_2018_paper.pdf).

[0081] When comparing the characteristic points extracted from the image captured by the camera 16 with the characteristic points Q stored in the map data D2, the control unit 11 may, for example, calculate the current position of the vehicle 1 as a tentative position based on the amount of movement of the vehicle 1 described above, and then, using this tentative position as a reference, narrow down the characteristic points Q stored in the map data D2 to be compared with the characteristic points extracted from the image captured by the camera 16.

[0082] Through these processes, the control unit 11 estimates current position information representing the current position of the vehicle 1 based on the map data D2 and the captured images of the surroundings of the vehicle 1 acquired by at least one camera 16, including the two-dimensional position (X coordinate, Y coordinate) of the vehicle 1 in real space and information related to the orientation of the vehicle 1.

[0083] The control unit 11 then controls the movement control device 12 so that the estimated current position of the vehicle 1 is a position on the teacher route R1 represented by the teacher route data D1, thereby causing the vehicle 1 to autonomously travel along the teacher route R1 from a predetermined position P1 toward a parking target position P2. The control unit 11 then stops the vehicle 1 at the parking target position P2.

[0084] When the vehicle 1 is autonomously traveling along the travel route R, the control unit 11 feedback controls the movement control device 12 so that the vehicle 1 moves along the teacher route R1 based on the estimated current position of the vehicle 1 and each position on the teacher route R1 represented by the teacher route data D1.

[0085] Here, while the vehicle 1 is moving autonomously along the teacher route R1 from a predetermined position P1 to the parking target position P2, an event may occur in which the user performs manual operation to avoid obstacles around the vehicle 1. If such an event occurs, the vehicle 1 will be at a distance greater than the first predetermined distance from the teacher route R1, making it impossible to estimate the position of the vehicle 1 relative to the teacher route R1, and making it difficult to resume or continue autonomous driving.

[0086] The first predetermined distance is a distance between the vehicle 1 and the teacher route R1 that makes it difficult to estimate the current position of the vehicle 1 using the characteristic points extracted from the image captured by the camera 16 and the characteristic points Q registered in the map data D2. In other words, when the vehicle 1 is away from the teacher route R1 by more than the first predetermined distance, the control unit 11 cannot estimate the position of the vehicle 1 relative to the teacher route R1, that is, the current position of the vehicle 1 relative to the teacher route R1 is lost.

[0087] Specifically, as described above, to estimate the current position of the vehicle 1 using the map data D2, the control unit 11 identifies N characteristic points (N is an integer equal to or greater than 1, e.g., 7 or greater) that are extracted from the image captured by the camera 16 and that can be matched with the characteristic points Q stored in the map data D2, and randomly selects several (e.g., 3 to 6) of the identified characteristic points to estimate the current position of the vehicle 1. If the number of characteristic points extracted from the image captured by the camera 16 that can be matched with the characteristic points Q stored in the map data D2 is less than N, the control unit 11 enters a state in which the current position of the vehicle 1 relative to the teacher route R1 has been lost. The first predetermined distance may be set in advance to a value equal to or greater than the lower limit of the distance between the vehicle 1 and the teacher route R1 that will result in the vehicle 1 being in this lost state. Furthermore, the first predetermined distance may be appropriately changeable by the user's operation using the operation device 20, etc., within a range that satisfies these conditions.

[0088] The control unit 11 determines whether the estimated current position of the vehicle 1 is away from the teacher route R1 by a first predetermined distance or more.

[0089] In detail, the control unit 11 estimates the current position of the vehicle 1 based on the map data D2 and the captured image of the surroundings of the vehicle 1 acquired by at least one camera 16, and determines whether the vehicle 1 is away from the teacher route R1 by a first predetermined distance or more using the estimated current position of the vehicle 1. In this case, the control unit 11 determines whether the shortest distance between the estimated current position of the vehicle 1 and the teacher route R1 represented by the teacher route data D1 is equal to or greater than the first predetermined distance, thereby determining whether the current position of the vehicle 1 is away from the teacher route R1 by a first predetermined distance or more.

[0090] In addition, the control unit 11 may determine whether the current position of the vehicle 1 is away from the teacher route R1 by, for example, a first predetermined distance or more by comparing characteristic points extracted from the image captured by the camera 16 with characteristic points Q stored in the map data D2 and determining whether the number of characteristic points that can be matched is less than N.

[0091] If the control unit 11 determines that the estimated current position of the vehicle 1 is not further than a first predetermined distance from the teacher route R1, it continues the process of controlling the mobile control device 12 so that the vehicle 1 travels autonomously along the teacher route R1.

[0092] On the other hand, if it is determined that the estimated current position of the vehicle 1 is more than a first predetermined distance away from the teacher route R1, the control unit 11 superimposes a returnable position from which the vehicle can return to the teacher route R1 on the surrounding image and displays it on the display device 22.

[0093] Displaying the returnable position superimposed on the peripheral image means at least one of superimposing the returnable position on the peripheral image and displaying it simultaneously with the peripheral image at the same time.

[0094] The returnable position is a position on the teacher route R1 to which the vehicle 1 can return.

[0095] For example, the returnable position is a position on a linear partial path on the teacher path R1 that is at least a second predetermined distance away. A partial path is a path that is a portion of the teacher path R1. The second predetermined distance may be at least the distance required for the vehicle 1 to return to the teacher path R1 from a position at least the first predetermined distance away from the teacher path R1 and travel in a state where the entire length of the vehicle 1 is approximately aligned in the direction along the extension direction of the teacher path R1. Specifically, for example, the second predetermined distance may be at least one or two times the entire length of the vehicle 1. The second predetermined distance may be changeable in response to an operation instruction by the user on the operation device 20, etc., as long as it satisfies the above condition.

[0096] The returnable position may also be a position on a partial path of the teacher path R1 with a curvature equal to or smaller than a predetermined value. The predetermined curvature may be equal to or smaller than the curvature required for the vehicle 1 to return to the teacher path R1 from a position at least a first predetermined distance away from the teacher path R1 and travel in a state where the entire length of the vehicle 1 is substantially aligned in the direction along the extension direction of the teacher path R1.

[0097] The returnable position may be a position on the teacher route R1 that is at least a third predetermined distance away from the parking target position P2. The third predetermined distance may be a distance at which the vehicle 1 can autonomously drive from a position that is the third predetermined distance away from the parking target position P2 on the teacher route R1 and provide parking assistance to the occupant. The third predetermined distance may be changeable as appropriate in response to an operation instruction from the user on the operation device 20, etc., within a range that satisfies the above-mentioned condition.

[0098] The returnable position may also be the entry position of the teacher route R1 in the partial map data D2P. When map data D2 consisting of a plurality of partial map data D2P is used as the map data D2, it becomes possible to resume autonomous driving using the feature point Q in the partial map data D2P. Therefore, the returnable position may also be the entry position IP of the partial map data D2P. Specifically, as shown in FIG. 7B , the control unit 11 may set the entry position IP of the teacher route R1 in the partial map data D2P as the returnable position P3.

[0099] The returnable position may also be the position on the teacher route R1 that is closest to the estimated current position of the vehicle 1.

[0100] The control unit 11 identifies as the return position P3 one of the following positions on the teacher route R1: a position on a partial route that is straight and has a distance equal to or greater than a second predetermined distance, a position on a partial route with a curvature equal to or less than a predetermined distance, a position that is a third predetermined distance or more away from the parking target position P2, the entry position IP of the teacher route R1 in the partial map data D2P, or a position that is closest to the estimated current position of the vehicle 1.

[0101] In addition, the control unit 11 may identify as the returnable position P3 a position that satisfies two or more of the following conditions: a position on a partial path on the teacher path R1 that is straight and has a distance equal to or greater than a second predetermined distance or a position on a partial path with a curvature equal to or less than a predetermined distance; a position that is apart from the parking target position P2 by a third predetermined distance; the entry position IP of the teacher path R1 in the partial map data D2P; and a position that is closest to the estimated current position of the vehicle 1.

[0102] Then, when the vehicle 1 moves away from the teacher route R1 by more than a first predetermined distance while moving from a predetermined position P1 to a parking target position P2 along the teacher route R1, the control unit 11 superimposes a returnable position P3 from which the vehicle 1 can return to the teacher route R1 on the surrounding image and displays it on the display device 22.

[0103] In detail, the control unit 11 displays on the display device 22 the returnable position P3, a document urging the user to move to the returnable position P3, and / or a document indicating that the user has moved away from the teacher route R1 by more than a first predetermined distance, superimposed on the surrounding image.

[0104] Furthermore, the control unit 11 may display, on the display device 22, a peripheral image corresponding to an image captured by at least one camera 16, and may superimpose the returnable position P3 on the peripheral image. The peripheral image on which the returnable position P3 is superimposed may be either a peripheral image captured by the camera 16 in real time or a peripheral image captured by the camera 16 in the past.

[0105] The control unit 11 may also superimpose the teacher route R1 and the return position P3 on the peripheral image. The control unit 11 may also superimpose at least one of a message urging the user to move to the return position P3 and a message indicating that the user has moved away from the teacher route R1 by a first predetermined distance or more on the peripheral image.

[0106] FIG. 9 is a schematic diagram of an example of the display screen 30. As shown in FIG.

[0107] For example, when the vehicle 1 moves away from the teacher route R1 by more than a first predetermined distance while moving from a predetermined position P1 to a parking target position P2 along the teacher route R1, the control unit 11 causes the display screen 30 to be displayed on the display device 22.

[0108] The display screen 30 is, for example, a screen in which a returnable position P3, a teacher route R1, and a message M are superimposed on a peripheral image 32. FIG. 9 shows an example in which the returnable position P3 is displayed by placing an icon representing the vehicle 1 at the returnable position P3 on the peripheral image 32. FIG. 9 also shows an example in which an arrow image representing the teacher route R1 is superimposed on the peripheral image 32. FIG. 9 also shows an example in which the message M includes an example of a message encouraging the vehicle to move to the returnable position P3, such as "Automatic parking can be resumed by moving the vehicle to the position on the screen," and an example of a message indicating that the vehicle has moved away from the teacher route R1 by more than a first predetermined distance, such as "Manual operation has caused deviation from the teacher route."

[0109] The display screen 30 may further include a screen in which an icon CP representing the current position and current attitude of the vehicle 1 is superimposed on the surrounding image 33 .

[0110] In addition, the control unit 11 may further output from a speaker provided in the vehicle 1 an audio signal representing the returnable position P3 and at least one of a document urging the vehicle to move to the returnable position P3 and a document indicating that the vehicle has moved away from the teacher route R1 by more than a first predetermined distance.

[0111] After displaying the returnable position P3 on the display device 22, if the vehicle 1 is driven by a user who is an occupant of the vehicle 1 and moves to the returnable position P3, the control unit 11 resumes autonomous driving along the teacher route R1 to the returnable position P3.

[0112] In detail, after the control unit 11 displays the returnable position P3 on the display device 22, when the control unit 11 receives driving operations of the vehicle 1 from the user, it cancels the autonomous driving mode and controls the movement control device 12 to drive in accordance with the driving operations of the user.

[0113] As described above, the returnable position P3 is displayed on the display device 22. Therefore, the driver of the vehicle 1 can easily move the vehicle 1 to the returnable position P3 and cause the vehicle 1 to resume autonomous driving by performing driving operations while visually checking the returnable position P3 displayed on the display device 22.

[0114] The control unit 11 then estimates the current position of the vehicle 1 in the same manner as above, and after determining that the vehicle 1 has moved to the returnable position P3, resumes autonomous driving when it receives an instruction to start the autonomous driving mode by the user operating the operation device 20. The control unit 11 then controls the movement control device 12 to autonomously drive the vehicle 1 along the teacher route R1 until it reaches the parking target position P2.

[0115] Next, an example of the flow of information processing executed by the control unit 11 of the vehicle control device 10 will be described.

[0116] FIG. 10 is a flowchart showing an example of the flow of information processing executed by the control unit 11 in the teacher running mode.

[0117] The control unit 11 determines whether a signal representing an instruction to start the teacher driving mode has been received from the operation device 20 (step S100). If the determination in step S100 is negative (step S100: No), the control unit 11 ends this routine. If the determination in step S100 is positive (step S100: Yes), the control unit 11 proceeds to step S102.

[0118] When the user starts driving the vehicle 1 to initiate teacher driving, the control unit 11 sequentially stores the map data D2 and the current position of the vehicle 1 in the storage device 18 (step S102). Specifically, the control unit 11 sequentially stores the current position of the vehicle 1 estimated along the teacher driving of the vehicle 1. At this time, the control unit 11 assigns an INDEX to the current position and sequentially stores the current position, which is the driving position, heading, driving direction, and reference driving information in association with each other. The control unit 11 also identifies feature points Q by analyzing images captured by the camera 16 during teacher driving and sequentially registers the feature points Q in the map data D2.

[0119] The control unit 11 determines whether an instruction to end the teacher driving mode has been received (step S104). The control unit 11 makes the determination in step S104 by determining whether a signal representing the end instruction has been received from the operation device 20 by the user using the operation device 20. If the control unit 11 makes a negative determination in step S104 (step S104: No), the control unit 11 returns to step S102. If the control unit 11 makes a positive determination in step S104 (step S104: Yes), the control unit 11 proceeds to step S106.

[0120] In step S106, the control unit 11 stores the teacher route data D1 and the map data D2 in the storage device 18 (step S106). The control unit 11 designates the travel route R during teacher travel represented by the group of sequentially estimated current positions stored in the processing of step S102 as a teacher route R1, and stores the teacher route data D1 representing the teacher route R1 in the storage device 18. The control unit 11 also stores the map data D2 in which the feature point Q identified in the processing of step S102 is registered in the storage device 18. Then, this routine ends.

[0121] FIG. 11 is a flowchart showing an example of the flow of information processing executed by the control unit 11 in the autonomous driving mode.

[0122] The control unit 11 determines whether a signal representing an instruction to start the autonomous driving mode has been received (step S200). The control unit 11 makes the determination in step S200 by determining whether a signal representing an instruction to start the autonomous driving mode has been received from the operation device 20. If the determination in step S200 is negative (step S200: No), the control unit 11 ends this routine. If the determination in step S200 is positive (step S200: Yes), the control unit 11 proceeds to step S202.

[0123] In step S202, the control unit 11 reads the teacher route data D1 and the map data D2 from the storage device 18 (step S202). Then, the control unit 11 controls the mobile control device 12 to perform autonomous traveling along the teacher route R1 represented by the teacher route data D1, thereby starting autonomous traveling along the teacher route R1 (step S204).

[0124] The control unit 11 estimates the current position of the vehicle 1 based on the map data D2 and the captured image of the surroundings of the vehicle 1 acquired by at least one camera 16 (step S206).

[0125] Then, the control unit 11 determines whether the current position of the vehicle 1 estimated in step S206 is away from the teacher route R1 by a first predetermined distance or more (step S208). If the determination in step S208 is affirmative (step S208: Yes), the process proceeds to step S210.

[0126] In step S210, the control unit 11 causes the display device 22 to display a returnable position P3 where the robot can return to the teacher route R1 (step S210).

[0127] The user, who is the driver of the vehicle 1, starts driving the vehicle 1 while visually checking the returnable position P3 displayed on the display device 22 so that the vehicle 1 reaches the returnable position P3.

[0128] Next, the control unit 11 repeats a negative determination (step S212: No) until it determines that the control unit 11 has received a driving operation from the user (step S212: Yes). If the control unit 11 makes a positive determination (step S212: Yes) in step S212, the control unit 11 proceeds to step S214. In step S214, the control unit 11 cancels the autonomous driving mode (step S214).

[0129] The processing of step S214 enables the user, who is the driver of the vehicle 1, to drive the vehicle 1, and the user drives the vehicle 1 so that the vehicle 1 reaches the returnable position P3 while visually checking the returnable position P3 displayed on the display device 22. The control unit 11 controls the movement control device 12 so that the vehicle 1 travels in accordance with the driving operation by the user.

[0130] The control unit 11 estimates the current position of the vehicle 1 in the same manner as in step S206, and determines whether the vehicle 1 has moved to the returnable position P3 (step S216). The control unit 11 repeats a negative determination (step S216: No) until a positive determination is made in step S216 (step S216: Yes). When the control unit 11 makes a positive determination in step S216 (step S216: Yes), the process returns to step S200.

[0131] On the other hand, if the control unit 11 determines a negative judgment in step S208 (step S208: No), the control unit 11 proceeds to step S218. In step S218, the control unit 11 controls the mobile control device 12 so that the current position of the vehicle 1 estimated in step S206 is a position on the teacher route R1 represented by the teacher route data D1, thereby causing the vehicle 1 to autonomously travel along the teacher route R1 from a predetermined position P1 toward the parking target position P2 (step S218). The control unit 11 then repeats a negative judgment (step S220: No) until it determines that the vehicle 1 has reached the parking target position P2 (step S220: Yes). If the control unit 11 determines a positive judgment in step S220 (step S220: Yes), the control unit 11 controls the mobile control device 12 to stop the vehicle 1 at the parking target position P2, and ends this routine.

[0132] As described above, the vehicle control device 10 of this embodiment is mounted on the vehicle 1, which includes the camera 16 that acquires a surrounding image, the display device 22 that is visible to the passengers of the vehicle 1, and the mobile control device 12 that controls at least steering. The vehicle control device 10 controls at least steering by the mobile control device 12, causing the vehicle 1 to autonomously travel to the parking target position P2 along a teacher path R1 obtained by teacher traveling from a predetermined position P1 to a parking target position P2. The vehicle control device 10 displays the surrounding image acquired by the camera 16 on the display device 22. When the vehicle 1 moves away from the teacher path R1 by more than a first predetermined distance while traveling from the predetermined position P1 to the parking target position P2 along the teacher path R1, the vehicle control device 10 displays a returnable position P3, at which the vehicle 1 can return to the teacher path R1, superimposed on the surrounding image on the display device 22.

[0133] In this way, when the vehicle 1 moves away from the teacher route R1 by more than a first predetermined distance while moving from a predetermined position P1 to a parking target position P2 along the teacher route R1, the vehicle control device 10 of this embodiment displays a returnable position P3 from which the vehicle can return to the teacher route R1 on the display device 22, superimposed on the surrounding image.

[0134] Therefore, when the vehicle 1 moves away from the teacher route R1 by a distance equal to or greater than a first predetermined distance and is unable to estimate the position of the vehicle 1 relative to the teacher route R1, that is, when the vehicle 1 loses its current position relative to the teacher route R1, the vehicle control device 10 of this embodiment displays a returnable position P3 on the display device 22. Therefore, the driver of the vehicle 1 can easily move the vehicle 1 to the returnable position P3 and cause the vehicle 1 to resume autonomous driving by performing driving operations while visually checking the returnable position P3 displayed on the display device 22.

[0135] Therefore, the vehicle control device 10 of this embodiment can provide more suitable parking assistance.

[0136] Next, the hardware configuration of the vehicle control device 10 of this embodiment will be described.

[0137] FIG. 12 is a block diagram showing an example of the hardware configuration of the vehicle control device 10.

[0138] The vehicle control device 10 has a hardware configuration that utilizes a normal computer, with a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, an I / F 11D for connecting to various devices, and the like interconnected by a bus 11E.

[0139] The CPU 11A is a calculation device that controls the overall processing of the vehicle control device 10. The RAM 11C stores data necessary for various processes by the CPU 11A. The ROM 11B stores programs and the like that realize various processes by the CPU 11A. The I / F 11D is an interface that is connected to an external device or external terminal via a communication line or the like and transmits and receives data to and from the connected external device or external terminal.

[0140] The programs for executing the various processes described above executed by the vehicle control device 10 are provided by being pre-installed in the ROM 11B, etc. The programs for executing the vehicle control method executed in this embodiment may be configured to be provided by being recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD) in a format that can be installed or executed by these devices.

[0141] The program for executing the vehicle control method according to the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program for executing the vehicle control method according to the present embodiment may be provided or distributed via a network such as the Internet.

[0142] Although an embodiment of the present disclosure has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment is included in the scope and spirit of the invention, as well as in the invention described in the claims and their equivalents.

[0143] 1 Vehicle 10 Vehicle control device 12 Movement control device 16 Camera 22 Display device

Claims

1. A vehicle control method executed by a vehicle control device mounted on a vehicle including a camera that acquires surrounding images, a display device visible to a passenger, and at least a movement control device that controls steering, and causing the movement control device to control at least the steering along a teaching route obtained by a teaching run from a predetermined position to a parking target position to autonomously drive the vehicle to the parking target position, wherein the surrounding images acquired by the camera are displayed on the display device; when the vehicle moves from the predetermined position to the parking target position along the teaching route and the vehicle is separated from the teaching route by a first predetermined distance or more, a returnable position where the vehicle can return to the teaching route is superimposed and displayed on the surrounding images on the display device.

2. The vehicle control method according to claim 1, wherein the vehicle further includes an operation device that receives an operation by a passenger; after the returnable position is superimposed and displayed on the surrounding images on the display device, when the passenger operates the operation device to move the vehicle to the returnable position, the autonomous driving is resumed along the teaching route to the parking target position.

3. The vehicle control method according to claim 1, when the vehicle is separated from the teaching route by the first predetermined distance or more, a document that prompts movement to the returnable position where the vehicle can return to the teaching route is displayed together with the surrounding images on the display device.

4. The vehicle control method according to claim 1, when the vehicle is separated from the teaching route by the first predetermined distance or more, a document indicating that the vehicle is separated from the teaching route by the first predetermined distance or more and the returnable position where the vehicle can return to the teaching route are displayed together with the surrounding images on the display device.

5. The vehicle control method according to claim 1, when the vehicle moves from the predetermined position to the parking target position along the teaching route and the vehicle is separated from the teaching route by the first predetermined distance or more, the returnable position where the vehicle can return to the teaching route and the teaching route are superimposed and displayed on the surrounding images on the display device.

6. A vehicle control method according to claim 1, wherein a current position of the vehicle is estimated based on three-dimensional positions of a plurality of feature points around the vehicle when traveling along the teacher route and map data in which feature amounts of the feature points are registered, and a captured image around the vehicle acquired by at least one camera, and it is determined that the vehicle is separated from the teacher route by a distance greater than or equal to the first predetermined distance from the current position of the vehicle. Vehicle control method.

7. A vehicle control method according to claim 1, wherein the returnable position is a position on a partial route that is linear by a distance greater than or equal to a second predetermined distance on the teacher route. Vehicle control method.

8. A vehicle control method according to claim 1, wherein the returnable position is a position on a partial route having a curvature equal to or less than a predetermined curvature on the teacher route. Vehicle control method.

9. A vehicle control method according to claim 1, wherein the returnable position is a position separated from the parking target position on the teacher route by a distance greater than or equal to a third predetermined distance. Vehicle control method.

10. A vehicle control device mounted on a vehicle including a camera that acquires a surrounding video, a display device visible to a passenger, and a movement control device that controls at least steering, and along a teacher route obtained by teacher driving from a predetermined position to a parking target position, the movement control device is controlled to at least steer, and the vehicle is autonomously driven to the parking target position. The vehicle control device includes: displaying the surrounding video acquired by the camera on the display device; and when the vehicle is separated from the teacher route by a distance greater than or equal to a first predetermined distance while moving from the predetermined position to the parking target position along the teacher route, displaying a returnable position where the vehicle can return to the teacher route by superimposing it on the surrounding video. Vehicle control device.

11. The vehicle control device according to claim 10, wherein the vehicle further includes an operating device that receives an operation of a passenger, and after the returnable position is displayed by superimposing it on the surrounding video on the display device, when the passenger operates the operating device and moves the vehicle to the returnable position, the autonomous driving is resumed along the teacher route to the parking target position. Vehicle control device.

12. The vehicle control device according to claim 10, wherein when the vehicle is separated from the teacher route by a distance equal to or greater than the first predetermined distance, the display device displays the returnable position where the vehicle can return to the teacher route and a document prompting movement to the returnable position together with the surrounding video.

13. The vehicle control device according to claim 10, wherein when the vehicle is separated from the teacher route by a distance equal to or greater than the first predetermined distance, the display device displays the returnable position where the vehicle can return to the teacher route and a document indicating that the vehicle is separated from the teacher route by a distance equal to or greater than the first predetermined distance together with the surrounding video.

14. The vehicle control device according to claim 10, wherein when the vehicle is separated from the teacher route by a distance equal to or greater than the first predetermined distance while moving from the predetermined position to the parking target position along the teacher route, the display device superimposes and displays the returnable position where the vehicle can return to the teacher route and the teacher route on the surrounding video.

15. The vehicle control device according to claim 10, wherein the current position of the vehicle is estimated based on the three-dimensional positions of each of a plurality of feature points around the vehicle and the feature amounts of the feature points registered in the map data during traveling along the teacher route, and the captured image of the surrounding of the vehicle acquired by at least one camera, and it is determined that the vehicle is separated from the teacher route by a distance equal to or greater than the first predetermined distance from the current position of the vehicle.

16. The vehicle control device according to claim 10, wherein the returnable position is a position on a partial route that is linear by a second predetermined distance or more on the teacher route.

17. The vehicle control device according to claim 10, wherein the returnable position is a position on a partial route having a curvature equal to or less than a predetermined curvature on the teacher route.

18. The vehicle control device according to claim 10, wherein the returnable position is a position separated from the parking target position on the teacher route by a third predetermined distance or more.

Images