Vehicle control method and vehicle control device

Abstract

To provide more suitable parking support.SOLUTION: In a vehicle control device 10, in an autonomous traveling mode, when an obstacle is detected in front of a vehicle 1 in a movement direction on the basis of an external situation acquired by a sensor 14 during movement by first autonomous traveling from a predetermined position to a parking target position along a teacher route R1, a traveling mode is switched to a temporary manual traveling mode different from a manual traveling mode, and an operation device 20 receives manual operation by an occupant. Then, in the vehicle control device 10, after the vehicle 1 is moved by the manual operation, when the obstacle is not detected in front of the vehicle 1 in the movement direction on the basis of the external situation acquired by the sensor 14, a display device 22 outputs a first screen 30C showing return to the teacher route by autonomous traveling. Then, the vehicle control device 10 allows second autonomous traveling to be performed to a halfway point between the predetermined position and the parking target position on the teacher route, and thereafter allows third autonomous traveling to be performed from the halfway point to the parking target position along the teacher route.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a vehicle control method and a vehicle control device. [Background technology]

[0002] Generally, parking spaces in homes are often narrow and can be difficult to park in, so there is a high demand for automated parking and exit.

[0003] There are known vehicle control devices that realize this type of automatic driving. For example, a technology has been disclosed in which a driver manually drives a vehicle from a predetermined position to a parking target position, the route traveled during this process is stored in advance as a training route, and the vehicle autonomously drives along the training route in subsequent parking situations (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-124898 Summary of the Invention [Problem to be solved by the invention]

[0005] However, during autonomous driving along the teacher route, obstacles that were not present during the teacher driving may appear on the teacher route. In such cases, with the conventional technology, automatic parking by autonomous driving may be difficult. In other words, with the conventional technology, it may be difficult to provide suitable parking assistance.

[0006] An object of the present disclosure is to provide a vehicle control method and a vehicle control device that can provide more suitable parking assistance. [Means for solving the problem]

[0007] A vehicle control method according to the present disclosure is a vehicle control method that is executed by a vehicle that includes a sensor that acquires external conditions, an operation device that accepts operations by a passenger, a steering device that accepts steering operations by the passenger, a display device that is visible to the passenger, and a movement control unit that controls at least steering, and that holds a teacher path obtained by teacher driving from a predetermined position to a parking target position. The vehicle control method according to the present disclosure includes: in a manual driving mode, causing the vehicle to travel in response to a steering operation of the steering device by the passenger; when the vehicle is within a predetermined range from the predetermined position and the operation device accepts a first operation, switching to an autonomous driving mode; thereafter, in the autonomous driving mode, controlling at least the steering along the teacher path; and, during movement by first autonomous driving from the predetermined position toward the parking target position, if an obstacle is detected ahead in the direction of movement of the vehicle based on the external conditions acquired by the sensor, switching to a temporary manual driving mode that is different from the manual driving mode, in which the operation device and the steering device accept manual operation by the passenger; and then, in the temporary manual driving mode, controlling the vehicle in response to the manual operation. Therefore, after the vehicle has moved, if the obstacle is not detected ahead in the direction of movement of the vehicle based on the external conditions acquired by the sensor, the display device outputs a first screen showing return to the teacher route by autonomous driving, and after performing a second autonomous driving to a midpoint between the predetermined position on the teacher route and the parking target position, performs a third autonomous driving along the teacher route from the midpoint to the parking target position, and in the manual driving mode, the vehicle is driven in response to steering operation of the steering device by the occupant, and if the vehicle is within a predetermined range from the predetermined position and the operating device does not receive a first operation, the vehicle continues to be driven in the manual driving mode to the parking target position in response to steering operation of the steering device by the occupant.In the vehicle control method, when the operating device receives a second operation in the temporary manual driving mode, the mode is switched to the manual driving mode, and when, in the manual driving mode, after the vehicle has moved by the manual operation, the obstacle is not detected ahead in the direction of movement of the vehicle based on the external conditions acquired by the sensor, the display device does not output the first screen indicating return to the teacher route by autonomous driving. [Effects of the Invention]

[0008] According to the vehicle control method and vehicle control device disclosed herein, more suitable parking assistance can be provided. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle. [Figure 2] FIG. 2 is a schematic diagram showing an example of the arrangement of the vehicle-mounted cameras. [Figure 3] FIG. 3 is a schematic diagram showing an example of the external configuration of a vehicle. [Figure 4] FIG. 4 is a diagram showing an example of the configuration near the driver's seat of the vehicle according to this embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a teacher route. [Figure 6] FIG. 6 is an explanatory diagram of an example of a process for estimating the current position of the vehicle executed by the control unit in the autonomous driving mode. [Figure 7] FIG. 7 is a diagram illustrating an example in which an obstacle is detected on the teacher route. [Figure 8A] FIG. 8A is a schematic diagram of an example of the third screen. [Figure 8B] FIG. 8B is a schematic diagram of an example of the third screen. [Figure 9] FIG. 9 is a schematic diagram of an example of the fourth screen. [Figure 10A] FIG. 10A is a schematic diagram of an example of the first screen. [Figure 10B] FIG. 10B is a schematic diagram of an example of the first screen. [Figure 11] FIG. 11 is an explanatory diagram of an example of generating a return path. [Figure 12A] FIG. 12A is a schematic diagram of an example of the second screen. [Figure 12B] FIG. 12B is a schematic diagram of an example of the second screen. [Figure 13] FIG. 13 is a schematic diagram of an example of the fifth screen. [Figure 14] FIG. 14 is a flowchart showing an example of the flow of information processing executed by the control unit in the teacher running mode. [Figure 15] FIG. 15 is a flowchart showing an example of the flow of information processing executed by the control unit in the autonomous driving mode. [Figure 16] FIG. 16 is a flowchart showing an example of the flow of interrupt processing executed by the control unit. [Figure 17] FIG. 17 is a block diagram showing an example of the hardware configuration of a vehicle control device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of a vehicle control method and a vehicle control device according to the present disclosure will be described with reference to the drawings.

[0011] FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle 1. As shown in FIG.

[0012] The vehicle 1 includes a vehicle control device 10, a movement control device 12, a sensor 14, a storage device 18, an operation device 20, a display device 22, and a steering device 27.

[0013] The vehicle control device 10 is connected to the movement control device 12, the sensor 14, the storage device 18, the operation device 20, the display device 22, and the steering device 27 so as to be able to exchange data or signals. In other words, the vehicle control device 10 is configured to be communicatively connected to at least the sensor 14, the operation device 20, the display device 22, the movement control device 12, and the steering device 27.

[0014] The mobility control device 12 controls at least the steering of the vehicle 1. The mobility control device 12 is a means for realizing the driving, braking, and turning movements required for the vehicle 1 to travel. For example, the mobility control device 12 is configured to include a drive motor, a power transmission mechanism, a brake device, a steering device, etc., and an electronic vehicle control device that controls them. The mobility control device 12 travels the vehicle 1, for example, by generating power using the drive motor and transmitting the power to the wheels via the power transmission mechanism. The power transmission mechanism is, for example, a propeller shaft, a differential gear, a drive shaft, etc.

[0015] Controlling at least the steering means that the mobile control device 12 controls at least one of the driving, braking, and turning motions required for the travel of the vehicle 1. In other words, controlling the steering means that the mobile control device 12 controls at least one of the turning direction by steering, the vehicle speed or acceleration by accelerator steering, and deceleration or stopping by braking. Controlling at least the acceleration and deceleration means that the mobile control device 12 controls at least one of the acceleration and deceleration of the vehicle 1.

[0016] Specifically, the movement control device 12 includes an auxiliary control device 12A, a brake control device 12B, an engine control device 12C, and a power steering control device 12D. The brake control device 12B, the engine control device 12C, and the power steering control device 12D can be collectively referred to as an actuator control unit that controls the operation of the vehicle 1.

[0017] The auxiliary control device 12A is a control device that monitors the transmission status of the vehicle control device 10 and operates as a backup to execute appropriate degeneration control in the event of a failure of the vehicle control device 10. Note that if safety can be ensured by providing a degeneration control function within the vehicle control device 10 even in the event of a failure of the vehicle control device 10, degeneration control is not necessary.

[0018] The brake control device 12B is a control device that controls the brakes of the vehicle 1. The brake control is sometimes referred to as braking force control. For example, the brake control device 12B controls the brakes of the vehicle 1 in accordance with the intensification and relaxation of the brake pedal operation by the passenger. Intensification of the brake pedal operation by the passenger specifically means the passenger depressing the brake pedal. Furthermore, the brake control device 12B controls the brakes in accordance with the surrounding image during autonomous driving.

[0019] The engine control device 12C is a control device that controls an engine that generates a driving force for the vehicle 1. The power steering control device 12D is a control device that controls the power steering of the vehicle 1.

[0020] The sensor 14 is mounted on the vehicle 1 and acquires at least the situation outside the vehicle 1. In detail, the sensor 14 is a variety of sensors that detect the running state of the vehicle 1 and the situation outside the vehicle 1. The external situation includes an image of the outside of the vehicle 1.

[0021] The sensor 14 includes at least a camera 16. The sensor 14 also includes at least one of a LiDAR (Light detection and ranging), radar, sonar, and ultrasonic sensor. The sensor 14 includes, for example, an accelerator opening sensor that detects an accelerator opening, a steering angle sensor that detects a steering angle of a steering device, an acceleration sensor that detects acceleration acting in the longitudinal direction of the vehicle 1, a torque sensor that detects torque acting on a power transmission mechanism between the wheels of the vehicle 1 and the drive motor, a vehicle speed sensor that detects the vehicle speed of the vehicle 1, a wheel speed sensor, a GPS (Global Positioning System), and the like. The sensor 14 outputs sensor information obtained by detection to the vehicle control device 10.

[0022] The camera 16 is a surrounding sensor mounted on the vehicle 1 and monitors the environment surrounding the vehicle 1. In this embodiment, the camera 16 captures images of the surroundings of the vehicle 1 and outputs the captured image data to the vehicle control device 10. Hereinafter, the captured image data may be simply referred to as image. In this embodiment, the camera 16 is also used to detect objects present around the vehicle 1 and estimate the location of the vehicle 1 from the positional relationship between the vehicle 1 and the objects present around the vehicle 1.

[0023] The positions, number of cameras 16, and shooting directions are adjusted in advance so that the cameras 16 can capture images of the surroundings of the vehicle 1. For example, the vehicle 1 is provided with four cameras 16 arranged so as to capture images in four directions, namely, the front, rear, left, and right directions of the vehicle 1. Note that the number of cameras 16 provided on the vehicle 1 is not limited to four.

[0024] The storage device 18 stores various types of data. In this embodiment, the storage device 18 stores data such as teacher route data 18A and map data 18B. That is, the storage device 18 holds a teacher route represented by teacher route data 18A. Details of the teacher route data 18A and the map data 18B will be described later.

[0025] The storage device 18 is, for example, an auxiliary storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. At least a part of the data included in the storage device 18 may be stored in an external storage device such as a server device that is provided outside the vehicle 1 and communicably connected to the vehicle control device 10.

[0026] The operation device 20 accepts operations by a passenger of the vehicle 1. The operation device 20 includes operation mechanisms related to driving operations, such as an accelerator pedal, a brake pedal, a turn signal lever, and a push switch, as well as input devices, such as a keyboard, a touch panel, and a switch.

[0027] At least one of the keyboard, the touch panel, and the switch functions as an automatic parking instruction unit that receives an instruction to start automatic parking.

[0028] The shift lever is an example of a forward / reverse operation unit. The forward / reverse operation unit switches the vehicle 1 at least between forward and reverse. The brake pedal is an example of a brake operation unit. The brake operation unit is an operation unit for a brake that suppresses the speed of the vehicle 1. In other words, the brake operation unit receives an instruction to decelerate the vehicle 1. The operation device 20 may constitute a part of an HMI (Human Machine Interface) or an IVI (In-Vehicle Infotainment).

[0029] The display device 22 is a display that outputs various images. The display device 22 is arranged in a position that is visible to the passengers of the vehicle 1. The display device 22 is installed in a position that is visible to the passengers of the vehicle 1. Examples of the display include a liquid crystal display (LCD), an organic electro-luminescence (EL) display, and a projector. The display device 22 may be a touch panel display that is an integral configuration of the display device 22 and the operation device 20. The display device 22 is an example of at least one of an HMI and an IVI.

[0030] The display device 22 is not limited to having only one display area, but may have multiple display areas, for example.

[0031] The vehicle 1 may also include multiple display devices 22. For example, the display device 22 may include a first display unit and a second display unit, where the first display unit and the second display unit are displays that output various images. The first display unit and the second display unit are display devices 22 configured as separate units. The first display unit and the second display unit may be located in different positions within the vehicle 1. For example, the first display unit may function as an IVI, and the second display unit may function as part of the instrument panel of the vehicle 1.

[0032] The steering device 27 receives steering operations from a passenger of the vehicle 1. The steering device 27 is, for example, a steering wheel. The steering wheel is sometimes called a handle. The steering angle of the steering device is adjusted by the passenger's operation of the steering device.

[0033] FIG. 2 is an explanatory diagram of an example of the arrangement of the sensor 14 and the camera 16.

[0034] Vehicle 1 is provided with, for example, four cameras 16 (camera 16A to camera 16D) so that the external situation of vehicle 1 can be acquired in at least four directions, for example, in front, behind, to the right, and to the left of vehicle 1.

[0035] Specifically, for example, camera 16 includes camera 16A, camera 16B, camera 16C, and camera 16D. Camera 16A is disposed at the front of vehicle 1 and photographs the area in front of vehicle 1. Camera 16A may be referred to as a front camera. Camera 16B is disposed at the right side of vehicle 1 and photographs the area to the right of vehicle 1. Camera 16C is disposed at the left side of vehicle 1 and photographs the area to the left of vehicle 1. Camera 16D is disposed at the rear of vehicle 1 and photographs the area to the rear of vehicle 1. Camera 16D may be referred to as a rear camera, rear camera, etc.

[0036] The number of cameras 16 provided on the vehicle 1 is not limited to four. It is also preferable that the placement positions and number of sensors included in the sensor 14, such as lidar, radar, sonar, and ultrasonic sensors, that detect objects are adjusted in advance so as to be able to acquire the external conditions on the right side, left side, front, and rear of the vehicle 1. For example, as shown in FIG. 2, the sensor 14 includes sensors 14A to 14F. These sensors 14A to 14F are arranged on the vehicle 1 so as to be able to acquire the external conditions on the right side, left side, front, and rear of the vehicle 1. The object-detecting sensors 14, such as lidar, radar, sonar, and ultrasonic sensors, may be arranged only in the rear of the vehicle 1.

[0037] Next, the configuration of the vehicle 1 will be described.

[0038] FIG. 3 is a schematic diagram showing an example of the external configuration of the vehicle 1. As shown in FIG.

[0039] The vehicle 1 includes a vehicle body 2 and two pairs of wheels 23 arranged along a predetermined direction on the vehicle body 2. The two pairs of wheels 23 include a pair of front tires 23F and a pair of rear tires 23R (see also FIG. 2). Note that FIGS. 2 and 3 show an example in which the vehicle 1 includes four wheels 23. However, the number of wheels 23 provided on the vehicle 1 is not limited to this.

[0040] Next, the configuration of the vicinity of the driver's seat of the vehicle 1 of this embodiment will be described.

[0041] FIG. 4 is a diagram showing an example of the configuration of the vicinity of the driver's seat 24A of the vehicle 1 according to this embodiment.

[0042] Vehicle 1 has a driver's seat 24A and a passenger seat 24B. In front of driver's seat 24A are provided a windshield 25, a dashboard 26, a steering device 27, operation buttons 20B, and a display device 22. Also, near driver's seat 24A is provided a shift lever 20C, which is a lever for changing gears in the transmission.

[0043] The steering device 27, the operation button 20B, and the shift lever 20C are examples of the operation device 20.

[0044] The steering device 27 is provided in front of the driver's seat 24A and can be operated by the passenger. As described above, the steering device 27 accepts steering operations from the passenger. The rotation angle of the steering device 27, i.e., the steering angle, is electrically or mechanically linked to the change in direction of the front tires 23F, which are the steered wheels. Note that the steered wheels may be the rear tires 23R, or both the front tires 23F and the rear tires 23R may be steered wheels.

[0045] The operation button 20B is a button that can accept operation by a passenger. The operation button 20B may include a direction indicator. The position of the operation button 20B is not limited to the example shown in FIG. 4, and may be provided, for example, on the steering device 27. Although one operation button 20B is illustrated in FIG. 4, a plurality of operation buttons 20B may be provided. When the display device 22 also functions as a touch panel, the display device 22 may be an example of the operation device 20.

[0046] Returning to Figure 1, we continue the explanation.

[0047] The vehicle control device 10 is an electronic control unit that controls each part of the vehicle 1 in an integrated manner.

[0048] The vehicle control device 10 uses sensor information and captured images received from the sensor 14 and the camera 16, respectively, to control the movement control device 12 so as to optimize the driving conditions of the vehicle 1. The vehicle control device 10 also controls the movement control device 12 to cause the vehicle 1 to drive autonomously.

[0049] The vehicle control device 10 includes a control unit 11. Part or all of the control unit 11 may be a software configuration realized by cooperation between a processor and various programs stored in memory. Alternatively, part or all of the control unit 11 may be a hardware configuration realized by a dedicated circuit or the like.

[0050] The control unit 11 controls each part of the vehicle 1 in an integrated manner.

[0051] In this embodiment, the control unit 11 is configured to be able to switch the driving mode between a supervised driving mode, an autonomous driving mode, a manual driving mode, and a temporary manual driving mode based on an input operation by the passenger on the operation device 20. Note that the driving modes executable by the vehicle 1 may include various driving modes other than the supervised driving mode, the autonomous driving mode, the manual driving mode, and the temporary manual driving mode.

[0052] The supervised driving mode is a mode for registering a supervised route for autonomous driving of the vehicle 1. The supervised route is a route obtained by supervised driving from a predetermined position to a target parking position. In the supervised driving mode, the vehicle 1 is controlled to travel by manual operation of the occupant. That is, in the supervised driving mode, the control unit 11 controls the movement control device 12 to travel in accordance with manual operation, which is a driving operation by the occupant.

[0053] The autonomous driving mode is a mode in which the vehicle 1 travels autonomously. In this embodiment, the autonomous driving mode refers to a mode in which the vehicle 1 travels autonomously along a teacher route. In the autonomous driving mode, the control unit 11 controls at least steering and controls the mobility control device 12 so that the vehicle travels along the teacher route. In the autonomous driving mode, the vehicle 1 is automatically controlled to travel by the vehicle control device 10 without manual operation by the passenger.

[0054] The manual driving mode is a driving mode in which the vehicle 1 is driven in response to a steering operation of the steering device 27 by the passenger.

[0055] The temporary manual driving mode is a driving mode in which manual driving is temporarily performed in the autonomous driving mode. The temporary manual driving mode is a driving mode different from the manual driving mode. In detail, the temporary manual mode is a driving mode in which, during autonomous driving, the operation device 20 and the steering device 27 temporarily accept manual operation by the occupant to drive the vehicle 1, and then, when a predetermined condition is satisfied, the vehicle 1 starts to drive autonomously.

[0056] FIG. 5 is an explanatory diagram of an example of the teacher route R1.

[0057] In the supervised driving mode, the vehicle is manually driven by the rider to steer the vehicle from a predetermined position P1 to a target parking position P2. The target parking position P2 may be, for example, a parking lot, but is not limited to this. The predetermined position P1 may be any position of the rider in real space.

[0058] The travel route R traveled during teacher driving is treated as a teacher route R1, and teacher route data 18A for the teacher route R1 is stored in the storage device 18. During teacher driving, the occupant may manually operate the vehicle 1 to travel from the parking target position P2 toward the predetermined position P1, or may manually operate the vehicle 1 to travel from the predetermined position P1 toward the parking target position P2. When the vehicle 1 travels from the parking target position P2 toward the predetermined position P1 in teacher driving mode, the control unit 11 may create teacher route data 18A for the teacher route R1, which is the opposite of the traveling direction during teacher driving for the travel route R. When the vehicle 1 travels from the predetermined position P1 toward the parking target position P2 during teacher driving, the control unit 11 may create teacher route data 18A for the teacher route R1, which is the same as the traveling direction during teacher driving for the travel route R. The creation of the teacher route data 18A will be described in detail later.

[0059] In the autonomous driving mode, the control unit 11 controls at least the steering along the driving route R obtained by the supervised driving, and causes the vehicle 1 to autonomously drive to the parking target position P2. Note that in the autonomous driving mode, the control unit 11 controls the steering and the front and rear acceleration / deceleration of the vehicle 1, but at least a part of the front and rear acceleration / deceleration control may be performed by the driver's operation.

[0060] Next, the control by the control unit 11 in the teacher driving mode and the autonomous driving mode will be described in detail.

[0061] [Teacher driving mode] First, the control of the control unit 11 in the teacher running mode will be described in detail.

[0062] When the control unit 11 receives a signal indicating an instruction to start the teacher-driven driving mode, for example, by the rider operating the operation device 20, the control unit 11 switches the driving mode to the teacher-driven driving mode. Then, when the control unit 11 is in the teacher-driven driving mode, the control unit 11 executes the following processing.

[0063] The control unit 11 acquires sensor information indicating the traveling state of the vehicle 1 from the sensor 14. Then, the control unit 11 estimates the current position of the vehicle 1 based on the temporal changes in the sensor values ​​represented by the sensor information. For example, the control unit 11 calculates the amount of movement of the vehicle 1 from a reference position, such as the traveling start position when the teacher traveling mode was started, based on the temporal changes in the vehicle speed and yaw rate represented by the sensor values, and estimates the current position of the vehicle 1 based on the amount of movement.

[0064] Note that the accuracy of estimating the current position based on the amount of movement may be low. Therefore, the control unit 11 may correct the estimated current position based on the video of the surroundings of the vehicle 1 captured by the camera 16, and use the corrected result as the current position.

[0065] The control unit 11 sequentially stores the current position of the vehicle 1 estimated sequentially as the vehicle 1 travels in the storage device 18. In detail, the control unit 11 defines the travel route R during teacher travel, which is represented by a group of current positions sequentially estimated from the time when an instruction to start the teacher travel mode is received until the time when an instruction to end the teacher mode is received, as a teacher route R1, and stores teacher route data 18A representing the teacher route R1 in the storage device 18.

[0066] The teacher route data 18A is composed of a group of driving information for each position, which is the current position that is sequentially estimated during teacher driving. The driving information includes an INDEX, a driving position, a bearing, a driving direction, and reference driving information. The INDEX is identification information for the driving information. The driving position is the estimated position of the vehicle 1. The bearing indicates the orientation of the vehicle 1 at that position. The driving direction indicates the direction of driving of the vehicle 1 at that position, and is expressed, for example, as forward or reverse. The reference driving information is information that indicates the driving state at that position. The reference driving information is, for example, information such as the steering angle and vehicle speed detected at each position during teacher driving.

[0067] Furthermore, when the vehicle 1 is traveling on a teacher's route, the control unit 11 creates map data 18B for estimating the current position of the vehicle 1 from the captured image taken by the camera 16. The method for estimating the current position of the vehicle 1 from the captured image may be a SLAM (Simultaneous Localization and Mapping) method or the like.

[0068] The map data 18B is map data in which a plurality of characteristic points around the vehicle 1 when traveling along the teaching route R1 are registered.

[0069] A feature point is a characteristic point obtained by image analysis of the video captured by the camera 16 during teacher driving. For example, a feature point is a portion of an object (e.g., a tree, wall, or pillar) that can serve as a landmark in the actual scene, from which a characteristic image pattern is obtained by analyzing the captured video. Such a portion is, for example, an edge portion of the object. The map data 18B includes a plurality of feature points, and each feature point is registered by being assigned an identification number so that it can be identified.

[0070] A feature point is represented by feature point data including a three-dimensional position and a feature amount. The three-dimensional position of a feature point is the three-dimensional position of the feature point in real space and is represented, for example, by a three-dimensional Cartesian coordinate system (X, Y, Z). The feature amount of a feature point is a characteristic amount of the feature point represented by image analysis of the captured video. The feature amount of a feature point is, for example, brightness or density in the captured video, SIFT (Scale Invariant Feature Transform) feature amount, SURF (Speeded Up Robust Features) feature amount, etc.

[0071] In the map data 18B, one feature point is registered for each identical three-dimensional position. Note that, for the same three-dimensional position, the map data 18B may register multiple feature points for each photographing position and photographing direction of the three-dimensional position by the camera 16. Furthermore, the feature point data of the feature points registered in the map data 18B may further include image data of an object having the feature point.

[0072] During supervised driving, the control unit 11 identifies the coordinates of feature points in the actual scene, for example, based on stereo photogrammetry. Specifically, the control unit 11 reads multiple captured images taken at different times and associates the same feature points that appear in the multiple captured images. The control unit 11 then estimates the virtual position of the vehicle 1 at the time the multiple captured images were taken and identifies the virtual coordinates of the feature points in the actual scene using the principle of triangulation. The control unit 11 then performs bundle adjustment, for example, using the virtual position of the vehicle 1 and the virtual coordinates of the feature points in the actual scene as reference information, to calculate the formal position of the vehicle 1 and the formal coordinates of the feature points in the actual scene so as to minimize the reprojection error when each feature point in the actual scene is projected onto all captured images. The control unit 11 then stores, in the storage device 18, map data 18B in which feature points are registered, represented by feature point data including the formal coordinates of the feature points in the actual scene as three-dimensional positions.

[0073] The three-dimensional positions of the feature points registered in the map data 18B may be positions measured in advance using a LiDAR or a stereo camera, instead of using the SLAM method. However, from the viewpoint of suppressing a decrease in the accuracy of position estimation, the SLAM method may be used.

[0074] As described above, the control unit 11 executes the above process in the teacher driving mode. Therefore, in the teacher driving mode, the control unit 11 generates teacher route data 18A of the teacher route R1 obtained by teacher driving from a predetermined position P1 to a parking target position P2, and map data 18B in which the three-dimensional positions of each of a plurality of feature points around the vehicle 1 while traveling along the teacher route R1 and the feature amounts of the feature points are registered, and stores these in the storage device 18.

[0075] [Autonomous driving mode] Next, the control of the control unit 11 in the autonomous driving mode will be described in detail.

[0076] When the control unit 11 receives a signal indicating an instruction to start the autonomous driving mode as a result of the operation device 20 receiving a first operation such as an operation of the operation device 20 by a passenger, the control unit 11 switches the driving mode to the autonomous driving mode. Then, when in the autonomous driving mode, the control unit 11 executes the following processing.

[0077] The control unit 11 reads the teacher route data 18A and the map data 18B from the storage device 18, and controls the mobile control device 12 so that the mobile device 12 performs autonomous traveling along the teacher route R1 represented by the teacher route data 18A.

[0078] The control unit 11 estimates the current position of the vehicle 1 based on the map data 18B and the video of the surroundings of the vehicle 1 captured by at least one camera 16.

[0079] FIG. 6 is an explanatory diagram of an example of the process of estimating the current position of the vehicle 1 executed by the control unit 11 in the autonomous driving mode.

[0080] 6, S1, S2, and S3 represent three characteristic points extracted from the image captured by camera 16, and points Q1, Q2, and Q3 are characteristic points stored in map data 18B and represent the three-dimensional positions of the characteristic points SA, SB, and SC in real space. RP1 represents the imaging plane of camera 16. Point P' represents the position of camera 16 (i.e., the position of vehicle 1) determined from the three characteristic points SA, SB, and SC extracted from the image captured by camera 16 and the characteristic points (Q1, Q2, and Q3) stored in map data 18B.

[0081] For example, the control unit 11 first compares characteristic points extracted from the video captured by the camera 16 with characteristic points stored in the map data 18B using pattern matching, feature search, etc. Then, the control unit 11 randomly selects several (e.g., 3 to 6) characteristic points from the characteristic points extracted from the video captured by the camera 16 that can be compared with the characteristic points stored in the map data 18B.

[0082] Then, the control unit 11 estimates the current position of the vehicle 1 in real space based on the positions of these several characteristic points in the captured image and the three-dimensional positions in real space of characteristic points registered in the map data 18B that correspond to the several characteristic points. At this time, the control unit 11 estimates the current position of the vehicle 1 by solving the PnP problem using a known method such as Lambda Twist (for example, literature: Mikael Persson et al. “Lambda Twist: An Accurate Fast Robust Perspective Three Point (P3P) Solver.”, ECCV 2018, pp. 334-349, published in 2018, http: / / openaccess.thecvf.com / content_ECCV_2018 / papers / Mikael_Persson_Lambda_Twist_An_ECCV_2018_paper.pdf).

[0083] When comparing the characteristic points extracted from the image captured by the camera 16 with the characteristic points stored in the map data 18B, the control unit 11 may, for example, calculate the current position of the vehicle 1 as a tentative position based on the amount of movement of the vehicle 1 described above, and, using this tentative position as a reference, narrow down the characteristic points stored in the map data 18B to be compared with the characteristic points extracted from the image captured by the camera 16.

[0084] Through these processes, the control unit 11 estimates current position information representing the current position of the vehicle 1 based on the map data 18B and the captured image of the surroundings of the vehicle 1 acquired by at least one camera 16, and the current position information includes the two-dimensional position (X coordinate, Y coordinate) of the vehicle 1 in real space and information related to the attitude, which is the orientation of the vehicle 1.

[0085] The control unit 11 then controls the movement control device 12 so that the estimated current position of the vehicle 1 is a position on the teacher route R1 represented by the teacher route data 18A, thereby causing the vehicle 1 to autonomously travel along the teacher route R1 from a predetermined position P1 toward the parking target position P2. The control unit 11 then stops the vehicle 1 at the parking target position P2.

[0086] When the vehicle 1 is autonomously traveling along the teacher route R1, the control unit 11 feedback controls the movement control device 12 so that the vehicle 1 moves along the teacher route R1 based on the estimated current position of the vehicle 1 and each position on the teacher route R1 represented by the teacher route data 18A.

[0087] In the following description, the autonomous traveling of the vehicle 1 from a predetermined position P1 to a parking target position P2 along the teacher route R1 will be referred to as a first autonomous traveling.

[0088] Here, while the vehicle 1 is moving by the first autonomous driving along the teacher path R1 from a predetermined position P1 to the parking target position P2, an obstacle may be detected on the teacher path R1.

[0089] FIG. 7 is an explanatory diagram of an example in which an obstacle B is detected on the teacher route R1.

[0090] As shown in FIG. 7, an obstacle B that was not present when the teacher route data 18A was registered may be present on the teacher route R1 during the first autonomous traveling from a predetermined position P1 to a parking target position P2.

[0091] Obstacle B is an object that obstructs the travel of vehicle 1. In detail, obstacle B is an object that may make it difficult for vehicle 1 to continue traveling if it comes into contact with the object, or may cause some damage to vehicle 1 or cause an event that obstructs travel if vehicle 1 tries to come into contact with or pass over the object without avoiding the object.

[0092] Therefore, in this embodiment, the control unit 11 determines whether or not an obstacle B has been detected ahead of the vehicle 1 in the direction of movement D, based on the external conditions acquired by the sensor 14 during movement by the first autonomous driving.

[0093] Specifically, in the manual driving mode, control unit 11 drives vehicle 1 in response to a steering operation of steering device 27 by the occupant. Then, when vehicle 1 is within a predetermined range from predetermined position P1 and operation device 20 receives a first operation, control unit 11 switches to the autonomous driving mode. Thereafter, in the autonomous driving mode, control unit 11 controls at least the steering along teacher route R1 to drive vehicle 1 toward parking target position P2.

[0094] The first operation may be a predetermined operation that indicates an instruction to start the autonomous driving mode.

[0095] The predetermined range from the predetermined position P1 may be within a predetermined range from the predetermined position P1. The predetermined range from the predetermined position P1 may be a range narrower than the range from the predetermined position P1 to the parking target position P2.

[0096] Then, the control unit 11 determines whether or not an obstacle B has been detected ahead in the moving direction D of the vehicle 1, based on the external situation acquired by the sensor 14 during the movement by the first autonomous traveling.

[0097] For example, the control unit 11 determines whether or not an obstacle B exists on the teacher path R1 by analyzing, by a known analysis process, at least one of the sensor information received from the sensor 14 and the video captured by the camera 16. Through this determination process, the control unit 11 determines whether or not an obstacle B has been detected ahead in the movement direction D.

[0098] More specifically, the control unit 11 determines whether or not an obstacle B is present on the teaching path R1 ahead in the moving direction D of the vehicle 1 by deriving the distance to the obstacle B and the position of the obstacle B by analyzing the detection results of the LiDAR, radar, ultrasonic sensor, or the like included in the sensor 14, and by analyzing the image of the obstacle B included in the captured video. Through this determination process, the control unit 11 determines whether or not an obstacle B has been detected ahead in the moving direction D of the vehicle 1.

[0099] In this embodiment, when the control unit 11 detects an obstacle B ahead in the moving direction D of the vehicle 1, the driving mode is temporarily switched to a manual driving mode, and the operating device 20 and the steering device 27 accept manual operation by the occupant.

[0100] For example, imagine a situation in which vehicle 1 is traveling by first autonomous driving along teacher route R1 from a predetermined position P1 to a parking target position P2, and when it reaches point S1 on teacher route R1, control unit 11 detects obstacle B ahead of vehicle 1 in the direction of movement D based on sensor information obtained by sensor 14.

[0101] In this case, the control unit 11 controls the movement control device 12 to stop the vehicle 1 from traveling and switch the traveling mode from the autonomous traveling mode to the temporary manual traveling mode. By the control unit 11 switching the traveling mode from the autonomous traveling mode to the temporary manual traveling mode, traveling according to manual operation by the occupant becomes temporarily possible during autonomous traveling. That is, in the temporary manual traveling mode, the operation device 20 and the steering device 27 accept manual operation by the occupant, and the vehicle 1 travels according to manual operation by the occupant.

[0102] Furthermore, when the control unit 11 detects an obstacle B ahead in the moving direction D, it causes the display device 22 to output a third screen. The third screen is a screen that prompts the passenger to perform a manual operation to avoid the obstacle. In detail, for example, the third screen is a screen that includes at least one of a message that prompts the passenger to perform a manual operation to avoid the obstacle and a schematic diagram that prompts the passenger to perform a manual operation to avoid the obstacle.

[0103] 8A and 8B are schematic diagrams showing an example of a third screen 30 A. The third screen 30 A is an example of the screen 30 output by the display device 22.

[0104] 8A is a schematic diagram of an example of the third screen 30A1. The third screen 30A1 is an example of the third screen 30A.

[0105] The third screen 30A1 is, for example, a screen 30 including a message M1, a schematic diagram D1, and a cancel button 31. FIG. 8A shows an example in which the message M1 includes, "The vehicle has stopped because there is an obstacle on the registered route. Please manually drive to a position that avoids the obstacle," which is an example of a message prompting the occupant to perform manual operation to avoid the obstacle. FIG. 8A also shows an example in which the schematic diagram D1 is an arrow image indicating a recommended driving direction for manual operation by the occupant to avoid the obstacle. Note that the message M1 may include any message prompting the occupant to perform manual operation to avoid the obstacle, and is not limited to the message shown in FIG. 8A. Similarly, the schematic diagram D1 may be any message prompting the occupant to perform manual operation to avoid the obstacle, and is not limited to the form shown in FIG. 8A.

[0106] The cancel button 31 is an image area for receiving an instruction from the passenger to switch from the autonomous driving mode or the temporary manual driving mode to the manual driving mode. When the cancel button 31 is operated by the passenger and a signal to switch to the manual driving mode is received from the operation device 20 as a result of the operation, the control unit 11 switches the driving mode of the vehicle 1 to the manual driving mode. Then, the control unit 11 controls the movement control device 12 so that the vehicle 1 travels in accordance with the manual operation of the passenger. The operation of the cancel button 31 by the passenger is an example of a second operation received by the operation device 20.

[0107] The third screen 30A1 may further include at least one of the captured image 40, at least one of text and a schematic diagram representing the current position and attitude of the vehicle 1, at least one of text and a schematic diagram representing the teacher route R1, and at least one of text and a schematic diagram representing the obstacle B. The captured image 40 included in the screen 30 may be either a captured image captured by the camera 16 in real time or a captured image captured by the camera 16 in the past.

[0108] Figure 8A shows an example of a third screen 30A1 in which an icon representing vehicle 1 is placed at a position corresponding to the current position of vehicle 1 in the captured image 40 so that its posture corresponds to the current posture of vehicle 1, and an arrow image representing the teacher route R1 and an icon representing obstacle B are superimposed on the captured image 40.

[0109] Furthermore, when an obstacle B is detected ahead in the travel direction D, the control unit 11 may further cause a speaker provided in the vehicle 1 to output a sound that prompts the passenger to perform a manual operation to avoid the obstacle.

[0110] 8B is a schematic diagram of an example of the third screen 30A2. The third screen 30A2 is an example of the third screen 30A.

[0111] 8A, the third screen 30A2 further includes an instruction button 32 for receiving an instruction to start manual driving from the rider. In this way, the third screen 30A may further include the instruction button 32.

[0112] Returning to FIG. 7, the explanation continues.

[0113] By the above process, when the control unit 11 detects an obstacle B ahead in the moving direction D while moving by the first autonomous driving, the third screen 30A is displayed on the display device 22. In detail, if the obstacle B is detected when the vehicle 1 reaches a point S1 on the teacher route R1, the vehicle 1 stops moving at the point S1, the autonomous driving mode is temporarily switched to the manual driving mode, and the third screen 30A is output to the display device 22.

[0114] Then, when the control unit 11 receives a manual operation of the operation device 20 by the passenger, it controls the movement control device 12 to travel in accordance with the manual operation by the passenger. When the command button 32 shown in Fig. 8B is operated by the passenger and a manual operation start signal due to the operation is received from the operation device 20, the control unit 11 may control the movement control device 12 to travel in accordance with the manual operation by the passenger.

[0115] When the occupant starts manual operation from point S1 where obstacle B is detected, the occupant operates the steering device 27 (steering wheel, handle) etc. to avoid obstacle B, and the vehicle 1 moves by manual operation of the occupant (see route R2 in Figure 7).

[0116] While accepting this manual operation, the control unit 11 causes the display device 22 to output a fourth screen that prompts the user to perform an operation indicating the completion of avoidance when the avoidance of the obstacle B is completed. For example, the fourth screen includes at least one of text and a schematic diagram that prompts the user to perform an operation indicating the completion of avoidance when the avoidance of the obstacle B is completed.

[0117] 9 is a schematic diagram of an example of the fourth screen 30B. The fourth screen 30B is an example of the screen 30 output by the display device 22.

[0118] The fourth screen 30B is a screen 30 including, for example, a message M2 and a cancel button 31. FIG. 9 shows an example in which the message M2 includes the phrase "Please apply the brakes when avoidance driving is complete." This phrase is an example of a message that urges the driver to perform an operation that indicates the completion of avoidance when avoidance of obstacle B is complete. In this case, the operation that indicates the completion of avoidance is "apply the brakes."

[0119] The fourth screen 30B may be a screen in which at least a message M2 and a cancel button 31 are superimposed on the captured image 40. The fourth screen 30B may further include at least one of the captured image 40, at least one of a wording and a schematic diagram representing the current position and attitude of the vehicle 1, at least one of a wording and a schematic diagram representing the teacher route R1, and at least one of a wording and a schematic diagram representing an obstacle B.

[0120] Figure 9 shows an example of the fourth screen 30B in which an icon representing vehicle 1 is placed at a position corresponding to the current position of vehicle 1 in the captured image 40 so that it assumes a posture corresponding to the current posture of vehicle 1, and an arrow image representing the teacher route R1 and an icon representing obstacle B are superimposed on the captured image 40.

[0121] Furthermore, when avoidance of the obstacle B is completed, the control unit 11 may further output, from a speaker provided in the vehicle 1, a sound that prompts the driver to perform an operation that indicates the completion of avoidance.

[0122] Returning to FIG. 7, the explanation will be continued. In the temporary manual driving mode, the control unit 11 causes the display device 22 to output the fourth screen 30B while the operation device 20 accepts manual operation by the passenger. Therefore, the display device 22 is switched to the temporary manual driving mode at the point S1 where the obstacle B is detected, and outputs the fourth screen 30B to the display device 22 while the display device 22 accepts manual operation after the passenger starts manual operation. That is, in the temporary manual driving mode, the display device 22 outputs the fourth screen 30B while the operation device 20 is operated by the passenger to avoid the obstacle B and the vehicle 1 moves along the route R2 by the passenger's manual operation. The route R2 is an example of a movement trajectory of the vehicle 1 moved by the passenger's manual operation.

[0123] When an obstacle B is detected ahead in the direction of movement D of the vehicle 1, the display device 22 outputs the fourth screen 30B while accepting manual operation by the occupant, thereby making it possible to prompt the occupant to perform an operation indicating that avoidance has been completed when avoidance of the obstacle B has been completed by manual operation.

[0124] Thereafter, in the temporary manual driving mode, if the control unit 11 does not detect an obstacle B ahead of the vehicle 1 in the moving direction D based on the external situation acquired by the sensor 14 after the vehicle 1 has been moved by manual operation by the occupant, the control unit 11 causes the display device 22 to output the first screen. The first screen is a screen 30 showing a return to the teacher route R1 by autonomous driving.

[0125] In detail, when the vehicle 1 starts moving due to manual operation by the passenger, the control unit 11 determines whether or not an obstacle B has been detected ahead in the direction of movement D of the vehicle 1, based on the external situation acquired by the sensor 14. The control unit 11 can determine whether or not an obstacle B exists ahead in the direction of movement D of the vehicle 1 in the same manner as described above, using the sensor information received from the sensor 14. Then, when the control unit 11 determines that an obstacle B has not been detected ahead in the direction of movement D of the vehicle 1, it causes the display device 22 to output the first screen.

[0126] For example, assume that the occupant starts manual operation from point S1 where obstacle B is detected, and then operates operation device 20 to avoid obstacle B, causing vehicle 1 to move by manual operation by the occupant (see route R2 in FIG. 7). Then, assume that when vehicle 1 reaches point S2, control unit 11 determines that obstacle B is not detected ahead of vehicle 1 in direction D of movement. In this case, control unit 11 causes display device 22 to output the first screen.

[0127] The control unit 11 may output the first screen to the display device 22 when it determines that no obstacle B is detected ahead in the direction of movement D of the vehicle 1 and when it receives a predetermined operation instruction from the occupant, such as stepping on the brakes.

[0128] 10A and 10B are schematic diagrams showing an example of a first screen 30C. The first screen 30C is an example of the screen 30 output by the display device 22.

[0129] 10A is a schematic diagram of an example of first screen 30C1. First screen 30C1 is an example of first screen 30C.

[0130] The first screen 30C1 is, for example, a screen 30 including a message M3 and a cancel button 31. FIG. 10A shows an example in which the message M3 includes the phrase "Release the brakes. Returning to autonomous driving." This phrase is an example of a phrase indicating a return to the teacher route R1 by autonomous driving.

[0131] The first screen 30C1 may be a screen 30 including at least one of a message indicating return to the teacher route R1 by autonomous driving and a schematic diagram indicating return to the teacher route R1 by autonomous driving.

[0132] In addition, the first screen 30C1 may further include at least one of the captured image 40, at least one of text and a schematic diagram representing the current position and attitude of the vehicle 1, at least one of text and a schematic diagram representing the teacher route R1, and at least one of text and a schematic diagram representing an obstacle B.

[0133] Figure 10A shows an example of a first screen 30C1 in which an icon representing vehicle 1 is placed at a position corresponding to the current position of vehicle 1 in the captured image 40 so that its posture corresponds to the current posture of vehicle 1, and an arrow image representing the teacher route R1 and an icon representing obstacle B are superimposed on the captured image 40.

[0134] 10B is a schematic diagram of an example of first screen 30C2. First screen 30C2 is an example of first screen 30C.

[0135] 10A, first screen 30C2 further includes command button 33 for receiving a command to start autonomous traveling from the passenger. In this way, first screen 30C may further include command button 33.

[0136] In addition, if, after the vehicle 1 has been moved by manual operation, the control unit 11 does not detect an obstacle B ahead of the vehicle 1 in the direction of movement D based on the external conditions acquired by the sensor 14, the control unit 11 may further output a sound from a speaker provided in the vehicle 1 indicating that the vehicle 1 is returning to the teacher route R1 by autonomous driving.

[0137] Returning to FIG. 7, the explanation continues.

[0138] By the above processing, the mode is temporarily switched to manual driving mode at point S1 where obstacle B is detected, and manual operation by the occupant is started, so that vehicle 1 moves by manual operation of operation device 20 and steering device 27 by the occupant (see route R2 in FIG. 7), and when vehicle 1 reaches point S2, for example, if it is determined that obstacle B is not detected ahead in the direction of movement D, display device 22 outputs first screen 30C. That is, in the example shown in FIG. 7, for example, when vehicle 1, which has been moved by manual operation, reaches point S2, display device 22 outputs first screen 30C.

[0139] By display device 22 outputting first screen 30C, it becomes possible to provide the occupant with a recognizable indication that when obstacle B is avoided by manual operation by the occupant, vehicle 1 will return to the teacher route R1 by autonomous driving.

[0140] Then, the control unit 11 controls the mobile control device 12 to perform a second autonomous driving to an intermediate point S3 between a predetermined position P1 on the teacher route R1 and the parking target position P2, and then to perform a third autonomous driving from the intermediate point S3 to the parking target position P2.

[0141] The intermediate point S3 may be any point on the teacher path R1 between the predetermined position P1 and the parking target position P2. In detail, the intermediate point S3 may be any point on the teacher path R1 between the avoided obstacle B and the parking target position P2.

[0142] The second autonomous driving refers to autonomous driving from point S2 to midpoint S3, where obstacle B is avoided by manually moving vehicle 1 through the passenger's manual operation and obstacle B is no longer detected ahead in the direction of travel D. The third autonomous driving refers to autonomous driving from midpoint S3 to parking target position P2.

[0143] The control unit 11 may control the mobile control device 12 to perform the second autonomous driving based on the difference between the teacher route R1 and the current position of the vehicle 1. That is, the control unit 11 performs feedback control so that the vehicle 1 returns to the teacher route R1 based on the estimated current position of the vehicle 1 and each position on the teacher route R1 represented by the teacher route data 18A. Specifically, the control unit 11 controls the mobile control device 12 so that the difference between the estimated current position of the vehicle 1 and the position on the teacher route R1 represented by the teacher route data 18A that is closest to the current position gradually decreases. The control unit 11 may also adjust the gain of the feedback control to reduce abrupt steering. For example, the control unit 11 may perform feedback control to prevent abrupt steering by adjusting the correction amount of the difference or the correction amount of the angle deviation so that it is equal to or less than a predetermined value for a section of a predetermined distance after starting traveling from point S2 toward the intermediate point S3.

[0144] Through this control, the control unit 11 controls the movement control device 12 so that the vehicle 1 gradually approaches the teacher path R1 by tracing a trajectory such as the return path R3, and reaches the waypoint S3 by reaching the teacher path R1.

[0145] In addition, the control unit 11 may generate a return route R3 from the point S2 to the intermediate point S3 before starting the second autonomous driving to the intermediate point S3, and control the mobile control device 12 to perform the second autonomous driving based on the return route R3.

[0146] 11 is an explanatory diagram of an example of generating a return route R3. For example, the control unit 11 generates a virtual circle C with a radius obtained by adding a predetermined value to the minimum turning radius of the vehicle 1, and places the center of the virtual circle C at a position on the teacher route R1 that is closest to the current position of the vehicle 1. The control unit 11 then identifies, as an intermediate point S3, a position on the teacher route R1 that is outside the virtual circle C on the teacher route R1 and within a section E where the route curvature and rate of change are stable. The control unit 11 then generates, as the return route R3, a curve that connects the current position of the vehicle 1 (for example, point S2) to the intermediate point S3 within a predetermined curvature range.

[0147] The control unit 11 may specify the intermediate point S3 by the above processing, calculate a route connecting the point S2 to the intermediate point S3 by a known method, and generate the calculated route as the return route R3. The control unit 11 may also specify the intermediate point S3 by a known method, calculate a route connecting the point S2 to the intermediate point S3 by a known method, and generate the calculated route as the return route R3.

[0148] Then, when the control unit 11 generates the return route R3, it may control the mobile control device 12 to perform the second autonomous traveling along the return route R3 from the point S2 toward the midpoint S3.

[0149] Returning to FIG. 7, the explanation continues.

[0150] The control unit 11 causes the display device 22 to output a second screen during at least a part of the time while the vehicle 1 is performing the second autonomous driving to the waypoint S3. The second screen is a screen 30 indicating that the vehicle is currently performing autonomous driving to return to the teacher route R1. For example, the second screen includes at least one of a statement indicating that the vehicle is currently performing autonomous driving to return to the teacher route R1 and a schematic diagram indicating that the vehicle is currently performing autonomous driving to return to the teacher route R1.

[0151] 12A and 12B are schematic diagrams showing an example of a second screen 30D. The second screen 30D is an example of the screen 30 output by the display device 22.

[0152] 12A is a schematic diagram of an example of a second screen 30D1. The second screen 30D1 is an example of the second screen 30D.

[0153] The second screen 30D1 is, for example, a screen 30 including a message M4, a schematic diagram D2, and a cancel button 31. FIG. 12A shows an example in which the message M4 includes the phrase "Currently returning to the registered route." This phrase is an example of phrase indicating that the vehicle is autonomously traveling to return to the teacher route R1. FIG. 12A also shows an example in which the schematic diagram D2 is an arrow image indicating that the vehicle is autonomously traveling to return to the teacher route R1. Note that the message M4 is not limited to the phrase shown in FIG. 12A as long as it includes phrase indicating that the vehicle is autonomously traveling to return to the teacher route R1. Similarly, the schematic diagram D2 is not limited to the form shown in FIG. 12A as long as it includes phrase indicating that the vehicle is autonomously traveling to return to the teacher route R1.

[0154] The second screen 30D1 may further include at least one of the captured image 40, at least one of text and a schematic diagram representing the current position and posture of the vehicle 1, at least one of text and a schematic diagram representing the teacher route R1, and at least one of text and a schematic diagram representing the obstacle B. The schematic diagram representing the current position of the vehicle 1 is an example of a second image representing the position of the vehicle 1 at that time. The schematic diagram representing the teacher route R1 is an example of a first image representing the teacher route R1.

[0155] Figure 12A shows an example of a second screen 30D1 in which an icon representing vehicle 1 is placed at a position corresponding to the current position of vehicle 1 in the captured image 40 so that its posture corresponds to the current posture of vehicle 1, and an arrow image representing the teacher route R1 and an icon representing obstacle B are further superimposed on the captured image 40.

[0156] Furthermore, when the control unit 11 generates the return route R3, the control unit 11 may cause the display device 22 to output the second screen 30D that further includes a third image representing the generated return route R3.

[0157] 12B is a schematic diagram of an example of the second screen 30D2. The second screen 30D2 is an example of the second screen 30D.

[0158] As shown in Fig. 12B, when the control unit 11 generates the return path R3, for example, a line image representing the return path R3 is generated as a third image, and the control unit 11 causes the display device 22 to output a second screen 30D2 in which the third image is further superimposed on the second screen 30D1 shown in Fig. 12A. In this case, the display device 22 outputs the second screen 30D2 shown in Fig. 12B.

[0159] In addition, the control unit 11 may further output, from a speaker provided in the vehicle 1, a sound indicating that the vehicle 1 is in autonomous driving to return to the teacher route R1, for at least a portion of the time that the vehicle 1 is in the second autonomous driving to the waypoint S3.

[0160] Returning to FIG. 7, the explanation continues.

[0161] The control unit 11 controls the movement control device 12 to cause the vehicle 1 to perform a second autonomous driving from point S2 to an intermediate point S3, and then controls the movement control device 12 to perform a third autonomous driving along the teacher route R1 from the intermediate point S3 to the parking target position P2.

[0162] During at least a part of the third autonomous driving, the control unit 11 may cause the display device 22 to output a fifth screen. The fifth screen is a screen 30 indicating that the autonomous driving will continue after returning to the teacher route R1. In detail, the fifth screen is a screen 30 including at least one of a message and a schematic diagram indicating that the autonomous driving will continue after returning to the teacher route R1.

[0163] 13 is a schematic diagram of an example of the fifth screen 30E. The fifth screen 30E is an example of the screen 30.

[0164] The fifth screen 30E is, for example, a screen 30 including a message M5 and a cancel button 31. FIG. 13 shows an example in which the message M5 includes the phrase "Returned to registered route. Continue autonomous driving." This phrase is an example of a phrase indicating that the robot will return to the teacher route R1 and continue autonomous driving. The fifth screen 30E may further include a schematic diagram indicating that the robot will return to the teacher route R1 and continue autonomous driving, instead of or together with this phrase.

[0165] In addition, the fifth screen 30E may further include at least one of the captured image 40, at least one of text and a schematic diagram representing the current position and posture of the vehicle 1, at least one of text and a schematic diagram representing the teacher route R1, and at least one of text and a schematic diagram representing an obstacle B.

[0166] When the vehicle 1 avoids the obstacle B through manual operation by the occupant and then returns to the teacher route R1, the display device 22 outputs the fifth screen 30E, thereby providing the occupant with information indicating that the vehicle has returned to the teacher route R1 and is continuing autonomous driving.

[0167] Returning to FIG. 7, the explanation continues.

[0168] On the other hand, in the manual driving mode, when vehicle 1 is driven in response to a steering operation of steering device 27 by the occupant and vehicle 1 is within a predetermined range from predetermined position P1, operation device 20 may not receive the first operation representing an instruction to start the autonomous driving mode. In this case, control unit 11 continues to drive vehicle 1 to parking target position P2 in the manual driving mode in response to a steering operation of steering device 27 by the occupant.

[0169] As described above, the screens 30 (third screen 30A to fifth screen 30E, see FIGS. 8A to 13) displayed in the temporary manual driving mode include the cancel button 31. As described above, the cancel button 31 is an image area for receiving an instruction to switch to the manual driving mode from the passenger. The operation of the cancel button 31 by the passenger is an example of a second operation received by the operation device 20.

[0170] In the temporary manual driving mode, when the operation device 20 receives a second operation, the control unit 11 switches the driving mode to the manual driving mode. Then, in the manual driving mode, if the control unit 11 does not detect an obstacle B ahead in the direction of movement of the vehicle 1 based on the external situation acquired by the sensor 14 after the vehicle 1 has moved by manual operation, the control unit 11 does not display the first screen 30C. As described above, the first screen 30C is the screen 30 representing the return to the teacher route R1 by autonomous driving. Therefore, in the temporary manual driving mode, the first screen 30C may be displayed, but in the manual driving mode, the first screen 30C is not displayed.

[0171] Next, an example of the flow of information processing executed by the control unit 11 of the vehicle control device 10 will be described.

[0172] FIG. 14 is a flowchart showing an example of the flow of information processing executed by the control unit 11 in the teacher running mode.

[0173] The control unit 11 determines whether or not a signal representing an instruction to start the teacher running mode has been received from the operation device 20 (step S100). If the determination in step S100 is negative (step S100: No), the control unit 11 ends this routine. If the determination in step S100 is positive (step S100: Yes), the control unit 11 proceeds to step S102.

[0174] When the vehicle 1 starts teacher driving by manually operating the vehicle 1 by the passenger, the control unit 11 sequentially stores the map data 18B and the current position of the vehicle 1 in the storage device 18 (step S102). Specifically, the control unit 11 sequentially stores the current position of the vehicle 1 estimated along the teacher driving of the vehicle 1. At this time, the control unit 11 assigns an INDEX to the current position, and sequentially stores the current position, which is the driving position, orientation, driving direction, and reference driving information in association with each other. The control unit 11 also identifies feature points by performing image analysis on the video captured by the camera 16 during teacher driving, and sequentially registers the feature points in the map data 18B.

[0175] The control unit 11 determines whether or not an instruction to end the teacher driving mode has been received (step S104). The control unit 11 makes the determination in step S104 by determining whether or not a signal representing the end instruction has been received from the operation device 20 by the passenger operating the operation device 20. If the control unit 11 makes a negative determination in step S104 (step S104: No), the control unit 11 returns to step S102. If the control unit 11 makes a positive determination in step S104 (step S104: Yes), the control unit 11 proceeds to step S106.

[0176] In step S106, the control unit 11 stores the teacher route data 18A and the map data 18B in the storage device 18 (step S106). The control unit 11 defines the travel route R during teacher travel represented by the group of sequentially estimated current positions stored by the processing of step S102 as a teacher route R1, and stores the teacher route data 18A representing the teacher route R1 in the storage device 18. The control unit 11 also stores the map data 18B in which the feature points identified by the processing of step S102 are registered in the storage device 18. Then, this routine ends.

[0177] FIG. 15 is a flowchart showing an example of the flow of information processing executed by the control unit 11 in the autonomous driving mode.

[0178] In the manual driving mode, the control unit 11 drives the vehicle 1 in response to a steering operation of the steering device 27 by the passenger (step S200).

[0179] The control unit 11 determines whether the position of the vehicle 1 is within a predetermined range from the predetermined position P1 (step S202). If the determination in step S202 is affirmative (step S202: Yes), the process proceeds to step S204.

[0180] The control unit 11 determines whether or not the operation device 20 has received a first operation (step S204). The control unit 11 executes the determination of step S204 by determining whether or not a signal representing an instruction to start the autonomous driving mode has been received from the operation device 20. If the determination in step S204 is affirmative (step S204: Yes), the process proceeds to step S206.

[0181] In step S206, the control unit 11 switches the driving mode from the manual driving mode to the autonomous driving mode (step S206). Then, the control unit 11 reads the teacher route data 18A and the map data 18B from the storage device 18 (step S208). Then, the control unit 11 controls the mobile control device 12 to start the first autonomous driving from a predetermined position P1 toward a parking target position P2 along the teacher route R1 represented by the teacher route data 18A (step S210).

[0182] When the first autonomous traveling starts, the control unit 11 determines whether or not an obstacle B has been detected ahead of the vehicle 1 in the traveling direction D, based on the external situation acquired by the sensor 14 (step S212). If the determination in step S212 is negative (step S212: No), the process proceeds to step S240, which will be described later. If the determination in step S212 is positive (step S212: Yes), the process proceeds to step S214.

[0183] The control unit 11 controls the movement control device 12 to stop the vehicle 1 from traveling (step S214) and switches the autonomous traveling mode to a temporary manual traveling mode (step S216). By the control unit 11 switching the autonomous traveling mode to the temporary manual traveling mode, the vehicle becomes capable of traveling according to manual operation by the passenger.

[0184] Next, the control unit 11 causes the display device 22 to output the third screen 30A (step S218). By the process of step S218, the display device 22 outputs the third screen 30A shown in Fig. 8A or 8B, for example.

[0185] The control unit 11 determines whether or not a manual operation by the passenger has been started (step S220). The control unit 11 makes the determination in step S220 by, for example, determining whether or not a manual operation of the operating device 20 by the passenger has been accepted. If the control unit 11 makes a negative determination in step S220 (step S220: No), the process returns to step S218. If the control unit 11 determines that a manual operation by the passenger has been started (step S220: Yes), the process proceeds to step S222.

[0186] In step S222, the control unit 11 causes the display device 22 to output the fourth screen 30B (step S222). By the process of step S222, the display device 22 outputs, for example, the fourth screen 30B shown in FIG.

[0187] Then, the control unit 11 determines whether or not an obstacle B has been detected ahead of the vehicle 1 in the moving direction D based on the external situation acquired by the sensor 14 (step S224). If the control unit 11 determines that an obstacle B has been detected (step S224: No), the control unit 11 returns to step S222. If the control unit 11 determines that an obstacle B has not been detected (step S224: Yes), the control unit 11 proceeds to step S226.

[0188] In step S226, the control unit 11 causes the display device 22 to output the first screen 30C (step S226). When the control unit 11 makes a positive determination in step S224 (step S224: Yes) and receives a predetermined operation instruction from the passenger, such as stepping on the brake, the control unit 11 may cause the display device 22 to output the first screen 30C. By the processing of step S226, the display device 22 outputs, for example, the first screen 30C shown in FIG. 10A or 10B.

[0189] Next, the control unit 11 determines whether or not an operation indicating completion of avoidance has been received from the occupant (step S228). The control unit 11 makes the determination in step S228 by determining whether or not a predetermined operation, such as the occupant releasing their foot from the brake, has been performed. If the control unit 11 determines negative in step S228 (step S282: No), the control unit 11 returns to step S226. If the control unit 11 determines positive in step S228 (step S228: Yes), the control unit 11 proceeds to step S230.

[0190] In step S230, the control unit 11 controls the mobile control device 12 to start the second autonomous traveling to a midpoint S3 between the predetermined position P1 and the parking target position P2 on the teacher route R1 (step S230).

[0191] When the second autonomous traveling is started, the control unit 11 causes the display device 22 to output the second screen 30D (step S232). By the processing of step S232, the display device 22 outputs, for example, the second screen 30D shown in FIG. 12A or 12B.

[0192] Then, the control unit 11 determines whether the vehicle 1 has reached a point on the teacher route R1, i.e., the waypoint S3 (step S234). For example, the control unit 11 makes the determination in step S234 by determining whether the current position of the vehicle 1 is at any point on the teacher route R1. If the control unit 11 determines negative in step S234 (step S234: No), the control unit 11 returns to step S232. If the control unit 11 determines positive in step S234 (step S234: Yes), the control unit 11 proceeds to step S236.

[0193] In step S236, the control unit 11 controls the mobile control device 12 to start the third autonomous traveling along the teacher route R1 from the midpoint S3 to the parking target position P2 (step S236).

[0194] During at least a part of the third autonomous driving, the control unit 11 causes the display device 22 to output the fifth screen 30E (step S238). By the processing in step S234, for example, the display device 22 outputs the fifth screen 30E shown in FIG.

[0195] Then, the control unit 11 determines whether the vehicle 1 has reached the parking target position P2 (step S240). If the determination in step S240 is negative (step S240: No), the process proceeds to step S212. If the determination in step S240 is positive (step S240: Yes), the present routine is terminated.

[0196] On the other hand, if a negative decision is made in step S202 (step S202: No), the process proceeds to step S242. Also, if a negative decision is made in step S204 (step S204: No), the process proceeds to step S242.

[0197] In step S242, manual traveling in which the vehicle 1 travels in response to the steering operation of the steering device 27 by the passenger is continued (step S242).

[0198] Next, the control unit 11 determines whether or not an obstacle B has been detected ahead of the vehicle 1 in the moving direction D, based on the external situation acquired by the sensor 14 (step S244). If the determination in step S244 is affirmative (step S244: Yes), the process proceeds to step S248. If the control unit 11 does not detect obstacle B (step S244: No), the control unit 11 does not output the first screen 30C, which shows the return to the teacher route R1 by autonomous driving, to the display device 22 (step S246). Then, the process proceeds to step S248.

[0199] In step S248, the control unit 11 determines whether the vehicle 1 has reached the parking target position P2 (step S248). If the determination in step S248 is negative (step S248: No), the process proceeds to step S202. If the determination in step S248 is positive (step S248: Yes), the present routine is terminated.

[0200] Next, an example of the flow of interrupt processing that the control unit 11 of this embodiment executes between steps S216 to S240 in the information processing shown in FIG. 15 will be described.

[0201] 16 illustrates an example of the flow of interrupt processing executed by the control unit 11 of this embodiment. The control unit 11 repeatedly executes the interrupt processing shown in FIG. 16 during the processing of steps S216 to S240 in the information processing shown in FIG.

[0202] The control unit 11 determines whether the operation device 20 has accepted the second operation (step S300). The control unit 11 makes the determination in step S300 by determining whether the cancel button 31 included in the screen 30 has been operated by the passenger. The control unit 11 repeats a negative determination (step S300: No) until a positive determination is made in step S300 (step S300: Yes). When the control unit 11 makes a positive determination in step S300 (step S300: Yes), it switches the driving mode of the vehicle 1 to the manual driving mode (step S302). The process returns to step S242 in FIG. 15 above.

[0203] Therefore, when the operation device 20 receives the second operation in the temporary manual driving mode, the driving mode is switched to the manual driving mode.

[0204] The cancel button 31 may be included on the screen 30 that is displayed when the driving mode is the autonomous driving mode or the temporary manual driving mode. Therefore, when in the autonomous driving mode or the temporary manual driving mode, the occupant can operate the cancel button 31 at any time to switch the driving mode of the vehicle 1 from the autonomous driving mode or the temporary manual driving mode to the manual driving mode.

[0205] As described above, the vehicle control device 10 of this embodiment includes a sensor 14 that acquires external conditions, an operation device 20 that accepts operations by the occupant, a steering device 27 that accepts steering operations by the occupant, a display device 22 that is visible to the occupant, and a movement control device 12 that controls at least steering, and controls a vehicle 1 that holds a teacher route R1 obtained by teacher driving from a predetermined position P1 to a parking target position P2.

[0206] In the manual driving mode, the vehicle control device 10 drives the vehicle 1 in response to a steering operation of the steering device 27 by the occupant, and when the vehicle 1 is within a predetermined range from a predetermined position P1 and the operation device 20 receives a first operation, the vehicle control device 10 switches to the autonomous driving mode. Then, in the autonomous driving mode, the vehicle control device 10 controls at least the steering along the teacher path R1. If the vehicle 1 detects an obstacle B ahead in the direction of travel D of the vehicle 1 based on external conditions acquired by the sensor 14 during first autonomous driving from the predetermined position P1 toward the parking target position P2, the vehicle control device 10 switches to a temporary manual driving mode different from the manual driving mode, and the operation device 20 and the steering device 27 receive manual operation by the occupant. Then, in the temporary manual driving mode, if the vehicle 1 does not detect an obstacle B ahead in the direction of travel D of the vehicle 1 based on external conditions acquired by the sensor 14 after the vehicle 1 has moved by manual operation, the display device 22 outputs a first screen 30C showing a return to the teacher path R1 by autonomous driving. The vehicle control device 10 causes the vehicle to perform a second autonomous driving to a midpoint S3 between a predetermined position P1 on the teacher path R1 and the parking target position P2, and then causes the vehicle to perform a third autonomous driving along the teacher path R1 from the midpoint S3 to the parking target position P2.

[0207] In the manual driving mode, the vehicle control device 10 drives the vehicle 1 in response to the steering operation of the steering device 27 by the occupant, and when the vehicle 1 is within a predetermined range from a predetermined position P1 and the operation device 20 does not receive a first operation, the vehicle control device 10 continues to drive the vehicle 1 in the manual driving mode to a parking target position P2 in response to the steering operation of the steering device 27 by the occupant.

[0208] When the operation device 20 receives a second operation in the temporary manual driving mode, the vehicle control device 10 switches to the manual driving mode, and when, in the manual driving mode, the vehicle 1 moves by manual operation, if an obstacle B is not detected ahead of the vehicle 1 in the direction of movement D based on the external conditions acquired by the sensor 14, the display device 22 does not output the first screen 30C indicating a return to the teacher route 41 by autonomous driving.

[0209] As described above, in the vehicle control device 10 of this embodiment, if an obstacle B is detected ahead in the direction of movement D of the vehicle 1 while moving by first autonomous driving along the teacher route R1 from a predetermined position P1 to the parking target position P2, the vehicle control device 10 switches to a temporary manual driving mode different from the manual driving mode, and the operation device 20 accepts manual operation by the occupant. Then, in the vehicle control device 10, if an obstacle B is not detected ahead in the direction of movement D of the vehicle 1 after the vehicle 1 has moved by manual operation, the display device 22 outputs a first screen 30C indicating a return to the teacher route R1 by autonomous driving. Then, the vehicle control device 10 causes the vehicle to perform a second autonomous driving to a midpoint S3 on the teacher route R1 between the predetermined position P1 and the parking target position P2, and then causes the vehicle to perform a third autonomous driving along the teacher route R1 from the midpoint S3 to the parking target position P2.

[0210] Therefore, in the autonomous driving mode, when the vehicle control device 10 of this embodiment is autonomously driving along the teacher route R1 and an obstacle B that was not present during the teacher driving is present on the teacher route R1, it switches to a temporary manual driving mode different from the manual driving mode, prompts the occupant to perform manual operation to avoid the obstacle B through that manual operation, and causes the vehicle 1 to autonomously drive from the point S2 where the obstacle B has been avoided towards the parking target position P2 via the intermediate point S3 on the teacher route R1.

[0211] Therefore, the vehicle control device 10 of this embodiment can provide more suitable parking assistance.

[0212] Furthermore, according to the vehicle control device 10 of this embodiment, in the temporary manual driving mode, the occupant performs a manual operation to avoid obstacle B, and after avoiding obstacle B, the vehicle 1 autonomously drives from the point of avoidance (point S2) toward the parking target position P2 via the intermediate point S3. Therefore, the vehicle control device 10 of this embodiment does not require the occupant to perform an operation to resume driving when the vehicle 1 returns to the teacher route R1. Therefore, in addition to the above effects, the vehicle control device 10 of this embodiment can seamlessly autonomously drive the vehicle 1 from point S2 where obstacle B has been avoided toward the parking target position P2.

[0213] Next, the hardware configuration of the vehicle control device 10 of this embodiment will be described.

[0214] FIG. 17 is a block diagram showing an example of the hardware configuration of the vehicle control device 10. As shown in FIG.

[0215] The vehicle control device 10 has a hardware configuration that utilizes a normal computer, with a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, an I / F 11D for connecting to various devices, and the like interconnected by a bus 11E.

[0216] The CPU 11A is a calculation device that controls the overall processing of the vehicle control device 10. The RAM 11C stores data necessary for various processes by the CPU 11A. The ROM 11B stores programs and the like that realize various processes by the CPU 11A. The I / F 11D is an interface that is connected to an external device or external terminal via a communication line or the like and transmits and receives data to and from the connected external device or external terminal.

[0217] The programs for executing the various processes described above executed by the vehicle control device 10 are provided by being pre-installed in the ROM 11B, etc. The programs for executing the vehicle control method executed in this embodiment may be configured to be provided by being recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disc (DVD) in a format that can be installed or executed by these devices.

[0218] The program for executing the vehicle control method according to the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program for executing the vehicle control method according to the present embodiment may be provided or distributed via a network such as the Internet.

[0219] Although an embodiment of the present disclosure has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment is included in the scope and spirit of the invention, as well as in the invention described in the claims and their equivalents. [Explanation of symbols]

[0220] 1 vehicle 10 Vehicle control device 11 Control section 12 Movement control device 14 Sensors 20 Operating device 22 Display device

Claims

1. A vehicle control method is carried out by a vehicle that is equipped with a sensor that acquires an external situation, an operation device that receives an operation by a passenger, a steering device that receives a steering operation by the passenger, a display device that is visible to the passenger, and a movement control unit that controls at least steering, and that holds a teacher route obtained by teacher driving from a predetermined position to a parking target position, In a manual driving mode, the vehicle is driven in response to a steering operation of the steering device by the passenger, and when the vehicle is within a predetermined range from the predetermined position and the operation device receives a first operation, the mode is switched to an autonomous driving mode; thereafter, in the autonomous driving mode, when an obstacle is detected ahead in the direction of movement of the vehicle based on the external situation acquired by the sensor during movement by first autonomous driving from the predetermined position toward the parking target position by controlling at least the steering along the teacher path, the mode is switched to a temporary manual driving mode different from the manual driving mode, and the operation device and the steering device accept manual operation by the occupant; thereafter, in the temporary manual driving mode, after the vehicle has been moved by the manual operation, if the obstacle is not detected ahead in the direction of movement of the vehicle based on the external situation acquired by the sensor, the display device outputs a first screen indicating return to the teacher route by autonomous driving; performing a second autonomous driving to a midpoint on the teacher path between the predetermined position and the parking target position, and then performing a third autonomous driving along the teacher path from the midpoint to the parking target position; In the manual driving mode, the vehicle is driven in response to a steering operation of the steering device by the occupant, and when the vehicle is within a predetermined range from the predetermined position and the operation device does not receive a first operation, the vehicle is driven to the parking target position in response to a steering operation of the steering device by the occupant, continuing in the manual driving mode; When the operation device receives a second operation in the temporary manual driving mode, the operation device switches to the manual driving mode, and when, in the manual driving mode, after the vehicle has moved by the manual operation, the obstacle is not detected ahead in the moving direction of the vehicle based on the external situation acquired by the sensor, the display device does not output the first screen indicating return to the teacher route by autonomous driving. Vehicle control method.

2. 2. The vehicle control method according to claim 1, the first screen includes at least one of a statement indicating return to the teacher route by autonomous driving and a schematic diagram indicating return to the teacher route by autonomous driving; Vehicle control method.

3. 2. The vehicle control method according to claim 1, The second autonomous driving is performed based on a difference between the teacher route and the current position of the vehicle. Vehicle control method.

4. 2. The vehicle control method according to claim 1, and during at least a part of the time while the vehicle is traveling in the second autonomous traveling mode to the waypoint, the display device outputs a second screen indicating that the vehicle is traveling in the second autonomous traveling mode to return to the teacher route. Vehicle control method.

5. 5. The vehicle control method according to claim 4, the second screen includes at least one of a statement indicating that the autonomous traveling vehicle is currently traveling to return to the teacher route and a schematic diagram indicating that the autonomous traveling vehicle is currently traveling to return to the teacher route; Vehicle control method.

6. 5. The vehicle control method according to claim 4, the second screen includes a first image representing the teacher route and a second image representing the current position of the vehicle; Vehicle control method.

7. 7. A vehicle control method according to claim 6, In the temporary manual driving mode, if the obstacle is not detected ahead in the direction of movement of the vehicle based on the external situation acquired by the sensor after the vehicle has been moved by the manual operation, a return route is generated, and the second autonomous driving is performed based on the return route; the second screen further includes a third image representing the return path; Vehicle control method.

8. 2. The vehicle control method according to claim 1, When an obstacle is detected ahead in the moving direction of the vehicle based on the external situation acquired by the sensor, the display device displays a third screen prompting the occupant to perform a manual operation to avoid the obstacle. Vehicle control method.

9. 9. A vehicle control method according to claim 8, The third screen displays a message prompting the passenger to perform a manual operation to avoid an obstacle, and a schematic diagram prompting the occupant to perform a manual operation to avoid an obstacle; Vehicle control method.

10. 2. The vehicle control method according to claim 1, In the autonomous driving mode, when an obstacle is detected ahead in the direction of movement of the vehicle based on the external situation acquired by the sensor during first autonomous driving from the predetermined position toward the parking target position by controlling at least the steering along the teacher path, the mode is switched to a temporary manual driving mode different from the manual driving mode, and while the operation device accepts the manual operation by the occupant, the display device outputs a fourth screen prompting the occupant to perform an operation indicating completion of avoidance when avoidance of the obstacle is completed. Vehicle control method.

11. A vehicle control device comprising: a sensor for acquiring an external situation; an operation device for receiving an operation by a passenger; a steering device for receiving a steering operation by the passenger; a display device visible to the passenger; and a movement control unit for controlling at least steering; and the vehicle control device controls a vehicle that holds a teacher route obtained by teacher driving from a predetermined position to a parking target position, In a manual driving mode, the vehicle is driven in response to a steering operation of the steering device by the passenger, and when the vehicle is within a predetermined range from the predetermined position and the operation device receives a first operation, the mode is switched to an autonomous driving mode; thereafter, in the autonomous driving mode, when an obstacle is detected ahead in the direction of movement of the vehicle based on the external situation acquired by the sensor during movement by first autonomous driving from the predetermined position toward the parking target position by controlling at least the steering along the teacher path, the mode is switched to a temporary manual driving mode different from the manual driving mode, and the operation device and the steering device accept manual operation by the occupant; thereafter, in the temporary manual driving mode, after the vehicle has been moved by the manual operation, if the obstacle is not detected ahead in the direction of movement of the vehicle based on the external situation acquired by the sensor, the display device outputs a first screen indicating return to the teacher route by autonomous driving; performing a second autonomous driving to a midpoint on the teacher path between the predetermined position and the parking target position, and then performing a third autonomous driving along the teacher path from the midpoint to the parking target position; In the manual driving mode, the vehicle is driven in response to a steering operation of the steering device by the occupant, and when the vehicle is within a predetermined range from the predetermined position and the operation device does not receive a first operation, the vehicle is driven to the parking target position in response to a steering operation of the steering device by the occupant, continuing in the manual driving mode; When the operation device receives a second operation in the temporary manual driving mode, the operation device switches to the manual driving mode, and when, in the manual driving mode, after the vehicle has moved by the manual operation, the obstacle is not detected ahead in the moving direction of the vehicle based on the external situation acquired by the sensor, the display device does not output the first screen indicating return to the teacher route by autonomous driving. Vehicle control device.

12. The vehicle control device according to claim 11, the first screen includes at least one of a statement indicating return to the teacher route by autonomous driving and a schematic diagram indicating return to the teacher route by autonomous driving; Vehicle control device.

13. The vehicle control device according to claim 11, The second autonomous driving is performed based on a difference between the teacher route and the current position of the vehicle. Vehicle control device.

14. The vehicle control device according to claim 11, and during at least a part of the time while the vehicle is traveling in the second autonomous traveling mode to the waypoint, the display device outputs a second screen indicating that the vehicle is traveling in the second autonomous traveling mode to return to the teacher route. Vehicle control device.

15. The vehicle control device according to claim 14, the second screen includes at least one of a statement indicating that the autonomous traveling vehicle is currently traveling to return to the teacher route and a schematic diagram indicating that the autonomous traveling vehicle is currently traveling to return to the teacher route; Vehicle control device.

16. The vehicle control device according to claim 14, the second screen includes a first image representing the teacher route and a second image representing the current position of the vehicle; Vehicle control device.

17. The vehicle control device according to claim 16, In the temporary manual driving mode, if the obstacle is not detected ahead in the direction of movement of the vehicle based on the external situation acquired by the sensor after the vehicle has been moved by the manual operation, a return route is generated, and the second autonomous driving is performed based on the return route; the second screen further includes a third image representing the return path; Vehicle control device.

18. The vehicle control device according to claim 11, When an obstacle is detected ahead in the moving direction of the vehicle based on the external situation acquired by the sensor, the display device displays a third screen prompting the occupant to perform a manual operation to avoid the obstacle. Vehicle control device.

19. 19. The vehicle control device according to claim 18, The third screen displays a message prompting the passenger to perform a manual operation to avoid an obstacle, and a schematic diagram prompting the occupant to perform a manual operation to avoid an obstacle; Vehicle control device.

20. The vehicle control device according to claim 11, In the autonomous driving mode, when an obstacle is detected ahead in the direction of movement of the vehicle based on the external situation acquired by the sensor during first autonomous driving from the predetermined position toward the parking target position by controlling at least the steering along the teacher path, the mode is switched to a temporary manual driving mode different from the manual driving mode, and while the operation device accepts the manual operation by the occupant, the display device outputs a fourth screen prompting the occupant to perform an operation indicating completion of avoidance when avoidance of the obstacle is completed. Vehicle control device.

Images