Parking assist method and parking assist apparatus

Abstract

To provide a parking support method and a parking support device for enabling a driver to easily grasp a position where the automatic driving of a vehicle based on a traveling route is possible.SOLUTION: A parking assistance method according to the present disclosure is a parking assistance method for performing automatic travel of a vehicle based on teaching travel by a driver, the parking assistance method including storing information about a travel route in the teaching travel by the driver, displaying the information on a display device in a first display mode before the vehicle reaches a position where the vehicle can join the travel route, displaying the information on the display device in a second display mode different from the first display mode after the vehicle reaches the position where the vehicle can join the travel route, and displaying a parking position image indicating a parking position on the display device.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to a parking assistance method and a parking assistance device. [Background technology]

[0002] Conventionally, parking assistance technologies that automatically move a vehicle when parking are known. One such parking assistance technology is a technology that learns a driving route based on a teacher driving by the driver and provides parking assistance using the learning results. This technology is used when the driver repeatedly parks in a fixed parking spot, such as a parking lot at the driver's home or workplace. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6022447 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when a driver parks a vehicle using parking assistance, if the driver stops the vehicle too far from the learned driving route, it may be difficult to move the vehicle using automatic driving based on the driving route.

[0005] The present disclosure provides a parking assistance method and a parking assistance device that allow a driver to easily grasp a position where automatic driving of a vehicle based on a driving route becomes possible. [Means for solving the problem]

[0006] The parking assistance method disclosed herein is a parking assistance method for automatically driving a vehicle based on supervised driving by a driver, which stores information regarding the driving route during the supervised driving by the driver, displays the information on a display device in a first display mode before the vehicle reaches a position where it can merge onto the driving route, and displays the information on a display device in a second display mode different from the first display mode after the vehicle reaches a position where it can merge onto the driving route, and displays a parking position image showing the parking position on the display device. [Effects of the Invention]

[0007] According to the parking assistance method and parking assistance device disclosed herein, it is possible to allow the driver to easily understand the position to which the vehicle can move to a parking position by automatic driving based on the driving route. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a vehicle equipped with a parking assistance device according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a configuration in the vicinity of the driver's seat of the vehicle according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of parking assistance according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a hardware configuration of the parking assistance device according to the first embodiment. [Figure 5] FIG. 5 is a block diagram showing an example of functions provided in the parking assistance device according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of an image showing a travel start position displayed on the head-up display according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of an image showing a running start position displayed on the display device according to the first embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of the flow of the parking assistance process executed by the parking assistance device according to the first embodiment. [Figure 9]FIG. 9 is a diagram illustrating an example of parking assistance according to the second embodiment. [Figure 10] FIG. 10 is a diagram showing an example of guidance information regarding a parking start possible area according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing another example of guidance information regarding the parking start possible area according to the second embodiment. [Figure 12] FIG. 12 is a flowchart showing an example of the flow of parking assistance processing executed by the parking assistance device according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of parking assistance according to the third embodiment. [Figure 14] FIG. 14 is a flowchart showing an example of the flow of parking assistance processing executed by the parking assistance device according to the third embodiment. [Figure 15] FIG. 15 is a diagram showing an example of an image showing a travel start position displayed on a head-up display according to the first modification. [Figure 16] FIG. 16 is a diagram showing an example of an image showing the travel start position displayed on a display device according to the second modification. [Figure 17] FIG. 17 is a diagram showing an example of an image showing the travel start position displayed on a display device according to the third modification. [Figure 18] FIG. 18 is a diagram showing an example of a second guide image displayed on a display device according to the fourth modification. [Figure 19] FIG. 19 is a diagram showing an example of a second guide image displayed on a display device according to the fifth modification. [Figure 20] FIG. 20 is a diagram showing an example of a first guide image displayed on a head-up display according to the sixth modification. [Figure 21] FIG. 21 is a diagram showing an example of guidance information that is displayed before the vehicle according to the fourth embodiment reaches a position where it can merge with the travel route. [Figure 22] FIG. 22 is a diagram showing an example of guidance information displayed after the vehicle according to the fourth embodiment reaches a position where it can merge with the travel route. [Figure 23] FIG. 23 is a diagram showing an example of guidance information that is displayed after the vehicle according to the fourth embodiment starts automatic parking along a travel route. [Figure 24] FIG. 24 is a diagram showing an example of a state before the vehicle according to the fourth embodiment reaches a position where it can merge with the travel route. [Figure 25] FIG. 25 is a diagram showing an example of a state after the vehicle according to the fourth embodiment has reached a position where it can merge with the travel route. [Figure 26] FIG. 26 is a flowchart showing an example of the flow of the parking assistance process executed by the parking assistance device according to the fourth embodiment. [Figure 27] FIG. 27 is a diagram illustrating an example of guidance information according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of a parking assistance method and a parking assistance device according to the present disclosure will be described with reference to the drawings.

[0010] (First embodiment) Fig. 1 is a diagram showing an example of a vehicle 1 equipped with a parking assistance device 100 according to the first embodiment. As shown in Fig. 1, the vehicle 1 includes a vehicle body 12 and two pairs of wheels 13 arranged along a predetermined direction on the vehicle body 12. The two pairs of wheels 13 include a pair of front tires 13f and a pair of rear tires 13r.

[0011] The front tire 13f shown in Fig. 1 is an example of a first wheel in this embodiment. The rear tire 13r is an example of a second wheel in this embodiment. Although the vehicle 1 shown in Fig. 1 has four wheels 13, the number of wheels 13 is not limited to this. For example, the vehicle 1 may be a two-wheeled vehicle.

[0012] The vehicle body 12 is coupled to wheels 13 and can move by the wheels 13. In this case, the predetermined direction in which the two pairs of wheels 13 are arranged is the running direction (movement direction) of the vehicle 1. The vehicle 1 can move forward or backward by switching gears (not shown) or the like. The vehicle 1 can also turn right or left by steering.

[0013] The vehicle body 12 has a front end F, which is the end on the front tire 13f side, and a rear end R, which is the end on the rear tire 13r side. The vehicle body 12 has a generally rectangular shape when viewed from above, and the four corners of the generally rectangular shape are sometimes called "ends." The vehicle 1 also has a display device, a speaker, and an operation unit, which are not shown in FIG. 1.

[0014] A pair of bumpers 14 are provided at the front and rear ends F, R of the vehicle body 12 near the lower end of the vehicle body 12. Of the pair of bumpers 14, the front bumper 14f covers the entire front surface and part of the side surface near the lower end of the vehicle body 12. Of the pair of bumpers 14, the rear bumper 14r covers the entire rear surface and part of the side surface near the lower end of the vehicle body 12.

[0015] Wave transmitting and receiving units 15f, 15r that transmit and receive sound waves such as ultrasonic waves are disposed at predetermined ends of the vehicle body 12. For example, one or more wave transmitting and receiving units 15f are disposed on the front bumper 14f, and one or more wave transmitting and receiving units 15r are disposed on the rear bumper 14r. Hereinafter, unless otherwise specified, the wave transmitting and receiving units 15f, 15r will be simply referred to as the wave transmitting and receiving unit 15. Furthermore, the number and positions of the wave transmitting and receiving units 15 are not limited to the example shown in FIG. 1. For example, the vehicle 1 may be provided with wave transmitting and receiving units 15 on both the left and right sides.

[0016] In this embodiment, sonar using ultrasonic waves will be described as an example of the wave transmitting and receiving unit 15, but the wave transmitting and receiving unit 15 may also be radar that transmits and receives electromagnetic waves. Alternatively, the vehicle 1 may be equipped with both sonar and radar. Furthermore, the wave transmitting and receiving unit 15 may simply be referred to as a sensor.

[0017] More specifically, the wave transmitting / receiving unit 15 includes a wave transmitting unit that transmits sound waves such as ultrasonic waves or electromagnetic waves, and a wave receiving unit that receives reflected sound waves that are the sound waves or electromagnetic waves transmitted from the wave transmitting unit and reflected by an object. The wave transmitting / receiving unit 15 detects obstacles around the vehicle 1 based on the results of transmitting and receiving the sound waves or electromagnetic waves. The wave transmitting / receiving unit 15 also measures the distance between the vehicle 1 and obstacles around the vehicle 1 based on the results of transmitting and receiving the sound waves or electromagnetic waves.

[0018] The vehicle 1 also includes a first imaging device 16a that captures an image in front of the vehicle 1, a second imaging device 16b that captures an image behind the vehicle 1, a third imaging device 16c that captures an image on the left side of the vehicle 1, and a fourth imaging device that captures an image on the right side of the vehicle 1. The fourth imaging device is not shown in the figure.

[0019] Hereinafter, when there is no need to distinguish between the first imaging device 16a, the second imaging device 16b, the third imaging device 16c, and the fourth imaging device, they will be simply referred to as imaging device 16. The positions and number of imaging devices are not limited to the example shown in FIG. 1. For example, the vehicle 1 may also be equipped with only two imaging devices, the first imaging device 16a and the second imaging device 16b. Alternatively, the vehicle 1 may have other imaging devices in addition to those in the above example.

[0020] The imaging device 16 is capable of capturing images of the surroundings of the vehicle 1, and is, for example, a camera that captures color images. The captured images captured by the imaging device 16 may be either moving images or still images. The imaging device 16 may also be a camera built into the vehicle 1, or may be a camera of a drive recorder that is retrofitted to the vehicle 1.

[0021] The vehicle 1 is also equipped with a parking assistance device 100. The parking assistance device 100 is an information processing device that can be installed in the vehicle 1, and is, for example, an ECU (Electronic Control Unit) or an OBU (On Board Unit) provided inside the vehicle 1. Alternatively, the parking assistance device 100 may be an external device installed near the dashboard of the vehicle 1. The parking assistance device 100 may also function as a car navigation device or the like.

[0022] Next, a description will be given of the configuration of the vicinity of the driver's seat of the vehicle 1 according to this embodiment. Fig. 2 is a diagram showing an example of the configuration of the vicinity of the driver's seat 130a of the vehicle 1 according to the first embodiment.

[0023] 2, the vehicle 1 has a driver's seat 130a and a passenger's seat 130b. In front of the driver's seat 130a, a windshield 180, a dashboard 190, a steering wheel 140, a display device 120, and operation buttons 141 are provided.

[0024] The display device 120 is a display provided on a dashboard 190 of the vehicle 1. For example, the display device 120 is located in the center of the dashboard 190 as shown in FIG. 2. The display device 120 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display device 120 may also function as a touch panel.

[0025] Furthermore, a projection device (not shown) projects an image onto the windshield 180, thereby serving as a head-up display capable of displaying an image. The display device 120 and the head-up display are examples of a display unit in this embodiment.

[0026] The steering wheel 140 is provided in front of the driver's seat 130a and can be operated by the driver. The rotation angle of the steering wheel 140, i.e., the steering angle, is electrically or mechanically linked to the change in the direction of the front tires 13f, which are the steered wheels. The steered wheels may be the rear tires 13r, or both the front tires 13f and the rear tires 13r may be steered wheels.

[0027] The operation button 141 is a button that can accept an operation by a user. In this embodiment, the user is, for example, the driver of the vehicle 1. When the operation button 141 is pressed by the driver, it accepts an operation from the driver to, for example, start parking assistance. The position of the operation button 141 is not limited to the example shown in FIG. 2 , and it may be provided on the steering wheel 140, for example. The operation button 141 is an example of an operation unit in this embodiment. In addition, if the display device 120 also functions as a touch panel, the display device 120 may be an example of an operation unit. In addition, an operation terminal that can transmit a signal to the vehicle 1 from outside the vehicle 1, such as a remote controller or electronic key (not shown), may be an example of an operation unit.

[0028] Next, functions of the vehicle 1 of this embodiment will be described. The vehicle 1 of this embodiment learns a driving route based on supervised driving by the driver and performs parking assistance using the learning results. In other words, the parking assistance method executed by the parking assistance device 100 mounted on the vehicle 1 of this embodiment is a method of automatically driving the vehicle 1 based on supervised driving by the driver. This parking assistance method is effective in reducing the driver's parking effort when repeatedly parking in a fixed parking spot, such as the driver's garage at home, a contracted parking spot at an apartment complex, or a designated parking spot in a parking lot at work, etc. For this reason, this type of parking assistance is called home zone parking or route memory type automatic parking. Note that supervised driving is an example of a parking operation that has been performed in the past in this embodiment.

[0029] 3 is a diagram illustrating an example of parking assistance according to the first embodiment. A travel route 80 shown in FIG.

[0030] The parking location 910 is located, for example, in a garage 920 at the home of the driver of the vehicle 1, but is not limited to this.

[0031] The parking assistance device 100 of this embodiment learns the travel route 80 and performs home zone parking, which parks the vehicle 1 in the parking position 910 based on the learned travel route 80.

[0032] Next, a description will be given of the hardware configuration of the parking assistance device 100. Fig. 4 is a diagram showing an example of the hardware configuration of the parking assistance device 100 according to the first embodiment. As shown in Fig. 4, the parking assistance device 100 has a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, an I / F (Interface) 11D, an HDD (Hard Disk Drive) 11E, etc., which are interconnected via a bus 11F, and has a hardware configuration using a normal computer.

[0033] The CPU 11A is an arithmetic unit that controls the entire ECU. The CPU 11A is an example of a processor in the parking assistance device 100 of this embodiment, and another processor or processing circuit may be provided instead of the CPU 11A. The ROM 11B stores programs and the like that realize various processes by the CPU 11A. The RAM 11C is, for example, a main storage device of the parking assistance device 100, and stores data necessary for various processes by the CPU 11A. The I / F 11D is an interface for transmitting and receiving data. The I / F 11D may also transmit and receive information to and from other ECUs mounted on the vehicle 1 via a CAN (Controller Area Network) or the like within the vehicle 1.

[0034] Next, a detailed description will be given of the functions of the parking assistance device 100 of this embodiment. Fig. 5 is a block diagram showing an example of functions provided in the parking assistance device 100 according to the first embodiment.

[0035] As shown in FIG. 5, the parking assistance device 100 of this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0036] The storage unit 108 is configured by, for example, a ROM 11B, a RAM 11C, or an HDD 11E. Although one storage unit 108 is illustrated as being included in the parking assistance device 100 in FIG. 5, a plurality of storage media may function as the storage unit 108.

[0037] The storage unit 108 stores programs and data used in various processes executed by the parking assistance device 100. For example, the program executed by the parking assistance device 100 of this embodiment has a modular configuration including the above-mentioned functional units (acquisition unit 101, extraction unit 102, learning unit 103, estimation unit 104, output control unit 105, vehicle control unit 106, and reception unit 107). In actual hardware, the CPU 11A reads and executes the program from the storage unit 108, thereby loading the above-mentioned units onto the RAM 11C, and the acquisition unit 101, extraction unit 102, learning unit 103, estimation unit 104, output control unit 105, vehicle control unit 106, and reception unit 107 are generated on the RAM 11C. Note that the processes realized by the functional units of the parking assistance device 100 are also referred to as steps.

[0038] The program executed by the parking assistance device 100 of this embodiment is provided as a file in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0039] The program executed by the parking assistance device 100 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program executed by the parking assistance device 100 of this embodiment may be provided or distributed via a network such as the Internet. The program executed by the parking assistance device 100 of this embodiment may be provided by being pre-installed in ROM 11B or the like.

[0040] The acquisition unit 101 acquires images of the surroundings of the vehicle 1 from the imaging device 16. Hereinafter, these images will be referred to as surrounding images. The acquisition unit 101 acquires multiple surrounding images when the vehicle 1 learns a driving route based on supervised driving and when the vehicle 1 performs automatic driving based on the learned driving route.

[0041] The surrounding image captured around the vehicle 1 while traveling during the instructor driving is an example of first sensor information in this embodiment.

[0042] Furthermore, a surrounding image captured around the vehicle 1 before or during automatic driving of the vehicle 1 based on the learned driving route is an example of second sensor information in this embodiment.

[0043] The acquisition unit 101 also acquires the presence or absence of an obstacle detected by the wave transmitting and receiving unit 15, and the distance between the vehicle 1 and the obstacle around the vehicle 1 measured by the wave transmitting and receiving unit 15. The presence or absence of an obstacle detected by the wave transmitting and receiving unit 15, and the distance between the vehicle 1 and the obstacle around the vehicle 1 measured by the wave transmitting and receiving unit 15 may be referred to as environmental information relating to the environment around the vehicle 1. The environmental information is not limited to this information, and may include, for example, a surrounding image.

[0044] Furthermore, the acquisition unit 101 acquires vehicle information of the vehicle 1 from various sensors or other ECUs of the vehicle 1. The vehicle information includes, for example, information relating to the speed, steering angle, and braking operation of the vehicle 1.

[0045] The extraction unit 102 extracts feature points from the peripheral image. The method by which the extraction unit 102 extracts feature points is not particularly limited, and any known method may be applied. For example, the extraction unit 102 extracts feature points using a method such as FAST (Features from Accelerated Segment Test) or ORB (Oriented FAST and Rotated BRIEF). Furthermore, when learning the travel route 80, the extraction unit 102 may preferentially record, among the extracted feature points, feature points that satisfy specified conditions. For example, among multiple consecutive peripheral images in time series, feature points extracted from an image in which the vehicle 1 traveled a longer distance during image capture may be preferentially selected as feature points.

[0046] The learning unit 103 stores information about the driving route 80 during the supervised driving in the memory unit 108. The learning unit 103 is also referred to as a storage processing unit or a recording unit. More specifically, the learning unit 103 learns the driving route 80 for parking the vehicle 1 at the parking position 910 based on the supervised driving by the driver and the surrounding images captured during the supervised driving. For example, when learning the driving route 80, the learning unit 103 learns changes in the position of the vehicle 1 based on changes in feature points extracted from the surrounding images captured during the supervised driving. The learning unit 103 also learns the speed, steering angle, and braking operation of the vehicle 1 during the supervised driving. The learning unit 103 also defines the environment around the vehicle 1 as a map based on the feature points extracted from the surrounding images captured during the supervised driving by the driver, and stores the map in the memory unit 108.

[0047] In this embodiment, the term "learning" includes storing collected information in the storage unit 108. Furthermore, the method of storing information about the travel route 80 and the above-described map in the storage unit 108 is not limited to the example given in this embodiment.

[0048] The supervisor driving in this embodiment is a driving in which the driver manually drives the vehicle 1 from a driving start position 900 outside the parking position to the parking position 910. In this embodiment, the learning unit 103 learns the driving start position 900 in the supervisor driving as the start position of the driving route 80. The distance from the driving start position 900 to the parking position 910 is, for example, about 50 m, but is not limited to this.

[0049] For example, the driver starts teacher driving when the vehicle 1 is stopped at a position different from the parking position 910, for example, a desired position outside the garage 920, and drives the vehicle 1 in reverse to the parking position 910 within the garage 920. Note that teacher driving is not limited to reverse driving, and may include forward driving or both forward and reverse driving.

[0050] During the supervised driving, the acquisition unit 101 acquires a peripheral image and vehicle information. During the supervised driving, the extraction unit 102 extracts feature points from the peripheral image acquired by the acquisition unit 101. The learning unit 103 records the route traveled by the vehicle 1 through manual driving as the driving route 80, based on the extracted feature points and the vehicle information at the time the peripheral image from which the feature points were extracted was captured. Specifically, when the driver performs supervised driving in which the vehicle 1 is parked in a parking position 910 by reversing, the driving route 80 also becomes the route through which the vehicle 1 is parked in the parking position 910 by reversing. In this embodiment, the position at which the supervised driving started is the driving start position 900 of the driving route 80. The learning unit 103 stores the driving route information defining the recorded driving route 80 in the storage unit 108.

[0051] The driving route information is stored in the storage unit 108 as information in which, for example, the speed, steering angle, braking operation of the vehicle 1 during the supervised driving, and feature points extracted from a plurality of surrounding images captured as the vehicle 1 moves during the supervised driving are associated in chronological order. Note that the method for learning the driving route 80 and the manner in which the driving route is defined are not limited to the above example.

[0052] The estimation unit 104 estimates the position of the vehicle 1 and a position to which the vehicle 1 can move to the parking position 910 by automatic driving based on the travel route 80, based on the surrounding image.

[0053] The position to which the vehicle 1 can move to the parking position 910 by autonomous driving is, in other words, a position from which the vehicle 1 can start autonomous driving by home zone parking. Automated driving by home zone parking is, in other words, automated driving based on the driving route 80 stored in the memory unit 108. The position to which the vehicle 1 can move to the parking position 910 by autonomous driving is an example of a target position in this embodiment. The target position is not limited to a single point position, but may be an area encompassing a wider range than the body 12 of the vehicle 1. The target position may also be referred to as a target area. In this embodiment, the target position to which the vehicle 1 can move to the parking position 910 is the driving start position 900. The parking position 910 is the goal of the driving route by home zone parking, and is also referred to as the final target position.

[0054] More specifically, when the driver's operation to start home zone parking is accepted by the accepting unit 107 (described later), the estimating unit 104 reads out from the memory unit 108 the driving route 80 and feature points (map) of the surrounding image at the time of learning the driving route 80. Then, based on the driving route 80 and the feature points extracted from the surrounding image by the extracting unit 102, the estimating unit 104 estimates the position of the vehicle 1 and a driving start position 900 from which the vehicle 1 can move to the parking position 910 by automatic driving based on the driving route 80.

[0055] The estimation unit 104 compares feature points of the peripheral image when learning the driving route 80 with feature points of the peripheral image at the time when the operation to start home zone parking is accepted. The estimation unit 104 estimates the position and orientation of the vehicle 1 on the map read from the storage unit 108, using the feature points of the peripheral image at the time when the operation to start home zone parking is accepted. Note that the estimation unit 104 does not need to specify the driving start position 900 and the current position of the vehicle 1 as absolute positions, but only needs to specify the relative positional relationship between the driving start position 900 and the current position of the vehicle 1. Note that the method by which the estimation unit 104 estimates the position of the vehicle 1 and the driving start position 900 is not limited to this example.

[0056] Furthermore, the estimation unit 104 determines whether the running start position 900 is within a specified distance from the vehicle 1 based on the estimated position of the vehicle 1 and the running start position 900. The specified distance is, for example, a distance at which the running start position 900 is included in the imaging range of the imaging device 16 of the vehicle 1, or a distance at which the running start position 900 is visible from the windshield 180 of the vehicle 1. As an example, the specified distance is about 5 m, but is not limited to this value. The specified distance is also referred to as a first specified distance.

[0057] When the estimation unit 104 determines that the running start position 900 is within a specified distance from the vehicle 1, it sends the estimated position of the vehicle 1 and the running start position 900 to the output control unit 105 described below. Note that the output control unit 105 may determine whether the running start position 900 is within a specified distance from the vehicle 1.

[0058] Furthermore, while the vehicle 1 is being driven automatically by the vehicle control unit 106 (described later), the estimation unit 104 estimates the position of the vehicle 1 based on feature points included in the driving route information and feature points extracted from surrounding images acquired during the automatic driving. The estimation unit 104 sends the estimated position of the vehicle 1 to the vehicle control unit 106.

[0059] The estimation unit 104 may continuously estimate the position of the vehicle 1 while the vehicle 1 is traveling, or may start after the user performs an operation to start home zone parking. For example, if the estimation process is performed outside the range of the stored map due to supervised traveling, the estimation unit 104 may not be able to identify the position of the vehicle 1. However, if the vehicle 1 reaches the range of the stored map, the estimation unit 104 may notify the output control unit 105 that the vehicle 1 is located near the traveling start position 900. Furthermore, if the current position of the vehicle 1 is the traveling start position 900, the estimation unit 104 may notify the output control unit 105 that the vehicle 1 is located at the traveling start position 900. The estimation unit 104 may determine that the vehicle 1 is located at the traveling start position 900 not only when the current position of the vehicle 1 and the traveling start position 900 completely match, but also when the distance between the current position of the vehicle 1 and the traveling start position 900 is equal to or smaller than a specified error.

[0060] The output control unit 105 causes the head-up display or the display device 120 to display guidance information relating to a position to which the vehicle 1 can move to the parking position 910 by autonomous driving.

[0061] The guidance information is information including a position where the vehicle 1 can travel automatically based on the travel route, and the orientation of the vehicle 1 in which the vehicle 1 can travel automatically when the vehicle is located at that position. In this embodiment, the guidance information is an image showing the travel start position 900. In this embodiment, this image is referred to as a guidance image.

[0062] For example, when the estimation unit 104 estimates that the vehicle 1 has reached the range of the stored map through supervised driving, the output control unit 105 suggests to the user that home zone parking should be started. The method of suggestion is not particularly limited, but the output control unit 105 may display a message on the display device 120 urging the user to start home zone parking, or may output the message as audio from a speaker of the vehicle 1.

[0063] Note that a configuration may be adopted in which such a notification is not provided and the user performs an operation to start home zone parking at any timing. For example, suppose that the user manually drives vehicle 1 to a location close to his or her home, and then desires to park vehicle 1 in his or her home garage 920 using home zone parking. In this case, the user performs an operation to start home zone parking, for example, by pressing operation button 141.

[0064] Furthermore, when the user manually drives the vehicle 1 to the travel start position 900 that the vehicle 1 has learned in advance and then performs an operation to start automatic travel by pressing the operation button 141, automatic travel is started by the vehicle control unit 106, which will be described later. Alternatively, the automatic travel may be started automatically when the user stops the vehicle 1.

[0065] When home zone parking is performed using this procedure, the user manually drives vehicle 1 to travel start position 900 as described above. If vehicle 1 stops at a position different from travel start position 900, and the distance between travel start position 900 and the actual stopping position of vehicle 1 is within an allowable range, correction can be made through feedback control by vehicle control unit 106, which will be described later. However, if the distance between travel start position 900 and the actual stopping position of vehicle 1 is so large that it exceeds the allowable range, it may be difficult for vehicle 1 to move to parking position 910 through automatic driving by vehicle control unit 106.

[0066] Therefore, the output control unit 105 displays an image indicating the travel start position 900 on the head-up display or the display device 120, thereby enabling the driver to grasp the travel start position 900. This makes it easy for the driver to stop the vehicle 1 at the travel start position 900.

[0067] Furthermore, the output control unit 105 may notify the user when the vehicle 1 is located at a position where the vehicle 1 can perform autonomous driving. For example, when the vehicle 1 moves to the driving start position 900, the output control unit 105 notifies the user that the vehicle 1 is located at the driving start position 900, that is, that autonomous driving can be started. The notification method is not particularly limited, but the output control unit 105 may display a message indicating that the vehicle 1 is located at the driving start position 900 on a head-up display or the display device 120. Alternatively, the output control unit 105 may output the message by voice from a speaker of the vehicle 1.

[0068] Fig. 6 is a diagram showing an example of an image displayed on the head-up display according to the first embodiment, which indicates the travel start position 900. As shown in Fig. 6, the output control unit 105 displays a first guide image 91a indicating the travel start position 900 on the windshield 180, which functions as a head-up display.

[0069] In the example shown in FIG. 6, the first guide image 91a is a rectangular shape that represents the size of the vehicle body of the vehicle 1. The orientation of the rectangle that is the first guide image 91a represents the orientation in which the vehicle 1 is capable of automatic driving at the driving start position 900. While the first guide image 91a is a rectangle surrounded by dashed lines in FIG. 6, it may also be a shape surrounded by solid lines instead of dashed lines. The first guide image 91a may also be an image in which the contour lines in the longitudinal direction are thicker than the contour lines in the lateral direction, as with the second guide image described later in FIG. 7. The display on the display device 120 is not shown in FIG. 6.

[0070] The output control unit 105 causes the display device 120 to display a second guide image indicating the travel start position on the front view or top view displayed thereon.

[0071] FIG. 7 is a diagram showing an example of an image showing a travel start position 900 displayed on the display device 120 according to the first embodiment. As shown in FIG. 7, the output control unit 105 displays a second guide image 92a on the front view 21, which represents the direction ahead of the vehicle 1. In the example shown in FIG. 7, the front view 21 includes a first vehicle image 2a representing the position of the body 12 of the vehicle 1. The output control unit 105 also displays a second guide image 92b on the top view 22, which represents the state of the vehicle 1 as seen from above. The top view 22 also includes a second vehicle image 2b representing the position of the body 12 of the vehicle 1.

[0072] In the example shown in Fig. 7, the second guide images 92a, 92b are rectangular shapes corresponding in size to the body of the vehicle 1. In the example shown in Fig. 7, the contour lines in the longer side direction of the second guide images 92a, 92b are thinner than the contour lines in the shorter side direction.

[0073] In this embodiment, the image displayed in both the front view 21 and the top view 22 is referred to as the second guide image, but the image on the top view 22 may also be referred to as the third image. Also, in FIG. 7, the output control unit 105 displays second guide images 92a, 92b in both the front view 21 and the top view 22, but it may display them in only one of them. Also, in the front view 21 shown in FIG. 7, a first vehicle image 2a representing the body 12 of the vehicle 1 is depicted within the front view 21, but the front view 21 does not have to include the body 12 of the vehicle 1.

[0074] In FIG. 7, the output control unit 105 displays the letter "P" on the second guide images 92a and 92b to inform the driver that the vehicle should stop at the travel start position 900, but this letter does not have to be displayed. Alternatively, the output control unit 105 may display a sentence indicating instructions to the driver, such as "Please stop here," on or near the second guide images 92a and 92b. The output control unit 105 may also display the letter "P" or a sentence indicating instructions to the driver on or near the first guide image 91a on the head-up display.

[0075] In addition, in this embodiment, the output control unit 105 displays the guidance information on both the head-up display and the display device 120, but it may also display the guidance information on only one of the head-up display and the display device 120.

[0076] Returning to FIG. 5 , the vehicle control unit 106 moves the vehicle 1 to the parking position 910 by automatic driving based on the travel route 80. The vehicle control unit 106 controls the steering, braking, and acceleration / deceleration of the vehicle 1 to cause the vehicle 1 to travel automatically. In this embodiment, the vehicle control unit 106 causes the vehicle 1 to travel automatically along the learned travel route 80. This type of travel control method is also referred to as regenerating the travel route 80.

[0077] Furthermore, during autonomous driving, the vehicle control unit 106 may cause the vehicle 1 to operate differently from the supervised driving based on the vehicle information and environmental information of the vehicle 1 acquired by the acquisition unit 101. As described above, the environmental information includes the presence or absence of an obstacle around the vehicle 1 and the distance between the vehicle 1 and the obstacle around the vehicle 1. For example, if an obstacle such as another vehicle approaches during autonomous driving, the vehicle control unit 106 executes control such as stopping the vehicle 1.

[0078] In this embodiment, the driver may be seated in the driver's seat 130a of the vehicle 1 or may be out of the vehicle 1 while the vehicle control unit 106 is running automatically.

[0079] Furthermore, when the vehicle 1 deviates from the travel route 80, the vehicle control unit 106 uses feedback control to move the vehicle 1 back onto the travel route 80. For example, the vehicle control unit 106 estimates the difference between the position of the vehicle 1 and the travel route 80 based on the position of the vehicle 1 estimated by the estimation unit 104, and causes the vehicle 1 to travel so as to reduce the difference.

[0080] 5 illustrates the parking assistance device 100 as including the vehicle control unit 106, the vehicle control unit 106 may be realized by another ECU outside the parking assistance device 100.

[0081] The reception unit 107 receives various operations from the driver. For example, the reception unit 107 receives an operation to start parking assistance when the operation button 141 is pressed. Furthermore, if the display device 120 is a touch panel, the reception unit 107 may receive an operation to start parking assistance when an image button on the touch panel is pressed.

[0082] Next, the flow of the parking assistance process executed by the parking assistance device 100 of this embodiment configured as above will be described.

[0083] 8 is a flowchart showing an example of the flow of parking assistance processing executed by the parking assistance device 100 according to the first embodiment. The processing of this flowchart starts, for example, when an operation to start parking assistance is accepted by the acceptance unit 107. It is assumed that the travel route 80 has been learned by the learning unit 103 before the processing of this flowchart is executed.

[0084] First, the acquisition unit 101 acquires a surrounding image from the imaging device 16 (S1).

[0085] Then, the extraction unit 102 extracts feature points from the surrounding image acquired by the acquisition unit 101 (S2).

[0086] Next, the estimation unit 104 reads out the driving route information from the memory unit 108 and estimates the position of the vehicle 1 based on the driving route 80 defined in the driving route information and the feature points extracted from the surrounding image by the extraction unit 102 (S3).

[0087] Furthermore, the estimation unit 104 estimates a driving start position 900 for automatic parking based on the driving route 80 and the feature points extracted from the surrounding image by the extraction unit 102 (S4).

[0088] Then, the estimation unit 104 determines whether the travel start position 900 is within a specified distance from the vehicle 1 based on the estimated position of the vehicle 1 and the travel start position 900 (S5).

[0089] If the estimation unit 104 determines that the travel start position 900 is not within the specified distance from the vehicle 1 (S5 "No"), the process returns to S1.

[0090] For example, if the vehicle 1 is located farther away from the travel start position 900 than a specified distance, the output control unit 105 cannot display the travel start position 900 on the head-up display or the display device 120. Therefore, in such a case, the estimation unit 104 repeats the processes of S1 to S5 and waits for the vehicle 1 to approach the travel start position 900. Note that even if the feature points extracted from the peripheral image in the process of S2 do not match at all with the feature points extracted when learning the travel route 80, the estimation unit 104 determines that the travel start position 900 is not within the specified distance from the vehicle 1.

[0091] When the estimation unit 104 determines that the travel start position 900 is within a specified distance from the vehicle 1 (S5 "Yes"), the output control unit 105 displays the travel start position 900.

[0092] Specifically, the output control unit 105 displays a first guide image 91a indicating the travel start position 900 on the windshield 180, which functions as a head-up display. The output control unit 105 also displays second guide images 92a and 92b indicating the travel start position 900 on the front view 21 and the top view 22 displayed on the display device 120 (S6). This display allows the driver to grasp the travel start position 900, and allows the driver to manually drive the vehicle 1 to the travel start position 900.

[0093] Furthermore, while the vehicle 1 moves to the travel start position 900, the acquisition unit 101 continues to acquire peripheral images from the imaging device 16. The extraction unit 102 extracts feature points from the acquired peripheral images. The acquisition unit 101 also acquires vehicle information, the presence or absence of obstacles around the vehicle 1, and the distance to the obstacles. The estimation unit 104 estimates the position of the vehicle 1, which changes as the vehicle 1 travels, based on the movement of the feature points extracted from the peripheral images or the vehicle information. Furthermore, the output control unit 105 moves the positions of the first guide image 91a and the second guide images 92a and 92b as the vehicle 1 moves.

[0094] Then, the vehicle control unit 106 determines whether the vehicle 1 has stopped at the travel start position 900 (S7). Note that this determination process may be executed by the estimation unit 104. If the vehicle 1 has not reached the travel start position 900 (S7 "No"), the vehicle control unit 106 waits for the vehicle 1 to reach the travel start position 900 and stop.

[0095] Then, when the vehicle 1 reaches the travel start position 900 and stops (S7 "Yes"), the vehicle control unit 106 starts automatic driving (S8). Then, the vehicle control unit 106 drives the vehicle 1 from the travel start position 900 along the travel route 80 to the parking position 910 by automatic driving. At this point, the processing of this flowchart ends. Note that in S7 and S8, automatic driving is started when the vehicle 1 reaches the travel start position 900 and stops, but automatic driving may also be started when an operation to start automatic driving is received from the user after the vehicle 1 reaches the travel start position 900 and stops.

[0096] Note that the conditions for starting autonomous driving are not limited to this, and autonomous driving may be started when the driver performs a start operation using the operation button 141 or touch panel in addition to the vehicle 1 stopping at the travel start position 900. Alternatively, autonomous driving may be started when the driver stops the vehicle 1 at the travel start position 900, gets out of the vehicle 1, and performs a start operation from outside the vehicle 1 using an operation terminal such as a remote controller or an electronic key.

[0097] In this way, in the parking assistance method performed by the parking assistance device 100 of this embodiment, information regarding the driving route 80 during teacher driving is stored in the memory unit 108, and guidance information regarding the target position at which the vehicle 1 can drive automatically based on the stored driving route 80 is displayed on the head-up display or the display device 120.Therefore, according to the parking assistance method performed by the parking assistance device 100 of this embodiment, the driver can easily understand the position at which the vehicle 1 can drive automatically based on the driving route 80.

[0098] Furthermore, in the parking assistance method executed by the parking assistance device 100 of this embodiment, a driving route 80 for parking the vehicle 1 at the parking position 910 is stored based on a surrounding image captured during supervised driving by the driver. Furthermore, the target position in this embodiment is a position to which the vehicle 1 can move to the parking position 910 by automatic driving. Furthermore, in the parking assistance method of this embodiment, the position of the vehicle 1 and a target position to which the vehicle 1 can move to the parking position 910 by automatic driving based on the driving route 80 are estimated based on second sensor information acquired of the surroundings of the vehicle 1. Therefore, according to the parking assistance method executed by the parking assistance device 100 of this embodiment, the driver can easily grasp the position to which the vehicle 1 can move to the parking position 910 by automatic driving based on the driving route 80.

[0099] Furthermore, the guidance information displayed on the head-up display or display device 120 in the parking assistance method executed by the parking assistance device 100 of this embodiment includes the target position and the orientation of the vehicle 1 that enables the vehicle 1 to travel automatically when the vehicle 1 is located at the target position. Therefore, according to the parking assistance method executed by the parking assistance device 100 of this embodiment, the driver can easily grasp the direction in which the vehicle 1 should face when moving the vehicle 1 to the target position.

[0100] Furthermore, in the parking assistance method executed by the parking assistance device 100 of this embodiment, the travel start position 900 of the travel route 80 is displayed on the head-up display or the display device 120 as a position to which the vehicle 1 can move to the parking position 910. Therefore, according to the parking assistance method executed by the parking assistance device 100 of this embodiment, it becomes easy for the driver to manually drive the vehicle 1 to the travel start position 900 by visually recognizing the display.

[0101] Furthermore, in the parking assistance method executed by parking assistance device 100 of this embodiment, first guide image 91a indicating traveling start position 900 is displayed on the head-up display. Therefore, according to the parking assistance method executed by parking assistance device 100 of this embodiment, the driver can grasp traveling start position 900 while facing forward.

[0102] Furthermore, in the parking assistance method executed by parking assistance device 100 of this embodiment, second guide images 92a, 92b indicating the traveling start position 900 are displayed on the front view 21 or the top view 22 displayed on the display device 120 provided on the dashboard 190 of the vehicle 1. Therefore, according to the parking assistance method executed by parking assistance device 100 of this embodiment, the driver can easily grasp the positional relationship between the vehicle 1 and the traveling start position 900.

[0103] Furthermore, as described above, in the parking assistance method executed by the parking assistance device 100 of this embodiment, when the vehicle 1 is located at the target position, the user may be notified that automatic driving can be started. By such notification, the user can easily understand that the vehicle 1 has reached a position where automatic driving can be started.

[0104] (Second embodiment) In the first embodiment described above, the parking assistance device 100 displays the travel start position 900 of the learned travel route 80. In contrast, in this second embodiment, the parking assistance device 100 displays an area where the vehicle 1 can merge onto the travel route 80 midway.

[0105] The configuration of vehicle 1 in this embodiment is the same as the configuration in the first embodiment described with reference to Figures 1 and 2. The hardware configuration and functional blocks of parking assistance device 100 in this embodiment are the same as those in the first embodiment described with reference to Figures 4 and 5. As in the first embodiment, parking assistance device 100 in this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0106] Furthermore, the acquisition unit 101, extraction unit 102, learning unit 103, and reception unit 107 of this embodiment have the same functions as those of the first embodiment.

[0107] In addition to the functions of the first embodiment, the estimation unit 104 of this embodiment estimates the position of the vehicle 1 and an area in which the vehicle 1 can merge midway onto the learned travel route 80, based on a surrounding image captured of the area around the vehicle 1. More specifically, the estimation unit 104 estimates an area in which the vehicle 1 can return to the travel route 80 and reach the parking position 910 by feedback control of the vehicle control unit 106.

[0108] FIG. 9 is a diagram illustrating an example of parking assistance according to the second embodiment. A strip-shaped area 801 along a travel route 80 shown in FIG. 9 is a range in which it is estimated that the vehicle 1 can merge en route with the learned travel route 80. When the vehicle 1 is located in this area 801, the vehicle control unit 106 can use feedback control to return the vehicle 1 to the travel route 80 and move it to a parking position 910. In other words, this area 801 is a position where the vehicle 1 can start autonomous driving using home zone parking. In this embodiment, this area 801 is referred to as a parking start area 801.

[0109] The width of the parking start possible region 801 becomes narrower as the distance to the parking position 910 becomes shorter. For example, in the example shown in FIG. 9 , the width d2 of the parking start possible region 801 at point A2, which is closer to the parking position 910, is narrower than the width d1 of the parking start possible region 801 at point A1, which is closer to the driving start position 900. Furthermore, as the distance to the parking position 910 becomes even shorter, the region 801 disappears. This is because if the parking position 910, which is the goal of the driving route 80, is too close, it is difficult to return the vehicle 1 to the driving route 80. The position at which the parking start possible region 801 disappears varies depending on, for example, the road conditions around the driving route 80, the size of the space, and the like. Furthermore, the width of the parking start possible region 801 may be determined in advance in association with the distance from the parking position 910.

[0110] In addition to the functions of the first embodiment, the output control unit 105 causes the head-up display or the display device 120 to display guidance information regarding the parking start possible area 801.

[0111] Fig. 10 is a diagram showing an example of guidance information related to the parking-start possible area 801 according to the second embodiment. In the example shown in Fig. 10, a route guidance image 901a showing the driving route 80 and an area guidance image 901b showing the parking-start possible area 801 are displayed on the head-up display. The route guidance image 901a and the area guidance image 901b are examples of guidance information in this embodiment. Note that only the area guidance image 901b may be an example of guidance information.

[0112] In the example shown in Fig. 10, it is assumed that an area guide image 901b corresponding to the parking start possible area 801 near point A1 described in Fig. 9 is displayed. As the distance to the parking position 910 becomes closer, the width of the area guide image 901b becomes narrower.

[0113] Fig. 11 is a diagram showing another example of guidance information related to the parking start possible area 801 according to the second embodiment. In the example shown in Fig. 11, it is assumed that an area guidance image 901b corresponding to the parking start possible area 801 near point A2 described in Fig. 9 is displayed. Because the width d2 of the parking start possible area 801 at point A2 is narrower than the width d1 of the parking start possible area 801 at point A1, the width of the area guidance image 901b displayed on the head-up display shown in Fig. 11 is narrower than the width of the area guidance image 901b shown in Fig. 10.

[0114] The display mode of the area guidance image 901b is not limited to the example shown in Fig. 10. For example, the output control unit 105 does not have to distinguish between the driving route 80 and the parking-start possible area 801. In this case, the output control unit 105 displays only the area guidance image 901b without displaying the route guidance image 901a. The output control unit 105 may also display the area guidance image 901b on the front view 21 or the top view 22 displayed on the display device 120.

[0115] In addition to the functions of the first embodiment, the vehicle control unit 106 performs automatic driving control to move the vehicle 1 stopped within the parking start area 801 so that it merges with the driving route 80 and is then parked at the parking position 910.

[0116] In this embodiment, an example is given in which a route guidance image 901a and an area guidance image 901b are displayed, but in addition, as in the first embodiment, a first guidance image 91a or a second guidance image 92a, 92b indicating the driving start position 900 may also be displayed.

[0117] Next, the flow of the parking assistance process executed by the parking assistance device 100 of this embodiment configured as above will be described.

[0118] FIG. 12 is a flowchart showing an example of the flow of the parking assistance process executed by the parking assistance device 100 according to the second embodiment.

[0119] The process from the process of acquiring a surrounding image in S1 to the process of estimating the position of the vehicle 1 in S3 is the same as in the first embodiment. Then, the estimation unit 104 estimates the parking start possible area 801 (S101).

[0120] The estimation unit 104 also determines whether the parking-start possible area 801 is within a specified distance from the vehicle 1 (S102). The specified distance in this embodiment is, for example, a distance at which the parking-start possible area 801 is included in the imaging range of the imaging device 16 of the vehicle 1, or a distance at which the parking-start possible area 801 is visible from the windshield 180 of the vehicle 1.

[0121] If the estimation unit 104 determines that the parking start possible area 801 is not present within the specified distance from the vehicle 1 (S102 "No"), the process returns to S1.

[0122] Furthermore, when the estimation unit 104 determines that the parking-start possible region 801 is present within a specified distance from the vehicle 1 (S102 "Yes"), the output control unit 105 displays the parking-start possible region 801 on the head-up display or the display device 120 (S103). For example, the output control unit 105 causes the head-up display or the display device 120 to display a route guidance image 901a and an area guidance image 901b.

[0123] Then, the vehicle control unit 106 determines whether the vehicle 1 has stopped in the parking start possible area 801 (S104). If the vehicle 1 has not reached the parking start possible area 801 (S104 "No"), the vehicle control unit 106 waits for the vehicle 1 to reach the parking start possible area 801 and stop.

[0124] Then, when the vehicle 1 reaches and stops in the parking start possible area 801 (S104 "Yes"), the vehicle control unit 106 starts automatic driving (S8). Then, the vehicle control unit 106 performs automatic driving so that the position of the vehicle 1 moves onto the driving route 80. Thereafter, the vehicle control unit 106 drives the vehicle 1 along the driving route 80 to the parking position 910. At this point, the processing of this flowchart ends.

[0125] As described above, the parking assistance method executed by the parking assistance device 100 of this embodiment learns the driving route 80 for parking the vehicle 1 at the parking position 910 based on the supervised driving by the driver and the surrounding images captured during the supervised driving, and then estimates the position of the vehicle 1 and the parking-start possible area 801 based on the surrounding images acquired before the start of parking assistance. The parking assistance method then displays guidance information regarding the parking-start possible area 801 on the head-up display or the display device 120. Therefore, in addition to the effects of the first embodiment, the parking assistance method executed by the parking assistance device 100 of this embodiment can easily allow the driver to grasp the area from which the vehicle can be moved to the parking position 910 by autonomous driving even from the middle of the driving route 80. The parking assistance method can allow the driver to grasp positions from which home zone parking can be started in addition to the driving start position 900 during the supervised driving, thereby increasing the driver's flexibility in determining where to stop the vehicle 1.

[0126] (Third embodiment) In the first and second embodiments described above, the parking assistance device 100 estimates the position of the vehicle 1 based on feature points extracted from a peripheral image. In this third embodiment, the parking assistance device 100 estimates the position of the vehicle 1 by combining other position estimation methods, and provides assistance to the driver according to the estimated position.

[0127] The vehicle 1 of this embodiment has the same configuration as the first embodiment described with reference to Figures 1 and 2. The vehicle 1 of this embodiment also has an antenna capable of receiving GPS (Global Positioning System) signals, and a GPS device that identifies GPS coordinates representing the position of the vehicle 1 based on the received GPS signals. The vehicle 1 also has a gyro sensor that detects the turning speed (angular velocity) of the vehicle 1, and a sensor that measures the number of rotations of the wheels 13 of the vehicle 1.

[0128] The hardware configuration and functional blocks of the parking assistance device 100 of this embodiment are the same as those of the first embodiment described with reference to Figures 4 and 5. As in the first embodiment, the parking assistance device 100 of this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0129] Furthermore, the extraction unit 102, learning unit 103, vehicle control unit 106, and reception unit 107 of this embodiment have the same functions as those of the first embodiment.

[0130] In addition to the functions of the first embodiment, the acquisition unit 101 of this embodiment acquires position information of the vehicle 1. More specifically, the acquisition unit 101 acquires GPS coordinates indicating the position of the vehicle 1, i.e., GPS position information, from a GPS device. The acquisition unit 101 also acquires the number of rotations or angular velocity of the wheels 13 of the vehicle 1 from a gyro sensor and other sensors. The acquisition unit 101 may acquire this information directly from various sensors or may acquire it from another ECU.

[0131] In addition to the functions of the first embodiment, the estimation unit 104 of this embodiment estimates the distance to the parking position 910 from the position of the vehicle 1 based on the position information acquired by the acquisition unit 101. Furthermore, the estimation unit 104 may estimate the position of the vehicle 1 based not only on the GPS position information but also on the amount of movement of the vehicle 1 since the time the GPS position information was acquired. The estimation unit 104 may estimate the amount of movement of the vehicle 1 from the number of rotations or angular velocity of the wheels 13 of the vehicle 1. For example, the estimation unit 104 may identify the coordinates of a reference point using the GPS position information, and then estimate the position of the vehicle 1 after it has moved by estimating the distance and direction that the vehicle 1 has moved from the reference point based on the number of rotations or angular velocity of the wheels 13 of the vehicle 1.

[0132] In addition to the functions of the first embodiment, the output control unit 105 of this embodiment notifies the user that the parking position 910 is close when the distance from the parking position 910 to the vehicle 1 is equal to or less than a threshold. The threshold is not particularly limited, but may be, for example, about 20 m. The threshold is also referred to as a second specified distance.

[0133] FIG. 13 is a diagram illustrating an example of parking assistance according to the third embodiment. For example, assume that the distance from point A4 shown in FIG. 13 to parking position 910 is equal to or less than a threshold. When vehicle 1 is located at point A4, output control unit 105 notifies the user that parking position 910 is nearby. The content of the notification is not particularly limited, and may be, for example, a message prompting the user to start parking assistance, such as "You are approaching your home. Would you like to start parking assistance?" The output control unit 105 may output the message as audio from a speaker of vehicle 1. Alternatively, the output control unit 105 may display the message on a head-up display or the display device 120. For example, if an operation to start automatic parking at parking position 910 by the driver is received after the notification of the message, the estimation unit 104 estimates the location of vehicle 1 and a location to which vehicle 1 can move to parking position 910 by automatic driving based on the driving route 80, based on the surrounding image, as in the first or second embodiment.

[0134] When the vehicle 1 approaches the parking position 910, the estimation unit 104 can estimate the position of the vehicle 1 based on feature points extracted from the peripheral image using the same method as in the first embodiment. For example, when the vehicle 1 is located at point A3 shown in Fig. 13, the estimation unit 104 estimates the position of the vehicle 1 based on feature points extracted from the peripheral image. The position at which the estimation unit 104 can estimate the position of the vehicle 1 based on feature points extracted from the peripheral image is, for example, about 5 m from the travel start position 900, but is not limited to this value.

[0135] Next, the flow of the parking assistance process executed by the parking assistance device 100 of this embodiment configured as above will be described.

[0136] FIG. 14 is a flowchart showing an example of the flow of the parking assistance process executed by the parking assistance device 100 according to the third embodiment.

[0137] First, the acquisition unit 101 acquires the location information of the vehicle 1 (S201).

[0138] Then, the estimation unit 104 estimates the distance from the position of the vehicle 1 based on the acquired position information to the parking position 910 (S202).

[0139] The estimation unit 104 determines whether the estimated distance is equal to or smaller than a threshold (S203). Note that this determination process may be performed by the output control unit 105.

[0140] If the estimation unit 104 determines that the estimated distance is longer than the threshold value (S203 "No"), the process returns to S1.

[0141] Furthermore, when the estimation unit 104 determines that the estimated distance is equal to or less than the threshold (S203 "Yes"), the output control unit 105 notifies the user that the parking position 910 is close (S204).

[0142] Then, the reception unit 107 determines whether or not an operation to start automatic parking by the driver has been received (S205). If the reception unit 107 has not received an operation to start parking (S205 "No"), the reception unit 107 repeats the process of S205 and waits for an operation from the driver.

[0143] If the reception unit 107 receives an operation to start parking (S205 "Yes"), the process proceeds to the acquisition of a surrounding image in S1. The processes from S1 to S8 are the same as those in the first embodiment described in FIG. 8. Note that the processes from S1 onwards may be the same as those in the second embodiment described in FIG. 12.

[0144] In this way, in the parking assistance method executed by the parking assistance device 100 of this embodiment, when the distance from the position of the vehicle 1 to the parking position 910 is equal to or less than a threshold, the driver is notified that the parking position 910 is close. As a result, the parking assistance method of this embodiment has the functions of the first embodiment and can make the driver easily know the appropriate timing to perform an operation to start automatic parking.

[0145] (Variation 1) In the above-described embodiments, examples of the display on the head-up display and the display device 120 have been described, but the manner in which the driving start position 900 or the parking start possible area 801 is displayed is not limited to the above-described examples.

[0146] FIG. 15 is a diagram showing an example of an image indicating a driving start position 900 displayed on a head-up display according to the first modification. The output control unit 105 of this modification displays a first guide image 91b indicating the driving start position on the windshield 180, which functions as a head-up display. In this modification, the first guide image 91b is in the shape of an arrow, and the starting point of the arrow corresponds to the driving start position. The arrow extends along the driving route 80. This display mode allows the driver to understand the direction in which the vehicle 1 will travel by autonomous driving from the driving start position. The arrow-shaped guide image may be displayed superimposed on the front view 21 or the top view 22 on the display device 120.

[0147] (Variation 2) 16 is a diagram showing an example of an image showing a travel start position 900 displayed on the display device 120 according to Modification 2. In the example shown in Fig. 16, the second guide image 92c on the front view 21 and the second guide image 92d on the top view 22 are both dashed lines indicating a position corresponding to the longitudinal direction of the vehicle body 12 when the vehicle body 12 of the vehicle 1 stops at the travel start position. In this display mode, the driver is more likely to be aware of driving the vehicle 1 so that the vehicle body 12 is positioned between the two dashed lines.

[0148] The second guide images 92c and 92d may be solid lines instead of dashed lines. Similarly, the first guide image may be displayed on a head-up display using dashed or solid lines that indicate a position corresponding to the longitudinal direction of the vehicle body 12.

[0149] (Variation 3) Furthermore, the first guide image and the second guide image do not have to have a shape that follows the shape of the vehicle body 12, and may be, for example, a circle or an oval.

[0150] Fig. 17 is a diagram showing an example of an image showing a travel start position 900 displayed on a display device 120 according to Modification 3. As shown in Fig. 17, a second guide image 92e on the front view 21 and a second guide image 92f on the top view 22 are ellipses corresponding in size to the body 12 of the vehicle 1. Note that a circular or elliptical first guide image may also be displayed on a head-up display.

[0151] (Variation 4) Furthermore, the guidance information may be an image that clearly indicates the direction in which the vehicle 1 should face in order to perform autonomous driving. For example, FIG. 18 is a diagram showing an example of a second guidance image 92g displayed on the display device 120 according to Modification Example 4. In the example shown in FIG. 18, the second guidance image 92g on the top view 22 includes an arrow image 921a that indicates the direction in which the vehicle 1 should face from the driving start position 900. Therefore, when the user manually drives the vehicle 1 to the driving start position 900, the user can easily understand the direction in which the vehicle 1 should face from the driving start position 900.

[0152] (Variation 5) The display mode for indicating the direction in which the vehicle 1 should face is not limited to the one exemplified in Modification 4. Fig. 19 is a diagram showing an example of a second guide image 92h displayed on the display device 120 according to Modification 5. In the example shown in Fig. 19, the second guide image 92h on the top view 22 represents the orientation of the vehicle 1 at the travel start position 900 and the change in orientation of the vehicle 1 along the travel route 80 extending from the travel start position 900. This display mode allows the user to easily understand the direction in which the vehicle 1 will travel due to autonomous driving starting from the travel start position 900.

[0153] Although Figures 18 and 19 show an example of a top view 22, the output control unit 105 may also display an arrow image 921a indicating the direction in which the vehicle 1 should face, or a second guide image 92h indicating a change in the direction of the vehicle 1, in the front view 21 or a head-up display.

[0154] (Variation 6) Fig. 20 is a diagram showing an example of a first guide image 91c displayed on a head-up display according to Modification Example 6. In the example shown in Fig. 20, the first guide image 91c shows the direction in which the vehicle 1 should face along the travel route 80 in the traveling direction of the vehicle 1. The first guide image 91c may also be an image that shows the route for the vehicle 1 to travel from its current position to the travel start position 900, together with the direction in which the vehicle 1 should face.

[0155] (Variation 7) In addition, in each of the above-described embodiments, the parking assistance device 100 uses the driving route in the teacher driving during learning as the driving route 80 for automatic driving in the home zone parking. However, the learning method is not limited to this.

[0156] For example, the teacher driving may be a driving in which the driver manually drives the vehicle 1 from the parking position 910 to a teacher driving end position outside the parking position. In this case, the learning unit 103 generates the driving route 80 for home zone parking by reproducing the teacher driving route in the reverse direction.

[0157] For example, the driver starts supervised driving from a state in which vehicle 1 is parked in parking position 910 within garage 920, and drives vehicle 1 forward to exit garage 920. Then, the driver drives vehicle 1 that has exited garage 920 further forward to a desired supervised driving end position, where vehicle 1 is stopped. Learning unit 103 then records the supervised driving based on steering, braking, acceleration / deceleration during the supervised driving, and feature points extracted from the surrounding image. Based on the recorded supervised driving, learning unit 103 generates a driving route 80 in which vehicle 1 travels backward from the supervised driving end position to parking position 910 within garage 920. In this method, the supervised driving end position becomes the driving start position 900 for the automatic driving. This method is also referred to as replaying the driving route in reverse.

[0158] Generally, in manual driving, it is easier to have vehicle 1 leave garage 920 than to have vehicle 1 enter garage 920. With such a learning method, even if the driver is not adept at reversing vehicle 1 into garage 920, it is possible to generate driving route 80 for home zone parking by supervised driving in which the driver drives vehicle 1 forward to leave garage 920. Note that, although this modified example has been described as an example in which the driver drives vehicle 1 forward in supervised driving, the driver may also drive vehicle 1 backward in supervised driving.

[0159] (Variation 8) In each of the above-described embodiments, an example of the first sensor information is a peripheral image captured around the vehicle 1 during the supervised driving. Also, in each of the above-described embodiments, an example of the second sensor information is a peripheral image captured around the vehicle 1 before or during the vehicle 1's automatic driving based on the learned driving route. However, the first sensor information and the second sensor information are not limited to these examples.

[0160] For example, the detection results by various sensors such as the wave transmitter / receiver 15 may be an example of the first sensor information or the second sensor information. The vehicle 1 may also be equipped with a distance measuring device such as LiDAR (Light Detection and Ranging, or Laser Imaging Detection and Ranging) or RADAR (Radio Detection and Ranging). In this case, the detection results of the surroundings of the vehicle 1 by the LiDAR or RADAR may be an example of the first sensor information or the second sensor information. The first sensor information and the second sensor information may also be a combination of two or more of a surrounding image, the detection results by the wave transmitter / receiver 15, the detection results by the LiDAR, and the detection results by the RADAR. The first sensor information and the second sensor information may also include information collected by other devices.

[0161] (Fourth embodiment) In the fourth embodiment, the parking assistance device 100 changes the display mode of the guidance information before and after the vehicle 1 reaches a position where it can merge with the travel route 80.

[0162] The configuration of vehicle 1 in this embodiment is the same as the configuration in the first embodiment described with reference to Figures 1 and 2. The hardware configuration and functional blocks of parking assistance device 100 in this embodiment are the same as those in the first embodiment described with reference to Figures 4 and 5. As in the first embodiment, parking assistance device 100 in this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0163] Furthermore, the acquisition unit 101, extraction unit 102, learning unit 103, vehicle control unit 106, and reception unit 107 of this embodiment have the same functions as those of the first embodiment.

[0164] The output control unit 105 has the functions of the first embodiment and changes the display mode of the guidance information before and after the vehicle 1 reaches a position where it can merge with the travel route 80. The change in the display mode of the guidance information will be described with reference to Figs. 21 to 23.

[0165] 21 is a diagram showing an example of guidance information that is displayed before vehicle 1 according to the fourth embodiment reaches a position where it can merge onto travel route 80. In the example shown in Fig. 21, both front view 21 and top view 22 are displayed on display device 120, but it is also possible to display only one of front view 21 and top view 22.

[0166] 21 is displayed when, for example, the estimation unit 104 determines that the driving start position 900 or an area where the vehicle 1 can merge onto the driving route 80 is within a specified distance from the vehicle 1. This embodiment is applicable to both cases where the vehicle 1 starts autonomous driving from the start position of the driving route 80 and cases where the vehicle 1 merges halfway along the driving route 80.

[0167] 21, the output control unit 105 displays, as guidance information, a driving route image 800, a driving area image 930a, and a second vehicle image 2b representing the position of the body 12 of the vehicle 1 on the top view 22. The output control unit 105 moves the display position of the second vehicle image 2b on the top view 22 as the vehicle 1 moves.

[0168] The output control unit 105 also displays a traveling area image 930b on the front view 21. The output control unit 105 may also display a first vehicle image 2a representing the position of the body 12 of the vehicle 1 on the front view 21.

[0169] Furthermore, the output control unit 105 displays a start button 121 on the display device 120. The start button 121 is an image button for instructing the start of automatic parking based on the travel route 80. Note that the display text and shape of the start button 121 are not limited to the example shown in FIG.

[0170] The driving route image 800 is an image that represents the driving route 80 during teacher driving. More specifically, the driving route image 800 is a line that indicates the movement history of the reference point of the vehicle 1 when the vehicle 1 travels along the driving route 80. The reference point of the vehicle 1 is, for example, the midpoint of the line connecting the left and right rear tires 13r of the vehicle 1, but is not limited to this.

[0171] The driving area image 930a is an image representing the driving area through which the outline of the vehicle 1 passes when the vehicle 1 travels along the driving route 80. The driving area image 930a may be a strip-shaped image with a width equal to the width of the vehicle 1 and centered on the driving route 80, or may be a strip-shaped image showing a circumscribed area, which is an area through which the outline of the vehicle 1 actually passed during teacher driving. The circumscribed area also includes the area through which the vehicle body 12 passes due to the difference between the inner and outer wheel diameters of the front tires 13f and rear tires 13r of the vehicle 1, etc.

[0172] The travel area is an example of an area and a target position where the vehicle 1 can merge onto the travel route 80 in this embodiment. The travel area images 930a and 930b are an example of guidance information in this embodiment.

[0173] Furthermore, the output control unit 105 may prompt the driver to move the vehicle 1 to the position represented by the traveling area images 930a, 930b by displaying a message M1 such as "Please park at the displayed parking start position" on the display device 120. The content and display position of the message M1 are not limited to the example shown in Fig. 21. Furthermore, the output control unit 105 may output the message M1 as a voice message from a speaker (not shown).

[0174] 21, a part of the body 12 of the vehicle 1 overlaps the travel area, but the vehicle 1 has not yet reached a position where it can merge with the travel route 80. In this case, the output control unit 105 displays the start button 121 in a state where it cannot be pressed. For example, the output control unit 105 displays the start button 121 in a grayed-out state. Furthermore, when a user operates the start button 121 that is displayed in a state where it cannot be pressed, the reception unit 107 does not accept the operation of the user.

[0175] Furthermore, when the vehicle 1 has not yet reached a position where it can merge with the travel route 80, the output control unit 105 causes the display device 120 to display the travel area images 930a and 930b in a first color. The first color is not particularly limited, but may be orange, for example. The first color is an example of a first display mode.

[0176] FIG. 22 is a diagram showing an example of guidance information displayed after the vehicle 1 according to the fourth embodiment reaches a position where it can merge with the travel route 80. In FIG.

[0177] In the example shown in Fig. 22, the vehicle 1 has reached a position where it can merge with the travel route 80. In this case, the output control unit 105 causes the display device 120 to display the travel area images 930a and 930b in a second color. The second color is not particularly limited, but is a color different from the first color, for example, green. The second color is an example of a second display mode.

[0178] Note that the change in color of the driving area images 930a, 930b is an example of a change in the display mode of the driving area images 930a, 930b, and the change in the display mode is not limited to this. For example, the output control unit 105 may cause the driving area images 930a, 930b to flash until the vehicle 1 reaches a position where it can merge with the driving route 80, and may stop the flashing after the vehicle 1 reaches a position where it can merge with the driving route 80.

[0179] Furthermore, when the vehicle 1 reaches a position where it can merge with the travel route 80, the output control unit 105 displays the start button 121 in a pressable state. For example, when the vehicle 1 reaches a position where it can merge with the travel route 80, the output control unit 105 displays the start button 121, which has been grayed out, in a normal state.

[0180] Furthermore, when the user operates the start button 121 that is displayed in a pressable state, the reception unit 107 receives the user's operation. When the start button 121 is pressed by the user, the vehicle control unit 106 starts automatic parking based on the driving route 80. Note that the user's operation to instruct the start of automatic parking based on the driving route 80 is not limited to pressing the start button 121, and may be voice input, operation of a physical button, or the like.

[0181] 23 is a diagram showing an example of guidance information displayed after the vehicle 1 according to the fourth embodiment starts automatic parking along the travel route 80. In the example shown in FIG. 23, when the vehicle 1 starts automatic parking along the travel route 80, the output control unit 105 ends the display of the travel area image 930a in the top view 22.

[0182] The output control unit 105 also continues to display the driving route image 800. As shown in FIG. 23, the output control unit 105 may display a parking position image 911 representing a parking position 910 on the top view 22. By having the output control unit 105 display the driving route image 800 and the parking position image 911, the user can understand the planned driving route of the vehicle 1 during automatic driving. Note that the output control unit 105 may also display the parking position image 911 before a user operation is input to instruct the start of automatic parking. For example, the output control unit 105 may display the parking position image 911 on the top view 22 shown in FIGS. 21 and 22.

[0183] Furthermore, after the start button 121 is pressed, the output control unit 105 displays the start button 121 in a display mode different from that before the start button 121 was pressed. For example, the output control unit 105 may change the color of the start button 121 before and after the start button 121 is pressed.

[0184] In the example shown in Figure 23, the output control unit 105 continues to display the driving area image 930b on the front view 21 even after the vehicle 1 has started automatic parking along the driving route 80, but this display may also be terminated.

[0185] Although the display device 120 is shown as an example in FIGS. 21 to 23, the output control unit 105 may also display the guidance information on a head-up display.

[0186] The estimation unit 104 of this embodiment has the functions of the first embodiment and determines whether the vehicle 1 has reached a position where it can merge with the travel route 80. For example, the estimation unit 104 determines that the vehicle 1 has reached a position where it can merge with the travel route 80 when the position and orientation of the vehicle 1 satisfy specified conditions.

[0187] The specified condition for the position of vehicle 1 is, for example, that the distance from the current position of the reference point of vehicle 1 to the travel route 80 is within a specified distance. Also, the specified condition for the orientation of vehicle 1 is, for example, that the difference between the current orientation of vehicle 1 and the orientation of vehicle 1 when vehicle 1 is located at a position on the travel route 80 closest to the reference point of vehicle 1 is equal to or less than a specified angle.

[0188] In this embodiment, the specified condition for the position of the vehicle 1 and the specified condition for the orientation of the vehicle 1 are AND conditions for determining that the vehicle 1 has reached a position where it can merge with the travel route 80. In other words, the vehicle control unit 106 can start automatic parking when the position and orientation of the vehicle 1 satisfy the specified conditions, but cannot start dynamic parking when the position and orientation of the vehicle 1 do not satisfy the specified conditions. Therefore, when the position and orientation of the vehicle 1 do not satisfy the specified conditions, the driver must manually move the vehicle 1 to a position where it can merge with the travel route 80. The values ​​of the specified distance and the specified angle are not particularly limited and vary depending on the steering characteristics of the vehicle 1, the size of the vehicle body 12, etc.

[0189] Here, a method for determining whether or not the vehicle 1 has reached a position where it can merge with the travel route 80 will be described with reference to FIGS. 24 and 25. FIG.

[0190] Fig. 24 is a diagram showing an example of a state before the vehicle 1 according to the fourth embodiment reaches a position where it can merge with the travel route 80. Fig. 25 is a diagram showing an example of a state after the vehicle 1 according to the fourth embodiment reaches a position where it can merge with the travel route 80. In Figs. 24 and 25, the positional relationship between the vehicle 1 and the travel route 80 is explained using a top view 22.

[0191] 24, the position of the vehicle 1 will be described based on a reference point 30, which is the midpoint of the line connecting the left and right rear tires 13r of the vehicle 1. The estimation unit 104 estimates a distance d10 between the reference point 30 of the vehicle 1 and a position 81 on the travel route 80 that is closest to the reference point 30, as the distance from the current position of the reference point 30 of the vehicle 1 to the travel route 80.

[0192] The estimation unit 104 also uses the orientation of a straight line 41 that passes through the reference point 30 of the vehicle 1 and perpendicularly intersects with a line connecting the left and right rear tires 13r of the vehicle 1 as the orientation of the vehicle 1. In FIG. 24 , the orientation of the vehicle 1 when the vehicle 1 is located at position 81 on the travel path 80 is represented as a straight line 42. The estimation unit 104 estimates the angle θ formed by the straight lines 41 and 42 as the difference between the current orientation of the vehicle 1 and the orientation of the vehicle 1 when the vehicle 1 is located at a position closest to the reference point of the vehicle 1 on the travel path 80. The estimation unit 104 may also detect the difference between the current orientation of the vehicle 1 and the orientation of the vehicle 1 during teacher travel as the difference in the heading direction of the vehicle 1.

[0193] In the example shown in FIG. 24 , the distance d10 from the position of the reference point 30 of the vehicle 1 to the travel route 80 is longer than a specified distance, and the angle θ between the straight lines 41 and 42 is larger than a specified angle. With such a position and orientation, the vehicle control unit 106 cannot start automatic parking. In this case, the estimation unit 104 determines that the vehicle 1 has not reached a position where it can merge with the travel route 80. As described above, the specified condition for the position of the vehicle 1 and the specified condition for the orientation of the vehicle 1 are AND conditions. Therefore, even if one of the conditions is satisfied, the estimation unit 104 determines that the vehicle 1 has not reached a position where it can merge with the travel route 80 if the other condition is not satisfied. This determination process is an example of a determination step in this embodiment.

[0194] 25, as the vehicle 1 moves, the distance d10 between the reference point 30 of the vehicle 1 and the position 81 on the travel route 80 that is closest to the reference point 30 becomes shorter than in the example shown in Fig. 24. Also, the angle θ between the straight lines 41 and 42 becomes smaller than in the example shown in Fig. 24.

[0195] 25, it is assumed that the distance d10 from the position of the reference point 30 of the vehicle 1 to the travel route 80 is equal to or less than a specified distance, and the angle θ between the straight lines 41 and 42 is greater than a specified angle. In this case, the estimation unit 104 determines that the vehicle 1 has reached a position where it can merge with the travel route 80. In addition, in this case, the vehicle control unit 106 can start automatic parking.

[0196] 24 and 25, the process of determining whether the vehicle 1 in the first to third embodiments has stopped at the travel start position 900 may also be performed using the position of the reference point 30 of the vehicle 1 and the orientation of the vehicle 1. For example, the estimation unit 104 may determine that the vehicle 1 has stopped at the travel start position 900 if the distance between the current reference point 30 of the vehicle 1 and the position of the reference point 30 when the vehicle 1 is located at the travel start position 900 during teacher-guided travel is within a specified distance and if the difference between the current orientation of the vehicle 1 and the orientation of the vehicle 1 when the vehicle 1 is located at the travel start position 900 during teacher-guided travel is equal to or smaller than a specified angle.

[0197] Furthermore, in the first to third embodiments, when the estimation unit 104 determines that the parking start possible area 801 is within a specified distance from the vehicle 1, the output control unit 105 displays the driving start position 900 or the parking start possible area 801 on the head-up display or the display device 120. However, in this processing, the condition for displaying the driving start position 900 or the parking start possible area 801 may further be added such that the orientation of the vehicle 1 satisfies a specified condition.

[0198] Next, the flow of the parking assistance process executed by the parking assistance device 100 of this embodiment configured as above will be described.

[0199] FIG. 26 is a flowchart showing an example of the flow of the parking assistance process executed by the parking assistance device 100 according to the fourth embodiment.

[0200] The process from the process of acquiring a surrounding image in S1 shown in FIG. 26 to the process of estimating the driving start position for automatic parking in S4 is the same as in the first embodiment.

[0201] Then, the estimation unit 104 determines whether or not there is an area within a specified distance from the vehicle 1 where the vehicle 1 can merge onto the travel route 80, based on the estimated position of the vehicle 1 and the travel start position 900 (S301). Note that in the processing of S5 in the processing flow of the first to third embodiments, whether or not there is an area within a specified distance from the vehicle 1 where the vehicle 1 can merge onto the travel route 80 may also be determined, as in S301.

[0202] 21, the output control unit 105 causes the display device 120 to display the driving route image 800 and the driving area images 930a and 930b in the first display mode. The output control unit 105 also causes the display device 120 to display the start button 121 in a state where it cannot be pressed (S302). At this time, the output control unit 105 may cause a speaker (not shown) to output a message M1 as a voice message, urging the driver to move the vehicle 1 to the positions represented by the driving area images 930a and 930b.

[0203] Then, the estimation unit 104 determines whether the position and orientation of the vehicle 1 satisfy specified conditions based on the feature points extracted from the acquired surrounding image and the driving route information read from the memory unit 108 (S303).

[0204] If the position and orientation of the vehicle 1 do not satisfy the specified conditions (S303 "No"), the estimation unit 104 determines that the vehicle 1 has not reached a position where it can merge with the travel route 80. In this case, the processing of S303 is repeatedly executed until the driver manually drives the vehicle 1 to a position where it can merge with the travel route 80.

[0205] Furthermore, if the position and orientation of the vehicle 1 satisfy the specified conditions (S303 "Yes"), the estimation unit 104 determines that the vehicle 1 has reached a position where it can merge with the travel route 80. In this case, the output control unit 105 changes the travel area images 930a and 930b to the second display mode, as shown in Fig. 22. Furthermore, the output control unit 105 causes the display device 120 to display the start button 121 in a state where it can be pressed (S304).

[0206] Furthermore, if the start button 121, which is displayed in a depressible state, is not pressed by the user (S305 "No"), the reception unit 107 waits for an operation by the user.

[0207] Then, when the start button 121, which is displayed in a pressable state, is pressed by the user (S305 "Yes"), the reception unit 107 receives an operation by the user to instruct the start of automatic parking based on the travel route 80. The reception unit 107 sends a notification to the output control unit 105 that the instruction to start automatic parking based on the travel route 80 has been received. In this case, the output control unit 105 ends the display of the travel area image 930a, as shown in FIG. 23, and displays the parking position image 911 on the display device 120 (S306). In this case, the vehicle control unit 106 starts automatic driving (S8). Here, the processing of this flowchart ends.

[0208] In this way, the parking assistance device 100 of this embodiment changes the display mode of the guidance information before and after the vehicle 1 reaches a position where it can merge onto the driving route 80, thereby achieving the same effect as the first embodiment and allowing the driver of the vehicle 1 to easily understand whether or not the vehicle 1 has reached a position where it can merge onto the driving route 80.

[0209] Furthermore, the parking assistance device 100 of this embodiment displays the driving area images 930a, 930b in a first color on the display device 120 before the vehicle 1 reaches a position where it can merge onto the driving route 80, and displays the driving area images 930a, 930b in a second color different from the first color on the display device 120 after the vehicle 1 reaches a position where it can merge onto the driving route 80. Therefore, the parking assistance device 100 of this embodiment makes it easy for the driver of the vehicle 1 to visually recognize whether or not the vehicle 1 has reached a position where it can merge onto the driving route 80.

[0210] Furthermore, when the position and orientation of the vehicle 1 satisfy specified conditions, the parking assistance device 100 of this embodiment determines that the vehicle 1 has reached a position where it can merge with the driving route 80. Therefore, the parking assistance device 100 of this embodiment can determine with high accuracy whether the vehicle 1 has reached a state where it can be automatically driven by the vehicle control unit 106.

[0211] The configuration of the parking assistance device 100 of this embodiment may be combined with the configurations of the above-described modified examples 1 to 8.

[0212] (Fifth embodiment) In this fifth embodiment, automatic parking based on a driving route 80 recorded by a method other than supervised driving will be described.

[0213] The configuration of vehicle 1 in this embodiment is the same as the configuration in the first embodiment described with reference to Figures 1 and 2. The hardware configuration and functional blocks of parking assistance device 100 in this embodiment are the same as those in the first embodiment described with reference to Figures 4 and 5. As in the first embodiment, parking assistance device 100 in this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0214] Furthermore, the acquisition unit 101, extraction unit 102, estimation unit 104, and reception unit 107 of this embodiment have the same functions as those of the first embodiment.

[0215] The vehicle control unit 106 of this embodiment has a function of automatic parking by automatic driving. More specifically, the vehicle control unit 106 generates a driving route for parking the vehicle 1 at a parking target position determined by the user, and performs automatic parking by moving the vehicle 1 to the parking target position based on the generated driving route. Note that the automatic parking by the vehicle control unit 106 of this embodiment can employ known technology. The parking operation by automatic driving is an example of a parking operation that has been performed in the past in this embodiment.

[0216] The target parking position during autonomous driving may be a section separated by a white line or the like, or may be a space without any markings such as a white line. If there is no marking such as a white line, the user may input the target parking position on the front view 21 or top view 22 displayed on the display device 120. The reception unit 107 receives a user operation to input the target parking position.

[0217] The learning unit 103 of this embodiment stores in the memory unit 108 the driving route 80 traveled by the vehicle 1 during automatic parking by the vehicle control unit 106, the driving start position 900, and the parking target position. For example, the learning unit 103 learns changes in the position of the vehicle 1 based on changes in feature points extracted from multiple surrounding images captured during automatic parking. The learning unit 103 also learns the speed, steering angle, and braking operation of the vehicle 1 during automatic parking. The learning unit 103 also defines the environment around the vehicle 1 as a map based on feature points extracted from surrounding images captured during automatic driving, and stores the map in the memory unit 108.

[0218] In addition, the learning unit 103 may store in the memory unit 108 only the driving route 80, driving start position 900, and parking target position for the automatic parking operation selected by the user as the one to be stored among the multiple times the automatic parking operation has been performed, and discard information about the driving route 80, driving start position 900, and parking target position for the operation not selected by the user as the one to be stored.

[0219] For example, the learning unit 103 may temporarily store in the memory unit 108 the driving route 80, driving start position 900, and parking target position traveled by the vehicle 1 during automatic parking by the vehicle control unit 106, and then, after automatic parking is completed, if the user selects the driving route 80, driving start position 900, and parking target position as driving route information to be used next time onwards.

[0220] The output control unit 105 of this embodiment displays, on a head-up display or a display device 120, guidance information based on driving route information recorded by the vehicle control unit 106 during past automatic parking.

[0221] Fig. 27 is a diagram showing an example of guidance information according to the fifth embodiment. In the example shown in Fig. 27, the output control unit 105 causes the display device 120 to display a top view 22. On the top view 22, a second vehicle image 2b, a driving area image 930a showing the driving area through which the outline of the vehicle 1 will pass when driving along a driving route 80 in a previous automatic parking performed by the vehicle control unit 106, and a parking position image 911 showing a parking position 910 are displayed.

[0222] The output control unit 105 may display the front view 21 or the driving route image 800.

[0223] Furthermore, the output control unit 105 of this embodiment changes the display mode of the guidance information before and after the vehicle 1 reaches a position where it can merge with the travel route 80, as in the fourth embodiment.

[0224] 27, as in the example shown in Fig. 22, the vehicle 1 has reached a position where it can merge with the travel route 80, so the output control unit 105 displays the travel area images 930a and 930b in the second display mode. Also, if the vehicle 1 has not yet reached a position where it can merge with the travel route 80, the output control unit 105 displays the travel area images 930a and 930b in the first display mode.

[0225] In this way, the parking assistance device 100 of this embodiment can achieve the same effect as the first embodiment, even when performing automatic parking based on a driving route 80 recorded by a parking operation performed by automatic driving in the past, and can also allow the driver of the vehicle 1 to easily understand whether the vehicle 1 has reached a position where it can merge with the driving route 80.

[0226] The configuration of the parking assistance device 100 of this embodiment may be combined with the configurations of the above-described modified examples 1 to 8.

[0227] In this embodiment, the parking assistance device 100 stores the driving route 80 for parking operation by autonomous driving, but the learning unit 103 of the parking assistance device 100 may store information on at least one of the driving route 80 and the parking target position in the storage unit 108. For example, the learning unit 103 may store only the parking target position for automatic parking without storing the driving route 80. For example, the parking target position is a position set by the user as the parking target position for the vehicle 1 during parking operation by autonomous driving.

[0228] In this case, the vehicle control unit 106 generates a driving route for automatically driving the vehicle 1 from the current position of the vehicle 1 to a previously registered parking target position. The output control unit 105 also displays guidance information based on the driving route generated by the vehicle control unit 106 on a head-up display or the display device 120. For example, the output control unit 105 may display a strip-shaped image representing a driving area along the generated driving route.

[0229] Furthermore, for example, the learning unit 103 may store the parking target position and the travel start position 900 for automatic parking. In this case, the output control unit 105 may display guidance information based on the travel start position 900 for automatic parking on the head-up display or the display device 120.

[0230] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0231] 1 vehicle 2a First vehicle image 2b Second vehicle image 12 Body 13f front tire 13r rear tire 13 wheels 15, 15f, 15r Transmitting and receiving unit 16 Imaging device 21 Front view 22 Top View 30 reference points 80 Driving Route 91a, 91b, 91c First guide image 92a, 92b, 92c, 92d, 92e, 92f, 92g, 92h Second guide image 100 Parking assistance device 101 Acquisition Department 102 Extraction part 103 Learning Department 104 Estimation part 105 Output control section 106 Vehicle control unit 107 Reception 108 Storage section 120 Display device 121 Start button 130a Driver's seat 140 steering wheel 141 Operation buttons 180 Windshield 190 Dashboard 800 driving route images 801 Parking start area 900 Running start position 901a Route guidance image 901b Area guide image 910 Parking Location 911 parking position image 920 Garage 930a, 930b Travel area images d10 distance M1 Message

Claims

1. A parking assistance method for automatically driving a vehicle based on supervised driving by a driver, comprising: storing information about a driving route in a teacher driving session by the driver; displaying information on a display device in a first display mode before the vehicle reaches a position where it can merge with the travel route; After the vehicle reaches a position where it can merge with the travel route, the information is displayed on the display device in a second display manner different from the first display manner; a parking position image showing the parking position is displayed on the display device; Parking assistance methods.

2. The information is a driving route image, The travel route image is a line showing a movement history of a reference point of the vehicle when the vehicle travels along the travel route. The parking assistance method according to claim 1 .

3. The information is a travel area image, The travel area image is an image representing a travel area through which an outer contour of the vehicle passes when the vehicle travels along the travel route. The parking assistance method according to claim 1 .

4. displaying the travel area image in a first color before the vehicle reaches a position where it can merge with the travel route, and displaying the travel area image in a second color different from the first color after the vehicle reaches a position where it can merge with the travel route. The parking assistance method according to claim 3 .

5. The traveling area image is a strip-shaped image having a width corresponding to the vehicle width of the vehicle. The parking assistance method according to claim 4.

6. The travel area image is a band-shaped image showing a circumscribed area through which the outer periphery of the vehicle passes during instructor travel. The parking assistance method according to claim 4.

7. The first color is orange and the second color is green. The parking assistance method according to claim 4.

8. The information is a vehicle image representing the position of the vehicle body. The parking assistance method according to claim 1 .

9. A parking assistance method for automatically driving a vehicle based on supervised driving by a driver, comprising: storing information about a driving route in a teacher driving session by the driver; before the vehicle starts automatic traveling based on the traveling route, displaying information on a display device in a first display mode; After the vehicle starts automatic traveling based on the traveling route, the information is displayed on a display device in a second display manner different from the first display manner; a parking position image showing the parking position is displayed on the display device; Parking assistance methods.

10. The information is a travel area image, The travel area image is an image representing a travel area through which an outer contour of the vehicle passes when the vehicle travels along the travel route. The parking assistance method according to claim 9.

11. The traveling area image is a strip-shaped image having a width corresponding to the vehicle width of the vehicle. The parking assistance method according to claim 10.

12. A parking assistance device that automatically drives a vehicle based on a teacher driving by a driver, a storage unit that stores information about a driving route in a teacher driving session by the driver; an output control unit that causes a display device to display information in a first display mode before the vehicle reaches a position where the vehicle can merge onto the travel route, and that causes the display device to display the information in a second display mode different from the first display mode after the vehicle reaches a position where the vehicle can merge onto the travel route; Equipped with The output control unit displays a parking position image indicating the parking position on the display device. Parking assistance device.

13. The information is a driving route image, The travel route image is a line showing a movement history of a reference point of the vehicle when the vehicle travels along the travel route.

13. The parking assistance device according to claim 12.

14. The information is a travel area image, The travel area image is an image representing a travel area through which an outer contour of the vehicle passes when the vehicle travels along the travel route.

13. The parking assistance device according to claim 12.

15. the output control unit displays the traveling area image in a first color before the vehicle reaches a position where the vehicle can merge onto the traveling route, and displays the traveling area image in a second color different from the first color after the vehicle reaches a position where the vehicle can merge onto the traveling route.

15. A parking assistance device according to claim 14.

16. The traveling area image is a strip-shaped image having a width corresponding to the vehicle width of the vehicle.

16. A parking assistance device according to claim 15.

17. The travel area image is a band-shaped image showing a circumscribed area through which the outer periphery of the vehicle passes during instructor travel.

16. A parking assistance device according to claim 15.

18. The first color is orange and the second color is green.

16. A parking assistance device according to claim 15.

19. The information is a vehicle image representing the position of the vehicle body.

13. The parking assistance device according to claim 12.

Images