Parking support method and parking support device

Abstract

To provide a parking support method and a parking support device that enable preparations for parking support to be started smoothly before a vehicle arrives near a target parking position.SOLUTION: A parking support method according to the present disclosure is a parking support method in which a parking support device performs automated travel of a vehicle on the basis of a travel route generated by teacher travel by a driver. In the parking support method, the parking support device acquires a position of the vehicle and an azimuth in a travel direction of the vehicle, and the parking support device receives operation to start parking support when the vehicle is located at a prescribed position associated with the travel route at a prescribed azimuth, on the basis of the position of the vehicle and the azimuth in the travel direction of the vehicle. The prescribed position is located within a range of a prescribed distance from a start position of the travel route.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to a parking assistance method and a parking assistance device. [Background technology]

[0002] Conventionally, parking assistance technologies that move a vehicle by automatic driving when parking a vehicle are known. One such parking assistance technology is a technology that learns a driving route based on a teacher driving by a driver and performs parking assistance using the learning results. This technology is used when a vehicle repeatedly parks in a fixed parking position, such as a parking lot at a user's home or workplace. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6022447 [Patent Document 2] Japanese Patent Application Publication No. 2019-137158 Summary of the Invention [Problem to be solved by the invention]

[0004] When a user uses such parking assistance, after the vehicle arrives near the desired parking position, the parking assistance function is activated and procedures for starting automatic parking are carried out, so it takes time for automatic parking to actually start. For example, Japanese Patent No. 6022447 describes that the system notifies the driver whether or not the vehicle can be parked in a parking space based on sensor data, and the driver receives the notification to start or cancel the parking process.

[0005] Furthermore, Japanese Patent Application Laid-Open Publication No. 2019-137158 mentions that preparation for parking assistance is started in advance, such as automatically informing the user that automatic parking is possible when the distance between the target parking position and the vehicle becomes equal to or less than a threshold. However, in this case, the user may feel annoyed because the parking assistance guidance is started even when the user does not actually intend to park.

[0006] The present disclosure provides a parking assistance method and a parking assistance device that can smoothly start preparations for parking assistance before a vehicle arrives near a target parking position. [Means for solving the problem]

[0007] A parking assistance method according to the present disclosure is a parking assistance method in which a parking assistance device automatically drives a vehicle based on a driving route generated by a driver through supervised driving, the parking assistance device acquiring a position and a direction of travel of the vehicle, and accepting an operation to start parking assistance when the vehicle is located at a specified position and in a specified direction associated with the driving route based on the position and the direction of travel of the vehicle, The specified position is within a specified distance from the start position of the driving route. [Effects of the Invention]

[0008] According to the parking assistance method and parking assistance device of the present disclosure, preparation for parking assistance can be smoothly started before the vehicle reaches the vicinity of the target parking position. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a vehicle equipped with a parking assistance device according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a configuration in the vicinity of the driver's seat of the vehicle according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of the parking assistance device according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of an environment to which the home zone parking according to the first embodiment is applied. [Figure 5] FIG. 5 is a block diagram showing an example of functions provided in the parking assistance device according to the first embodiment. [Figure 6] FIG. 6 is a diagram schematically illustrating an example of a plurality of driving histories according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a call origination point and a specified direction according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a registration format of a call origination point and a specified direction according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a state in which a vehicle is located at a second call point according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a state in which a vehicle is located at a first call point according to the first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a notification prompting an operation to start parking assistance according to the first embodiment. [Figure 12] FIG. 12 is a flowchart showing an example of the flow of a driving history recording process executed by the parking assistance device according to the first embodiment. [Figure 13] FIG. 13 is a flowchart showing an example of the flow of the call point registration process executed by the parking assistance device according to the first embodiment. [Figure 14] FIG. 14 is a flowchart showing an example of the flow of a process for notifying that parking assistance can be started and for parking assistance, which is executed by the parking assistance device according to the first embodiment. [Figure 15] FIG. 15 is a flowchart showing an example of the flow of the self-position estimation and parking assistance process executed by the parking assistance device according to the first embodiment. [Figure 16] FIG. 16 is a flowchart showing an example of the flow of processing for automatic start of self-position estimation and parking assistance executed by the parking assistance device according to the second embodiment. [Figure 17]FIG. 17 is a flowchart showing an example of the flow of the self-position estimation and parking assistance process executed by the parking assistance device according to the second embodiment. [Figure 18] FIG. 18 is a diagram illustrating an example of a guide image according to the third embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of the display timing of the guide image according to the first modification. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of a parking assistance method and a parking assistance device according to the present disclosure will be described with reference to the drawings.

[0011] (First embodiment) Fig. 1 is a diagram showing an example of a vehicle 1 equipped with a parking assistance device 100 according to the first embodiment. As shown in Fig. 1, the vehicle 1 includes a vehicle body 12 and two pairs of wheels 13 arranged along a predetermined direction on the vehicle body 12. The two pairs of wheels 13 include a pair of front tires 13f and a pair of rear tires 13r.

[0012] The front tire 13f shown in Fig. 1 is an example of a first wheel in this embodiment. The rear tire 13r is an example of a second wheel in this embodiment. Although the vehicle 1 shown in Fig. 1 has four wheels 13, the number of wheels 13 is not limited to this. For example, the vehicle 1 may be a two-wheeled vehicle.

[0013] The vehicle body 12 is coupled to wheels 13 and can move by the wheels 13. In this case, the predetermined direction in which the two pairs of wheels 13 are arranged is the traveling direction of the vehicle 1. The vehicle 1 can move forward or backward by switching gears (not shown) or the like. The vehicle 1 can also turn right or left by steering.

[0014] The vehicle body 12 has a front end F, which is the end on the front tire 13f side, and a rear end R, which is the end on the rear tire 13r side. The vehicle body 12 has a generally rectangular shape when viewed from above, and the four corners of the generally rectangular shape are sometimes called "ends." Although not shown in FIG. 1, the vehicle 1 also includes a display device, a speaker, an operation unit, a direction sensor, a GPS (Global Positioning System) antenna, and a GPS device.

[0015] A pair of bumpers 14 are provided at the front and rear ends F, R of the vehicle body 12 near the lower end of the vehicle body 12. Of the pair of bumpers 14, the front bumper 14f covers the entire front surface and part of the side surface near the lower end of the vehicle body 12. Of the pair of bumpers 14, the rear bumper 14r covers the entire rear surface and part of the side surface near the lower end of the vehicle body 12.

[0016] Wave transmitting and receiving units 15f, 15r that transmit and receive sound waves such as ultrasonic waves are disposed at predetermined ends of the vehicle body 12. For example, one or more wave transmitting and receiving units 15f are disposed on the front bumper 14f, and one or more wave transmitting and receiving units 15r are disposed on the rear bumper 14r. Hereinafter, unless otherwise specified, the wave transmitting and receiving units 15f, 15r will be simply referred to as the wave transmitting and receiving unit 15. Furthermore, the number and positions of the wave transmitting and receiving units 15 are not limited to the example shown in FIG. 1. For example, the vehicle 1 may be provided with wave transmitting and receiving units 15 on both the left and right sides.

[0017] In this embodiment, sonar using ultrasonic waves will be described as an example of the wave transmitting and receiving unit 15, but the wave transmitting and receiving unit 15 may also be radar that transmits and receives electromagnetic waves. Alternatively, the vehicle 1 may be equipped with both sonar and radar. Furthermore, the wave transmitting and receiving unit 15 may simply be referred to as a sensor.

[0018] The wave transmitting and receiving unit 15 detects obstacles around the vehicle 1 based on the results of transmitting and receiving sound waves or electromagnetic waves. The wave transmitting and receiving unit 15 also measures the distance between the vehicle 1 and obstacles around the vehicle 1 based on the results of transmitting and receiving sound waves or electromagnetic waves.

[0019] The vehicle 1 also includes a first imaging device 16a that captures an image in front of the vehicle 1, a second imaging device 16b that captures an image behind the vehicle 1, a third imaging device 16c that captures an image on the left side of the vehicle 1, and a fourth imaging device that captures an image on the right side of the vehicle 1. The fourth imaging device is not shown in the figure.

[0020] Hereinafter, when there is no need to distinguish between the first imaging device 16a, the second imaging device 16b, the third imaging device 16c, and the fourth imaging device, they will be simply referred to as imaging device 16. The positions and number of imaging devices are not limited to the example shown in FIG. 1. For example, the vehicle 1 may also be equipped with only two imaging devices, the first imaging device 16a and the second imaging device 16b. Alternatively, the vehicle 1 may have other imaging devices in addition to those in the above example.

[0021] The imaging device 16 is capable of capturing images of the surroundings of the vehicle 1, and is, for example, a camera that captures color images. The captured images captured by the imaging device 16 may be either moving images or still images. The imaging device 16 may also be a camera built into the vehicle 1, or may be a camera of a drive recorder that is retrofitted to the vehicle 1.

[0022] The vehicle 1 is also equipped with a parking assistance device 100. The parking assistance device 100 is an information processing device that can be installed in the vehicle 1, and is, for example, an ECU (Electronic Control Unit) or an OBU (On Board Unit) provided inside the vehicle 1. Alternatively, the parking assistance device 100 may be an external device installed near the dashboard of the vehicle 1. The parking assistance device 100 may also function as a car navigation device or the like.

[0023] Next, a description will be given of the configuration of the vicinity of the driver's seat of the vehicle 1 according to this embodiment. Fig. 2 is a diagram showing an example of the configuration of the vicinity of the driver's seat 130a of the vehicle 1 according to the first embodiment.

[0024] 2, the vehicle 1 has a driver's seat 130a and a passenger's seat 130b. In front of the driver's seat 130a, a windshield 180, a dashboard 190, a steering wheel 140, a display device 120, and operation buttons 141 are provided.

[0025] The display device 120 is a display provided on a dashboard 190 of the vehicle 1. For example, the display device 120 is located in the center of the dashboard 190 as shown in FIG. 2. The display device 120 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display device 120 may also function as a touch panel. The display device 120 is an example of a display unit in this embodiment.

[0026] The steering wheel 140 is provided in front of the driver's seat 130a and can be operated by the driver. The rotation angle of the steering wheel 140, i.e., the steering angle, is electrically or mechanically linked to the change in the direction of the front tires 13f, which are the steered wheels. The steered wheels may be the rear tires 13r, or both the front tires 13f and the rear tires 13r may be steered wheels.

[0027] The operation button 141 is a button that can accept an operation by a user. In this embodiment, the user is, for example, the driver of the vehicle 1. When the operation button 141 is pressed by the driver, it accepts an operation from the driver to, for example, start parking assistance. The position of the operation button 141 is not limited to the example shown in FIG. 2 , and it may be provided on the steering wheel 140, for example. The operation button 141 is an example of an operation unit in this embodiment. In addition, if the display device 120 also functions as a touch panel, the display device 120 may be an example of an operation unit. In addition, an operation terminal that can transmit a signal to the vehicle 1 from outside the vehicle 1, such as a tablet terminal, smartphone, remote controller, or electronic key (not shown), may be an example of an operation unit.

[0028] Next, a description will be given of the hardware configuration of the parking assistance device 100. Fig. 3 is a diagram showing an example of the hardware configuration of the parking assistance device 100 according to the first embodiment.

[0029] As shown in FIG. 3, the parking assistance device 100 has a hardware configuration that utilizes a normal computer, in which a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, an I / F (Interface) 11D, an HDD (Hard Disk Drive) 11E, etc. are interconnected by a bus 11F.

[0030] The CPU 11A is a calculation device that controls the entire ECU. The CPU 11A is an example of a processor in the parking assistance device 100 of this embodiment, and another processor or processing circuit may be provided instead of the CPU 11A.

[0031] The ROM 11B stores programs and the like that realize various processes performed by the CPU 11A.

[0032] The RAM 11C is, for example, a main storage device of the parking assistance device 100, and stores data necessary for various processes performed by the CPU 11A.

[0033] The I / F 11D is an interface for transmitting and receiving data. The I / F 11D transmits and receives data to and from other devices provided in the vehicle 1, such as the display device 120. The I / F 11D also acquires the heading of the vehicle 1 from the heading sensor 21.

[0034] The direction sensor 21 is a sensor that measures the direction of travel of the vehicle 1 from, for example, the difference in rotation between the left and right wheels 13 of the vehicle 1, geomagnetism, a gas rate gyro, an optical fiber gyro, etc. The method for determining the direction of travel of the vehicle 1 can be any known method and is not particularly limited.

[0035] Furthermore, the I / F 11D acquires location information indicating the current location of the vehicle 1 from the GPS device 22. The location information is, for example, latitude and longitude values indicating the absolute location of the vehicle 1.

[0036] The GPS device 22 is a device that identifies GPS coordinates that indicate the position of the vehicle 1 based on a GPS signal received by the GPS antenna 23. The GPS antenna 23 is an antenna that can receive GPS signals.

[0037] In addition, I / F11D may send and receive information between other ECUs installed in vehicle 1 via a CAN (Controller Area Network) within vehicle 1, or may communicate with an information processing device outside vehicle 1 via a network such as the Internet.

[0038] 3, the direction sensor 21, the GPS device 22, and the GPS antenna 23 are illustrated as not being included in the parking assistance device 100, but some or all of these may be included in the parking assistance device 100. Furthermore, in FIGS. 1 to 3, the display device 120 is illustrated as a device separate from the parking assistance device 100, but the display device 120 may be included in the parking assistance device 100.

[0039] The parking assistance device 100 of this embodiment learns a driving route based on supervised driving by the driver and provides parking assistance using the learning results. In other words, the parking assistance method executed by the parking assistance device 100 is a method of automatically driving the vehicle 1 based on a driving route generated by the supervised driving by the driver. This parking assistance method is effective in reducing the driver's parking effort when repeatedly parking in a fixed parking spot, such as the driver's garage at home, a contracted parking spot at an apartment complex, or a designated parking spot in a parking lot at work, for example. This type of parking assistance is called home zone parking.

[0040] Fig. 4 is a diagram showing an example of an environment to which the home zone parking according to the first embodiment is applied. In the example shown in Fig. 4, a parking space 910 located near a home 70 of the driver of the vehicle 1 is the parking position of the vehicle 1. In Fig. 4, directions are indicated with north up, south down, east to the right, and west to the left.

[0041] Parking space 910 faces road 40a. Road 40b connects to road 40a. Road 40c also exists near parking space 910, but road 40c does not connect to parking space 910.

[0042] 4, the driving routes that allow vehicle 1 to park in parking space 910 include a first driving route 80a that starts on the west side of parking space 910 and a second driving route 80b that starts on the east side of parking space 910. The first driving route 80a and the second driving route 80b are driving routes recorded by parking assistance device 100 based on supervised driving by the user. Hereinafter, the first driving route 80a and the second driving route 80b will be collectively referred to simply as driving route 80.

[0043] The first driving route 80a is a route that the vehicle 1 travels along from a first start position 900a on the road 40a to a parking space 910. The second driving route 80b is a route that the vehicle 1 travels along from a second start position 900b on the road 40a to the parking space 910. Hereinafter, the first start position 900a and the second start position 900b will be collectively referred to simply as the start position 900. Note that the number and shape of the driving routes 80 shown in FIG. 4 are merely examples and are not limited to these.

[0044] In this embodiment, the details of the functions of the parking assistance device 100 will be specifically described using the environment shown in FIG. 4 as an example.

[0045] FIG. 5 is a block diagram showing an example of functions provided in the parking assistance device 100 according to the first embodiment.

[0046] As shown in FIG. 5, the parking assistance device 100 of this embodiment includes an acquisition unit 101, a reception unit 102, a driving history recording unit 103, a registration processing unit 104, a change processing unit 105, a determination unit 106, an output control unit 107, an extraction unit 108, an estimation unit 109, a vehicle control unit 110, a route recording unit 111, and a memory unit 112.

[0047] The storage unit 112 is configured by, for example, a ROM 11B, a RAM 11C, or an HDD 11E. Although one storage unit 112 is included in the parking assistance device 100 in FIG. 5, multiple storage media may function as the storage unit 112.

[0048] The storage unit 112 stores programs and data used in various processes executed by the parking assistance device 100. For example, the program executed by the parking assistance device 100 of this embodiment has a modular configuration including the above-mentioned functional units (acquisition unit 101, reception unit 102, driving history recording unit 103, registration processing unit 104, change processing unit 105, determination unit 106, output control unit 107, extraction unit 108, estimation unit 109, vehicle control unit 110, and route recording unit 111), and as actual hardware, the CPU 11A reads and executes the program from the storage unit 112, whereby the above-mentioned units are loaded onto the RAM 11C, and the acquisition unit 101, reception unit 102, driving history recording unit 103, registration processing unit 104, change processing unit 105, determination unit 106, output control unit 107, extraction unit 108, estimation unit 109, vehicle control unit 110, and route recording unit 111 are generated on the RAM 11C. The processes implemented by the functional units of the parking assistance device 100 are also referred to as steps.

[0049] The program executed by the parking assistance device 100 of this embodiment is provided as a file in an installable or executable format recorded on a computer-readable recording medium such as a flash memory, CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0050] The program executed by the parking assistance device 100 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program executed by the parking assistance device 100 of this embodiment may be provided or distributed via a network such as the Internet. The program executed by the parking assistance device 100 of this embodiment may be provided by being pre-installed in ROM 11B or the like.

[0051] The acquisition unit 101 acquires the absolute position and heading direction of the vehicle 1. This acquisition process is an example of a first acquisition step in this embodiment. More specifically, the acquisition unit 101 acquires the absolute position of the vehicle 1 from the GPS device 22 via the I / F 11D. The acquisition unit 101 also acquires the heading direction of the vehicle 1 from the heading sensor 21 via the I / F 11D.

[0052] The acquisition unit 101 also acquires an image of the surroundings of the vehicle 1. Since the captured image is an image of the surroundings of the vehicle 1, it is referred to as a surrounding image in this embodiment. This acquisition process is an example of a second acquisition step in this embodiment. In the second acquisition step, the acquisition unit 101 may further acquire information regarding the distance between the vehicle 1 and objects surrounding the vehicle 1 during instructor driving.

[0053] The reception unit 102 receives various operations from the user. For example, when the operation button 141 is pressed after the travel route 80 has been registered, the reception unit 102 receives an operation by the user to start parking assistance. This reception process is an example of a first reception step in this embodiment.

[0054] The reception unit 102 also receives a user's operation to register a call origination point and a specified direction via, for example, the display device 120 and the I / F 11D. The call origination point and the specified direction will be described later. The reception unit 102 also receives a user's operation to modify the call origination point and the specified direction. This reception process is an example of a second reception step in this embodiment.

[0055] Furthermore, the reception unit 102 receives operations by the user to start and end recording of the supervised driving, for example, when the operation button 141 is pressed. In this embodiment, the start operation reception step, which is a process of receiving an operation to start recording of the supervised driving, and the end operation reception step, which is a process of receiving an operation to end recording of the supervised driving, are collectively referred to as the start / end operation reception step.

[0056] There is no particular limitation on the means by which the reception unit 102 receives a user operation. For example, if the display device 120 is a touch panel, the reception unit 102 may receive an operation to start or end recording of a supervised driving or to start parking assistance when an image button on the touch panel is pressed.

[0057] The driving history recording unit 103 records the absolute position and traveling direction of the vehicle 1 acquired in chronological order by the acquisition unit 101 in the storage unit 112 as the driving history of the vehicle 1. More specifically, when the vehicle 1 performs parking assistance, the driving history recording unit 103 records the chronological history of the absolute position of the vehicle 1 within a specified distance from the start position 900 of the driving route 80 used in the parking assistance, and the traveling direction direction of the vehicle 1 at each position, in association with the driving route 80 used in the parking assistance.

[0058] FIG. 6 is a diagram illustrating an example of a plurality of driving histories according to the first embodiment. In the example illustrated in FIG. 6, before using parking assistance along a first driving route 80a, the user manually drives the vehicle 1 along routes 81a, 82a, and 83a to a first start position 900a. The driving history recording unit 103 records the routes 81a, 82a, and 83a and the orientation of the vehicle 1 along the routes 81a, 82a, and 83a as driving history data, in association with the first driving route 80a. Furthermore, before using parking assistance along a second driving route 80b, the user manually drives the vehicle 1 along routes 81b, 82b, and 83b to a second start position 900b. The driving history recording unit 103 records the routes 81b, 82b, and 83b and the orientation of the vehicle 1 along the routes 81b, 82b, and 83b as driving history data, in association with the second driving route 80b.

[0059] Furthermore, if the vehicle 1 does not perform parking assistance, the driving history recording unit 103 deletes the records of the absolute position and heading direction of the vehicle 1 without saving them.

[0060] 5, the registration processing unit 104 identifies a call point and a specified direction at the call point based on multiple past driving histories of the vehicle 1, and registers the identified call point and the specified direction at the call point in association with the driving route 80. This registration processing is an example of a first registration step.

[0061] The call point is a location where the output control unit 107, which will be described later, outputs a notification to the user urging the user to perform an operation to start parking assistance. The call point is located within a specified distance from the start position 900 of the travel route 80, and is a location where the user is likely to park the vehicle 1 in the parking space 910 after the vehicle 1 passes that location. The specified distance is, for example, about 100 to 200 m, but is not limited to this. The call point is an example of a specified location in this embodiment.

[0062] The specified direction is the direction of travel of the vehicle 1 at the call point.

[0063] More specifically, the registration processing unit 104 identifies, as the call point, a location within a specified distance from the starting position of the driving route 80, among the locations that the vehicle 1 has commonly passed through in multiple past driving histories before the automatic driving was performed.

[0064] 6, the first call point 920a is illustrated as a location that the vehicle 1 commonly passes before reaching the first start position 900a of the first travel route 80a in the past travel history and that is approximately 100 to 200 m away from the first start position 900a. Also, in the past travel history, the vehicle 1 is facing south at the first call point 920a.

[0065] FIG. 7 is a diagram illustrating an example of a call point 920 and a specified direction according to the first embodiment. As shown in FIG. 7, the registration processing unit 104 registers the absolute position of the first call point 920a and the direction of travel of the vehicle 1, "south," in the storage unit 112 in association with the first travel route 80a. In other words, when the vehicle 1 is located at the first call point 920a and the direction of travel of the vehicle 1 is facing south, the vehicle 1 satisfies the conditions for notification by the output control unit 107, which will be described later. Note that in this embodiment, for ease of explanation, the direction of travel is expressed as north, south, east, and west. However, in practice, the registration processing unit 104 registers the direction of travel as a numerical value, such as 360 degrees clockwise, with north being 0 degrees.

[0066] 6 illustrates a second call point 920b as a location that, in the past travel history, the vehicle 1 commonly passed before reaching the second start position 900b of the second travel route 80b and that is approximately 100 to 200 meters away from the second start position 900b. In this case, as shown in FIG. 7, the registration processing unit 104 registers in the storage unit 112 the absolute location of the second call point 920b and the heading "west" of the vehicle 1 at the second call point 920b in association with the second travel route 80b. Hereinafter, when there is no particular need to distinguish between the first call point 920a and the second call point 920b, they will simply be referred to as the call point 920.

[0067] Fig. 8 is a diagram showing an example of a registration format for a call origination point 920 and a specified direction according to the first embodiment. As shown in Fig. 8, the latitude and longitude representing the absolute position of the call origination point 920, the direction representing the specified direction, and the travel route 80 are registered in association with each other. Note that, although Fig. 8 shows that one call origination point is registered for one travel route, multiple call origination points may be registered for one travel route.

[0068] 5, the change processing unit 105 changes the registered call point 920 and the specified direction based on a user operation to modify the call point 920 and the specified direction. This change processing is an example of a change step. For example, when the call point 920 and the specified direction are displayed on the display device 120 by the output control unit 107 (described later), the user can modify the call point 920 and the specified direction by a touch operation or the like.

[0069] The determination unit 106 determines whether the vehicle 1 is located facing a specified direction at a call point 920 registered in association with the travel route 80, based on the absolute position and heading direction of the vehicle 1. This determination process is an example of a determination step.

[0070] More specifically, the determination unit 106 determines that the vehicle 1 is located at the call point 920 facing in a specified direction if the difference between the acquired absolute position of the vehicle 1 and the call point 920 is within a first specified range, and the difference between the acquired direction of travel of the vehicle 1 and the specified direction is within a second specified range.

[0071] For example, the determination unit 106 compares the latitude and longitude of the acquired absolute position of the vehicle 1 with the latitude and longitude of the call point 920 registered in the memory unit 112. The first specified range is not particularly limited, but may be, for example, a range of about several meters. The determination unit 106 also compares the acquired heading of the traveling direction of the vehicle 1 with the specified heading registered in the memory unit 112. The second specified range is not particularly limited, but may be, for example, a difference of up to several degrees. The first specified range and the second specified range may be changeable by the user.

[0072] 9 is a diagram showing an example of a state in which the vehicle 1 is located at the second call point 920b according to the first embodiment. In the example shown in FIG. 9, the absolute position of the vehicle 1 is the same as that of the second call point 920b, out of the registered first call point 920a and second call point 920b. In addition, the heading of the vehicle 1's direction of travel is the same as the default heading "west" associated with the second call point 920b. In this case, the determination unit 106 determines that the vehicle 1 is located at the second call point 920b and is facing the specified heading.

[0073] FIG. 10 is a diagram illustrating an example of a state in which the vehicle 1 is located at the first call point 920a according to the first embodiment. In the example illustrated in FIG. 10, the absolute position of the vehicle 1 is the same as that of the first call point 920a, which is one of the registered first and second call points 920a and 920b. However, the direction of travel of the vehicle 1 is "north," which differs from the default direction of "south" associated with the first call point 920a. In this case, the determination unit 106 determines that the vehicle 1 is located at the first call point 920a but is not facing the specified direction. In this way, the determination unit 106 makes a determination based not only on the location but also on the direction of travel of the vehicle 1. This makes it possible to distinguish between a case in which the vehicle 1 leaves the parking space 910 and departs, and a case in which the vehicle 1 returns from a destination and heads toward the parking space 910.

[0074] 5, when the determination unit 106 determines that the vehicle 1 is located facing a specified direction at the call point 920, the output control unit 107 outputs a notification to prompt the user to perform an operation to start parking assistance. This output processing is an example of a first output step.

[0075] Fig. 11 is a diagram showing an example of a notification prompting an operation to start parking assistance according to the first embodiment. In the example shown in Fig. 11, the output control unit 107 causes the display device 120 to display a message M1 as a notification prompting an operation to start parking assistance. The content of the message M1 includes information that parking assistance can be started and a prompt for an operation, such as "Parking assistance can be started. Do you want to start?"

[0076] Furthermore, the output control unit 107 may cause the display device 120 to display a start button 1201 and a cancel button 1202. When the user presses the start button 1201, the reception unit 102 receives an operation by the user to start parking assistance. When the user presses the cancel button 1202, the reception unit 102 receives an operation by the user to cancel the start of parking assistance.

[0077] The notification method is not limited to a screen display, but may be a voice notification or the like.

[0078] Furthermore, the output control unit 107 displays various operation screens on the display device 120. For example, the output control unit 107 causes the display device 120 to display the call origination point 920 identified by the registration processing unit 104 and the specified direction.

[0079] Furthermore, when parking assistance starts, output control unit 107 causes display device 120 to display a guide image indicating start position 900 of travel route 80. The user is able to ascertain start position 900 of travel route 80 from the guide image displayed on display device 120. The display mode of the guide image is not particularly limited, and may be, for example, an image in which an image indicating start position 900 of travel route 80 is superimposed on an overhead image including vehicle 1 and the area around vehicle 1.

[0080] Returning to FIG. 5 , the extraction unit 108 extracts feature points from the peripheral image. The method by which the extraction unit 108 extracts feature points is not particularly limited, and any known method may be applied. For example, the extraction unit 108 extracts feature points using a method such as FAST (Features from Accelerated Segment Test) or ORB (Oriented FAST and Rotated BRIEF). Furthermore, when learning the driving route 80, the extraction unit 108 may preferentially record, among the extracted feature points, feature points that satisfy a specified condition. For example, among a plurality of consecutive peripheral images in time series, feature points extracted from an image in which the vehicle 1 traveled a longer distance during image capture may be preferentially selected as feature points.

[0081] The estimation unit 109 estimates the position and orientation of the vehicle 1 based on the surrounding image when the vehicle 1 is automatically traveling based on a travel route 80 under the control of a vehicle control unit 110 described later. This estimation process is an example of an estimation step.

[0082] For example, the estimation unit 109 estimates the position of the vehicle 1 by comparing feature points of a peripheral image captured during supervised driving with feature points of a current peripheral image. The position estimated by the estimation unit 109 may be a relative positional relationship with the start position 900 of the registered driving route 80, and does not have to be an absolute position. Note that the method by which the estimation unit 109 estimates the position and orientation of the vehicle 1 is not limited to this example.

[0083] The feature points of the surrounding image captured during supervised driving are an example of data generated from the surrounding image. The feature points of the surrounding image captured during supervised driving are stored as a feature point map in the storage unit 112 by the route recording unit 111 described below, for example. Specifically, the estimation unit 109 reads out the feature point map from the storage unit 112 and compares it with the feature points of the current surrounding image, thereby estimating the position of the vehicle 1.

[0084] The vehicle control unit 110 moves the vehicle 1 to the parking space 910 by automatic driving based on the driving route 80. The vehicle control unit 110 controls the steering, braking, and acceleration / deceleration of the vehicle 1 to cause the vehicle 1 to automatically drive along the registered driving route 80. This driving control method is also referred to as reproducing the driving route 80. The automatic driving process by the vehicle control unit 110 is an example of a driving control step.

[0085] The route recording unit 111 records the travel route 80 of the vehicle 1 during supervised travel. The route recording unit 111 estimates the position of the vehicle 1 during supervised travel, for example, based on changes in feature points extracted from multiple surrounding images and vehicle information about the vehicle 1, and identifies the travel route 80 from changes in the position over time.

[0086] More specifically, the route recording unit 111 identifies a change in the position of the vehicle 1 based on a time series change in feature points extracted from a plurality of surrounding images captured during supervised driving. The route recording unit 111 may also correct the position of the vehicle 1 identified from the feature points based on the acquired vehicle information.

[0087] In this embodiment, information that chronologically associates the driving route 80 for automatic driving, the speed, steering angle, braking operation of the vehicle 1 traveling on the driving route 80, and feature points extracted from multiple surrounding images captured as the vehicle 1 moves during supervised driving, is referred to as driving route information. The route recording unit 111 stores the driving route information in the storage unit 112. The driving route information is used by the vehicle control unit 110 during automatic driving, which will be described later. Note that the recording method of the driving route 80 and the definition of the driving route information are not limited to this example.

[0088] Furthermore, the route recording unit 111 defines the environment around the vehicle 1 as a map based on feature points extracted from the surrounding image captured during supervised driving, and stores the map in the storage unit 112. The process of recording the driving route 80 based on supervised driving may also be called a learning process.

[0089] Next, a flow of the driving history recording process executed by the parking assistance device 100 of this embodiment configured as above will be described.

[0090] 12 is a flowchart showing an example of the flow of the driving history recording process executed by the parking assistance device according to the first embodiment. As a prerequisite for this process, it is assumed that a driving route 80 based on a teacher driving has already been registered.

[0091] First, the acquisition unit 101 acquires the absolute position and heading of the vehicle 1 (S101).

[0092] Next, the driving history recording unit 103 determines whether parking assistance has been performed by the vehicle control unit 110, and if parking assistance has been performed (S102 "Yes"), records the absolute position and heading direction of the vehicle 1 acquired in chronological order by the acquisition unit 101 in the memory unit 112 as the driving history of the vehicle 1 (S103).

[0093] Furthermore, if parking assistance has not been performed (S102 "No"), the driving history recording unit 103 does not record the absolute position and traveling direction of the vehicle 1 acquired in chronological order by the acquisition unit 101. Here, the processing of this flowchart ends.

[0094] Next, the flow of the call point registration process executed by the parking assistance device 100 of this embodiment will be described.

[0095] FIG. 13 is a flowchart showing an example of the flow of the call point registration process executed by the parking assistance device 100 according to the first embodiment.

[0096] First, the reception unit 102 determines whether or not a call point registration start operation by the user has been received (S201). If a call point registration start operation by the user has not been received (S201 "No"), the reception unit 102 waits while repeating the process of S201.

[0097] When the reception unit 102 receives a user's operation to start registering a call point (S201 "Yes"), the registration processing unit 104 identifies the call point 920 and the specified direction from multiple past driving histories of the vehicle 1 stored in the memory unit 112 (S202).

[0098] Then, the output control unit 107 causes the display device 120 to display the call origination point 920 and the specified direction identified by the registration processing unit 104, for example, by superimposing them on a map image (S203).

[0099] Then, if the reception unit 102 receives a user's operation to modify the call point 920 or the specified direction (S204 "Yes"), the registration processing unit 104 changes the call point 920 and the specified direction in accordance with the user's operation (205), and registers the changed call point 920 and the specified direction in the memory unit 112 (S206).

[0100] Furthermore, if the reception unit 102 has not received a user's operation to correct the call origination point 920 and the specified direction (S204 "No"), the registration processing unit 104 registers the call origination point 920 and the specified direction identified in S202 in the storage unit 112 (S207). Here, the processing of this flowchart ends.

[0101] 13, the user's correction operation is accepted immediately after the call origination point 920 and the specified direction are identified, but the timing of the user's correction operation is not limited to this. For example, the user may correct the call origination point 920 and the specified direction after the start of operation using the registered call origination point 920 and the specified direction.

[0102] Next, a flow of processing for notifying that parking assistance can be started and parking assistance executed by the parking assistance device 100 of this embodiment will be described.

[0103] FIG. 14 is a flowchart showing an example of the flow of the process of notifying that parking assistance can be started and parking assistance executed by the parking assistance device 100 according to the first embodiment.

[0104] First, the acquisition unit 101 acquires the absolute position and heading of the vehicle 1 (S301).

[0105] Then, the determination unit 106 determines whether the vehicle 1 is located at the call point 920 and facing in the specified direction based on the acquired absolute position and heading direction of the vehicle 1 and the call points 920 and specified headings registered in the storage unit 112 (S302). If multiple call points 920 and heading directions are registered, the determination unit 106 compares each of the registered multiple call points 920 and heading directions with the acquired absolute position and heading direction of the vehicle 1.

[0106] If the determination unit 106 determines that the vehicle 1 is not located at the call origination point 920 or that the vehicle 1 is not facing in the specified direction (S302 "No"), the process returns to S301.

[0107] When the determination unit 106 determines that the vehicle 1 is located at the call point 920 and is facing in a specified direction (S302 "Yes"), the output control unit 107 outputs a notification prompting the user to operate to start parking assistance (S303). This notification allows the user to understand that the parking assistance device 100 is in a state where it can start parking assistance.

[0108] Then, for example, when the start button 1201 displayed on the display device 120 is pressed by the user, the reception unit 102 receives an operation to instruct the start of parking assistance (S304 "Yes"). In this case, the estimation unit 109, the vehicle control unit 110, etc. start self-location estimation and parking assistance processing (S305). The self-location estimation and parking assistance processing will be described in detail later.

[0109] Furthermore, if the reception unit 102 does not receive an operation to instruct the start of parking assistance, for example, if the user presses the cancel button 1202 displayed on the display device 120 (S304 "No"), the self-position estimation and parking assistance processing will not start, and the processing of this flowchart will end.

[0110] FIG. 15 is a flowchart showing an example of the flow of the self-position estimation and parking assistance process executed by the parking assistance device 100 according to the first embodiment.

[0111] First, the acquisition unit 101 acquires a surrounding image obtained by capturing an image of the surroundings of the vehicle 1 from the imaging device 16 (S401).

[0112] Next, the extraction unit 108 extracts feature points from the surrounding image acquired by the acquisition unit 101 (S402).

[0113] Next, the estimation unit 109 reads out the feature points of the surrounding image captured during the supervised driving from the memory unit 112, and estimates the position of the vehicle 1 based on the feature points and the feature points extracted from the surrounding image by the extraction unit 108 (S403).

[0114] Furthermore, the estimation unit 109 estimates the start position 900 of the driving route 80 for automatic parking based on the driving route 80 and the feature points extracted from the surrounding image by the extraction unit 108 (S404).

[0115] Then, the estimation unit 109 determines whether the start position 900 of the travel route 80 is within a specified distance from the vehicle 1 based on the estimated position of the vehicle 1 and the start position 900 of the travel route 80 (S405).

[0116] If the estimation unit 109 determines that the start position 900 of the travel route 80 is not within the specified distance from the vehicle 1 (S405 "No"), the process returns to S401.

[0117] Furthermore, if the estimation unit 109 determines that the start position 900 of the travel route 80 is within a specified distance from the vehicle 1 (S405 "Yes"), the output control unit 107 causes the display device 120 to display the start position 900 of the travel route 80 (S406). This display allows the driver to accurately grasp the start position 900 of the travel route 80, and can manually drive the vehicle 1 to the start position 900 of the travel route 80.

[0118] While the vehicle 1 moves to the starting position 900 of the travel route 80, the acquisition unit 101 continues to acquire peripheral images from the imaging device 16. The extraction unit 108 extracts feature points from the acquired peripheral images. The estimation unit 109 estimates the position of the vehicle 1, which changes as the vehicle 1 travels, based on the movement of the feature points extracted from the peripheral images or vehicle information.

[0119] Then, the estimation unit 109 determines whether the vehicle 1 has stopped at the start position 900 of the travel route 80 (S407). If the vehicle 1 has not reached the start position 900 of the travel route 80 (S407 "No"), the estimation unit 109 continues the process of estimating the position of the vehicle 1 and repeats the determination of S407.

[0120] Then, when the vehicle 1 reaches and stops at the start position 900 of the travel route 80 (S407 "Yes"), the vehicle control unit 110 starts automatic driving (S408). Then, the vehicle control unit 110 drives the vehicle 1 by automatic driving from the start position 900 of the travel route 80 along the travel route 80 to the parking space 910. At this point, the processing of this flowchart ends. Note that, although the example described here is one in which automatic driving is started by the vehicle stopping near the start position 900, a predetermined operation by the user (such as pressing a specific button or releasing the foot brake) may also be used as a trigger to start automatic driving.

[0121] In this way, the parking assistance method executed by the parking assistance device 100 of this embodiment determines, based on the absolute position and heading of the vehicle 1, whether the vehicle 1 is located facing a specified direction at the call point 920 registered in association with the travel route 80. Therefore, the parking assistance method of this embodiment can smoothly start preparation for parking assistance before the vehicle 1 arrives near the target parking space 910.

[0122] In addition, in the parking assistance method executed by the parking assistance device 100 of this embodiment, when it is determined that the vehicle 1 is located at the call point 920 facing in a specified direction, a notification is output to prompt the user to perform an operation to start parking assistance.

[0123] For example, if the parking assistance function is activated after the vehicle 1 reaches the vicinity of the starting position 900 of the driving route 80 and the vehicle 1 is manually driven to the parking assistance starting position 900 based on the displayed on-screen guidance, unnecessary driving operations such as waiting for the process to start and adjusting the location within a short distance to move to the starting position 900 may occur. This may result in a long time until parking or may cause psychological stress for the user. Furthermore, even if the self-location estimation process using image recognition is started at a point too far from the starting position 900, the feature points registered during the supervised driving may not match the feature points of the surrounding image, making it difficult to estimate the position of the vehicle 1, or other scenery similar to the surroundings of the home 70 may be mistakenly recognized as the surroundings of the home 70. Furthermore, if the self-location estimation process using image recognition is constantly executed while the vehicle 1 is driving, the processing load may increase and power consumption may increase.

[0124] In contrast, in the parking assistance method of this embodiment, a notification is output when vehicle 1 reaches call point 920, before vehicle 1 reaches the vicinity of start position 900, allowing the user to start parking assistance at an appropriate point, reducing the waiting time for processing startup and enabling smooth guidance to start position 900.

[0125] Furthermore, in the parking assistance method of this embodiment, the absolute position of vehicle 1 is used to determine whether vehicle 1 is located at call point 920, and therefore, compared to position determination based on feature points of the surrounding image, the occurrence of erroneous determination can be reduced, for example, when vehicle 1 is located at a different point in a landscape similar to the landscape surrounding call point 920.

[0126] Furthermore, in the parking assistance method of this embodiment, when the vehicle 1 is located at the call point 920 and the direction of travel of the vehicle 1 is a specified direction, a notification urging the user to perform an operation to start parking assistance is output. Therefore, in the parking assistance method of this embodiment, the notification is not output when the vehicle 1 is not on a route to the parking space 910, such as when the vehicle 1 is traveling on the road 40c shown in FIG. 4. Furthermore, in the parking assistance method of this embodiment, even when the vehicle 1 is located at the call point 920, the notification is not output when the direction of travel of the vehicle 1 is not a specified direction. Therefore, according to the parking assistance method of this embodiment, the notification urging the user to perform an operation to start parking assistance can be output only when the vehicle 1 is heading toward the parking space 910, thereby reducing the inconvenience to the user caused by notifications being output at unnecessary times.

[0127] Furthermore, in the parking assistance method of this embodiment, when a user operation to start parking assistance is accepted, self-location estimation is started based on the surrounding image and the surrounding image captured during supervised driving, and the vehicle 1 is moved to the parking space 910 by automatic driving based on the position of the vehicle 1 estimated by the self-location estimation and the registered driving route 80. Therefore, according to the parking assistance method of this embodiment, it is possible to improve user convenience by automatically parking the vehicle 1 in response to the user operation.

[0128] Furthermore, in the parking assistance method of this embodiment, if the difference between the acquired absolute position of the vehicle 1 and the position of the call point 920 is within a first specified range, and the difference between the acquired heading of the traveling direction of the vehicle 1 and the specified heading is within a second specified range, it is determined that the vehicle 1 is located at a specified position and facing in a specified heading. Generally, when the user manually drives the vehicle, errors may occur in the traveling position and the heading of the traveling direction of the vehicle 1, but according to the parking assistance method of this embodiment, an error within a specified range is allowed, making it possible to output a notification at an appropriate position.

[0129] Furthermore, in the parking assistance method of this embodiment, the call point 920 and the specified direction are identified based on a plurality of past driving histories of the vehicle 1, and are registered in association with the driving route 80. Therefore, according to the parking assistance method of this embodiment, it is possible to register an appropriate call point 920 and specified direction that are in line with the actual driving position of the vehicle 1 when manually driven by the user.

[0130] Furthermore, in the parking assistance method of this embodiment, the registered call point 920 and the specified orientation are changed based on a user operation to modify the call point 920 and the specified orientation. Therefore, according to the parking assistance method of this embodiment, the user can adjust the call point 920 and the specified orientation to a desired position and orientation. Furthermore, for example, if there is a location where the GPS signal is unstable due to a building or roof, the user may avoid such a location and specify as the call point 920 a location where GPS position information can be stably obtained.

[0131] (Second embodiment) In the first embodiment, the parking assistance device 100 outputs a notification prompting the user to perform an operation to start parking assistance when the vehicle 1 is located at a specified position, that is, a call point 920, and is facing in a specified direction. In this second embodiment, the parking assistance device 100 starts processing of self-position estimation for parking assistance when the vehicle 1 is located at a specified position and is facing in a specified direction.

[0132] The vehicle 1 of this embodiment has the same configuration as that of the first embodiment described with reference to Figures 1 and 2. The hardware configuration of the parking assistance device 100 of this embodiment is the same as that of the first embodiment described with reference to Figure 3.

[0133] Similar to the first embodiment, the parking assistance device 100 of this embodiment includes an acquisition unit 101, a reception unit 102, a driving history recording unit 103, a registration processing unit 104, a change processing unit 105, a determination unit 106, an output control unit 107, an extraction unit 108, an estimation unit 109, a vehicle control unit 110, a route recording unit 111, and a memory unit 112. The reception unit 102, the driving history recording unit 103, the registration processing unit 104, the change processing unit 105, the output control unit 107, the extraction unit 108, the vehicle control unit 110, and the route recording unit 111 have the same functions as those of the first embodiment.

[0134] The determination unit 106 of this embodiment has the same functions as the first embodiment, and determines whether the vehicle 1 is located facing a specified direction at the self-position estimation start point registered in correspondence with the driving route 80, based on the absolute position and heading direction of the vehicle 1.

[0135] The self-localization start point is an example of a predetermined position in this embodiment. The method of registering and correcting the self-localization start point is the same as the method of registering and correcting the call point 920 in the first embodiment.

[0136] The estimation unit 109 of this embodiment has the same functions as the first embodiment, and when the judgment unit 106 judges that the vehicle 1 is located at the self-position estimation start point facing a specified direction, it starts self-position estimation based on the surrounding image acquired by the acquisition unit 101 and the surrounding image captured during teacher driving.

[0137] The acquisition unit 101 of this embodiment has the same functions as those of the first embodiment, and when it is determined that the vehicle 1 is located at a specified position facing a specified direction, acquires an image of the surroundings of the vehicle 1. This acquisition process is an example of a fourth acquisition step in this embodiment.

[0138] FIG. 16 is a flowchart showing an example of the flow of processing for automatic start of self-position estimation and parking assistance executed by the parking assistance device 100 according to the second embodiment.

[0139] The process of acquiring the absolute position and heading of the vehicle 1 in S501 is similar to the process of S301 in the process of the first embodiment described with reference to FIG.

[0140] Then, based on the acquired absolute position and heading direction of vehicle 1 and the self-position estimation start point and specified direction registered in memory unit 112, judgment unit 106 judges whether vehicle 1 is located at the self-position estimation start point and whether vehicle 1 is facing in the specified direction (S502).

[0141] If the determination unit 106 determines that the vehicle 1 is not located at the self-position estimation start point or that the vehicle 1 is not facing in the specified direction (S502 "No"), the process returns to S501.

[0142] If the determination unit 106 determines that the vehicle 1 is located at the self-position estimation start point and that the vehicle 1 is facing in a specified direction (S502 "Yes"), the estimation unit 109, the vehicle control unit 110, etc. start self-position estimation and parking assistance processing (S503). Then, the processing of this flowchart ends.

[0143] FIG. 17 is a flowchart showing an example of the flow of the self-position estimation and parking assistance process executed by the parking assistance device 100 according to the second embodiment.

[0144] The process from acquiring a surrounding image in S601 to determining whether the vehicle 1 has stopped at the start position of the travel route 80 in S607 is the same as the process from S401 to S407 in the first embodiment described in Figure 15.

[0145] Then, when the vehicle 1 reaches and stops at the start position 900 of the travel route 80 (S607 "Yes"), the output control unit 107 notifies the user that the vehicle 1 is located at the start position 900 of the travel route 80 (S608). For example, the output control unit 107 may cause the display device 120 to display a start button 1201 that can accept an operation by the user to start parking assistance, and a cancel button 1202 that can accept an operation by the user to cancel the start of parking assistance.

[0146] In the first embodiment, it was possible to confirm whether the user intended to use parking assistance at the time of the call point 920, but in this embodiment, the self-location estimation process is performed from the self-location estimation start point regardless of the user's intention, so whether the user intended to use parking assistance is confirmed before the start of automatic driving.

[0147] Then, for example, when the start button 1201 displayed on the display device 120 is pressed by the user, the reception unit 102 receives an operation to instruct the start of parking assistance (S609 "Yes"). In this case, the vehicle control unit 110 starts automatic driving (S610). Then, the vehicle control unit 110 drives the vehicle 1 by automatic driving from the start position 900 of the driving route 80 to the parking space 910 along the driving route 80.

[0148] Furthermore, if the reception unit 102 does not receive an operation to instruct the start of parking assistance, for example, if the user presses the cancel button 1202 displayed on the display device 120 (S609 "No"), parking assistance by automatic driving will not start and the processing of this flowchart will end.

[0149] In this way, in the parking assistance method executed by the parking assistance device 100 of this embodiment, when it is determined that the vehicle 1 is located at the self-location estimation start point facing a specified direction, a peripheral image of the area around the vehicle 1 is acquired, and self-location estimation is performed based on the acquired peripheral image and the peripheral image acquired during supervised driving. Therefore, in the parking assistance method of this embodiment, preparation for parking assistance can be started smoothly before the vehicle 1 arrives near the target parking space 910.

[0150] (Third embodiment) In the third embodiment, the parking assistance device 100 further guides the user to a start position 900 of the travel route 80 before the vehicle 1 reaches the vicinity of the home 70.

[0151] The vehicle 1 of this embodiment has the same configuration as that of the first embodiment described with reference to Figures 1 and 2. The hardware configuration of the parking assistance device 100 of this embodiment is the same as that of the first embodiment described with reference to Figure 3.

[0152] Similar to the first embodiment, the parking assistance device 100 of this embodiment includes an acquisition unit 101, a reception unit 102, a driving history recording unit 103, a registration processing unit 104, a change processing unit 105, a determination unit 106, an output control unit 107, an extraction unit 108, an estimation unit 109, a vehicle control unit 110, a route recording unit 111, and a memory unit 112. The reception unit 102, the driving history recording unit 103, the registration processing unit 104, the change processing unit 105, the extraction unit 108, the vehicle control unit 110, and the route recording unit 111 have the same functions as those of the first embodiment.

[0153] The output control unit 107 of this embodiment has the same functions as those of the first embodiment, and when it is determined that the vehicle 1 is located at the call point 920 facing in a specified direction, causes the display device 120 to display an image indicating the start position 900 of the travel route 80. In this embodiment, the image indicating the start position 900 of the travel route 80 is referred to as a guidance image. The process of displaying the guidance image is an example of a second output step.

[0154] Fig. 18 is a diagram showing an example of a guide image 600 according to the third embodiment. In the example shown in Fig. 18, the output control unit 107 causes the display device 120 to display a first guide image 600a, a second guide image 600b, and a message M2. Note that the first guide image 600a and the second guide image 600b are collectively referred to simply as guide image 600.

[0155] The first guide image 600a is, for example, a bird's-eye view image depicting the surroundings of the vehicle 1 from a virtual viewpoint located directly above the vehicle 1. The bird's-eye view image is, for example, a composite image synthesized based on images of the surroundings of the vehicle 1 acquired by the acquisition unit 101 during supervised driving. The composite image may be generated by the route recording unit 111 when recording the driving route 80, or a separate image processing unit may be provided.

[0156] The first guide image 600a also depicts a vehicle image 61a representing the vehicle 1, a first frame 62a representing the starting position 900 of the travel route 80, and a second frame 63a representing a parking space 910. The first frame 62a and the second frame 63a are displayed in different ways so that they can be easily distinguished by the user. While Fig. 18 is monochrome, the first frame 62a may be in green and the second frame 63a in black, for example, and the colors may be different.

[0157] Furthermore, second guide image 600b is, for example, a bird's-eye view image depicting the surroundings of vehicle 1 from a virtual viewpoint positioned diagonally above vehicle 1. Like first guide image 600a, second guide image 600b is a composite image synthesized based on images of the surroundings of vehicle 1 acquired by acquisition unit 101 during supervised driving. Second guide image 600b also depicts a vehicle image 61b representing vehicle 1, a first frame 62b representing start position 900 of driving route 80, and a second frame 63b representing a parking space 910.

[0158] Furthermore, the message M2 provides guidance on the start position 900 where the vehicle 1 should stop when parking assistance starts, such as "When parking at home, please stop the vehicle in the green frame position." In the example shown in Fig. 18, the "green frame" refers to the first frames 62a and 62b.

[0159] If multiple travel routes 80 have been registered, the output control unit 107 causes the guide image 600 to display the start position 900 of the travel route 80 associated with the call point 920 where it has been determined that the vehicle 1 is located.

[0160] 18, the vehicle 1 stopping at the start position 900 is depicted in a first guide image 600a and a second guide image 600b from different virtual viewpoints, making it easier for the user to understand the position where the vehicle 1 should be stopped. Note that the number and display mode of the guide images 600 displayed are not limited to those in the example shown in FIG.

[0161] The timing for displaying the guide image 600 by the output control unit 107 is, for example, after the notification prompting the user to perform an operation to start parking assistance described in Fig. 11 is output and then the user presses the start button 1201. Alternatively, the output control unit 107 may display the guide image 600 shown in Fig. 18 instead of the notification prompting the user to perform an operation to start parking assistance described in Fig. 11.

[0162] As described above, in the parking assistance method executed by the parking assistance device 100 of this embodiment, when it is determined that the vehicle 1 is located at the call point 920 facing in a specified direction, the guidance image 600 indicating the start position 900 of the travel route 80 is displayed on the display device 120. Therefore, according to the parking assistance method of this embodiment, it is possible to guide the user to stop the vehicle 1 at the start position 900, and automatic parking can be started smoothly.

[0163] In this embodiment, the parking assistance device 100 displays the guidance image 600 when the vehicle 1 is located at the call point 920, but a location other than the call point 920 may also be registered as a point for displaying the guidance image 600.

[0164] (Variation 1) In the above-described third embodiment, when it is determined that the vehicle 1 is located at the call point 920 facing a specified direction, the guide image 600 is displayed on the display device 120, but the timing of displaying the guide image 600 is not limited to this.

[0165] For example, when the receiving unit 102 receives a user operation specifying home as the destination of the car navigation system, the output control unit 107 may cause the display device 120 to display a guide image 600 indicating a starting position 900 of the driving route 80. This receiving process is an example of a fourth receiving step. The display process of the guide image 600 in this modification is an example of a third output step.

[0166] 19 is a diagram illustrating an example of the display timing of guide image 600 according to Modification 1. As shown in Fig. 19, when vehicle 1 is heading from point 930 away from home 70 to home 70, the user specifies home as the destination in the car navigation system. When receiving unit 102 receives such a user operation, output control unit 107 causes display device 120 to display first guide image 600a, second guide image 600b, and message M2, as shown in Fig. 18, for example.

[0167] If multiple travel routes 80 have been registered, the output control unit 107 provides guidance to the start position 900 of the travel route 80 whose start position 900 is closest to the absolute position of the vehicle 1 at the time the user performs an operation to specify their home as the destination in the car navigation system. In the example shown in Fig. 19, the first start position 900a is closer to the point 930 than the second start position 900b, so the output control unit 107 displays the guidance image 600 indicating the position of the first start position 900a.

[0168] Alternatively, the output control unit 107 may display a list of multiple driving routes 80 on the display device 120, allowing the user to select a desired driving route 80. In this case, the names of each driving route 80 may be displayed, such as "a route that enters from the right side of the parking space" or "a route that enters from the left side of the parking space," so that the user can easily recognize the differences between the multiple driving routes 80. The names may also be set by the user. The output control unit 107 may also display an image showing the shapes of the multiple driving routes 80 superimposed on an overhead image or a map, allowing the user to select a driving route 80 from the displayed image.

[0169] In the parking assistance method executed by the parking assistance device 100 of this modified example, the starting position 900 of the driving route 80 is presented to the user when it becomes clear that the vehicle 1 is heading to the home 70, making it easy for the user to park the vehicle 1 at the starting position 900 and allowing automatic parking to begin smoothly.

[0170] The configuration of this modified example may be combined with the configuration of the parking assistance device 100 of the first embodiment, or may be combined with the configuration of the parking assistance device 100 of the second embodiment. Alternatively, the configuration of this modified example may be applied even when the parking assistance device 100 does not set the call point 920 and the self-location estimation point.

[0171] (Variation 2) In the first embodiment described above, a method for registering the call point 920 based on a past driving history has been described, but the method for registering the call point 920 is not limited to this. For example, the user may register the call point 920 and a specified direction on the screen of the display device 120. The display device 120 is an example of an input device provided near the driver's seat of the vehicle 1.

[0172] More specifically, the reception unit 102 receives input of the call point 920 and the specified direction by the user. This reception process is an example of a third reception step. Furthermore, the registration processing unit 104 registers the call point 920 and the specified direction input by the user in the storage unit 112 in association with the travel route 80 registered by the supervised travel. This registration process is an example of a second registration step.

[0173] Furthermore, the output control unit 107 of this modified example causes the display device 120 to display an operation screen that allows the user to specify a call point 920 and a specified direction on a map.

[0174] Also, for example, the user may perform the registration operation while the vehicle 1 is located at a position that the user wants to register as the call point 920. The operation to register the call point 920 is, for example, pressing the operation button 141 or operating a button on the screen of the display device 120. In this case, the registration processing unit 104 registers, as the call point 920, the absolute position of the vehicle 1 at the time when the reception unit 102 received the user's operation to register the call point 920. Furthermore, the registration processing unit 104 registers, as the specified direction, the direction of travel of the vehicle 1 at the time when the reception unit 102 received the user's operation to register the call point 920.

[0175] More specifically, when the reception unit 102 receives an instruction from the user to register the call point 920 and a specified direction, the acquisition unit 101 acquires the absolute position and direction of the vehicle 1. This acquisition process is an example of a third acquisition step.

[0176] The registration processing unit 104 registers the absolute position and heading direction of the vehicle 1 acquired by the acquisition unit 101 as a call origination point 920 and a specified heading, in association with the travel route 80. Note that if multiple travel routes 80 have been registered, the output control unit 107 may display a list of the multiple travel routes 80 on the display device 120, allowing the user to select the travel route 80 for which the user wishes to register the call origination point 920. In this case, the registration processing unit 104 registers the absolute position and heading direction of the vehicle 1 acquired by the acquisition unit 101 in association with the travel route 80 selected by the user.

[0177] When the user performs the operation to register the call point 920, the vehicle 1 may be stopped or may be moving.

[0178] The method for registering the call origination point 920 in this modified example can also be used for registering the self-location estimation start point in the second embodiment.

[0179] (Variation 3) In each of the above-described embodiments, the parking assistance device 100 acquires the absolute position of the vehicle 1 based on a GPS signal, but the absolute position of the vehicle 1 may be determined using a technology other than GPS. For example, the parking assistance device 100 may acquire the absolute position of the vehicle 1 based on known map matching or the amount of movement of the vehicle 1 from a reference position determined by a vehicle speed pulse. Specifically, a dead-reckoning (DR) technology using a wheel encoder, an inertial sensor, or the like may be employed.

[0180] (Variation 4) In the above-described embodiments, the display device 120 and the operation unit are mounted on the vehicle 1, but they may be provided outside the vehicle 1. For example, the display device 120 and the operation unit may be a mobile terminal such as a tablet terminal or a smartphone that can communicate with the parking assistance device 100 wirelessly or via a wired connection. Note that it is sufficient that the mobile terminal is not directly connected to the parking assistance device 100 of the vehicle 1, but can indirectly transmit and receive data and control signals to and from the parking assistance device 100 via a server device or a cloud environment.

[0181] If the display device 120 and the operation unit are a tablet terminal or the like, the driver may use the tablet terminal or the like inside the vehicle 1, or may use the tablet terminal or the like after getting out of the vehicle 1.

[0182] For example, in the above-described third modification, the operation screen on which the user can specify the call point 920 and the specified direction on the map may be displayed on the display of a mobile terminal. In this case, the reception unit 102 receives the call point 920 and the specified direction input by the user from a mobile terminal capable of communicating with the parking assistance device of the vehicle 1. For example, the user may use the mobile terminal inside the vehicle 1, or outside the vehicle 1, for example, at home 70.

[0183] Alternatively, if the vehicle 1 is equipped with a head-up display, the head-up display may be an example of a display unit. For example, the windshield 180 of the vehicle 1 may become a head-up display capable of displaying an image by projecting an image onto it using a projection device (not shown).

[0184] (Variation 5) Furthermore, some of the functions of the parking assistance device 100 in each of the above-described embodiments and modifications may be executed by an information processing device provided outside the vehicle 1. The information processing device provided outside the vehicle 1 is, for example, a mobile terminal such as a smartphone, a PC, a server device, etc. The information processing device may be provided in a cloud environment.

[0185] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0186] 1 vehicle 12 Body 15, 15f, 15r Transmitting and receiving unit 16, 16a to 16d Imaging device 21 Orientation sensor 22 GPS device 23 GPS antenna 40a~40c road 61a, 61b Vehicle images 62a, 62b First frame 63a, 63b Second frame 70 Home 80 Driving Route 80a First driving route 80b Second driving route 100 Parking assistance device 101 Acquisition Department 102 Reception 103 Driving history recording unit 104 Registration processing unit 105 Change processing section 106 Judgment section 107 Output control section 108 Extraction part 109 Estimation section 110 Vehicle control unit 111 Route Recording Unit 112 Storage section 120 Display device 141 Operation buttons 600 Guide Image 600a First guide image 600b Second guide image 900 starting position 900a First starting position 900b Second starting position 910 Parking Spaces 920 Call Point 920a First Call Point 920b Secondary Call Point 930 points 1201 Start button 1202 Cancel button

Claims

1. A parking assistance method in which a parking assistance device automatically drives a vehicle based on a driving route generated by a driver through training driving, the parking assistance device acquires the position of the vehicle and the direction of travel of the vehicle; the parking assistance device accepts an operation to start parking assistance when the vehicle is located at a specified position and in a specified orientation associated with the travel route based on the position of the vehicle and the orientation of the vehicle's traveling direction, The specified position is within a specified distance from the start position of the travel route. Parking assistance methods.

2. The parking assistance device acquires an image of the surroundings of the vehicle, the parking assistance device estimates its own position based on the image and a surrounding image captured during teacher driving or data generated from the surrounding image; the parking assistance device moves the vehicle to a parking position by automatic driving based on the position of the vehicle estimated by the self-position estimation and the driving route; The parking assistance method according to claim 1 .

3. the parking assistance device determines that the vehicle is located at the specified position and facing the specified orientation when a difference between the acquired position of the vehicle and the specified position is within a first specified range and a difference between the acquired heading of the traveling direction of the vehicle and the specified orientation is within a second specified range; The parking assistance method according to claim 1 or 2.

4. The parking assistance device identifies the specified position and the specified direction based on a plurality of past driving histories of the vehicle, and associates the specified position and the specified direction with the driving route. A parking assistance method according to any one of claims 1 to 3.

5. the parking assistance device receives a user's operation to correct the specified position and the specified orientation; The parking assistance device changes the registered specified position and specified orientation based on a user's operation to correct the specified position and the specified orientation. The parking assistance method according to claim 4.

6. the parking assistance device accepts an input of the specified position and the specified direction by a user; The parking assistance device associates the specified position and the specified orientation input by the user with the driving route registered by the teacher driving. A parking assistance method according to any one of claims 1 to 3.

7. the parking assistance device receives the specified position and the specified orientation input by the user from an input device provided near a driver's seat of the vehicle; The parking assistance method according to claim 6.

8. When it is determined that the vehicle is located at the specified position and facing the specified direction, the parking assistance device displays an image indicating a start position of the travel route on a display unit. A parking assistance method according to any one of claims 1 to 7.

9. When the parking assistance device determines, based on the position of the vehicle and the direction of travel of the vehicle, that the vehicle is not located at a specified position associated with the travel route in a specified direction, the parking assistance device does not output a notification prompting the user to perform an operation to start the parking assistance. A parking assistance method according to any one of claims 1 to 8.

10. The specified orientation is the traveling direction of the vehicle at the specified position. A parking assistance method according to any one of claims 1 to 9.

11. A parking assistance device that performs parking assistance for automatic driving of a vehicle based on a driving route generated by a driver through teacher driving, an acquisition unit for acquiring the position of the vehicle and the direction of travel of the vehicle; a reception unit that receives an operation to start parking assistance when the vehicle is located at a specified position and in a specified orientation associated with the travel route based on the position of the vehicle and the orientation of the vehicle's traveling direction, The specified position is within a specified distance from the start position of the travel route. Parking assistance device.

12. the acquisition unit acquires an image of the surroundings of the vehicle; and an estimation unit estimates a self-position based on the image and a surrounding image captured during teacher driving or data generated from the surrounding image; a vehicle control unit that moves the vehicle to a parking position by automatic driving based on the position of the vehicle estimated by the self-position estimation and the driving route, 12. The parking assistance device according to claim 11.

13. a determination unit that determines that the vehicle is located at the specified position and facing the specified direction when a difference between the acquired vehicle position and the specified position is within a first specified range and a difference between the acquired heading of the vehicle's traveling direction and the specified direction is within a second specified range, 13. A parking assistance device according to claim 11 or 12.

14. and a registration processing unit that identifies the specified position and the specified direction based on a plurality of past driving histories of the vehicle and associates the specified position and the specified direction with the driving route. A parking assistance device according to any one of claims 11 to 13.

15. the accepting unit accepts a user operation to correct the specified position and the specified orientation; a change processing unit that changes the registered specified position and specified orientation based on a user's operation to correct the specified position and the specified orientation, 15. A parking assistance device according to claim 14.

16. the receiving unit receives an input of the specified position and the specified orientation by a user; a registration processing unit that associates the specified position and the specified orientation input by the user with the travel route registered by the supervised travel, A parking assistance device according to any one of claims 11 to 13.

17. the receiving unit receives the specified position and the specified orientation input by the user from an input device provided near a driver's seat of the vehicle; 17. A parking assistance device according to claim 16.

18. an output control unit that, when it is determined that the vehicle is located at the specified position and facing the specified direction, causes an image indicating the start position of the travel route to be displayed on a display unit.

18. A parking assistance device according to any one of claims 11 to 17.

19. the output control unit does not output a notification prompting the user to start the parking assistance when the parking assistance device determines, based on the position of the vehicle and the direction of travel of the vehicle, that the vehicle is not located at a specified position and in a specified direction associated with the travel route.

19. A parking assistance device according to claim 18.

20. The specified orientation is the traveling direction of the vehicle at the specified position.

20. A parking assistance system according to any one of claims 11 to 19.

Images