Parking assistance method and parking assistance device

Abstract

The present invention provides a parking assistance method and a parking assistance device that allow the driver to easily understand the position where the vehicle can be driven autonomously based on its travel path. [Solution] The parking assistance method according to this disclosure is a parking assistance method that performs automatic driving of a vehicle based on a driver's instructional driving, and stores information about the driving route during the driver's instructional driving, and displays guidance information on a display unit based on a target position from which the vehicle can move to the parking position and merge onto the driving route by automatic driving based on the driving route. The width of the second area from which the vehicle can merge onto the driving route at the second point is narrower than the width of the first area from which the vehicle can merge onto the driving route at the first point. The second point is a point on the driving route that is closer to the parking position than the first point.

Description

Technical Field

[0001] The present disclosure relates to a parking assistance method and a parking assistance device.

Background Art

[0002] Conventionally, a parking assistance technology for moving a vehicle by automatic driving when parking the vehicle is known. One such parking assistance is a technology that learns a driving route based on a teacher run by a driver and uses the learning result to perform parking assistance. This technology is used, for example, when repeatedly parking at a fixed parking position such as a parking lot at the driver's home or workplace.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when a driver parks a vehicle using parking assistance and stops the vehicle at a position far from the learned driving route, it may be difficult to move the vehicle by automatic driving based on the driving route.

[0005] The present disclosure provides a parking assistance method and a parking assistance device that can easily allow a driver to grasp a position where automatic driving of a vehicle based on a driving route is possible.

Means for Solving the Problems

[0006] The parking assistance method relating to this disclosure is a parking assistance method that performs automatic driving of a vehicle based on a driver's instructional driving, and stores information about the driving path during the driver's instructional driving, and displays guidance information on a display unit based on a target position from which the vehicle can move to the parking position and merge onto the driving path by automatic driving based on the driving path. The width of the second area from which the vehicle can merge onto the driving path at the second point is narrower than the width of the first area from which the vehicle can merge onto the driving path at the first point. The second point is a point on the driving path that is closer to the parking position than the first point. [Effects of the Invention]

[0007] According to the parking assistance method and parking assistance device disclosed herein, the driver can easily understand the position in which the vehicle can move to a parking position through automated driving based on the travel route. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of a vehicle equipped with a parking assist device according to the first embodiment. [Figure 2] Figure 2 shows an example of the configuration near the driver's seat of a vehicle according to the first embodiment. [Figure 3] Figure 3 illustrates an example of parking assistance according to the first embodiment. [Figure 4] Figure 4 shows an example of the hardware configuration of the parking assistance device according to the first embodiment. [Figure 5] Figure 5 is a block diagram showing an example of the functions of the parking assistance device according to the first embodiment. [Figure 6] Figure 6 shows an example of an image indicating the starting position of driving, displayed on the head-up display according to the first embodiment. [Figure 7] Figure 7 shows an example of an image indicating the starting position of travel, displayed on the display device according to the first embodiment. [Figure 8]Figure 8 is a flowchart showing an example of the flow of the parking assistance process performed by the parking assistance device according to the first embodiment. [Figure 9] Figure 9 illustrates an example of parking assistance according to the second embodiment. [Figure 10] Figure 10 shows an example of guidance information regarding the parking start area according to the second embodiment. [Figure 11] Figure 11 shows another example of guidance information regarding the parking start area according to the second embodiment. [Figure 12] Figure 12 is a flowchart showing an example of the flow of the parking assistance process performed by the parking assistance device according to the second embodiment. [Figure 13] Figure 13 illustrates an example of parking assistance according to the third embodiment. [Figure 14] Figure 14 is a flowchart showing an example of the flow of the parking assistance process performed by the parking assistance device according to the third embodiment. [Figure 15] Figure 15 shows an example of an image indicating the starting position of driving, displayed on a head-up display according to Modification 1. [Figure 16] Figure 16 shows an example of an image indicating the starting position of travel, displayed on the display device according to Modification 2. [Figure 17] Figure 17 shows an example of an image indicating the starting position of travel, displayed on the display device according to Modification 3. [Figure 18] Figure 18 shows an example of a second guidance image displayed on the display device according to Modification 4. [Figure 19] Figure 19 shows an example of a second guidance image displayed on the display device according to Modification 5. [Figure 20] Figure 20 shows an example of the first guidance image displayed on the head-up display according to Modification 6. [Figure 21] Figure 21 shows an example of guidance information displayed before a vehicle according to the fourth embodiment reaches a position where it can merge onto a travel path. [Figure 22]FIG. 22 is a diagram showing an example of guidance information displayed after a vehicle according to the fourth embodiment reaches a position where it can merge into a driving route. [Figure 23] FIG. 23 is a diagram showing an example of guidance information displayed after a vehicle according to the fourth embodiment starts automatic parking along a driving route. [Figure 24] FIG. 24 is a diagram showing an example of a state before a vehicle according to the fourth embodiment reaches a position where it can merge into a driving route. [Figure 25] FIG. 25 is a diagram showing an example of a state after a vehicle according to the fourth embodiment reaches a position where it can merge into a driving route. [Figure 26] FIG. 26 is a flowchart showing an example of the flow of parking support processing executed by a parking support device according to the fourth embodiment. [Figure 27] FIG. 27 is a diagram showing an example of guidance information according to the fifth embodiment.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of a parking support method and a parking support device according to the present disclosure will be described while referring to the drawings.

[0010] (First Embodiment) FIG. 1 is a diagram showing an example of a vehicle 1 including a parking support device 100 according to the first embodiment. As shown in FIG. 1, the vehicle 1 includes a vehicle body 12 and two pairs of wheels 13 arranged along a predetermined direction on the vehicle body 12. The two pairs of wheels 13 include a pair of front tires 13f and a pair of rear tires 13r.

[0011] The front tire 13f shown in FIG. 1 is an example of the first wheel in the present embodiment. Also, the rear tire 13r is an example of the second wheel in the present embodiment. Note that the vehicle 1 shown in FIG. 1 includes four wheels 13, but the number of wheels 13 is not limited thereto. For example, the vehicle 1 may be a two-wheeled vehicle.

[0012] The vehicle body 12 is connected to the wheels 13 and is movable by the wheels 13. In this case, the predetermined direction in which the two pairs of wheels 13 are arranged becomes the direction of travel (direction of movement) of the vehicle 1. The vehicle 1 can move forward or backward by switching gears (not shown), etc. The vehicle 1 can also turn left or right by steering.

[0013] Furthermore, the vehicle body 12 has a front end F, which is the end on the front tire 13f side, and a rear end R, which is the end on the rear tire 13r side. The vehicle body 12 is roughly rectangular in shape when viewed from above, and the four corners of the roughly rectangular shape are sometimes referred to as ends. Although not shown in Figure 1, the vehicle 1 is also equipped with a display device, a speaker, and an operating unit.

[0014] The front and rear ends F and R of the vehicle body 12 are provided with a pair of bumpers 14 near the lower end of the vehicle body 12. Of the pair of bumpers 14, the front bumper 14f covers the entire front and part of the side near the lower end of the vehicle body 12. Of the pair of bumpers 14, the rear bumper 14r covers the entire rear and part of the side near the lower end of the vehicle body 12.

[0015] Transmitting and receiving units 15f and 15r for transmitting and receiving sound waves such as ultrasonic waves are arranged at predetermined ends of the vehicle body 12. For example, one or more transmitting and receiving units 15f are arranged on the front bumper 14f, and one or more transmitting and receiving units 15r are arranged on the rear bumper 14r. Hereinafter, unless otherwise specified, the transmitting and receiving units 15f and 15r will simply be referred to as the transmitting and receiving unit 15. Furthermore, the number and location of the transmitting and receiving units 15 are not limited to the example shown in Figure 1. For example, the vehicle 1 may have transmitting and receiving units 15 on both the left and right sides.

[0016] In this embodiment, a sonar using ultrasound is described as an example of the transmitter / receiver unit 15, but the transmitter / receiver unit 15 may also be a radar that transmits and receives electromagnetic waves. Alternatively, the vehicle 1 may be equipped with both sonar and radar. Furthermore, the transmitter / receiver unit 15 may simply be referred to as a sensor.

[0017] More specifically, the transmitter / receiver unit 15 comprises a transmitter that transmits sound waves such as ultrasonic waves or electromagnetic waves, and a receiver that receives reflected sound waves that have been reflected by an object from the sound waves or electromagnetic waves transmitted from the transmitter. Based on the results of transmitting and receiving sound waves or electromagnetic waves, the transmitter / receiver unit 15 detects obstacles around the vehicle 1. The transmitter / receiver unit 15 also measures the distance between the vehicle 1 and the obstacles around the vehicle 1 based on the results of transmitting and receiving sound waves or electromagnetic waves.

[0018] Vehicle 1 also includes a first imaging device 16a for imaging the area in front of Vehicle 1, a second imaging device 16b for imaging the area behind Vehicle 1, a third imaging device 16c for imaging the area to the left of Vehicle 1, and a fourth imaging device for imaging the area to the right of Vehicle 1. The fourth imaging device is not shown in the illustration.

[0019] Hereinafter, the first imaging device 16a, the second imaging device 16b, the third imaging device 16c, and the fourth imaging device will simply be referred to as imaging device 16 unless otherwise specified. The location and number of imaging devices are not limited to the example shown in Figure 1. For example, vehicle 1 may also be equipped with only two imaging devices, the first imaging device 16a and the second imaging device 16b. Alternatively, vehicle 1 may have other imaging devices in addition to those described above.

[0020] The imaging device 16 is capable of capturing images of the area around the vehicle 1, and is, for example, a camera that captures color images. The images captured by the imaging device 16 may be video or still images. The imaging device 16 may be a camera built into the vehicle 1, or it may be a camera of a drive recorder that is attached to the vehicle 1 afterwards.

[0021] Furthermore, vehicle 1 is equipped with a parking assist device 100. The parking assist device 100 is an information processing device that can be mounted on vehicle 1, and is, for example, an ECU (Electronic Control Unit) or OBU (On Board Unit) located inside vehicle 1. Alternatively, the parking assist device 100 may be an external device installed near the dashboard of vehicle 1. The parking assist device 100 may also function as a car navigation system or the like.

[0022] Next, the configuration of the area near the driver's seat of the vehicle 1 in this embodiment will be described. Figure 2 is a diagram showing an example of the configuration of the area near the driver's seat 130a of the vehicle 1 in the first embodiment.

[0023] As shown in Figure 2, the vehicle 1 is equipped with a driver's seat 130a and a passenger seat 130b. In front of the driver's seat 130a are a windshield 180, a dashboard 190, a steering wheel 140, a display device 120, and control buttons 141.

[0024] The display device 120 is a display mounted on the dashboard 190 of the vehicle 1. For example, the display device 120 is located in the center of the dashboard 190, as shown in Figure 2. The display device 120 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display device 120 may also function as a touch panel.

[0025] Furthermore, the windshield 180 becomes a head-up display capable of displaying images when an image is projected onto it by a projection device (not shown). The display device 120 and the head-up display are examples of display units in this embodiment.

[0026] Furthermore, the steering wheel 140 is located in front of the driver's seat 130a and can be operated by the driver. The rotation angle of the steering wheel 140, i.e., the steering angle, is electrically or mechanically linked to the change in direction of the front tires 13f, which are the steering wheels. Note that the steering wheels may also be the rear tires 13r, or both the front tires 13f and the rear tires 13r may be steering wheels.

[0027] The operation button 141 is a button that can accept input from the user. In this embodiment, the user is, for example, the driver of vehicle 1. By receiving a press from the driver, the operation button 141 accepts an operation from the driver, for example, to start parking assistance. The location of the operation button 141 is not limited to the example shown in Figure 2, and may be provided on the steering wheel 140, for example. The operation button 141 is an example of an operation unit in this embodiment. If the display device 120 also functions as a touch panel, the display device 120 may also be an example of an operation unit. Furthermore, an operation terminal capable of transmitting signals from outside vehicle 1 to vehicle 1, such as a remote controller or electronic key (not shown), may also be an example of an operation unit.

[0028] Next, the functions of the vehicle 1 of this embodiment will be described. The vehicle 1 of this embodiment learns a driving route based on a training drive performed by the driver and uses the learned result to provide parking assistance. In other words, the parking assistance method performed by the parking assistance device 100 installed in the vehicle 1 of this embodiment is a method of automatically driving the vehicle 1 based on a training drive performed by the driver. Such a parking assistance method is effective in reducing the effort required for the driver to park when repeatedly parking in a fixed parking location, such as the driver's home garage, a contracted parking space in an apartment building, or a designated parking space in a parking lot at their workplace. For this reason, such parking assistance is called home zone parking or route memory type automatic parking. The training drive is an example of a parking operation that has been performed in the past in this embodiment.

[0029] Figure 3 is a diagram illustrating an example of parking assistance according to the first embodiment. The travel path 80 shown in Figure 3 is the path by which vehicle 1 moves from the starting position 900 to the parking position 910.

[0030] The parking space 910 is located, for example, within the garage 920 at the driver's home of vehicle 1, but is not limited to this.

[0031] The parking assistance device 100 of this embodiment learns a driving path 80 and performs home zone parking, which parks the vehicle 1 at the parking position 910 based on the learned driving path 80.

[0032] Next, the hardware configuration of the parking assist device 100 will be described. Figure 4 is a diagram showing an example of the hardware configuration of the parking assist device 100 according to the first embodiment. As shown in Figure 4, the parking assist device 100 has a CPU (Central Processing Unit) 11A, ROM (Read Only Memory) 11B, RAM (Random Access Memory) 11C, I / F (Interface) 11D, HDD (Hard Disk Drive) 11E, etc., which are interconnected by a bus 11F, and has a hardware configuration that uses a normal computer.

[0033] The CPU 11A is a processing unit that controls the entire ECU. Note that the CPU 11A is an example of a processor in the parking assist device 100 of this embodiment, and other processors or processing circuits may be provided instead of the CPU 11A. The ROM 11B stores programs and the like that realize various processes performed by the CPU 11A. The RAM 11C is, for example, the main memory of the parking assist device 100, and stores data necessary for various processes performed by the CPU 11A. The I / F 11D is an interface for sending and receiving data. The I / F 11D may also send and receive information with other ECUs mounted in the vehicle 1 via a CAN (Controller Area Network) or the like within the vehicle 1.

[0034] Next, the functions of the parking assist device 100 of this embodiment will be described in detail. Figure 5 is a block diagram showing an example of the functions of the parking assist device 100 according to the first embodiment.

[0035] As shown in Figure 5, the parking assistance device 100 of this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0036] The storage unit 108 is composed of, for example, a ROM 11B, a RAM 11C, or an HDD 11E. Although Figure 5 shows a single storage unit 108 included in the parking assist device 100, multiple storage media may function as the storage unit 108.

[0037] The memory unit 108 stores programs and data used in various processes performed by the parking assist device 100. For example, the program performed by the parking assist device 100 in this embodiment has a modular configuration including the above-mentioned functional units (acquisition unit 101, extraction unit 102, learning unit 103, estimation unit 104, output control unit 105, vehicle control unit 106, and reception unit 107). In actual hardware, the CPU 11A reads the program from the memory unit 108 and executes it, loading each of the above units onto the RAM 11C, and generating the acquisition unit 101, extraction unit 102, learning unit 103, estimation unit 104, output control unit 105, vehicle control unit 106, and reception unit 107 on the RAM 11C. The processes realized by each functional unit of the parking assist device 100 are also called steps.

[0038] The program executed by the parking assistance device 100 of this embodiment is provided as a file in an installable or executable format, recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0039] Furthermore, the program executed by the parking assistance device 100 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. Alternatively, the program executed by the parking assistance device 100 of this embodiment may be provided or distributed via a network such as the Internet. Alternatively, the program executed by the parking assistance device 100 of this embodiment may be pre-installed in a ROM 11B or the like and provided.

[0040] The acquisition unit 101 acquires images of the area around the vehicle 1 from the imaging device 16. Hereinafter, these images will be referred to as surrounding images. The acquisition unit 101 acquires multiple surrounding images when the vehicle 1 learns a driving path based on a teacher's driving, and when the vehicle 1 performs automatic driving based on the learned driving path.

[0041] The surrounding image captured of the area around vehicle 1 during the teacher's run is an example of the first sensor information in this embodiment.

[0042] Furthermore, surrounding images captured before or during automatic driving based on a learned driving path are an example of the second sensor information in this embodiment.

[0043] Furthermore, the acquisition unit 101 acquires the presence or absence of obstacles detected by the transmitter / receiver unit 15, and the distance between the vehicle 1 and obstacles around the vehicle 1, as measured by the transmitter / receiver unit 15. The presence or absence of obstacles detected by the transmitter / receiver unit 15, and the distance between the vehicle 1 and obstacles around the vehicle 1, as measured by the transmitter / receiver unit 15, may also be referred to as environmental information regarding the environment around the vehicle 1. The environmental information is not limited to this information and may include, for example, surrounding images.

[0044] Furthermore, the acquisition unit 101 acquires vehicle information of vehicle 1 from various sensors or other ECUs of vehicle 1. The vehicle information includes, for example, information regarding the speed, steering angle, and braking operation of vehicle 1.

[0045] The extraction unit 102 extracts feature points from the surrounding images. The method used by the extraction unit 102 to extract feature points is not particularly limited, and any known method may be applied. For example, the extraction unit 102 may extract feature points using methods such as FAST (Features from Accelerated Segment Test) or ORB (Oriented FAST and Rotated BRIEF). Furthermore, when learning the driving route 80, the extraction unit 102 may prioritize recording feature points that satisfy specified conditions from among the extracted feature points. For example, among a series of consecutive surrounding images, feature points extracted from images where the vehicle 1 traveled a long distance during imaging may be given priority as feature points.

[0046] The learning unit 103 stores information regarding the driving route 80 during the teacher's drive in the storage unit 108. The learning unit 103 is also called the memory processing unit or recording unit. More specifically, the learning unit 103 learns the driving route 80 that parks vehicle 1 at the parking position 910 based on the teacher's drive by the driver and the surrounding images captured during the teacher's drive. For example, when learning the driving route 80, the learning unit 103 learns the change in the position of vehicle 1 based on the change in feature points extracted from multiple surrounding images captured during the teacher's drive. The learning unit 103 also learns the speed, steering angle, and braking operation of vehicle 1 during the teacher's drive. Furthermore, the learning unit 103 defines the environment around vehicle 1 as a map based on feature points extracted from surrounding images captured during the teacher's drive by the driver, and stores the map in the storage unit 108.

[0047] In this embodiment, "learning" includes storing the collected information in the storage unit 108. Furthermore, the method for storing the information regarding the travel route 80 and the aforementioned map in the storage unit 108 is not limited to that exemplified in this embodiment.

[0048] In this embodiment, the teacher drive is a drive in which the driver manually moves the vehicle 1 from a starting position 900 outside the parking position to the parking position 910. In this embodiment, the learning unit 103 learns the starting position 900 in the teacher drive as the starting position of the driving path 80. The distance from the starting position 900 to the parking position 910 is, for example, about 50m, but is not limited to this.

[0049] For example, the driver starts a training run with vehicle 1 stopped at a location different from the parking position 910, such as a desired location outside the garage 920, and drives vehicle 1 in reverse to the parking position 910 inside the garage 920. Note that the training run is not limited to reverse, and may include forward movement, or both forward and reverse movement.

[0050] During the teacher run, the acquisition unit 101 acquires surrounding images and vehicle information. Also during the teacher run, the extraction unit 102 extracts feature points from the surrounding images acquired by the acquisition unit 101. Based on the extracted feature points and the vehicle information at the time the surrounding images from which the feature points were extracted were taken, the learning unit 103 records the route taken by vehicle 1 under manual driving as the driving route 80. Specifically, if the driver performs a teacher run in which vehicle 1 is parked at the parking position 910 by reversing, the driving route 80 will also be the route in which vehicle 1 is parked at the parking position 910 by reversing. In this embodiment, the position where the teacher run begins becomes the starting position 900 of the driving route 80. The learning unit 103 stores the driving route information that defines the recorded driving route 80 in the storage unit 108.

[0051] The driving path information is stored in the memory unit 108 as information that associates, for example, the speed of vehicle 1 during training driving, steering angle, braking action, and feature points extracted from multiple surrounding images captured along with the movement of vehicle 1 during training driving, in a time-series manner. Note that the method for learning the driving path 80 and the manner in which the driving path is defined are not limited to this example.

[0052] The estimation unit 104 estimates the position of vehicle 1 and the position in which vehicle 1 can move to the parking position 910 by automatic driving based on the driving path 80, based on the surrounding image.

[0053] The position to which the vehicle can move to the parking position 910 by autonomous driving is, in other words, the position from which the vehicle 1 can start autonomous driving using home zone parking. Autonomous driving using home zone parking is, that is, autonomous driving based on the driving route 80 stored in the memory unit 108. Furthermore, the position to which the vehicle 1 can move to the parking position 910 by autonomous driving is an example of a target position in this embodiment. Note that the target position is not limited to a single point, but may be an area that encompasses a wider range than the vehicle body 12 of the vehicle 1. The target position may also be called a target area. In this embodiment, the target position to which the vehicle can move to the parking position 910 is the driving start position 900. The parking position 910 is the goal of the driving route using home zone parking, and is also called the final target position.

[0054] More specifically, when the driver has requested to start home zone parking via the reception unit 107 (described later), the estimation unit 104 reads the driving route 80 and the feature points (map) of the surrounding image used when learning the driving route 80 from the storage unit 108. Then, based on the driving route 80 and the feature points extracted from the surrounding image by the extraction unit 102, the estimation unit 104 estimates the position of the vehicle 1 and the starting position 900 from which the vehicle 1 can move to the parking position 910 by automatic driving based on the driving route 80.

[0055] The estimation unit 104 compares the feature points of the surrounding image during the learning of the driving route 80 with the feature points of the surrounding image at the time the operation to start home zone parking is received. The estimation unit 104 uses the feature points of the surrounding image at the time the operation to start home zone parking is received to estimate the position and orientation of vehicle 1 on the map read from the memory unit 108. The estimation unit 104 does not need to specify the driving start position 900 and the current position of vehicle 1 as absolute positions; it is sufficient to specify the relative positional relationship between the driving start position 900 and the current position of vehicle 1. The method used by the estimation unit 104 to estimate the position of vehicle 1 and the driving start position 900 is not limited to this example.

[0056] Furthermore, the estimation unit 104 determines whether the starting position 900 is within a specified distance from the vehicle 1, based on the estimated position of the vehicle 1 and the starting position 900. The specified distance is, for example, the distance at which the starting position 900 is included in the imaging range of the vehicle 1's imaging device 16, and the distance at which the starting position 900 is visible from the vehicle 1's windshield 180. As an example, the specified distance is approximately 5m, but it is not limited to this value. This specified distance is also called the first specified distance.

[0057] If the estimation unit 104 determines that the starting position 900 is within a specified distance from the vehicle 1, it sends the estimated position of the vehicle 1 and the starting position 900 to the output control unit 105, which will be described later. The determination of whether or not the starting position 900 is within a specified distance from the vehicle 1 may be performed by the output control unit 105.

[0058] Furthermore, even while the vehicle 1 is being driven automatically by the vehicle control unit 106 (described later), the estimation unit 104 estimates the position of the vehicle 1 based on feature points included in the driving route information and feature points extracted from surrounding images acquired during automatic driving. The estimation unit 104 sends the estimated position of the vehicle 1 to the vehicle control unit 106.

[0059] The estimation process for determining the position of vehicle 1 by the estimation unit 104 may be performed continuously while vehicle 1 is in motion, or it may be started after the user initiates home zone parking. For example, if the estimation process is performed outside the range of the map stored by the training drive, the estimation unit 104 cannot determine the position of vehicle 1. However, if vehicle 1 reaches the range of the map, the estimation unit 104 may notify the output control unit 105 that vehicle 1 is located near the starting position 900. Also, if the current position of vehicle 1 is the starting position 900, the estimation unit 104 may notify the output control unit 105 that vehicle 1 is located at the starting position 900. The estimation unit 104 may determine that vehicle 1 is located at the starting position 900 not only when the current position of vehicle 1 and the starting position 900 perfectly coincide, but also when the distance between the current position of vehicle 1 and the starting position 900 is less than or equal to a specified error.

[0060] The output control unit 105 causes the head-up display or display device 120 to display guidance information regarding the position in which the vehicle 1 can automatically move to the parking position 910.

[0061] The guidance information includes information that allows for automatic driving of vehicle 1 based on the driving route, and the orientation of vehicle 1 that enables automatic driving when the vehicle is located at that position. In this embodiment, the guidance information is an image showing the driving start position 900. In this embodiment, this image is referred to as the guidance image.

[0062] For example, the output control unit 105, when the estimation unit 104 estimates that vehicle 1 has reached the area of ​​the map stored through training driving, suggests to the user that they start home zone parking. The proposed method is not particularly limited, but the output control unit 105 may display a message prompting the user to start home zone parking on the display device 120, or it may output the message as an audible message from the speaker of vehicle 1.

[0063] Alternatively, a configuration may be adopted in which no such notification is sent, and the user initiates the Home Zone Parking operation at their own discretion. For example, suppose a user has driven vehicle 1 to near their home by manual driving, and then wishes to park vehicle 1 in their home garage 920 using Home Zone Parking. In this case, the user initiates Home Zone Parking by, for example, pressing operation button 141.

[0064] Furthermore, after the user manually moves vehicle 1 to a pre-programmed starting position 900, the user initiates an operation to start automatic driving by pressing the operation button 141 or similar, which initiates automatic driving by the vehicle control unit 106 described later. Alternatively, the system may be configured so that automatic driving starts automatically when the user stops vehicle 1.

[0065] When home zone parking is performed using this procedure, as described above, the user manually moves vehicle 1 to the starting position 900. If vehicle 1 stops at a position different from the starting position 900, correction is possible through feedback control by the vehicle control unit 106, as described later, provided that the distance between the starting position 900 and the actual stopping position of vehicle 1 is within an acceptable range. However, if the distance between the starting position 900 and the actual stopping position of vehicle 1 is significantly greater than the acceptable range, it may be difficult for vehicle 1 to move to the parking position 910 through automatic driving by the vehicle control unit 106.

[0066] Therefore, the output control unit 105 displays an image indicating the starting position 900 on the head-up display or display device 120, allowing the driver to understand the starting position 900. This makes it easier for the driver to stop the vehicle 1 at the starting position 900.

[0067] Furthermore, the output control unit 105 may notify the user when vehicle 1 is positioned in a location where it can perform automatic driving. For example, when vehicle 1 moves to the starting position 900, the output control unit 105 notifies the user that it is positioned at the starting position 900, that is, that automatic driving can begin. The method of notification is not particularly limited, but the output control unit 105 may display a message on the head-up display or display device 120 indicating that vehicle 1 is positioned at the starting position 900. Alternatively, the output control unit 105 may output the message as an audio from the speaker of vehicle 1.

[0068] Figure 6 shows an example of an image indicating the driving start position 900 displayed on the head-up display according to the first embodiment. As shown in Figure 6, the output control unit 105 causes the first guidance image 91a indicating the driving start position 900 to be displayed on the windshield 180, which functions as a head-up display.

[0069] In the example shown in Figure 6, the first guidance image 91a is a rectangular shape representing the size of the vehicle body 1. The orientation of the rectangle in the first guidance image 91a indicates the orientation in which vehicle 1 can automatically drive at the starting position 900. In Figure 6, the first guidance image 91a is a rectangle enclosed by a dashed line, but it could also be a shape enclosed by a solid line instead of a dashed line. Furthermore, the first guidance image 91a could be an image in which the longitudinal contour line is thicker than the transverse contour line, as shown in the second guidance image in Figure 7 below. Note that the display on the display device 120 is not shown in Figure 6.

[0070] The output control unit 105 causes a second guidance image indicating the starting position of the vehicle to be driven to be displayed on the front view or top view displayed on the display device 120.

[0071] Figure 7 is a diagram showing an example of an image indicating the starting position 900 of the vehicle as displayed on the display device 120 according to the first embodiment. As shown in Figure 7, the output control unit 105 causes the second guidance image 92a to be displayed on the front view 21, which represents the front direction of the vehicle 1. In the example shown in Figure 7, the front view 21 includes the first vehicle image 2a, which represents the position of the vehicle body 12 of the vehicle 1. The output control unit 105 also causes the second guidance image 92b to be displayed on the top view 22, which represents the vehicle 1 as seen from above. The top view 22 also includes the second vehicle image 2b, which represents the position of the vehicle body 12 of the vehicle 1.

[0072] In the example shown in Figure 7, the second guide images 92a and 92b are rectangular shapes representing the size of the vehicle body of vehicle 1. In the example shown in Figure 7, the longitudinal contour lines of the second guide images 92a and 92b are thinner than the transverse contour lines.

[0073] In this embodiment, the images displayed in both the front view 21 and the top view 22 are referred to as the second guide images, but the image on the top view 22 may also be referred to as the third image. Also, in Figure 7, the output control unit 105 displays the second guide images 92a and 92b in both the front view 21 and the top view 22, but they may be displayed in only one of them. Furthermore, in the front view 21 shown in Figure 7, the first vehicle image 2a representing the body 12 of vehicle 1 is depicted within the front view 21, but the front view 21 does not necessarily have to include the body 12 of vehicle 1.

[0074] In Figure 7, the output control unit 105 displays "P" on the second guidance images 92a and 92b to inform the driver that they should stop at the starting position 900, but this character does not have to be displayed. Alternatively, the output control unit 105 may display a message indicating an instruction to the driver, such as "Stop here," on or near the second guidance images 92a and 92b. Furthermore, the output control unit 105 may also display the character "P" or a message indicating an instruction to the driver on or near the first guidance image 91a on the head-up display.

[0075] Furthermore, in this embodiment, the output control unit 105 displays guidance information on both the head-up display and the display device 120, but it may also display the information on only one of the head-up display or the display device 120.

[0076] Returning to Figure 5, the vehicle control unit 106 moves vehicle 1 to the parking position 910 by automatic driving based on the driving path 80. The vehicle control unit 106 makes vehicle 1 drive automatically by controlling the steering, braking, and acceleration / deceleration of vehicle 1. In this embodiment, the vehicle control unit 106 makes vehicle 1 drive automatically along the learned driving path 80. This method of driving control is also called reproducing the driving path 80.

[0077] Furthermore, during automatic driving, the vehicle control unit 106 may cause vehicle 1 to perform actions different from those of the trained vehicle, based on the vehicle information and environmental information acquired by the acquisition unit 101. As described above, the environmental information includes the presence or absence of obstacles around vehicle 1 and the distance between vehicle 1 and obstacles around vehicle 1. For example, if an obstacle such as another vehicle approaches during automatic driving, the vehicle control unit 106 will perform control such as stopping vehicle 1.

[0078] In this embodiment, the driver may be seated in the driver's seat 130a of the vehicle 1 or may be out of the vehicle 1 while the vehicle control unit 106 is operating automatically.

[0079] Furthermore, if the vehicle 1 deviates from the travel path 80, the vehicle control unit 106 uses feedback control to move the vehicle 1 back onto the travel path 80. For example, based on the position of the vehicle 1 estimated by the estimation unit 104, the vehicle control unit 106 estimates the difference between the position of the vehicle 1 and the travel path 80, and drives the vehicle 1 to minimize this difference.

[0080] Furthermore, although Figure 5 shows the parking assist device 100 as comprising a vehicle control unit 106, the vehicle control unit 106 may be implemented by another ECU outside of the parking assist device 100.

[0081] The reception unit 107 receives various operations from the driver. For example, the reception unit 107 receives an operation to start parking assistance when, for example, the operation button 141 is pressed. Alternatively, if the display device 120 is a touch panel, the reception unit 107 may also receive an operation to start parking assistance when an image button on the touch panel is pressed.

[0082] Next, the flow of the parking assistance process performed by the parking assistance device 100 of this embodiment, configured as described above, will be explained.

[0083] Figure 8 is a flowchart showing an example of the flow of parking assistance processing performed by the parking assistance device 100 according to the first embodiment. This flowchart starts, for example, when the reception unit 107 receives a request to start parking assistance. It is assumed that the driving path 80 has been learned by the learning unit 103 before this flowchart is executed.

[0084] First, the acquisition unit 101 acquires a peripheral image from the imaging device 16 (S1).

[0085] Then, the extraction unit 102 extracts feature points from the surrounding image acquired by the acquisition unit 101 (S2).

[0086] Next, the estimation unit 104 reads the travel route information from the storage unit 108 and estimates the position of the vehicle 1 based on the travel route 80 defined in the travel route information and the feature points extracted from the surrounding image by the extraction unit 102 (S3).

[0087] Furthermore, the estimation unit 104 estimates the starting position 900 for automatic parking based on the driving path 80 and the feature points extracted from the surrounding image by the extraction unit 102 (S4).

[0088] Then, the estimation unit 104 determines whether or not the starting position 900 is within a specified distance from the vehicle 1, based on the estimated position of the vehicle 1 and the starting position 900 (S5).

[0089] If the estimation unit 104 determines that there is no starting position 900 within a specified distance from vehicle 1 (S5 "No"), the process returns to S1.

[0090] For example, if vehicle 1 is located further than a specified distance from the starting position 900, the output control unit 105 cannot display the starting position 900 on the head-up display or display device 120. In such cases, the estimation unit 104 repeats the processes S1 to S5 and waits for vehicle 1 to approach the starting position 900. In addition, if the feature points extracted from the surrounding image in the S2 process do not match the feature points extracted during the learning of the driving path 80, the estimation unit 104 will determine that the starting position 900 is not within a specified distance from vehicle 1.

[0091] If the estimation unit 104 determines that the starting position 900 is within a specified distance from the vehicle 1 (S5 "Yes"), the output control unit 105 displays the starting position 900.

[0092] Specifically, the output control unit 105 displays a first guidance image 91a indicating the starting position 900 on the windshield 180, which functions as a head-up display. The output control unit 105 also displays second guidance images 92a and 92b indicating the starting position 900 on the front view 21 and top view 22 displayed on the display device 120 (S6). This display allows the driver to understand the starting position 900 and move the vehicle 1 to the starting position 900 by manual driving.

[0093] Furthermore, while vehicle 1 moves to the starting position 900, the acquisition unit 101 continuously acquires surrounding images from the imaging device 16. The extraction unit 102 extracts feature points from the acquired surrounding images. The acquisition unit 101 also acquires vehicle information, the presence or absence of obstacles around vehicle 1, and the distance to any obstacles. The estimation unit 104 estimates the position of vehicle 1 as it moves, based on the movement of feature points extracted from the surrounding images or the vehicle information. The output control unit 105 moves the positions of the first guidance image 91a and the second guidance images 92a and 92b in accordance with the movement of vehicle 1.

[0094] The vehicle control unit 106 then determines whether or not vehicle 1 has stopped at the starting position 900 (S7). This determination process may also be performed by the estimation unit 104. If vehicle 1 has not reached the starting position 900 (S7 "No"), the vehicle control unit 106 waits for vehicle 1 to reach the starting position 900 and stop.

[0095] Then, when vehicle 1 reaches the starting position 900 and stops (S7 "Yes"), the vehicle control unit 106 starts automatic driving (S8). The vehicle control unit 106 then drives vehicle 1 from the starting position 900 along the driving path 80 to the parking position 910 using automatic driving. At this point, the processing of this flowchart ends. Note that in S7 and S8, automatic driving is started when vehicle 1 reaches the starting position 900 and stops, but it may also be stated that automatic driving is started when the user requests to start automatic driving after vehicle 1 has reached the starting position 900 and stopped.

[0096] However, the conditions for initiating autonomous driving are not limited to these. In addition to vehicle 1 stopping at the starting position 900, autonomous driving may also be initiated when the driver performs a start operation using the operation button 141 or touch panel. Alternatively, autonomous driving may be initiated when the driver stops vehicle 1 at the starting position 900, exits vehicle 1, and performs a start operation from outside vehicle 1 using an operation terminal such as a remote controller or electronic key.

[0097] Thus, in the parking assistance method performed by the parking assistance device 100 of this embodiment, information regarding the driving path 80 during the training drive is stored in the storage unit 108, and guidance information regarding a target position where the vehicle 1 can be driven automatically based on the stored driving path 80 is displayed on the head-up display or display device 120. For this reason, the parking assistance method performed by the parking assistance device 100 of this embodiment makes it easy for the driver to understand the position where the vehicle 1 can be driven automatically based on the driving path 80.

[0098] Furthermore, in the parking assistance method performed by the parking assistance device 100 of this embodiment, a driving path 80 for parking the vehicle 1 at the parking position 910 is stored based on surrounding images captured during the driver's test drive. In this embodiment, the target position is the position from which the vehicle 1 can move to the parking position 910 by automatic driving. In addition, in the parking assistance method of this embodiment, the position of the vehicle 1 and the target position from which the vehicle 1 can move to the parking position 910 by automatic driving based on the driving path 80 are estimated based on second sensor information acquired from the area around the vehicle 1. Therefore, according to the parking assistance method performed by the parking assistance device 100 of this embodiment, the driver can easily understand the position from which the vehicle 1 can move to the parking position 910 by automatic driving based on the driving path 80.

[0099] Furthermore, in the parking assistance method performed by the parking assistance device 100 of this embodiment, the guidance information displayed on the head-up display or display device 120 includes the target position and the orientation of the vehicle 1 so that the vehicle 1 can be driven automatically when the vehicle 1 is located at the target position. Therefore, according to the parking assistance method performed by the parking assistance device 100 of this embodiment, the driver can easily understand the direction in which the vehicle 1 should face when moving the vehicle 1 to the target position.

[0100] Furthermore, in the parking assistance method performed by the parking assistance device 100 of this embodiment, the starting position 900 of the driving path 80 is displayed on the head-up display or display device 120 as the position from which the vehicle 1 can move to the parking position 910. Therefore, with the parking assistance method performed by the parking assistance device 100 of this embodiment, it becomes easy for the driver to manually drive the vehicle 1 to the starting position 900 by visually confirming the display.

[0101] Furthermore, in the parking assistance method performed by the parking assistance device 100 of this embodiment, a first guidance image 91a indicating the starting position 900 is displayed on the head-up display. Therefore, according to the parking assistance method performed by the parking assistance device 100 of this embodiment, the driver can grasp the starting position 900 while facing forward.

[0102] Furthermore, in the parking assistance method performed by the parking assistance device 100 of this embodiment, a second guidance image 92a, 92b indicating the starting position 900 is displayed on the front view 21 or top view 22 displayed on the display device 120 provided on the dashboard 190 of the vehicle 1. Therefore, according to the parking assistance method performed by the parking assistance device 100 of this embodiment, the driver can easily grasp the positional relationship between the vehicle 1 and the starting position 900.

[0103] Furthermore, as described above, in the parking assistance method performed by the parking assistance device 100 of this embodiment, the user may be notified that automatic driving can begin when the vehicle 1 is located at the target position. Such notification allows the user to easily understand that the vehicle 1 has reached a position where automatic driving can begin.

[0104] (Second embodiment) In the first embodiment described above, the parking assist device 100 displayed the starting position 900 of the learned driving path 80. In contrast, in this second embodiment, the parking assist device 100 displays an area where the vehicle 1 can merge onto the driving path 80 midway.

[0105] The configuration of the vehicle 1 in this embodiment is the same as that of the first embodiment described in Figures 1 and 2. Furthermore, the hardware configuration and functional blocks of the parking assist device 100 in this embodiment are the same as those of the first embodiment described in Figures 4 and 5. Similar to the first embodiment, the parking assist device 100 in this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0106] Furthermore, the acquisition unit 101, extraction unit 102, learning unit 103, and reception unit 107 of this embodiment have the same functions as those of the first embodiment.

[0107] In addition to the functions of the first embodiment, the estimation unit 104 of this embodiment estimates the position of the vehicle 1 and the region in which the vehicle 1 can merge onto the learned driving path 80 midway, based on a surrounding image captured of the area around the vehicle 1. More specifically, the estimation unit 104 estimates the range in which the vehicle 1 can be returned to the driving path 80 and reach the parking position 910 by feedback control of the vehicle control unit 106.

[0108] Figure 9 is a diagram illustrating an example of parking assistance according to the second embodiment. The strip-shaped region 801 along the driving path 80 shown in Figure 9 is the range in which it is estimated that the vehicle 1 can merge with the learned driving path 80 midway. When the vehicle 1 is located in this region 801, the vehicle control unit 106 can use feedback control to return the vehicle 1 to the driving path 80 and move it to the parking position 910. In other words, this region 801 is the position in which the vehicle 1 can start automatic driving using home zone parking. In this embodiment, this region 801 is called the parking start area 801.

[0109] The width of the parking area 801 narrows as the distance to the parking position 910 decreases. For example, in the example shown in Figure 9, the width d2 of the parking area 801 at point A2, which is closer to the parking position 910, is narrower than the width d1 of the parking area 801 at point A1, which is closer to the driving start position 900. Furthermore, if the distance to the parking position 910 becomes even shorter, the area 801 disappears. This is because if the parking position 910, which is the goal of the driving route 80, is too close, it becomes difficult to return vehicle 1 to the driving route 80. The location where the parking area 801 disappears varies depending on, for example, the road conditions and the size of the space around the driving route 80. The width of the parking area 801 may also be predetermined in correspondence with the distance from the parking position 910.

[0110] In addition to the functions of the first embodiment, the output control unit 105 causes guidance information regarding the parking start area 801 to be displayed on the head-up display or display device 120.

[0111] Figure 10 shows an example of guidance information regarding the parking start area 801 according to the second embodiment. In the example shown in Figure 10, the head-up display shows a route guidance image 901a showing the driving route 80 and an area guidance image 901b showing the parking start area 801. The route guidance image 901a and the area guidance image 901b are examples of guidance information in this embodiment. Note that only the area guidance image 901b may be used as an example of guidance information.

[0112] In the example shown in Figure 10, for example, the area guidance image 901b corresponding to the parking start area 801 near point A1, as explained in Figure 9, is displayed. As the distance to the parking position 910 decreases, the width of the area guidance image 901b narrows.

[0113] Figure 11 shows another example of guidance information regarding the parking start area 801 according to the second embodiment. In the example shown in Figure 11, for example, a region guidance image 901b corresponding to the parking start area 801 near point A2, as described in Figure 9, is displayed. Since the width d2 of the parking start area 801 at point A2 is narrower than the width d1 of the parking start area 801 at point A1, the width of the region guidance image 901b displayed on the head-up display in Figure 11 is narrower than the width of the region guidance image 901b shown in Figure 10.

[0114] The display mode of the area guidance image 901b is not limited to the example shown in Figure 10. For example, the output control unit 105 does not need to distinguish between the driving route 80 and the parking start area 801. In this case, the output control unit 105 displays only the area guidance image 901b without displaying the route guidance image 901a. Alternatively, the output control unit 105 may display the area guidance image 901b on the front view 21 or top view 22 displayed on the display device 120.

[0115] In addition to the functions of the first embodiment, the vehicle control unit 106 performs automatic driving control to move the vehicle 1, which has stopped in the parking start area 801, so as to merge with the driving path 80, and then park it at the parking position 910.

[0116] In this embodiment, an example was given in which a route guidance image 901a and an area guidance image 901b are displayed. However, similar to the first embodiment, a first guidance image 91a or a second guidance image 92a, 92b indicating the starting position 900 may also be displayed.

[0117] Next, the flow of the parking assistance process performed by the parking assistance device 100 of this embodiment, configured as described above, will be explained.

[0118] Figure 12 is a flowchart showing an example of the flow of the parking assistance process performed by the parking assistance device 100 according to the second embodiment.

[0119] The process from acquiring the surrounding image in S1 to estimating the position of vehicle 1 in S3 is the same as in the first embodiment. Then, the estimation unit 104 estimates the parking start area 801 (S101).

[0120] Furthermore, the estimation unit 104 determines whether or not the parking start area 801 is within a specified distance from the vehicle 1 (S102). In this embodiment, the specified distance is, for example, the distance at which the parking start area 801 is included in the imaging range of the vehicle 1's imaging device 16, and the distance at which the parking start area 801 is visible from the vehicle 1's windshield 180.

[0121] If the estimation unit 104 determines that there is no parking start area 801 within a specified distance from the vehicle 1 (S102 "No"), the process returns to S1.

[0122] Furthermore, if the estimation unit 104 determines that there is a parking start area 801 within a specified distance from the vehicle 1 (S102 "Yes"), the output control unit 105 displays the parking start area 801 on the head-up display or display device 120 (S103). For example, the output control unit 105 causes the head-up display or display device 120 to display a route guidance image 901a and an area guidance image 901b.

[0123] Then, the vehicle control unit 106 determines whether or not vehicle 1 has stopped in the parking start area 801 (S104). If vehicle 1 has not reached the parking start area 801 (S104 "No"), the vehicle control unit 106 waits for vehicle 1 to reach the parking start area 801 and stop.

[0124] Then, when vehicle 1 reaches the parking start area 801 and stops (S104 "Yes"), the vehicle control unit 106 starts automatic driving (S8). The vehicle control unit 106 then performs automatic driving to move vehicle 1 along the travel path 80. After that, the vehicle control unit 106 drives vehicle 1 along the travel path 80 to the parking position 910. At this point, the processing of this flowchart ends.

[0125] As described above, in the parking assistance method performed by the parking assistance device 100 of this embodiment, the driving route 80 for parking the vehicle 1 at the parking position 910 is learned based on the driver's training drive and surrounding images taken during the training drive. Then, the position of the vehicle 1 and the parking start area 801 are estimated based on surrounding images acquired before the start of parking assistance. In this parking assistance method, guidance information regarding the parking start area 801 is displayed on the head-up display or display device 120. Therefore, in addition to the effects of the first embodiment, the parking assistance method performed by the parking assistance device 100 of this embodiment allows the driver to easily understand the area in which the vehicle can move to the parking position 910 by automatic driving, even from the middle of the driving route 80. With this parking assistance method, the driver can understand positions other than the driving start position 900 during the training drive where home zone parking can be started, and the degree of freedom of the position in which the driver stops the vehicle 1 can be increased.

[0126] (Third embodiment) In the first and second embodiments described above, the parking assist device 100 estimated the position of the vehicle 1 based on feature points extracted from the surrounding image. In this third embodiment, the parking assist device 100 estimates the position of the vehicle 1 by combining other position estimation methods and provides assistance to the driver according to the estimated position.

[0127] Vehicle 1 of this embodiment has the same configuration as the first embodiment described in Figures 1 and 2. In addition, Vehicle 1 of this embodiment is further equipped with an antenna capable of receiving GPS (Global Positioning System) signals and a GPS device that identifies GPS coordinates representing the position of Vehicle 1 based on the received GPS signals. Vehicle 1 is also equipped with a gyro sensor that detects the turning speed (angular velocity) of Vehicle 1 and a sensor that measures the rotation speed of the wheels 13 of Vehicle 1.

[0128] Furthermore, the hardware configuration and functional blocks of the parking assistance device 100 in this embodiment are the same as those of the first embodiment described in Figures 4 and 5. The parking assistance device 100 in this embodiment, like the first embodiment, includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0129] Furthermore, the extraction unit 102, learning unit 103, vehicle control unit 106, and reception unit 107 of this embodiment have the same functions as those of the first embodiment.

[0130] In addition to the functions of the first embodiment, the acquisition unit 101 of this embodiment acquires location information of the vehicle 1. More specifically, the acquisition unit 101 acquires GPS coordinates, i.e., GPS location information, representing the position of the vehicle 1 from a GPS device. The acquisition unit 101 also acquires the rotational speed or angular velocity of the vehicle 1's wheels 13 from a gyro sensor and other sensors. The acquisition unit 101 may acquire this information directly from the various sensors or from other ECUs.

[0131] In addition to the functions of the first embodiment, the estimation unit 104 of this embodiment estimates the distance from the position of vehicle 1 based on the position information acquired by the acquisition unit 101 to the parking position 910. Furthermore, the estimation unit 104 may estimate the position of vehicle 1 not only based on GPS position information, but also based on the amount of movement of vehicle 1 since the time the GPS position information was acquired. The estimation unit 104 may estimate the amount of movement of vehicle 1 from the rotation speed or angular velocity of the vehicle 1's wheels 13. For example, after identifying the coordinates of a reference point using GPS position information, the estimation unit 104 may estimate the position of vehicle 1 after movement by estimating the distance and direction of movement of vehicle 1 from the reference point based on the rotation speed or angular velocity of the vehicle 1's wheels 13.

[0132] In addition to the functions of the first embodiment, the output control unit 105 of this embodiment notifies the user that the parking position 910 is close when the distance from the parking position 910 to the vehicle 1 is below a threshold. The threshold is not particularly limited, but may be, for example, about 20m. This threshold is also called the second specified distance.

[0133] Figure 13 illustrates an example of parking assistance according to the third embodiment. For example, the distance from point A4 to parking position 910 shown in Figure 13 is assumed to be below a threshold. When vehicle 1 is located at point A4, the output control unit 105 notifies the user that parking position 910 is approaching. The content of the notification is not particularly limited, but it may be a message prompting the user to start parking assistance, such as "You are approaching home. Shall we start parking assistance?". The output control unit 105 may output the message as voice from the speaker of vehicle 1. Alternatively, the output control unit 105 may display the message on a head-up display or display device 120. For example, if the driver initiates automatic parking to parking position 910 after the notification of the message, the estimation unit 104 estimates the position of vehicle 1 and the position to which vehicle 1 can move to parking position 910 by automatic driving based on the driving path 80, based on the surrounding image, similar to the first or second embodiment.

[0134] Furthermore, as vehicle 1 approaches parking position 910, the estimation unit 104 can estimate the position of vehicle 1 based on feature points extracted from the surrounding image using the same method as in the first embodiment. For example, if vehicle 1 is located at point A3 shown in Figure 13, the estimation unit 104 estimates the position of vehicle 1 based on feature points extracted from the surrounding image. The position at which the estimation unit 104 can estimate the position of vehicle 1 based on feature points extracted from the surrounding image is, for example, about 5m from the starting position 900, but is not limited to this value.

[0135] Next, the flow of the parking assistance process performed by the parking assistance device 100 of this embodiment, configured as described above, will be explained.

[0136] Figure 14 is a flowchart showing an example of the flow of the parking assistance process performed by the parking assistance device 100 according to the third embodiment.

[0137] First, the acquisition unit 101 acquires the location information of vehicle 1 (S201).

[0138] Then, the estimation unit 104 estimates the distance from the position of vehicle 1 based on the acquired position information to the parking position 910 (S202).

[0139] The estimation unit 104 determines whether the estimated distance is below a threshold (S203). This determination process may also be performed by the output control unit 105.

[0140] If the estimation unit 104 determines that the estimated distance is longer than the threshold (S203 "No"), the process returns to S1.

[0141] Furthermore, if the estimation unit 104 determines that the estimated distance is below a threshold (S203 "Yes"), the output control unit 105 notifies the user that the parking position 910 is close (S204).

[0142] The reception unit 107 then determines whether or not it has received an operation from the driver to start automatic parking (S205). If the reception unit 107 has not received an operation to start parking (S205 "No"), it repeats the process in S205 and waits for an operation from the driver.

[0143] If the reception unit 107 receives the operation to start parking (S205 "Yes"), the process proceeds to S1, which is the process of acquiring the surrounding image. The processes from S1 to S8 are the same as those of the first embodiment described in Figure 8. The processes from S1 onward may also be the same as those of the second embodiment described in Figure 12.

[0144] Thus, in the parking assistance method performed by the parking assistance device 100 of this embodiment, when the distance from the position of the vehicle 1 to the parking position 910 is below a threshold, the driver is notified that the parking position 910 is close. As a result, the parking assistance method of this embodiment provides the functions of the first embodiment while allowing the driver to easily understand the appropriate timing for initiating automatic parking.

[0145] (Variation 1) Although examples of displays in the head-up display and display device 120 have been described in each of the embodiments described above, the manner in which the driving start position 900 or the parking start area 801 is displayed is not limited to the examples described above.

[0146] Figure 15 shows an example of an image indicating the starting position 900 of the vehicle as displayed on the head-up display according to Modification 1. The output control unit 105 of this modification displays a first guidance image 91b indicating the starting position of the vehicle as displayed on the windshield 180, which functions as a head-up display. In this modification, the first guidance image 91b is in the shape of an arrow, and the starting point of the arrow corresponds to the starting position of the vehicle as it is

[0147] (Modification 2) Figure 16 is a diagram showing an example of an image indicating the starting position 900 of driving, displayed on the display device 120 according to Modification 2. In the example shown in Figure 16, the second guidance image 92c on the front view 21 and the second guidance image 92d on the top view 22 are both dashed lines representing the position corresponding to the longitudinal direction of the vehicle body 12 when the vehicle body 12 of the vehicle 1 is stopped at the starting position of driving. In this display configuration, the driver can more easily be aware of driving the vehicle 1 so that the vehicle body 12 is positioned between the two dashed lines.

[0148] Furthermore, the second guidance images 92c and 92d may be solid lines instead of dashed lines. Similarly, the first guidance image may be displayed on the head-up display using dashed or solid lines to represent positions corresponding to the longitudinal direction of the vehicle body 12.

[0149] (Variation 3) Furthermore, the first and second guidance images do not necessarily have to be shaped to match the shape of the vehicle body 12; for example, they may be circular or elliptical.

[0150] Figure 17 shows an example of an image indicating the starting position 900 of the vehicle displayed on the display device 120 according to the modified example 3. As shown in Figure 17, the second guidance image 92e on the front view 21 and the second guidance image 92f on the top view 22 are ellipses corresponding to the size of the vehicle body 12 of the vehicle 1. Similarly, a circular or elliptical first guidance image may also be displayed on the head-up display.

[0151] (Modification 4) Furthermore, the guidance information may also be an image that clearly indicates the direction that vehicle 1 should face in order to perform automatic driving. For example, Figure 18 is a diagram showing an example of a second guidance image 92g displayed on the display device 120 according to Modification 4. In the example shown in Figure 18, the second guidance image 92g on the top view 22 includes an arrow image 921a that indicates the direction that vehicle 1 should face at the starting position 900. Therefore, when a user manually drives vehicle 1 to the starting position 900, the user can easily understand the direction that vehicle 1 should face at the starting position 900.

[0152] (Variation 5) The display method for clearly indicating the direction that vehicle 1 should be facing is not limited to that exemplified in Modification 4. Figure 19 shows an example of a second guidance image 92h displayed on the display device 120 according to Modification 5. In the example shown in Figure 19, the second guidance image 92h on the top view 22 represents the orientation of vehicle 1 at the starting position 900 and the change in the orientation of vehicle 1 along the driving path 80 extending from the starting position 900. With this display method, the user can easily understand the direction in which vehicle 1 will be traveling by the automated driving starting from the starting position 900.

[0153] Although Figures 18 and 19 illustrate the top view 22, the output control unit 105 may similarly display an arrow image 921a indicating the direction the vehicle 1 should face, or a second guidance image 92h indicating a change in the direction of the vehicle 1, in the front view 21 or the head-up display.

[0154] (Experimental variation 6) Figure 20 shows an example of the first guidance image 91c displayed on the head-up display according to the modified example 6. In the example shown in Figure 20, the first guidance image 91c represents the direction that vehicle 1 should face along the travel path 80 in the direction of travel of vehicle 1. Alternatively, the first guidance image 91c may be an image that represents the path for vehicle 1 to move from its current position to the starting position 900, along with the direction that vehicle 1 should face.

[0155] (Example 7) Furthermore, in each of the embodiments described above, the parking assistance device 100 used the driving path during the teacher's driving phase during learning as the driving path 80 for automatic driving in the home zone parking area. However, the learning method is not limited to this.

[0156] For example, the teacher drive may be a drive in which the driver manually moves vehicle 1 from the parking position 910 to the teacher drive end position outside the parking position. In this case, the learning unit 103 generates a drive path 80 for home zone parking by replaying the teacher drive path in reverse.

[0157] For example, the driver starts a training run with vehicle 1 stopped at parking position 910 in garage 920, and drives vehicle 1 forward to exit garage 920. The driver then drives vehicle 1 forward to a desired training run end position and stops vehicle 1. The learning unit 103 then records the training run based on steering, braking, acceleration / deceleration, and feature points extracted from surrounding images during the training run. Based on the recorded training run, the learning unit 103 generates a driving path 80 in which vehicle 1 reverses from the training run end position to the parking position 910 in garage 920. In this method, the training run end position becomes the automatic driving start position 900. This method is also called playing the driving path in reverse.

[0158] Generally, in manual driving, it is easier to move vehicle 1 out of garage 920 than to move vehicle 1 into garage 920. If such a learning method is used, even if the driver is not good at reversing vehicle 1 into garage 920, a driving path 80 for home zone parking can be generated by a training run in which the driver moves vehicle 1 forward to exit garage 920. In this modified example, the case in which the driver moves vehicle 1 forward during the training run was explained as an example, but the driver may also move vehicle 1 in reverse during the training run.

[0159] (Variation 8) In the embodiments described above, a peripheral image captured around the vehicle 1 during a training run was used as an example of the first sensor information. In addition, in the embodiments described above, a peripheral image captured around the vehicle 1 before or during automatic driving based on a learned driving path was used as an example of the second sensor information. However, the first and second sensor information are not limited to these examples.

[0160] For example, detection results from various sensors such as the transmitter / receiver unit 15 may be used as an example of the first sensor information or the second sensor information. Vehicle 1 may also be equipped with a ranging device such as LiDAR (Light Detection and Ranging, or Laser Imaging Detection and Ranging) or RADAR (Radio Detection and Ranging). In this case, detection results of the area around vehicle 1 by LiDAR or RADAR may be used as an example of the first sensor information or the second sensor information. Furthermore, the first sensor information and the second sensor information may be a combination of two or more of the following: surrounding images, detection results from the transmitter / receiver unit 15, detection results from LiDAR, or detection results from RADAR. The first sensor information and the second sensor information may also include information collected by other devices.

[0161] (Fourth embodiment) In this fourth embodiment, the parking assistance device 100 changes the display mode of the guidance information before and after the vehicle 1 reaches a position where it can merge onto the driving path 80.

[0162] The configuration of the vehicle 1 in this embodiment is the same as that of the first embodiment described in Figures 1 and 2. Furthermore, the hardware configuration and functional blocks of the parking assist device 100 in this embodiment are the same as those of the first embodiment described in Figures 4 and 5. Similar to the first embodiment, the parking assist device 100 in this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0163] Furthermore, the acquisition unit 101, extraction unit 102, learning unit 103, vehicle control unit 106, and reception unit 107 of this embodiment have the same functions as those of the first embodiment.

[0164] The output control unit 105 has the functions of the first embodiment and changes the display mode of the guidance information before and after the vehicle 1 reaches a position where it can merge onto the travel path 80. The changes in the display mode of the guidance information will be explained using Figures 21 to 23.

[0165] Figure 21 shows an example of guidance information displayed before the vehicle 1 according to the fourth embodiment reaches a position where it can merge onto the travel path 80. In the example shown in Figure 21, both the front view 21 and the top view 22 are displayed on the display device 120, but either the front view 21 or the top view 22 may be displayed.

[0166] The timing for displaying the guidance information shown in Figure 21 is, for example, when the estimation unit 104 determines that there is a driving start position 900 or an area where vehicle 1 can merge onto the driving path 80 within a specified distance from vehicle 1. This embodiment is applicable both when vehicle 1 starts automatic driving from the starting position of the driving path 80 and when vehicle 1 merges onto the driving path 80 from the middle of it.

[0167] As shown in Figure 21, the output control unit 105 displays the following as guidance information on the top view 22: a driving route image 800, a driving area image 930a, and a second vehicle image 2b representing the position of the vehicle body 12 of vehicle 1. The output control unit 105 moves the display position of the second vehicle image 2b on the top view 22 as vehicle 1 moves.

[0168] Furthermore, the output control unit 105 displays the driving area image 930b on the front view 21. Alternatively, the output control unit 105 may display a first vehicle image 2a on the front view 21, representing the position of the vehicle body 12 of the vehicle 1.

[0169] Furthermore, the output control unit 105 displays a start button 121 on the display device 120. The start button 121 is an image button for instructing the start of automatic parking based on the driving route 80. Note that the displayed text and shape of the start button 121 are not limited to the example shown in Figure 21.

[0170] The driving path image 800 is an image representing the driving path 80 during the training run. More specifically, the driving path image 800 is a line showing the movement history of the reference point of vehicle 1 as vehicle 1 travels along the driving path 80. The reference point of vehicle 1 is, for example, the midpoint of the line connecting the left and right rear tires 13r of vehicle 1, but is not limited to this.

[0171] The driving area image 930a is an image representing the driving area through which the outer shape of vehicle 1 passes when vehicle 1 travels along the driving path 80. The driving area image 930a may be a strip-shaped image with a width equal to the width of vehicle 1 centered on the driving path 80, or it may be a strip-shaped image showing the circumscribing area, which is the area that the outer shape of vehicle 1 actually passed through during the training run. The circumscribing area shall also include the area through which the vehicle body 12 passes due to the inner and outer wheel differences of the front tires 13f and rear tires 13r of vehicle 1.

[0172] The driving area is an example of the area and target location in which the vehicle 1 in this embodiment can merge onto the driving path 80. The driving area images 930a and 930b are examples of guidance information in this embodiment.

[0173] Furthermore, the output control unit 105 may prompt the driver to move the vehicle 1 to the position represented by the driving area images 930a and 930b by displaying a message M1 such as "Please stop at the displayed parking start position" on the display device 120. The content and display position of the message M1 are not limited to the example shown in Figure 21. The output control unit 105 may also output the message M1 as an audio message from a speaker (not shown).

[0174] In the example shown in Figure 21, a portion of the vehicle body 12 of vehicle 1 overlaps with the driving area, but vehicle 1 has not yet reached a position where it can merge onto the driving path 80. In this case, the output control unit 105 displays the start button 121 in a state where it cannot be pressed. For example, the output control unit 105 displays the start button 121 in a grayed-out state. Furthermore, if the user attempts to operate the start button 121, which is displayed in a state where it cannot be pressed, the reception unit 107 will not accept the user's operation.

[0175] Furthermore, if the vehicle 1 has not yet reached a position where it can merge onto the travel path 80, the output control unit 105 causes the travel area images 930a and 930b to be displayed on the display device 120 in a first color. The first color is not particularly limited, but for example, it may be orange. The first color is an example of a first display mode.

[0176] Figure 22 shows an example of guidance information displayed after the vehicle 1 according to the fourth embodiment has reached a position where it can merge onto the travel path 80.

[0177] In the example shown in Figure 22, vehicle 1 has reached a position where it can merge onto the travel path 80. In this case, the output control unit 105 causes the travel area images 930a and 930b to be displayed on the display device 120 in a second color. The second color is not particularly limited, but it is a different color from the first color, for example, green. The second color is an example of a second display mode.

[0178] The color changes in the driving area images 930a and 930b are just one example of a change in the display mode of the driving area images 930a and 930b, and the changes in the display mode are not limited to this. For example, the output control unit 105 may make the driving area images 930a and 930b blink until the vehicle 1 reaches a position where it can merge onto the driving path 80, and then stop blinking after the vehicle 1 reaches a position where it can merge onto the driving path 80.

[0179] Furthermore, the output control unit 105 displays the start button 121 in a pressable state when vehicle 1 reaches a position where it can merge onto the travel path 80. For example, when vehicle 1 reaches a position where it can merge onto the travel path 80, the output control unit 105 displays the grayed-out start button 121 in its normal state.

[0180] Furthermore, if the user operates the start button 121, which is displayed in a pressable state, the reception unit 107 accepts the user's operation. When the user presses the start button 121, the vehicle control unit 106 starts automatic parking based on the driving route 80. Note that the user's operation to instruct the start of automatic parking based on the driving route 80 is not limited to pressing the start button 121, but may also be voice input or operation of a physical button, etc.

[0181] Figure 23 shows an example of guidance information displayed after the vehicle 1 according to the fourth embodiment starts automatic parking along the driving path 80. In the example shown in Figure 23, when the vehicle 1 starts automatic parking along the driving path 80, the output control unit 105 terminates the display of the driving area image 930a in the top view 22.

[0182] Furthermore, the output control unit 105 continues to display the driving route image 800. The output control unit 105 may also display a parking position image 911 representing the parking position 910 in the top view 22, as shown in Figure 23. By displaying the driving route image 800 and the parking position image 911, the user can understand the planned route of the vehicle 1 during automatic driving. The output control unit 105 may also display the parking position image 911 before the user inputs an instruction to start automatic parking. For example, the output control unit 105 may display the parking position image 911 in the top view 22 shown in Figures 21 and 22.

[0183] Furthermore, the output control unit 105 displays the start button 121 in a different display mode after it has been pressed than it did before. For example, the output control unit 105 may change the color of the start button 121 before and after it has been pressed.

[0184] In the example shown in Figure 23, the output control unit 105 continues to display the driving area image 930b on the front view 21 even after the vehicle 1 has started automatic parking along the driving path 80, but this display may also be terminated.

[0185] Figures 21 to 23 illustrate the display device 120, but the output control unit 105 may also display guidance information on a head-up display.

[0186] The estimation unit 104 of this embodiment has the functions of the first embodiment and determines whether or not the vehicle 1 has reached a position where it can merge onto the travel path 80. For example, the estimation unit 104 determines that the vehicle 1 has reached a position where it can merge onto the travel path 80 if the position and orientation of the vehicle 1 satisfy the specified conditions.

[0187] The prescribed condition for the position of vehicle 1 is, for example, that the distance from the current reference point of vehicle 1 to the travel path 80 is within a prescribed distance. The prescribed condition for the orientation of vehicle 1 is, for example, that the difference between the current orientation of vehicle 1 and the orientation of vehicle 1 when vehicle 1 is located at the closest position to the reference point of vehicle 1 on the travel path 80 is less than or equal to a prescribed angle.

[0188] In this embodiment, the specified conditions for the position of vehicle 1 and the specified conditions for the orientation of vehicle 1 are AND conditions for determining that vehicle 1 has reached a position where it can merge onto the travel path 80. In other words, the vehicle control unit 106 can start automatic parking when the position and orientation of vehicle 1 meet the specified conditions, but cannot start automatic parking if the position and orientation of vehicle 1 do not meet the specified conditions. Therefore, if the position and orientation of vehicle 1 do not meet the specified conditions, the driver must manually move vehicle 1 to a position where it can merge onto the travel path 80. Note that the values ​​of the specified distance and specified angle are not particularly limited and will vary depending on the steering characteristics of vehicle 1 and the size of the vehicle body 12, etc.

[0189] Here, the method for determining whether or not vehicle 1 has reached a position where it can merge onto the travel path 80 will be explained using Figures 24 and 25.

[0190] Figure 24 shows an example of the state before the vehicle 1 according to the fourth embodiment reaches a position where it can merge onto the travel path 80. Figure 25 shows an example of the state after the vehicle 1 according to the fourth embodiment has reached a position where it can merge onto the travel path 80. In Figures 24 and 25, the positional relationship between the vehicle 1 and the travel path 80 is explained using a top view 22.

[0191] In the example shown in Figure 24, the position of vehicle 1 is described with reference to a reference point 30, which is the midpoint of the line connecting the left and right rear tires 13r of vehicle 1. The estimation unit 104 estimates the distance d10 between the reference point 30 of vehicle 1 and the position 81 on the travel path 80 closest to the reference point 30 as the distance from the current position of the reference point 30 of vehicle 1 to the travel path 80.

[0192] Furthermore, the estimation unit 104 adopts the direction of a straight line 41 that passes through the reference point 30 of the vehicle 1 and intersects perpendicularly with the line connecting the left and right rear tires 13r of the vehicle 1 as the direction of the vehicle 1. Also, in Figure 24, the direction of the vehicle 1 when the vehicle 1 is located at position 81 on the travel path 80 is represented as a straight line 42. The estimation unit 104 estimates the angle θ between the straight lines 41 and the straight line 41 as the difference between the current direction of the vehicle 1 and the direction of the vehicle 1 when the vehicle 1 is located at the position closest to the reference point of the vehicle 1 on the travel path 80. The estimation unit 104 may also detect the difference between the current direction of the vehicle 1 and the direction of the vehicle 1 during the training run as the difference in the direction the vehicle 1 is facing.

[0193] In the example shown in Figure 24, the distance d10 from the reference point 30 of vehicle 1 to the travel path 80 is longer than the specified distance, and the angle θ between the straight line 41 and the straight line 42 is greater than the specified angle. In such a position and orientation, the vehicle control unit 106 cannot start automatic parking. In this case, the estimation unit 104 determines that vehicle 1 has not reached a position where it can merge onto the travel path 80. As mentioned above, the specified conditions for the position of vehicle 1 and the specified conditions for the orientation of vehicle 1 are AND conditions, so even if one of the conditions is met, if the other condition is not met, the estimation unit 104 determines that vehicle 1 has not reached a position where it can merge onto the travel path 80. This determination process is an example of the determination steps in this embodiment.

[0194] Furthermore, in the example shown in Figure 25, as the vehicle 1 moves, the distance d10 between the reference point 30 of the vehicle 1 and the position 81 on the travel path 80 closest to the reference point 30 becomes shorter than in the example shown in Figure 24. Also, the angle θ between the straight line 41 and the straight line 42 is smaller than in the example shown in Figure 24.

[0195] In the example shown in Figure 25, the distance d10 from the reference point 30 of vehicle 1 to the travel path 80 is assumed to be less than or equal to a specified distance, and the angle θ between straight line 41 and straight line 42 is greater than a specified angle. In this case, the estimation unit 104 determines that vehicle 1 has reached a position where it can merge onto the travel path 80. In this case, the vehicle control unit 106 can also start automatic parking.

[0196] Furthermore, the process for determining whether or not vehicle 1 has stopped at the starting position 900 in the first to third embodiments may also be performed using the position of the reference point 30 of vehicle 1 and the orientation of vehicle 1, as shown in Figures 24 and 25. For example, the estimation unit 104 may determine that vehicle 1 has stopped at the starting position 900 if the distance between the current reference point 30 of vehicle 1 and the position of the reference point 30 when vehicle 1 is located at the starting position 900 during the teacher run is within a specified distance, and the difference between the current orientation of vehicle 1 and the orientation of vehicle 1 when vehicle 1 is located at the starting position 900 during the teacher run is less than or equal to a specified angle.

[0197] Furthermore, in the first to third embodiments, when the estimation unit 104 determined that there was a parking start area 801 within a predetermined distance from the vehicle 1, the output control unit 105 displayed the driving start position 900 or the parking start area 801 on the head-up display or display device 120. However, in this process as well, the condition that the orientation of the vehicle 1 satisfies a predetermined condition may be added as a condition for displaying the driving start position 900 or the parking start area 801.

[0198] Next, the flow of the parking assistance process performed by the parking assistance device 100 of this embodiment, configured as described above, will be explained.

[0199] Figure 26 is a flowchart showing an example of the flow of the parking assistance process performed by the parking assistance device 100 according to the fourth embodiment.

[0200] The process from acquiring the surrounding image of S1 shown in Figure 26 to estimating the starting position of the automatic parking drive in S4 is the same as in the first embodiment.

[0201] Then, the estimation unit 104 determines, based on the estimated position of vehicle 1 and the starting position 900, whether there is an area within a predetermined distance from vehicle 1 where the starting position 900 or vehicle 1 can merge onto the travel path 80 (S301). Note that in the processing of S5 in the processing flow of the first to third embodiments, the determination may also be made whether there is an area within a predetermined distance from vehicle 1 where it can merge onto the travel path 80, similar to S301.

[0202] Next, the output control unit 105 causes the display device 120 to display the driving route image 800 and the driving area images 930a and 930b in the first display mode, as shown in Figure 21. The output control unit 105 also causes the display device 120 to display the start button 121 in a state where it cannot be pressed (S302). At this time, the output control unit 105 may output a message M1 as an audio message from a speaker (not shown) prompting the driver to move the vehicle 1 to the position represented by the driving area images 930a and 930b.

[0203] Then, the estimation unit 104 determines whether the position and orientation of vehicle 1 meet the specified conditions based on the feature points extracted from the acquired surrounding image and the driving path information read from the storage unit 108 (S303).

[0204] If the position and orientation of vehicle 1 do not meet the specified conditions (S303 "No"), the estimation unit 104 determines that vehicle 1 has not reached a position where it can merge onto the travel path 80. In this case, the process in S303 is repeatedly executed until the driver manually drives vehicle 1 to a position where it can merge onto the travel path 80.

[0205] Furthermore, if the position and orientation of vehicle 1 meet the specified conditions (S303 "Yes"), the estimation unit 104 determines that vehicle 1 has reached a position where it can merge onto the travel path 80. In this case, the output control unit 105 changes the travel area images 930a and 930b to a second display mode, as shown in Figure 22. The output control unit 105 also displays the start button 121 on the display device 120 in a state where it can be pressed (S304).

[0206] Furthermore, if the start button 121, which is displayed as pressable, is not pressed by the user (S305 "No"), the reception unit 107 waits for user input.

[0207] Then, when the user presses the start button 121, which is displayed in a pressable state (S305 "Yes"), the reception unit 107 receives an operation from the user instructing the start of automatic parking based on the driving path 80. The reception unit 107 sends a message to the output control unit 105 indicating that it has received the instruction to start automatic parking based on the driving path 80. In this case, as shown in Figure 23, the output control unit 105 ends the display of the driving area image 930a and displays the parking position image 911 on the display device 120 (S306). Then, in this case, the vehicle control unit 106 starts automatic driving (S8). At this point, the processing of this flowchart ends.

[0208] Thus, the parking assistance device 100 of this embodiment changes the display mode of the guidance information before and after the vehicle 1 reaches a position where it can merge onto the driving path 80. Therefore, it achieves the same effects as the first embodiment, while also allowing the driver of the vehicle 1 to easily understand whether or not the vehicle 1 has reached a position where it can merge onto the driving path 80.

[0209] Furthermore, the parking assist device 100 of this embodiment displays the driving area images 930a and 930b in a first color on the display device 120 before the vehicle 1 reaches a position where it can merge onto the driving path 80, and displays the driving area images 930a and 930b in a second color different from the first color on the display device 120 after the vehicle 1 has reached a position where it can merge onto the driving path 80. Therefore, with the parking assist device 100 of this embodiment, it becomes easy for the driver of the vehicle 1 to visually recognize whether or not the vehicle 1 has reached a position where it can merge onto the driving path 80.

[0210] Furthermore, the parking assist device 100 of this embodiment determines that the vehicle 1 has reached a position where it can merge onto the driving path 80 when the position and orientation of the vehicle 1 meet the specified conditions. Therefore, the parking assist device 100 of this embodiment can determine with high accuracy whether or not the vehicle 1 is in a state where it can be driven automatically by the vehicle control unit 106.

[0211] Furthermore, the configuration of the parking assistance device 100 in this embodiment may be combined with the configurations of the above-described modifications 1 to 8.

[0212] (Fifth embodiment) This fifth embodiment describes automatic parking based on a driving path 80 recorded by a method other than teacher driving.

[0213] The configuration of the vehicle 1 in this embodiment is the same as that of the first embodiment described in Figures 1 and 2. Furthermore, the hardware configuration and functional blocks of the parking assist device 100 in this embodiment are the same as those of the first embodiment described in Figures 4 and 5. Similar to the first embodiment, the parking assist device 100 in this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, an output control unit 105, a vehicle control unit 106, a reception unit 107, and a storage unit 108.

[0214] Furthermore, the acquisition unit 101, extraction unit 102, estimation unit 104, and reception unit 107 of this embodiment have the same functions as those of the first embodiment.

[0215] The vehicle control unit 106 of this embodiment is equipped with an automatic parking function based on autonomous driving. More specifically, the vehicle control unit 106 generates a driving path for parking the vehicle 1 at a parking target position determined by the user, and performs automatic parking by moving the vehicle 1 to the parking target position based on the generated driving path. Note that the automatic parking by the vehicle control unit 106 of this embodiment can employ known technologies. The parking operation by autonomous driving is an example of a parking operation that has been performed in the past in this embodiment.

[0216] The target parking position during autonomous driving may be a designated area marked by white lines, or it may be a space without any markings such as white lines. If there are no markings such as white lines, the user may input the target parking position on the front view 21 or top view 22 displayed on the display device 120. The reception unit 107 receives the user's input of the target parking position.

[0217] In this embodiment, the learning unit 103 stores the travel path 80, the starting position 900, and the parking target position traveled by the vehicle 1 during automatic parking by the vehicle control unit 106 in the storage unit 108. For example, the learning unit 103 learns the change in the position of the vehicle 1 based on the change in feature points extracted from multiple surrounding images captured during automatic parking. The learning unit 103 also learns the speed, steering angle, and braking operation of the vehicle 1 during automatic parking. Furthermore, the learning unit 103 defines the environment around the vehicle 1 as a map based on feature points extracted from surrounding images captured during automatic driving, and stores the map in the storage unit 108.

[0218] Furthermore, the learning unit 103 may store in the storage unit 108 only the driving route 80, driving start position 900, and parking target position for each of the multiple automatic parking operations that have been selected by the user as items to be stored, while discarding information about the driving route 80, driving start position 900, and parking target position for each operation that was not selected by the user as items to be stored.

[0219] For example, the learning unit 103 may temporarily store the driving route 80, driving start position 900, and parking target position traveled by the vehicle 1 during automatic parking by the vehicle control unit 106 in the storage unit 108. After the automatic parking is completed, if the user selects these items to be stored, the learning unit 103 may permanently register the driving route 80, driving start position 900, and parking target position as driving route information to be used from the next time onward.

[0220] In this embodiment, the output control unit 105 causes the vehicle control unit 106 to display guidance information based on the driving route information recorded by past automatic parking operations on the head-up display or display device 120.

[0221] Figure 27 is a diagram showing an example of guidance information according to the fifth embodiment. In the example shown in Figure 27, the output control unit 105 displays a top view 22 on the display device 120. The top view 22 displays a second vehicle image 2b, a driving area image 930a representing the driving area through which the vehicle 1 passes when driving along the driving path 80 in past automatic parking by the vehicle control unit 106, and a parking position image 911 representing the parking position 910.

[0222] The output control unit 105 may display the front view 21 or the driving route image 800.

[0223] Furthermore, similar to the fourth embodiment, the output control unit 105 of this embodiment changes the display mode of the guidance information before and after the vehicle 1 reaches a position where it can merge onto the travel path 80.

[0224] In the example shown in Figure 27, similar to the example shown in Figure 22, since vehicle 1 has reached a position where it can merge onto the travel path 80, the output control unit 105 displays the travel area images 930a and 930b in the second display mode. If vehicle 1 has not yet reached a position where it can merge onto the travel path 80, the output control unit 105 displays the travel area images 930a and 930b in the first display mode.

[0225] Thus, even when the parking assistance device 100 of this embodiment performs automatic parking based on a driving path 80 recorded by a parking operation performed by automated driving in the past, it achieves the same effects as the first embodiment, and allows the driver of vehicle 1 to easily understand whether or not the vehicle 1 has reached a position where it can merge onto the driving path 80.

[0226] Furthermore, the configuration of the parking assistance device 100 in this embodiment may be combined with the configurations of the above-described modifications 1 to 8.

[0227] In this embodiment, the parking assist device 100 stores the driving path 80 during the parking operation by automatic driving. However, the learning unit 103 of the parking assist device 100 only needs to store information regarding at least one of the driving path 80 and the parking target position in the storage unit 108. For example, the learning unit 103 may not store the driving path 80, but only store the parking target position during automatic parking. For example, the parking target position is a position set by the user as the parking target position for vehicle 1 during the parking operation by automatic driving.

[0228] In this case, the vehicle control unit 106 generates a driving route that automatically drives the vehicle 1 from its current position to a previously registered parking target position. The output control unit 105 then displays guidance information based on the driving route generated by the vehicle control unit 106 on the head-up display or display device 120. For example, the output control unit 105 may display a strip-shaped image representing the driving area along the generated driving route.

[0229] Furthermore, for example, the learning unit 103 may store the target parking position and the starting position 900 for automatic parking. In this case, the output control unit 105 may display guidance information based on the starting position 900 for automatic parking on the head-up display or display device 120.

[0230] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0231] 1 vehicle 2a Image of the first vehicle 2b Second vehicle image 12 car bodies 13ft front tire 13r rear tire 13 wheels 15, 15f, 15r Transmitter / receiver section 16 Imaging device 21 Front View 22 Top View 30 reference points 80 Route 91a, 91b, 91c First guide image 92a,92b,92c,92d,92e,92f,92g,92h Second guide image 100 Parking assist system 101 Acquisition Department 102 Extraction part 103 Learning Department 104 Estimation part 105 Output control unit 106 Vehicle Control Unit 107 Reception Department 108 Storage section 120 Display device 121 Start button 130a Driver's seat 140 Steering Wheel 141 Operation Buttons 180 Windshield 190 Dashboard 800 Driving route images 801 Parking start area 900 Starting position 901a Route guidance image 901b Area Guide Image 910 Parking spot 911 Parking location image 920 Garage 930a, 930b Driving area images d10 distance M1 Message

Claims

1. A parking assistance method that performs automatic vehicle driving based on driver-guided driving, The system stores information regarding the driving route during the driver's instructional run. Based on the aforementioned driving path, the vehicle can move to the parking position through the automated driving, and guidance information is displayed on the display unit based on the target position from which the vehicle can rejoin the aforementioned driving path. The width of the second area at the second point from which a vehicle can merge onto the aforementioned driving path is narrower than the width of the first area at the first point from which a vehicle can merge onto the aforementioned driving path. The second point is a point on the travel route that is closer to the parking position than the first point. Parking assistance methods.

2. The width of the first region and the width of the second region are the widths in the vehicle width direction along the travel path. The parking assistance method according to claim 1.

3. The aforementioned target position is the starting position of the travel path. The parking assistance method according to claim 1 or 2.

4. The first region and the second region are strip-shaped regions along the travel path. The parking assistance method according to any one of claims 1 to 3.

5. To control the vehicle to the aforementioned target position. Parking assistance method according to any one of claims 1 to 4.

6. Based on first sensor information regarding the surroundings of the vehicle acquired by sensors mounted on the vehicle during the driver's test run, the driving path is stored. The parking assistance method according to any one of claims 1 to 5.

7. Based on second sensor information acquired around the vehicle, the position of the vehicle and the target position are estimated. The parking assistance method according to claim 6.

8. The aforementioned display unit is a head-up display. The parking assistance method according to any one of claims 1 to 7.

9. The display unit is a display installed on the dashboard of the vehicle. The parking assistance method according to any one of claims 1 to 7.

10. A parking assist system that performs automatic vehicle driving based on driver-guided driving, A storage unit that stores information regarding the driving route during the driver's training run, The system includes an output control unit that displays guidance information on a display unit based on a target position where the vehicle can move to a parking position by the automatic driving based on the aforementioned driving path and where the vehicle can rejoin the aforementioned driving path. The width of the second area at the second point from which a vehicle can merge onto the aforementioned driving path is narrower than the width of the first area at the first point from which a vehicle can merge onto the aforementioned driving path. The second point is a point on the travel route that is closer to the parking position than the first point. Parking assist system.

11. The first region and the second region are strip-shaped regions along the travel path. Parking assistance device according to claim 10.

12. The aforementioned target position is the starting position of the travel path. Parking assistance device according to claim 10 or 11.

13. The first region and the second region are strip-shaped regions along the travel path. A parking assistance device according to any one of claims 10 to 12.

14. The vehicle control unit further comprises a vehicle control unit that causes the vehicle to be controlled to the target position. A parking assistance device according to any one of claims 10 to 13.

15. The storage unit stores the driving path based on first sensor information regarding the surroundings of the vehicle, which is acquired by sensors mounted on the vehicle during the driver's test drive. A parking assistance device according to any one of claims 10 to 14.

16. The system further includes an estimation unit that estimates the position of the vehicle and the target position based on second sensor information acquired from the area around the vehicle. The parking assistance device according to claim 15.

17. The aforementioned display unit is a head-up display. A parking assistance device according to any one of claims 10 to 16.

18. The display unit is a display installed on the dashboard of the vehicle. A parking assistance device according to any one of claims 10 to 16.

Images