Parking support device and parking support method

JP2023095423A5Inactive Publication Date: 2024-11-11PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2021211302
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2021-12-24
Publication Date
2024-11-11
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing parking assistance systems fail to maintain user convenience when vehicles deviate from pre-registered travel routes to avoid obstacles, leading to user confusion and inconvenience.

Method used

A parking assistance device equipped with a travel control unit, detection unit, and correction unit that adjusts travel routes based on real-time object detection, allowing vehicles to navigate around obstacles while displaying corrected routes on a display.

Benefits of technology

Ensures user convenience by maintaining awareness of route changes and avoiding obstacles, reducing unexpected vehicle behavior and user anxiety during automatic parking.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

To provide a parking support device and a parking support method capable of preventing deterioration in convenience of a user even when a vehicle travels on a route which is different from a preliminarily registered route.SOLUTION: A parking support device comprises a travel control part, a detection part, a correction part and a display control part. The travel control part allows a vehicle to enter or leave by making the vehicle automatically travel on the basis of a travel route recorded on the basis of past parking operations. The detection part detects objects in the periphery of the vehicle. The correction part corrects the travel route on the basis of a detection result of the objects. The display control part displays the travel route corrected by the correction part in a display part during the execution of the automatic travel.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a parking assist device and a parking assist method.

Background Art

[0002] Conventionally, there has been known a technology of path memory type automatic parking in which a driving route is recorded based on a teaching drive by a driver and the vehicle is automatically driven based on the recorded driving route. This technology is used, for example, when entering or leaving a parking lot.

[0003] In addition, among vehicles that perform such path memory type automatic parking, there are some that have a function of automatically performing a detour operation for stopping or avoiding when detecting an unknown obstacle that is stationary.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when the vehicle travels on a route different from the pre-registered driving route due to detouring of an obstacle or the like, the user cannot grasp the route, which may impair the user's convenience.

[0006] The present disclosure provides a parking assist device and a parking assist method capable of reducing impairment of user convenience even when the vehicle travels on a route different from a pre-registered driving route.

Means for Solving the Problems

[0007] The parking assistance device according to this disclosure comprises a driving control unit, a detection unit, a correction unit, and a display control unit. The driving control unit automatically drives the vehicle in or out of a parking space based on a driving path recorded based on past parking operations. The detection unit detects objects around the vehicle. The correction unit corrects the driving path based on the object detection results. The display control unit displays the driving path corrected by the correction unit on the display unit while automatic driving is being performed. [Effects of the Invention]

[0008] According to the parking assistance device and parking assistance method described herein, even when a vehicle travels a route different from a pre-registered route, it is possible to reduce the inconvenience to the user. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of a vehicle equipped with a parking assist device according to the first embodiment. [Figure 2] Figure 2 shows an example of the configuration near the driver's seat of a vehicle according to the first embodiment. [Figure 3] Figure 3 shows an example of the hardware configuration of the parking assistance device according to the first embodiment. [Figure 4] Figure 4 is a block diagram showing an example of the functions provided by the parking assistance device according to the first embodiment. [Figure 5] Figure 5 shows an example of a driving route recorded by a teacher driving according to the first embodiment. [Figure 6] Figure 6 shows an example of a correction path according to the first embodiment. [Figure 7] Figure 7 shows an example of a correction path image displayed on the display device according to the first embodiment. [Figure 8] Figure 8 shows an example of a corrected path image different from that shown in Figure 7. [Figure 9]FIG. 9 is a flowchart showing an example of the flow of the automatic driving and driving route correction processes executed by the parking support device according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a corrected route image displayed on the display of the smartphone according to Modification 1 of the first embodiment. [Figure 11] FIG. 11 is a block diagram showing an example of the functions provided by the parking support device according to the second embodiment. [Figure 12] FIG. 12 is a diagram showing an example of the driving route database according to the second embodiment. [Figure 13] FIG. 13 is a diagram showing an example of the user information database according to the second embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a driving route notification screen displayed on the display of the smartphone according to the second embodiment. [Figure 15] FIG. 15 is a diagram showing an example of a driving route selection screen displayed on the display of the smartphone according to the second embodiment. [Figure 16] FIG. 16 is a sequence diagram showing an example of the flow of the processes executed by the parking support device and the smartphone according to the second embodiment. [Figure 17] FIG. 17 is a diagram showing an example of the configuration of a vehicle usage schedule management system according to Modification 1 of the second embodiment. [Figure 18] FIG. 18 is a diagram showing an example of the usage schedule database stored in the cloud server according to Modification 1 of the second embodiment. [Figure 19] FIG. 19 is a diagram showing an example of the driving route database according to Modification 2 of the second embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

[0010] Hereinafter, embodiments of the parking support method and the parking support device according to the present disclosure will be described with reference to the drawings.

[0011] (First Embodiment) FIG. 1 is a diagram showing an example of a vehicle 1 equipped with a parking assistance device 100 according to a first embodiment. As shown in FIG. 1, the vehicle 1 includes a vehicle body 12 and two pairs of wheels 13 arranged along a predetermined direction on the vehicle body 12. The two pairs of wheels 13 include a pair of front tires 13f and a pair of rear tires 13r.

[0012] The front tire 13f shown in FIG. 1 is an example of a first wheel in the present embodiment. The rear tire 13r is an example of a second wheel in the present embodiment. The two front tires 13f and the two rear tires 13r of the vehicle 1 are collectively referred to as the wheels 13. Although the vehicle 1 shown in FIG. 1 includes four wheels 13, the number of wheels 13 is not limited thereto. For example, the vehicle 1 may be a two-wheeled vehicle.

[0013] The vehicle body 12 is coupled to the wheels 13 and is movable by the wheels 13. In this case, the predetermined direction in which the two pairs of wheels 13 are arranged is the traveling direction of the vehicle 1. The vehicle 1 can move forward or backward by switching gears (not shown) or the like. The vehicle 1 can also turn right or left by steering.

[0014] The vehicle body 12 also has a front end portion F which is an end portion on the front tire 13f side and a rear end portion R which is an end portion on the rear tire 13r side. The vehicle body 12 is substantially rectangular in top view, and the four substantially rectangular corner portions may also be referred to as end portions. Although not shown in FIG. 1, the vehicle 1 includes a display device, a speaker, a microphone, and an operation unit.

[0015] A pair of bumpers 14 are provided near the lower ends of the front and rear end portions F and R of the vehicle body 12. Of the pair of bumpers 14, the front bumper 14f covers the entire front surface and a part of the side surface near the lower end portion of the vehicle body 12. Of the pair of bumpers 14, the rear bumper 14r covers the entire rear surface and a part of the side surface near the lower end portion of the vehicle body 12.

[0016] Transmitting and receiving units 15f and 15r for transmitting and receiving sound waves such as ultrasonic waves are arranged at predetermined ends of the vehicle body 12. For example, one or more transmitting and receiving units 15f are arranged on the front bumper 14f, and one or more transmitting and receiving units 15r are arranged on the rear bumper 14r. Hereinafter, unless otherwise specified, the transmitting and receiving units 15f and 15r will simply be referred to as the transmitting and receiving unit 15. Furthermore, the number and location of the transmitting and receiving units 15 are not limited to the example shown in Figure 1. For example, the vehicle 1 may have transmitting and receiving units 15 on the left and right sides.

[0017] In this embodiment, a sonar using ultrasound is described as an example of the transmitter / receiver unit 15, but the transmitter / receiver unit 15 may be a radar that transmits and receives electromagnetic waves. Alternatively, the vehicle 1 may be equipped with both sonar and radar. Furthermore, the transmitter / receiver unit 15 may simply be referred to as a sensor.

[0018] The transmitting / receiving unit 15 detects obstacles around the vehicle 1 based on the results of transmitting and receiving sound waves or electromagnetic waves. The transmitting / receiving unit 15 also measures the distance between the vehicle 1 and the obstacles around the vehicle 1 based on the results of transmitting and receiving sound waves or electromagnetic waves.

[0019] Vehicle 1 is also equipped with a first onboard camera 16a that photographs the front of Vehicle 1, a second onboard camera 16b that photographs the rear of Vehicle 1, a third onboard camera 16c that photographs the left side of Vehicle 1, and a fourth onboard camera that photographs the right side of Vehicle 1. The fourth onboard camera is not shown in the illustration.

[0020] Hereinafter, unless otherwise specified, the first in-vehicle camera 16a, the second in-vehicle camera 16b, the third in-vehicle camera 16c, and the fourth in-vehicle camera will simply be referred to as in-vehicle camera 16. The location and number of in-vehicle cameras are not limited to the example shown in Figure 1. For example, vehicle 1 may also be equipped with only two cameras, the first in-vehicle camera 16a and the second in-vehicle camera 16b. Alternatively, vehicle 1 may have other in-vehicle cameras in addition to those described above.

[0021] The in-vehicle camera 16 is capable of capturing images of the area around the vehicle 1, and is, for example, a camera that captures color images. The images captured by the in-vehicle camera 16 may be videos or still images. The in-vehicle camera 16 may be a camera built into the vehicle 1, or it may be a camera of a drive recorder or the like that which is attached to the vehicle 1.

[0022] Furthermore, the vehicle 1 is equipped with a parking assist device 100. The parking assist device 100 is an information processing device that can be mounted on the vehicle 1, and is, for example, an ECU (Electronic Control Unit) or OBU (On Board Unit) located inside the vehicle 1. Alternatively, the parking assist device 100 may be an external device installed near the dashboard of the vehicle 1. The parking assist device 100 may also function as a car navigation system or the like.

[0023] The parking assistance device 100 of this embodiment learns a driving route based on a driver's training drive and uses the learned result to provide parking assistance. Such parking assistance is effective in reducing the effort required for the driver to park when repeatedly parking in a fixed parking location, such as the driver's home garage, a contracted parking space in an apartment building, or a designated parking space in a parking lot at their workplace. Such parking assistance is called home zone parking or route memory type automatic parking.

[0024] In addition to the configuration shown in Figure 1, vehicle 1 may also be equipped with various sensors, such as a gyro sensor and wheel speed sensors (not shown). The gyro sensor measures the rotational behavior of vehicle 1, including forward / backward movement, left / right movement, and turning. The wheel speed sensors measure the wheel speed of each wheel 13 of vehicle 1.

[0025] Next, the configuration of the area near the driver's seat of the vehicle 1 in this embodiment will be described. Figure 2 is a diagram showing an example of the configuration of the area near the driver's seat 130a of the vehicle 1 in the first embodiment.

[0026] As shown in Figure 2, the vehicle 1 is equipped with a driver's seat 130a and a passenger seat 130b. In front of the driver's seat 130a are a windshield 180, a dashboard 190, a steering wheel 140, a display device 120, and control buttons 141.

[0027] The display device 120 is a display provided on the dashboard 190 of the vehicle 1. For example, as shown in Figure 2, the display device 120 is located in the center of the dashboard 190. The display device 120 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display device 120 may also function as a touch panel. The display device 120 is an example of a display unit in this embodiment.

[0028] Furthermore, the steering wheel 140 is located in front of the driver's seat 130a and can be operated by the driver. The rotation angle of the steering wheel 140, i.e., the steering angle, is electrically or mechanically linked to the change in direction of the front tires 13f, which are the steering wheels. Note that the steering wheels may also be the rear tires 13r, or both the front tires 13f and the rear tires 13r may be steering wheels.

[0029] The operation button 141 is a button that can accept input from the user. In this embodiment, the user is, for example, the driver of vehicle 1. The location of the operation button 141 is not limited to the example shown in Figure 2, and may be provided on the steering wheel 140, for example. Also, although Figure 2 shows one operation button 141, multiple operation buttons 141 may be provided. The operation button 141 is an example of an operation unit. Furthermore, if the display device 120 also functions as a touch panel, the display device 120 may be an example of an operation unit.

[0030] Next, the hardware configuration of the parking assist device 100 will be described.

[0031] Figure 3 shows an example of the hardware configuration of the parking assistance device 100 according to the first embodiment. As shown in Figure 3, the parking assistance device 100 has a CPU (Central Processing Unit) 11A, ROM 11B, RAM 11C, device I / F (interface) 11D, CAN (Controller Area Network) I / F 11E, NW (Network) I / F 11F, HDD (Hard Disk Drive) 11G, etc., all interconnected by a bus 11H, and has a hardware configuration that uses a normal computer.

[0032] The CPU 11A is a processing unit that controls the entire parking assistance system 100. Note that the CPU 11A is just one example of a processor in the parking assistance system 100 of this embodiment, and other processors or processing circuits may be provided instead of the CPU 11A.

[0033] ROM11B, RAM11C, and HDD11G function as memory units. For example, ROM11B stores programs that implement various processes performed by CPU11A. RAM11C is, for example, the main memory of the parking assist device 100 and stores data necessary for various processes performed by CPU11A.

[0034] Device I / F 11D is an interface that can connect to various devices. For example, Device I / F 11D connects to a GPS device 11l and obtains GPS location information indicating the current position of vehicle 1 from the GPS device 11l. This GPS location information is, for example, the latitude and longitude values ​​indicating the absolute position of vehicle 1.

[0035] The GPS device 11l is a device that identifies GPS coordinates representing the position of vehicle 1 based on GPS signals received by the GPS antenna 11J. The GPS antenna 11J is an antenna capable of receiving GPS signals.

[0036] Furthermore, the device I / F 11D acquires images and detection results from the on-board camera 16 and the transmitter / receiver 15. The device I / F 11D may also acquire measurement results from a gyro sensor and wheel speed sensor (not shown) mounted on the vehicle 1.

[0037] The CAN I / F11E is an interface for sending and receiving information between the vehicle 1 and other ECUs installed in the vehicle 1 via the CAN bus within the vehicle 1. Note that communication standards other than CAN may also be used.

[0038] The NW I / F11F is a communication device that can communicate with information processing equipment outside of Vehicle 1 via a network. The NW I / F11F can communicate with information processing equipment outside of Vehicle 1 via public networks such as LTE (Long Term Evolution) (registered trademark), Wi-Fi (registered trademark), or short-range communication such as Bluetooth (registered trademark). However, the communication format between Vehicle 1 and information processing equipment outside of Vehicle 1 is not limited to these. Information processing equipment outside of Vehicle 1 can be, for example, a server or a mobile terminal such as a smartphone.

[0039] In Figures 1-3, the display device 120 is shown as a separate device from the parking assist device 100, but the display device 120 may be included in the parking assist device 100.

[0040] Furthermore, the parking assist device 100 may also include a speaker and a microphone (not shown) in addition to the configuration shown in Figure 3. Alternatively, the device I / F 11D of the parking assist device 100 may perform input and output of audio data with a speaker and a microphone (not shown) provided in the vehicle 1 as a separate device from the parking assist device 100.

[0041] The parking assistance device 100 of this embodiment learns a driving path based on a driver's instructional driving and uses the learned result to provide parking assistance. In other words, the parking assistance method performed by the parking assistance device 100 is a method of automatically driving the vehicle 1 based on a driver's instructional driving. Such a parking assistance method is effective in reducing the effort required for parking when repeatedly parking in a fixed parking location, such as the driver's home garage, a contracted parking space in an apartment building, or a designated parking space in a parking lot at work.

[0042] Next, the functions of the parking assist device 100 of this embodiment will be described in detail. Figure 4 is a block diagram showing an example of the functions of the parking assist device 100 according to the first embodiment.

[0043] As shown in Figure 4, the parking assistance device 100 of this embodiment includes a reception unit 101, an acquisition unit 102, an extraction unit 103, a route recording unit 104, an estimation unit 105, a detection unit 106, a correction unit 107, an image generation unit 108, a display control unit 109, an audio output unit 110, a driving control unit 111, and a storage unit 150.

[0044] The storage unit 150 is composed of, for example, a ROM 11B, a RAM 11C, or an HDD 11G. Although Figure 4 shows one storage unit 150 included in the parking assist device 100, multiple storage media may function as the storage unit 150.

[0045] The memory unit 150 stores programs and data used in various processes performed by the parking assist device 100. For example, the program performed by the parking assist device 100 in this embodiment is a module configuration including the above-mentioned functional units (reception unit 101, acquisition unit 102, extraction unit 103, route recording unit 104, estimation unit 105, detection unit 106, correction unit 107, image generation unit 108, display control unit 109, audio output unit 110, and driving control unit 111). In actual hardware, the CPU 11A reads the program from the memory unit 150 and executes it, loading each of the above units onto the RAM 11C, and generating the reception unit 101, acquisition unit 102, extraction unit 103, route recording unit 104, estimation unit 105, detection unit 106, correction unit 107, image generation unit 108, display control unit 109, audio output unit 110, and driving control unit 111 onto the RAM 11C. Furthermore, the processing performed by each functional unit of the parking assistance device 100 is also called a step.

[0046] The program executed by the parking assist device 100 of this embodiment is provided as an installable or executable file recorded on a computer-readable recording medium such as flash memory, CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0047] Furthermore, the program executed by the parking assistance device 100 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the program executed by the parking assistance device 100 of this embodiment may be provided or distributed via a network such as the Internet. Alternatively, the program executed by the parking assistance device 100 of this embodiment may be pre-installed in a ROM 11B or the like and provided.

[0048] The reception unit 101 receives various operations from the user. For example, when the operation button 141 or an image button on the touch panel is pressed, the reception unit 101 receives the user's operation to start and stop recording the teacher's run.

[0049] Furthermore, the reception unit 101 may also accept various operations from the user based on the user's voice input from a microphone installed in the vehicle 1.

[0050] The acquisition unit 102 acquires multiple images taken in chronological order around the vehicle 1 as the vehicle 1 moves, from multiple in-vehicle cameras 16a to 16c. These images are referred to as peripheral images in this embodiment because they capture the area around the vehicle 1.

[0051] Furthermore, the acquisition unit 102 acquires information regarding the distance between the vehicle 1 and objects around it. This information includes, for example, the presence or absence of obstacles detected by the transmitter / receiver unit 15, and the length of the distance between the detected obstacles around the vehicle 1 and the vehicle 1. The presence or absence of obstacles detected by the transmitter / receiver unit 15, the distance between the vehicle 1 and obstacles around the vehicle 1 measured by the transmitter / receiver unit 15, and the surrounding image may be referred to as environmental information regarding the environment around the vehicle 1. However, environmental information is not limited to this information.

[0052] Furthermore, the acquisition unit 102 acquires GPS location information from the GPS device 11l via the device I / F 11D.

[0053] Furthermore, the acquisition unit 102 acquires vehicle information of vehicle 1 from various sensors or other ECUs of vehicle 1. The vehicle information includes, for example, information regarding the speed, steering angle, and braking operation of vehicle 1. Each piece of information included in the vehicle information of vehicle 1 is stored in the storage unit 150 in association with the time when each piece of information was detected. The vehicle information of vehicle 1 may also further include wheel speed or rotational speed of wheels 13, acceleration of vehicle 1 measured by a gyro sensor, etc.

[0054] The extraction unit 103 extracts feature points around the vehicle 1 from surrounding images captured by the on-board camera 16, or from the results of sensing the area around the vehicle 1 by the transmitter / receiver unit 15. The method used by the extraction unit 103 to extract feature points is not particularly limited, and any known method may be applied. For example, the extraction unit 103 may extract feature points using methods such as FAST (Features from Accelerated Segment Test) or ORB (Oriented FAST and Rotated BRIEF). Furthermore, when learning the driving route 80, the extraction unit 103 may prioritize recording feature points that satisfy specified conditions from among the extracted feature points. For example, among a series of consecutive surrounding images, feature points extracted from images where the vehicle 1 traveled a long distance during shooting may be prioritized as feature points.

[0055] The route recording unit 104 records the route taken by vehicle 1 during the driver's training run. The route recording unit 104 estimates the position of vehicle 1 during the training run based on changes in feature points extracted by the extraction unit 103 and vehicle information of vehicle 1, and identifies the route taken from the time-series changes in that position.

[0056] More specifically, the route recording unit 104 identifies changes in the position of vehicle 1 based on the time-series changes in feature points extracted during the training run. The route recording unit 104 may also correct the position of vehicle 1 identified from the feature points based on acquired vehicle information. Furthermore, the route recording unit 104 may estimate the position of vehicle 1 during the training run from the change in the absolute position of vehicle 1 based on the GPS position information of vehicle 1.

[0057] In this embodiment, the driving route for automatic driving, the speed of vehicle 1 traveling along the driving route, steering angle, braking operation, and information that associates feature points extracted during training driving in a time series are referred to as driving route information. Since the driving route of vehicle 1 is defined by the driving route information, the process of collecting driving route information is the process of recording the driving route of vehicle 1. The route recording unit 104 stores the driving route information in the storage unit 150. The driving route information, or corrected route information obtained by correcting the driving route information by the correction unit 107 described later, is used by the driving control unit 111 during automatic driving described later. Note that the method of recording the driving route and the definition of driving route information are not limited to this example.

[0058] Furthermore, the route recording unit 104 defines the environment around vehicle 1 as map information based on feature points extracted from surrounding images taken during the training run, and stores this map information in the storage unit 150. The process of recording the driving route based on the training run may also be called the learning process. The driving route recorded by the training run is used as a reference for correction by the correction unit 107 described later, and is therefore also called the reference route.

[0059] Figure 5 shows an example of a driving path 80 recorded by a teacher driving according to the first embodiment. In the example shown in Figure 5, the driver of vehicle 1 manually drives vehicle 1 from the starting position 23 to the parking position 27 during the teacher driving. The parking position 27 is, for example, located in a parking space 19 within a garage 18. The garage 18 has side walls 20 and 21. In other words, the parking space 19 is defined by the side walls 20 and 21. Note that the parking space 19 is not limited to the garage 18, but may be, for example, an area defined by white lines. The location of the parking space 19 may also be indicated by signs, etc. Furthermore, the parking space 19 may be an open space that is not demarcated by white lines, etc.

[0060] In this embodiment, when the parking assist device 100 is switched to learning mode by the driver, the route recording unit 104 performs a training drive. Switching to learning mode is performed, for example, by the driver operating the operation button 141 or the touch panel.

[0061] In learning mode, the route recording unit 104 learns about the area around the parking space 19. The area around the parking space 19 also includes the lateral boundary of the parking space 19, i.e., the side walls 20 and 21 in Figure 5. During the teacher run in learning mode, the route recording unit 104 starts recording map information of the surrounding area from the starting position 23 where the vehicle 1 begins to move, and stores the recorded map information in the storage unit 150.

[0062] In the example shown in Figure 5, a building 24 and trees 25 are located around the parking space 19. The side walls 20, 21, building 24, and trees 25 are depicted in the surrounding image and are also sensed by the transmitter / receiver 15. Therefore, the route recording unit 104 records the feature points of the side walls 20, 21, building 24, and trees 25 as map information. In this case, the side walls 20, 21, building 24, and trees 25 become known objects. These known objects serve as a reference for the driving route 80 and are therefore also called reference objects.

[0063] In this embodiment, the route recording unit 104 records information about the trajectory or shape of the driving route 80, the relative position of the starting position 23 with respect to feature points representing surrounding objects, and the relative position of the parking position 27 with respect to feature points representing surrounding objects. However, the information to be recorded is not limited to this. For example, the route recording unit 104 may record only the relative position of the parking position 27 with respect to feature points representing surrounding objects, without recording the trajectory or shape of the driving route 80.

[0064] Furthermore, the map information is not limited to that composed of feature points; the route recording unit 104 may also record information about the relative positions, colors, and outlines of multiple objects around the parking space 19.

[0065] Returning to Figure 4, the estimation unit 105 estimates the position and orientation of vehicle 1 based on the feature points extracted by the extraction unit 103 when vehicle 1 is automatically driven by the driving control unit 111 (described later) based on the driving path 80 or the corrected path 81.

[0066] For example, the estimation unit 105 estimates the position of vehicle 1 by comparing feature points of the surrounding image captured by the on-board camera 16 during the training run with feature points of the current surrounding image. Note that the method used by the estimation unit 105 to estimate the position and orientation of vehicle 1 is not limited to this example. Furthermore, the estimation unit 105 may correct the estimated position of vehicle 1 based on the absolute position of vehicle 1 using the vehicle 1's GPS position information.

[0067] The detection unit 106 detects objects around the vehicle 1. More specifically, before the start of automatic driving with parking assistance and during automatic driving, the detection unit 106 detects objects around the vehicle 1 based on the presence or absence of obstacles detected by the transmitter / receiver unit 15 acquired by the acquisition unit 102, the distance between the vehicle 1 and obstacles around the vehicle 1 measured by the transmitter / receiver unit 15, and environmental information such as surrounding images. In this embodiment, the term "object" includes other vehicles, buildings, trees, and people.

[0068] Objects detected around vehicle 1 during automated driving with parking assistance include known objects that were present during the training run and unknown objects that were not present during the training run. Among the unknown objects that were not present during the training run, objects whose distance from vehicle 1 is below a threshold when vehicle 1 is driving based on the driving path 80 recorded during the training run become obstacles. The threshold is not particularly limited.

[0069] For example, during automated driving with parking assistance, if the detection unit 106 detects an object included in the map information corresponding to the driving route 80 recorded by the route recording unit 104, it determines that a known object has been detected. Also, for example, during automated driving with parking assistance, if the detection unit 106 detects an object not included in the map information corresponding to the driving route 80 recorded by the route recording unit 104, it determines that an unknown object has been detected.

[0070] The correction unit 107 corrects the travel path 80 recorded by the training run based on the object detection results. More specifically, if the object detected by the detection unit 106 is not one that was detected during the training run, the correction unit 107 corrects the travel path 80 so that the distance to the object is greater than or equal to a threshold.

[0071] Figure 6 shows an example of a corrected path 81 according to the first embodiment. In the example shown in Figure 6, when the parking assist device 100 performs parking assist, a trash can 30a is left near the parking space 19. The trash can 30a was not present during the training run, so it is an unknown object to the parking assist device 100. In this case, the trash can 30a is detected by the detection unit 106. The correction unit 107 then corrects the driving path 80 so that the distance between the trash can 30a and the vehicle 1 remains above a threshold while the vehicle 1 moves from the starting position 23 to the parking position 27. The driving path corrected by the correction unit 107 is called the corrected path 81.

[0072] In this embodiment, the correction unit 107 does not generate the entire path to avoid the trash can 30a from the beginning, but rather generates a corrected path 81 by, for example, replacing a portion of the travel path 80 recorded during the training run with a new path. For example, in the example shown in Figure 6, the beginning and end of the corrected path 81 overlap with the travel path 80, and only the middle portion differs from the travel path 80. Therefore, according to the correction method of this embodiment, it is possible to avoid unknown objects without generating the entire path to avoid the trash can 30a.

[0073] Returning to Figure 4, the image generation unit 108 generates a corrected path image representing the corrected path 81 generated by the correction unit 107. The corrected path image is, for example, an image illustrating the parking space 19, the corrected path 81, the driving path 80, the starting position 23 of the driving path 80, the parking position 27, and objects around the parking space 19. Note that the corrected path image does not necessarily have to include the driving path 80. The corrected path image may be generated from surrounding images captured by the onboard camera 16, or it may be an illustration depicting the outlines of objects, etc.

[0074] The display control unit 109 causes the corrected driving path 80, i.e., the corrected path 81, corrected by the correction unit 107, to be displayed on the display device 120 at least before the start of automatic driving with parking assistance and during automatic driving. In this embodiment, the display control unit 109 causes the corrected path 81 to be displayed on the display device 120 at least during automatic driving. Specifically, in this embodiment, the display control unit 109 causes the corrected path image generated by the image generation unit 108 to be displayed on the display device 120.

[0075] Figure 7 shows an example of a corrected path image 121a displayed on the display device 120 according to the first embodiment. As shown in Figure 7, the corrected path image 121a shows a parking space 19, a corrected path 81, a driving path 80, the starting position 23 of the driving path 80, a parking position 27, side walls 20, 21, a building 24, a tree 25, and a trash can 30a. In Figure 7, both the uncorrected driving path 80 and the corrected path 81 are depicted in the corrected path image 121a, but only the corrected path 81 may be depicted. The display control unit 109 may also highlight which of the objects depicted in the corrected path image 121a is an obstacle. The display control unit 109 may further display the parking action in a video-like manner using XR (Extended Reality) technology such as AR (Augmented Reality) or CG (Computer Graphics). This technology allows for a faithful reproduction of actual car behavior in videos, making it visually easier to understand and allowing users to better assess the dangers of obstacles and other hazards.

[0076] Furthermore, the corrected route image 121a may include a message M1 explaining to the driver the reason for the change in the driving route 80. In the example shown in Figure 7, the display control unit 109 displays the message M1, "The route will be corrected due to an obstacle."

[0077] Furthermore, in the example shown in Figure 7, since the trash can 30a is included in the imaging range of the vehicle's onboard camera 16 and the measurement range of the transmitter / receiver unit 15 at the starting position 23 of the driving path 80, the display control unit 109 can display the corrected path image 121a before the start of automatic driving with parking assistance.

[0078] Furthermore, as vehicle 1 moves during automatic driving, the imaging range of the onboard camera 16 and the measurement range of the transmitter / receiver unit 15 also change, which may cause the detection unit 106 to detect a newly discovered unknown object. If the newly detected unknown object is an obstacle, the correction unit 107 corrects the driving path 80. Alternatively, if vehicle 1 is automatically driving along an already corrected path 81, the correction unit 107 further corrects the path 81 to avoid the newly detected obstacle. When the driving path 80 or the corrected path 81 is corrected during automatic driving in this way, the image generation unit 108 generates a corrected path image representing the newly corrected path 81. The display control unit 109 then displays the newly generated corrected path image on the display device 120.

[0079] Figure 8 shows an example of a corrected path image 121b that differs from Figure 7. In the example shown in Figure 8, a trash can 30b is left inside the garage 18. The trash can 30b is an unknown object that was not present in that location during the teacher's run. Furthermore, there is another unknown object 31 at the back of the garage 18. The trash can 30b and the other unknown object 31 are not included in the imaging range of the on-board camera 16 and the measurement range of the transmitter / receiver 15 unless vehicle 1 enters the garage 18.

[0080] In this case, the detection unit 106 detects the trash can 30b and other unknown objects 31 when the vehicle 1 enters the garage 18 and the trash can 30b and other unknown objects 31 come into the imaging range of the on-board camera 16 and the measurement range of the transmitter / receiver unit 15. The detection unit 106 also detects the side walls 20 and 21, which are known objects.

[0081] The correction unit 107 then corrects the travel path 80 to generate a corrected path 81, while maintaining a state in which the distances d1, d2, d4 to the side walls 20, 21 are longer than the threshold, so that the distances d3, d5 to the trash can 30b and other unknown objects 31 are longer than the threshold. In the case shown in Figure 8, since the trash can 30b and other unknown objects 31 are inside the garage 18, the correction unit 107 may also change the parking position 27 of the vehicle 1. Note that in Figure 8, the travel path 80, starting position 23, and parking position 27 are not shown. In this case, the image generation unit 108 generates a corrected path image 121b representing the corrected path 81 generated by the correction unit 107. Hereafter, unless otherwise distinguished, the individual corrected path images 121a and 121b will simply be referred to as the corrected path image 121.

[0082] As shown in Figure 8, the display control unit 109 displays the corrected path image 121b on the display device 120. In this way, the display control unit 109 displays the corrected path 81 generated in response to an unknown object newly detected during automatic driving, allowing the user to confirm the latest state of the corrected path 81. Note that the corrected path image 121 shown in Figures 7 and 8 is just an example, and the display method is not limited to this.

[0083] Furthermore, the display control unit 109 notifies the driver that parking assistance is possible when the vehicle 1 is in a position where parking assistance is possible while the vehicle 1 is in motion, by displaying this information on the display device 120. A position where parking assistance is possible is, for example, a position within a specified distance where the starting position 23 of the driving route 80 or a point where it is possible to merge onto the driving route 80 is located.

[0084] Returning to Figure 4, the audio output unit 110 causes sound or voice to be output to a speaker installed in the vehicle 1. For example, Figures 7 and 8 illustrate an example in which the display control unit 109 displays message M1 on the display device 120, but the method of notifying message M1 is not limited to display. For example, the audio output unit 110 may output message M1 as an audio message from the speaker.

[0085] Furthermore, the audio output unit 110 may notify the driver that parking assistance is available when the vehicle 1 is in a position where parking assistance is possible while the vehicle 1 is in motion, by outputting an audio message from the speaker.

[0086] The driving control unit 111 automatically drives the vehicle 1 based on a driving path 80 recorded based on past parking operations to move the vehicle into or out of the parking space. In this embodiment, the driving path 80 recorded by the training drive is an example of a driving path 80 recorded based on past parking operations. The driving control unit 111 moves the vehicle 1 into the parking space 19 by automatic driving based on the driving path 80 or a corrected path 81 obtained by correcting the driving path 80. For example, the driving control unit 111 controls the steering, braking, and acceleration / deceleration of the vehicle 1 to automatically drive the vehicle 1 along the driving path 80 or the corrected path 81.

[0087] During automatic driving by the driving control unit 111, the driver may be seated in the driver's seat 130a of vehicle 1, or may be out of vehicle 1. In this embodiment, the explanation will be based on the assumption that the driver is seated in the driver's seat 130a of vehicle 1.

[0088] Furthermore, although Figure 4 shows the parking assist device 100 as comprising a driving control unit 111, the driving control unit 111 may be implemented by a different ECU than the parking assist device 100.

[0089] Next, the flow of the correction process for the travel path 80 performed by the parking assistance device 100 of this embodiment, configured as described above, will be explained.

[0090] Figure 9 is a flowchart showing an example of the processing flow for automatic driving and correction of the driving path 80 performed by the parking assistance device 100 according to the first embodiment. It is assumed that the driving path 80 based on the teacher driving has been registered at the time the processing in this flowchart is executed.

[0091] First, the acquisition unit 102 acquires the absolute position of vehicle 1, surrounding image, speed, steering angle, braking operation, and distance measurement results (S101). Although not shown in this flowchart, the acquisition of the absolute position of vehicle 1, surrounding image, speed, steering angle, braking operation, and distance measurement results by the acquisition unit 102 continues throughout the execution of this flowchart.

[0092] Then, the extraction unit 103 extracts feature points from the multiple surrounding images acquired (S102).

[0093] The estimation unit 105 performs a self-position estimation process to estimate the position of the vehicle 1 by comparing the extracted feature points with the feature points of the surrounding images captured by the on-board camera 16 during the training run (S103). The estimation unit 105 may also perform a self-position estimation process based on the distance measurement results obtained from the transmitter / receiver unit 15, or GPS position information obtained from the GPS device 11l.

[0094] The estimation unit 105 determines, based on the result of self-position estimation, whether there is a starting position 23 of the travel route 80 or a point where the vehicle can merge onto the travel route 80 within a specified distance from the vehicle 1 (S104). If there is no starting position 23 of the travel route 80 or a point where the vehicle can merge onto the travel route 80 within the specified distance from the vehicle 1 (S104 "No"), the process returns to S101. The specified distance is the distance at which the travel control unit 111 can automatically drive the vehicle 1 to merge onto the travel route 80. For example, if the current position of the vehicle 1 is too far from the starting position 23, it becomes difficult for the travel control unit 111 to perform automatic driving based on the travel route 80. The specific value of the specified distance is not particularly limited. In addition, there may be cases where the vehicle 1 can merge onto the travel route 80 from a location other than the starting position 23 of the travel route 80.

[0095] Furthermore, if the starting position 23 of the driving route 80 or a point where it is possible to merge onto the driving route 80 is within a specified distance from the vehicle 1 (S104 "Yes"), the display control unit 109 notifies the driver that parking assistance is available by displaying it on the display device 120 (S105). Alternatively, the voice output unit 110 may notify the driver that parking assistance is available by outputting a voice message from the speaker.

[0096] The reception unit 101 then determines whether or not it has received a request from the user to start parking assistance (S106). If it has not received a request from the user to start parking assistance (S106 "No"), the reception unit 101 waits for a user operation.

[0097] When the reception unit 101 receives a user's request to start parking assistance (S106 "Yes"), the detection unit 106 determines whether or not it has detected an obstacle around the vehicle 1 (S107). For example, the detection unit 106 determines that an obstacle has been detected if it detects an unknown object that was not present during the training run, and whose distance from the vehicle 1 is below a threshold when the vehicle 1 is traveling based on the driving path 80 recorded by the training run.

[0098] If the detection unit 106 detects an obstacle (S107 "Yes"), the correction unit 107 corrects the travel path 80 so that the distance to the detected obstacle is greater than or equal to a threshold, thereby generating a corrected path 81 (S108).

[0099] Then, the image generation unit 108 generates a correction path image 121 representing the correction path 81 generated by the correction unit 107 (S109).

[0100] Then, the display control unit 109 causes the corrected path image 121 generated by the image generation unit 108 to be displayed on the display device 120 (S110).

[0101] Then, the driving control unit 111 performs automatic driving based on the corrected path 81 (S111).

[0102] Then, if the detection unit 106 detects an obstacle during automatic driving (S112 "Yes"), the process returns to S108, and the correction unit 107 further corrects the correction path 81 so that the distance to the newly detected obstacle is greater than or equal to the threshold. Then, the processes S109 to S111 are executed.

[0103] Furthermore, if the detection unit 106 does not detect an obstacle during automatic driving (S112 "No"), the driving control unit 111 continues automatic driving based on the corrected path 81 (S113).

[0104] While vehicle 1 has not reached parking space 19 (S114 "No"), processes S112 to S114 are repeated, and if an obstacle is detected, process S108 is executed.

[0105] Then, when vehicle 1 reaches parking space 19 (S114 "Yes"), the driving control unit 111 instructs vehicle 1 to park in parking space 19 (S115). If vehicle 1 can park in parking position 27 in the teacher's run, the driving control unit 111 instructs vehicle 1 to park in parking position 27 within parking space 19. Furthermore, as in the example in Figure 8, if vehicle 1 cannot park in parking position 27 in the teacher's run due to an obstacle, the driving control unit 111 instructs vehicle 1 to stop at a position where the distance between the obstacle and vehicle 1 is greater than or equal to a threshold.

[0106] Furthermore, if the detection unit 106 does not detect an obstacle (S107 "No"), the correction unit 107 performs automatic driving based on the driving path 80 recorded by the training drive (S116). After the process in S116, the process proceeds to S112, and if the detection unit 106 detects an obstacle during automatic driving, the process proceeds to S108. If the detection unit 106 does not detect an obstacle during automatic driving, the process proceeds to S113, and automatic driving based on the driving path 80 continues. Then, in S114, when vehicle 1 reaches the parking space 19, the driving control unit 111 proceeds to the process in S115, and vehicle 1 parks itself in the parking space 19. At this point, the process in this flowchart ends.

[0107] Thus, the parking assistance device 100 of this embodiment corrects the driving path 80 based on the detection results of objects during automatic driving with parking assistance based on the driving path 80 recorded based on past parking operations, and displays the corrected driving path 80, i.e., the corrected path 81, on the display device 120, thereby allowing the user to understand the corrected path 81. For this reason, with the parking assistance device 100 of this embodiment, even when the vehicle 1 travels a path different from the pre-registered driving path 80, it is possible to reduce the occurrence of events that impair user convenience, such as surprising the user due to unexpected movements of the vehicle 1.

[0108] Furthermore, in this embodiment, past parking operations are parking operations performed by the driver of vehicle 1 through manual driving. In other words, in the parking assistance device 100 of this embodiment, when vehicle 1, which is in autonomous driving mode, travels a route different from the driving route 80 based on the driver's training drive, the system makes the driver aware that the route has changed, thereby reducing anxiety and distress for the driver.

[0109] Furthermore, if the detected object in this embodiment is not one that was detected during the recording of the driving path 80, the parking assist device 100 corrects the driving path 80 so that the distance to the object is greater than or equal to a threshold. Therefore, according to the parking assist device 100 of this embodiment, even if an unknown object is detected, contact between the object and the vehicle 1 can be avoided.

[0110] In this embodiment, an example of vehicle 1 entering garage 18 has been described, but the parking assistance device 100 of this embodiment can also be applied when vehicle 1 is leaving the garage.

[0111] Furthermore, the parking assist device 100 may use image recognition technology to identify unknown objects, and if an object is identified as having no effect on driving (for example, a puddle), it may notify the user whether or not to perform correction by displaying it on the display device and by voice, and the user may choose whether or not to perform correction.

[0112] (Modification 1 of the first embodiment) In the first embodiment described above, the parking assist device 100 displayed the corrected path image 121 on the display device 120 inside the vehicle 1. In this modified example 1, the parking assist device 100 displays the corrected path image 121 on the display of a mobile device such as a smartphone.

[0113] The modified parking assist device 100 has the same hardware configuration as the first embodiment. Furthermore, the modified parking assist device 100 also includes, like the first embodiment, a reception unit 101, an acquisition unit 102, an extraction unit 103, a route recording unit 104, an estimation unit 105, a detection unit 106, a correction unit 107, an image generation unit 108, a display control unit 109, an audio output unit 110, a driving control unit 111, and a storage unit 150.

[0114] The reception unit 101, acquisition unit 102, extraction unit 103, route recording unit 104, estimation unit 105, detection unit 106, correction unit 107, image generation unit 108, audio output unit 110, driving control unit 111, and storage unit 150 have the same functions as in the first embodiment.

[0115] Furthermore, in this modified example, the display control unit 109 displays the corrected driving path 80, i.e., the corrected path 81, corrected by the correction unit 107, on the display of a mobile terminal such as a smartphone, at least before the start of automatic driving with parking assistance and during automatic driving. For example, the display control unit 109 displays the corrected path image 121 generated by the image generation unit 108 on the display of a mobile terminal such as a smartphone.

[0116] For example, the display control unit 109 transmits the corrected path image 121 to a mobile terminal such as a smartphone via the NW I / F 11F, and displays the corrected path image 121 on the display.

[0117] The display of a mobile device such as a smartphone is an example of a display unit in this embodiment. Furthermore, if the display of a mobile device such as a smartphone has the functionality of a touch panel, the touch panel is an example of an operation unit.

[0118] Figure 10 shows an example of a corrected path image 121a displayed on the display 210 of a smartphone 200 according to Modification 1 of the First Embodiment. The content of the corrected path image 121a is the same as that of the First Embodiment described in Figure 7. The smartphone 200 may belong to the driver of the vehicle 1, or to a passenger other than the driver.

[0119] For example, when the driver of vehicle 1 has exited vehicle 1 and the vehicle 1 is being automatically driven into a parking lot, the driver can confirm the route that vehicle 1 will take by looking at the corrected route image 121a displayed on the display 210 of the smartphone 200. Therefore, according to this modified example, even when vehicle 1 is driving along the corrected route 81 instead of the pre-registered driving route 80, the driver can still understand the corrected route 81 using the smartphone 200.

[0120] Furthermore, when the driver of vehicle 1 has disembarked from vehicle 1 and the vehicle 1 is automatically driven out of the garage, the driver and passenger of vehicle 1 can see the corrected route image 121a displayed on the display 210 of the smartphone 200 to understand the position where vehicle 1 will exit the garage 18 and stop, as well as the route it will take to stop. In particular, if the stopping position of vehicle 1 differs from the stopping position of the pre-registered driving route 80 due to an obstacle during departure, the driver and passenger can see the corrected stopping position using the corrected route image 121a, allowing them to wait for vehicle 1 at the appropriate position.

[0121] (Modification 2 of the first embodiment) In the first embodiment described above, the driving control unit 111 of the parking assistance device 100 automatically drives the vehicle 1 based on a driving path 80 based on a driver-guided driving pattern, thereby parking or exiting the parking space. However, the method for recording the driving path 80 is not limited to a driver-guided driving pattern. For example, the driving path 80 may be recorded based on parking operations performed by the vehicle 1 through automatic driving.

[0122] More specifically, in addition to the functional unit shown in Figure 4, the parking assistance device 100 may also include a parking path generation unit that generates a parking path to a target parking position. In this case, the parking path generation unit generates a driving path 80 through calculations and other processing. The parking path generation unit then stores the generated driving path 80 in the storage unit 150. The parking path generation unit may generate multiple driving paths 80, and the user may select a driving path 80 to be registered from among these multiple driving paths 80.

[0123] (Second embodiment) In the first embodiment described above, the case where there is one type of pre-registered driving route 80 was explained, but multiple driving routes 80 may be registered. In this embodiment, the parking assist device 100 identifies the driving route 80 on which the vehicle 1 will travel from among the multiple pre-registered driving routes 80, based on at least one of environmental information regarding the environment around the vehicle 1 and user information regarding the user riding in the vehicle 1.

[0124] The parking assist device 100 of this modified example has the same hardware configuration as the first embodiment.

[0125] Figure 11 is a block diagram showing an example of the functions of the parking assistance device according to the second embodiment. As shown in Figure 4, the parking assistance device 100 of this embodiment includes a reception unit 101, an acquisition unit 1102, an extraction unit 103, a route recording unit 1104, an estimation unit 105, a detection unit 106, a correction unit 107, an image generation unit 108, a display control unit 1109, an audio output unit 110, a driving control unit 111, a specific unit 112, and a storage unit 1150.

[0126] The reception unit 101, extraction unit 103, estimation unit 105, detection unit 106, correction unit 107, image generation unit 108, and audio output unit 110 have the same functions as in the first embodiment.

[0127] The storage unit 1150 of this embodiment is configured, similarly to the first embodiment, with, for example, a ROM 11B, a RAM 11C, or an HDD 11G. In addition to the same features as the first embodiment, the storage unit 1150 of this embodiment stores multiple travel routes 80 in association with at least one of environmental information and user information.

[0128] In the first embodiment, the presence or absence of obstacles detected by the transmitter / receiver 15, the distance between the vehicle 1 and obstacles around the vehicle 1 measured by the transmitter / receiver 15, and the surrounding image were given as examples of environmental information regarding the environment around the vehicle 1, but the environmental information is not limited to these. For example, the environmental information may include feature points around the vehicle 1 extracted from the detection results of the transmitter / receiver 15 or the surrounding image, the position coordinates of the vehicle 1, and the weather around the vehicle 1 or the parking space 19.

[0129] Furthermore, user information refers to information about the driver or passenger of vehicle 1, such as the driver's or passenger's age and physical information. Physical information includes, for example, whether or not they use a wheelchair and whether or not they have other physical disabilities. However, user information is not limited to these. When referring to the driver and passenger collectively, the term "user" is used. The driver's or passenger's age and physical information are also called user attribute information. User attribute information is not limited to the examples above, but it shall include at least information about the user's age and the user's physical disability.

[0130] More specifically, the storage unit 1150 of this embodiment stores a travel route database in which multiple travel routes 80 are associated with at least one of the conditions related to environmental information and the conditions related to user information.

[0131] Figure 12 shows an example of a travel route database 90 according to the second embodiment. In the example shown in Figure 12, the travel route database 90 is a database that associates the travel route numbers of multiple travel routes 80, environmental information conditions, and user information conditions.

[0132] The route number is an example of identifying information that can specify each route 80. The environmental information conditions are conditions that define the environmental information corresponding to the route 80. The user information conditions are conditions that define the user information corresponding to the route 80. In the example shown in Figure 12, the route 80 with route number "1" is associated with the environmental information conditions "Surrounding feature points: xxx, presence of obstacles: none, weather: sunny or cloudy". That is, if the environment surrounding vehicle 1 or parking space 19 matches these conditions, the route 80 with route number "1" is recommended. Note that the multiple conditions registered as environmental information conditions in Figure 12 may be AND conditions or OR conditions.

[0133] Furthermore, in the example shown in Figure 12, the route 80 with route number "1" is associated with the user information condition "Healthy person, age: 10-60s". Similarly, the route 80 with route number "2" is associated with the user information condition "Wheelchair user, age: 70s or older". In addition, the multiple conditions registered as user information conditions in Figure 12 may be AND conditions or OR conditions.

[0134] For example, if the user is a wheelchair user, a route 80 in which vehicle 1 exits parking space 19 and stops near the ramp of building 24 is highly convenient for the user. Also, if it is raining, a route 80 in which vehicle 1 exits parking space 19 and stops near the entrance / exit of building 24 is highly convenient for the user. Furthermore, if the user is a healthy person with no particular difficulty walking, a route 80 in which vehicle 1 is parked in a position that makes it easy to start the vehicle after the driver gets in is highly convenient for the user.

[0135] Furthermore, the route database 90 may contain only one of either environmental information conditions or user information conditions.

[0136] Furthermore, the storage unit 1150 of this embodiment stores a user information database 91 in which a user ID, the physical information of the user identified by the user ID, and the age are associated.

[0137] Figure 13 shows an example of a user information database 91 according to the second embodiment.

[0138] In Figures 12 and 13, the route database 90 and user information database 91 are shown in table format, but the data format is not limited as long as the relevant information is associated. Furthermore, the user information database 91 may also register whether each user is the driver or a passenger of vehicle 1.

[0139] Returning to Figure 11, the route recording unit 1104 in this embodiment records the route 80 of vehicle 1 during a driver-led training run, associating it with at least one of environmental information and user information. This recording process generates a route database 90. The route database 90 may be generated by an information processing device or the like outside of vehicle 1, or it may be registered by the user via the touch panel of the display device 120 or a mobile terminal such as a smartphone 200.

[0140] The acquisition unit 1102 of this embodiment has the same functions as in the first embodiment, and acquires at least one of environmental information regarding the environment surrounding the vehicle 1 and user information regarding the users riding in the vehicle 1.

[0141] For example, the acquisition unit 1102 acquires the presence or absence of obstacles detected by the transmitter / receiver unit 15, the distance between the vehicle 1 and obstacles around the vehicle 1 measured by the transmitter / receiver unit 15, surrounding images captured by the onboard camera 16, and GPS location information of the vehicle 1 measured by the GPS device 11l. The acquisition unit 1102 may also acquire weather information from an external information processing device of the vehicle 1 via a network. Alternatively, the acquisition unit 1102 may acquire the weather around the vehicle 1 from surrounding images captured by the onboard camera 16.

[0142] The identification unit 112 identifies a driving route 80 from among a plurality of driving routes 80 pre-registered in the storage unit 1150 that corresponds to at least one of the acquired environmental information and user information. In this embodiment, both environmental information and user information are used as conditions for identifying the driving route 80. For example, if the acquired environmental information satisfies the conditions "surrounding feature points: xxx, presence or absence of obstacles: none, weather: sunny or cloudy" and the acquired user information satisfies the conditions "healthy person, age: 10-60s", the identification unit 112 identifies the driving route 80 with driving route number "1" as illustrated in Figure 12 as the driving route 80 to be used for the automatic driving of vehicle 1.

[0143] Furthermore, if there are multiple driving routes 80 among the multiple driving routes 80 pre-registered in the storage unit 1150 that correspond to at least one of the acquired environmental information and user information, the identification unit 112 identifies the multiple driving routes 80 that correspond to at least one of the acquired environmental information and user information as candidates for driving routes 80 to be used for the automatic driving of vehicle 1.

[0144] Furthermore, if the identification unit 112 identifies multiple travel routes 80 as candidates, it may calculate the recommendation level of each identified travel route 80 according to at least one of the environmental information and user information. Alternatively, the identification unit 112 may calculate the recommendation level of each identified travel route 80 according to the frequency with which each of the identified travel routes 80 has been selected by a user in the past.

[0145] The display control unit 1109 of this embodiment has the same functions as the first embodiment and the first modified example of the first embodiment, and displays the travel route 80 or candidate travel route 80 identified by the identification unit 112 on the display 210 of the smartphone 200. The display 210 of the smartphone 200 is an example of a display unit in this embodiment.

[0146] Figure 14 shows an example of a driving route notification screen 122 displayed on the display 210 of a smartphone 200 according to a second embodiment. The driving route notification screen 122 is a screen that displays an image representing the driving route 80 identified by the identification unit 112, for example. In the example shown in Figure 14, the driving route notification screen 122 represents the driving route 80 from when vehicle 1 exits the parking space 19 in the garage 18 and stops at the stopping position 28. The driving route notification screen 122 may also show objects such as buildings 24 surrounding the parking space 19 and the stopping position 28.

[0147] In addition, in the example shown in Figure 14, the driving route notification screen 122 displays the message M2, "Do you want to exit the depot using this driving route?". The display control unit 1109 may also display a message M2 prompting the user to approve the driving route 80, although this is not limited to the above.

[0148] In the example shown in Figure 14, the route notification screen 122 displays an OK button 231 and a Cancel button 232. The OK button 231 is an image button that accepts the user's operation to approve the automatic driving of vehicle 1 along the route 80 displayed on the route notification screen 122. The Cancel button 232 is an image button that accepts the user's operation to cancel the automatic driving along the route 80 displayed on the route notification screen 122. When the OK button 231 or the Cancel button 232 is operated by the user, the reception unit 101 accepts the user's operation.

[0149] If the user presses the cancel button 232, the display control unit 1109 may, for example, display a list of pre-registered driving routes 80 on the display 210 and allow the user to select one. Alternatively, if the user presses the cancel button 232, the parking assist device 100 may cancel the automatic driving of the vehicle 1 and wait for manual driving by the driver.

[0150] Furthermore, as described above, among the multiple travel routes 80 pre-registered in the storage unit 1150, there may be multiple travel routes 80 that correspond to at least one of the acquired environmental information and user information. When the identification unit 112 identifies multiple travel routes 80, the display control unit 1109 displays the identified multiple travel routes 80 on the display 210 of the smartphone 200. For example, the display control unit 1109 displays a travel route selection screen on the display 210 of the smartphone 200, which displays the identified multiple travel routes as multiple travel route candidates for the user to select.

[0151] Figure 15 shows an example of a route selection screen 123 displayed on the display 210 of a smartphone 200 according to a second embodiment. As shown in Figure 15, the route selection screen 123 is an image representing multiple routes 80a and 80b. In the example shown in Figure 15, routes 80a and 80b stop at different stopping positions 28a and 28b, respectively. Hereafter, if routes 80a and 80b are not specifically distinguished, they will simply be referred to as route 80. Similarly, if stopping positions 28a and 28b are not specifically distinguished, they will simply be referred to as stopping position 28.

[0152] Furthermore, in the example shown in Figure 15, the message M3 "Please select a route" is displayed on the route selection screen 123. The display control unit 1109 may also display a message M2 prompting the user to select a route 80, regardless of the content of the message M2. Messages M2 and M3 may also be output as voice messages by the voice output unit 110.

[0153] In the example shown in Figure 15, a confirmation button 233 is displayed on the route selection screen 123. The confirmation button 233 is an image button that accepts the user's operation to select either route 80a or 80b displayed on the route selection screen 123. For example, the user selects the route 80a or route 80b that was pressed by pressing either the image representing route 80a or route 80b on the display 210, and then pressing the confirmation button 233. When the confirmation button 233 is operated by the user, the reception unit 101 accepts the user's operation.

[0154] If the user selects either route 80a or 80b, the display control unit 1109 may display the route notification screen 122 for the user to reconfirm the selected route 80. Alternatively, if the user selects either route 80a or 80b, the driving control unit 111 may start automatic driving.

[0155] Furthermore, when the display control unit 1109 presents multiple travel routes 80 as candidates, as shown in Figure 15, it may arrange each travel route 80 on the travel route selection screen 123 in order of the recommendation level of each travel route 80 calculated by the identification unit 112.

[0156] Furthermore, when a user selects one of the driving routes 80, the reception unit 101 stores the driving route 80 selected by the user in the storage unit 1150, associating it with the user's user ID. The information associating the driving route 80 selected by the user with the user's user ID is an example of the user's past usage history of vehicle 1. Then, when the identification unit 112 identifies multiple driving routes 80 for the second time or later, the display control unit 1109 displays the identified multiple driving routes on the smartphone 200's display 210 in an order based on the user's past usage history. For example, the display control unit 1109 displays the driving routes 80 that the user has selected most frequently in the past at the top of the list. This display makes it easy for the user to find their desired driving route 80 even when multiple driving route candidates exist.

[0157] The display control unit 1109 may display the route notification screen 122 and the route selection screen 123 only if the user of the smartphone 200 is the driver and not a passenger in the vehicle 1. Alternatively, if the user of the smartphone 200 is a passenger in the vehicle 1, the display control unit 1109 may display only the route notification screen 122. In this case, the OK button 231 and the Cancel button 232 are not displayed on the route notification screen 122 displayed on the passenger's smartphone 200.

[0158] Returning to Figure 11, the driving control unit 111 of this embodiment has the same functions as in the first embodiment, and drives the vehicle 1 to the stopping position 28 corresponding to the driving path 80 identified by the identification unit 112. Furthermore, if there are multiple driving paths 80 corresponding to at least one of the acquired environmental information and user information, the driving control unit 111 drives the vehicle 1 to the stopping position 28 corresponding to the driving path 80 selected by the user from among the multiple identified driving paths 80.

[0159] Figure 16 is a sequence diagram showing an example of the processing flow performed by the parking assist device 100 and smartphone 200 according to the second embodiment.

[0160] First, the smartphone 200 transmits to the parking assist device 100 the user ID that the user of the smartphone 200 has for using the vehicle 1 (S201).

[0161] In this sequence diagram, the acquisition unit 1102 of the parking assist device 100 acquires the user ID from the smartphone 200, but it may also acquire user information such as the age and physical information of the user of the smartphone 200. In this case, the acquisition unit 1102 may, for example, acquire user information for all users, including the driver and passengers, from the smartphone 200 of the driver of vehicle 1, or it may acquire user information for each user separately from each user's smartphone 200. Furthermore, even when acquiring the user ID from the smartphone 200 as in S201, the acquisition unit 1102 may acquire the user IDs of all users, including the driver and passengers, from the smartphone 200 of the driver of vehicle 1, or it may acquire the user ID for each user separately from each user's smartphone 200.

[0162] Furthermore, the acquisition unit 1102 of the parking assistance device 100 acquires environmental information regarding the environment surrounding the vehicle 1 (S202).

[0163] Then, the identification unit 112 of the parking assistance device 100 obtains the user's physical information and age corresponding to the acquired user ID from the user information database 91, and identifies a driving route 80 that satisfies the conditions from a plurality of driving routes 80 pre-registered in the storage unit 1150 based on the acquired user's physical information and age and the environmental information acquired by the acquisition unit 1102 (S203). Here, it is assumed that there are multiple driving routes 80 among the plurality of driving routes 80 pre-registered in the storage unit 1150 that correspond to the acquired environmental information and user information.

[0164] Then, the display control unit 1109 of the parking assist device 100 transmits the multiple driving routes 80 identified by the identification unit 112 to the smartphone 200 via the network and displays them on the display 210 (S204).

[0165] The smartphone 200 then displays a route selection screen 123 on the display 210, which shows multiple route selection screens 80 transmitted from the parking assist device 100 (S205).

[0166] Then, when the user presses either the image showing the driving route 80a or the image showing the driving route 80b on the display 210 of the smartphone 200 and then presses the confirm button 233 (S206), the smartphone 200 notifies the parking assist device 100 of the driving route 80 selected by the user (S207). The reception unit 101 of the parking assist device 100 receives the user's selection operation for the driving route 80 via the smartphone 200.

[0167] Then, the display control unit 1109 of the parking assist device 100 displays the driving route notification screen 122, which represents the driving route 80 selected by the user, by sending a display instruction to the smartphone 200 (S208).

[0168] Then, the smartphone 200 displays the driving route notification screen 122 on the display 210 (S209).

[0169] Then, when the user performs an approval operation by pressing the OK button 231 on the display 210 of the smartphone 200 (S210), the smartphone 200 notifies the parking assist device 100 that the user has approved the start of automatic driving based on the driving route 80 (S211).

[0170] Then, the driving control unit 111 of the parking assist device 100 performs automatic driving based on the driving route 80 selected and approved by the user (S212). In addition, before the start of automatic driving of the vehicle 1 and during automatic driving, the parking assist device 100 performs correction processing of the driving route 80 according to the obstacle detection results, similar to the first embodiment. Furthermore, when the parking assist device 100 corrects the driving route 80, it displays the corrected route 81 on the display 210 of the smartphone 200, similar to the modification 1 of the first embodiment. At this point, the processing in this sequence diagram ends.

[0171] If an obstacle has already been detected based on the environmental information acquired in S202, the correction unit 107 may correct the travel path 80 identified by the identification unit 112 before the processing from S204 onwards is executed.

[0172] Thus, the parking assistance device 100 of this embodiment displays on the smartphone 200's display 210 a driving route 80 that corresponds to at least one of the environmental information and user information from among a plurality of pre-registered driving routes 80. Therefore, in addition to the effects of the first embodiment, the parking assistance device 100 of this embodiment can improve convenience for the user by suggesting a driving route 80 to the user that corresponds to the environmental information or user information.

[0173] For example, the parking assistance device 100 of this embodiment identifies a suitable driving route 80 from among a plurality of pre-registered driving routes 80 based on information regarding the user's age and / or physical disability. Therefore, according to the parking assistance device 100 of this embodiment, even in usage scenarios where multiple users share one vehicle 1, it is possible to suggest a driving route 80 that is appropriate for the individual user's condition.

[0174] In this embodiment, the example illustrates the case where vehicle 1 exits parking space 19, but the configuration of this embodiment can also be applied when vehicle 1 enters parking space 19.

[0175] (Modification 1 of the second embodiment) In the second embodiment described above, user information was defined as the age and physical information of the driver or passenger, but user information is not limited to this. For example, user information may be the driver and passenger information registered in the usage schedule of vehicle 1.

[0176] For example, there is a usage scenario in which multiple users share one vehicle 1. In such a usage scenario, the usage schedule for vehicle 1 is registered, for example, on a cloud server 300 located in a cloud environment. This usage schedule includes the user IDs of the users who use vehicle 1 for each time slot.

[0177] Figure 17 shows an example of the configuration of the vehicle usage schedule management system S according to Modification 1 of the second embodiment. The management system S includes, for example, a cloud server 300, a terminal device such as a smartphone 200 or a PC (Personal Computer), and a parking assistance device 100 installed in the vehicle 1. Note that a regular server device or a PC may be used instead of the cloud server 300.

[0178] As shown in Figure 17, the user registers their usage schedule on the smartphone 200. The smartphone 200 then sends the usage schedule registered by the user to the cloud server 300. The usage schedule is stored on the cloud server 300. The acquisition unit 1102 of the parking assist device 100 then retrieves the driver and passenger information registered in the usage schedule of vehicle 1 from the cloud server 300.

[0179] Furthermore, the smartphone 200 and the parking assist device 100 of the vehicle 1 do not need to be directly connected, and data and control signals may be sent and received via a cloud server 300 or other server device.

[0180] Furthermore, as described in the second embodiment, the parking assist device 100 transmits the identified candidate driving routes to the smartphone 200 and displays the driving route selection screen 123 and the driving route notification screen 122.

[0181] Figure 18 shows an example of a usage schedule database 92 stored in a cloud server 300 according to Modification 1 of the second embodiment. The usage schedule database 92 is a database in which, for example, a vehicle ID that can identify the vehicle 1 to be reserved, the reservation time, and the user ID are registered in association with each other. In Figure 18, the usage schedule database 92 includes a vehicle ID as an example of a case where reservations for multiple vehicles 1 are registered, but if there is only one vehicle 1 to be registered for scheduling, the vehicle ID does not need to be included in the items of the usage schedule database 92.

[0182] In this modified version of the parking assistance device 100, the identification unit 112 obtains the user's physical information and age corresponding to the user ID registered in the usage schedule database 92 from the user information database 91. Based on the obtained user's physical information and age, and the environmental information obtained by the acquisition unit 1102, it identifies a driving route 80 that satisfies the conditions from a plurality of driving routes 80 pre-registered in the storage unit 1150.

[0183] With this configuration, the parking assistance device 100 of this modified version can suggest a suitable driving route 80 to the user simply by the user making a reservation for vehicle 1. Therefore, the parking assistance device 100 of this modified version simplifies user operation and improves convenience.

[0184] (Modification 2 of the second embodiment) Furthermore, user information may be the user ID itself. For example, in the second embodiment, the travel route database 90 is a database that associates the travel route numbers of multiple travel routes 80, environmental information conditions, and user information conditions, but it may also be a database that associates multiple travel routes 80 with user IDs.

[0185] Figure 19 shows an example of a travel route database 1090 according to Modification 2 of the second embodiment. As shown in Figure 19, the travel route database 1090 of this modification is a database that associates the travel route numbers of multiple travel routes 80 with user IDs.

[0186] In this modified example, the identification unit 112 of the parking assistance device 100 searches the driving route database 1090 for a driving route number corresponding to the user ID of the vehicle 1 acquired by the acquisition unit 1102, and identifies the driving route 80 corresponding to the driving route number as the driving route 80 corresponding to the user information.

[0187] (Modification 3 of the second embodiment) Furthermore, in the second embodiment, the multiple travel routes 80 are stored in the storage unit 1150 of the parking assist device 100, but the storage location is not limited to this. For example, the multiple travel routes 80 may be stored in a storage device such as a cloud server 300 located outside the vehicle 1.

[0188] In this modified example, the acquisition unit 1102 acquires multiple travel routes 80 from a storage device located outside the vehicle 1, along with at least one of the environmental information and user information corresponding to each of the multiple travel routes 80. For example, the acquisition unit 1102 acquires this information from a storage device such as a cloud server 300, with the travel route numbers, environmental information conditions, and user information conditions of the multiple travel routes 80 associated with each other.

[0189] Then, the identification unit 112 of this modified example identifies a travel route 80 from among a plurality of travel routes 80 stored in a storage device such as a cloud server 300 that corresponds to at least one of the current environmental information and user information.

[0190] (Modification 4 of the second embodiment) Furthermore, the acquisition unit 1102 may acquire both the travel route 80 stored in a storage device such as a cloud server 300 located outside the vehicle 1 and the travel route 80 stored in the storage unit 1150 of the parking assist device 100.

[0191] In this modified example, the identification unit 112 identifies a driving route 80 and parking location from among a plurality of driving routes 80 stored in the storage unit 1150 and an external storage device that corresponds to at least one of the current environmental information and user information.

[0192] Furthermore, the multiple travel routes 80 may be stored in a storage device within the vehicle 1, which is provided separately from the parking assist device 100.

[0193] (Modification 5 of the second embodiment) In the second embodiment described above, the parking assist device 100 was described as displaying the corrected path 81 on the display 210 of the smartphone 200, similar to the first embodiment or Modification 1 of the first embodiment. However, the parking assist device 100 of the second embodiment does not necessarily have the function of displaying the corrected path 81. That is, the specific function of the driving path 80 according to environmental information or user information in the second embodiment may be applied independently to a conventional parking assist device.

[0194] (Modification 6 of the second embodiment) In the second embodiment described above, the display 210 of the smartphone 200 was used as an example of a display unit, but as in the first embodiment, the display device 120 may also be an example of a display unit. For example, the display control unit 1109 of the parking assist device 100 may display the driving route notification screen 122 and the driving route selection screen 123 on the display device 120 when the driver is in the vehicle 1, and display the driving route notification screen 122 and the driving route selection screen 123 on the display 210 of the smartphone 200 when the driver has gotten out of the vehicle 1.

[0195] (Modifications of the first and second embodiments) In the first and second embodiments described above, a display device 120 provided in the vehicle 1 or the display 210 of a mobile terminal such as a smartphone 200 was used as an example of a display unit, but the display device 120 is not limited to these. For example, a head-up display projected onto the windshield 180 of the vehicle 1 or a transparent plate near the windshield 180 may be used as a display unit.

[0196] Furthermore, an operating terminal capable of transmitting signals to vehicle 1 from outside vehicle 1, such as a remote controller or electronic key, may also be used as an example of an operating unit.

[0197] Furthermore, some of the functions of the parking assistance device 100 in the first and second embodiments described above may be performed by an information processing device located outside the vehicle 1. Examples of information processing devices located outside the vehicle 1 include mobile terminals such as smartphones 200, PCs, cloud servers 300, and other server devices.

[0198] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0199] 1 vehicle 15, 15f, 15r Transmitter / receiver section 16, 16a~16c In-car camera 18 Garage 19 Parking Spaces 20,21 side wall 24 Buildings 25 Trees 27 Parking location 28,28a,28b Stop position 30a, 30b Trash cans 31 Unknown Object Routes 80, 80a, 80b 81 Correction Path 90,1090 Driving Route Database 91 User Information Database 92 Usage Schedule Database 100 Parking assist system 101 Reception Department 102,1102 Acquisition Department 103 Extraction part 104,1104 Route recording section 105 Estimation part 106 Detection unit 107 Correction Unit 108 Image generation unit 109,1109 Display Control Unit 110 Audio output section 111 Driving Control Unit 112 Specific section 120 Display device 121,121a,121b Corrected path images 122 Driving route notification screen 123 Route Selection Screen 150,1150 storage section 200 Smartphones 210 displays 300 cloud servers S Management System

Claims

1. A parking assistance device that automatically drives a vehicle based on a driving route in a previous parking operation performed by a driver, an acquisition unit that acquires at least one of environmental information related to an environment around the vehicle and user information related to a user who gets on the vehicle; a storage unit that stores a plurality of travel routes associated with at least one of the environmental information and the user information; an identification unit that identifies a travel route along which the vehicle will automatically travel, based on at least one of the environmental information and the user information, from among the plurality of stored travel routes; a display control unit that causes a display unit to display information prompting approval of the travel route identified by the identification unit; a driving control unit that drives the vehicle based on a driving route approved by the driver; A parking assistance device comprising:

2. A detection unit that detects objects around the vehicle; A correction unit that corrects the travel path based on the detection result of the object, The display control unit causes the display unit to display the driving route corrected by the correction unit during the execution of the autonomous driving.

2. The parking assistance device according to claim 1.

3. The correction unit corrects the travel route so that a distance to the object is equal to or greater than a threshold value when the object detected by the detection unit is not detected when the travel route is recorded.

3. The parking assistance device according to claim 2.

4. The acquisition unit acquires, as the environmental information, a measurement result of a sensor provided in the vehicle and a surrounding image captured by a camera provided in the vehicle.

2. The parking assistance device according to claim 1.

5. The acquisition unit acquires, as the user information, attribute information of the user of the vehicle, information of a driver and a passenger registered in a usage schedule of the vehicle, or a user ID capable of identifying the user of the vehicle.

2. The parking assistance device according to claim 1.

6. The user's attribute information includes at least information regarding either the user's age or the user's physical handicap.

6. A parking assistance device according to claim 5.

7. the acquisition unit acquires the plurality of travel routes together with at least one of the environmental information and the user information corresponding to each of the plurality of travel routes from a storage device provided outside the vehicle; The identification unit identifies a travel route corresponding to at least one of the environmental information and the user information from among the plurality of travel routes stored in the storage device.

2. The parking assistance device according to claim 1.

8. The past parking operation is a parking operation performed manually by a driver of the vehicle.

2. The parking assistance device according to claim 1.

9. The display unit is a display of a smartphone.

2. The parking assistance device according to claim 1.

10. An acquisition step of acquiring at least one of environmental information regarding an environment around a vehicle and user information regarding a user riding in the vehicle; a storage step of storing a plurality of driving routes in a storage unit in association with at least one of the environmental information and the user information, the driving routes being used in past parking operations by the driver; a specifying step of specifying a travel route along which the vehicle will automatically travel, based on at least one of the environmental information and the user information, from among the plurality of travel routes stored; a display control step of displaying information on a display unit to prompt approval of the travel route identified by the identification step; a travel control step of causing the vehicle to travel based on a travel route approved by the driver; A parking assistance method comprising:

11. A detection step of detecting objects around the vehicle; and correcting the travel path based on the detection result of the object, In the display control step, the driving route corrected in the correction step is displayed on the display unit during execution of the automatic driving. The parking assistance method according to claim 10.

12. In the correction step, if the object detected in the detection step was not detected when recording the travel route, the travel route is corrected so that the distance to the object is equal to or greater than a threshold value. The parking assistance method according to claim 11.

13. In the acquisition step, a measurement result of a sensor provided in the vehicle and a surrounding image captured by a camera provided in the vehicle are acquired as the environmental information. The parking assistance method according to claim 10.

14. In the acquisition step, attribute information of the user of the vehicle, information of a driver and passengers registered in a usage schedule of the vehicle, or a user ID capable of identifying the user of the vehicle is acquired as the user information. The parking assistance method according to claim 10.

15. The attribute information of the user includes at least information regarding either the age of the user or a physical handicap of the user.

15. The parking assistance method according to claim 14.

16. In the obtaining step, the plurality of driving routes are obtained from a storage device provided outside the vehicle together with at least one of the environmental information and the user information corresponding to each of the plurality of driving routes; In the identifying step, a travel route corresponding to at least one of the environmental information and the user information is identified from among the plurality of travel routes stored in the storage device. The parking assistance method according to claim 10.

17. The past parking operation is a parking operation performed manually by a driver of the vehicle. The parking assistance method according to claim 10.

18. The display unit is a display of a smartphone. The parking assistance method according to claim 10.