Parking assistance device and parking assistance method

Abstract

This parking assistance device (5) comprises a path generation unit (55) that generates a target path (TP) to be traveled by a vehicle during the parking of the vehicle on the basis of path information including a travel path of the vehicle when a parking operation of the vehicle (V) is performed by a user and information about the surroundings of the vehicle on the travel path. The parking assistance device comprises: a tracking control unit (57) that performs a tracking control process to automatically move the vehicle to a planned parking position along the target path; and an information providing unit (58) that provides information to the user. The information providing unit provides information relating to the planned parking position included in the path information, in a visual mode to the user before the start of the tracking control process.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Patent Application No. 2021-136598 filed on August 24, 2021, the contents of which are incorporated herein by reference. [Technical field]

[0002] The present disclosure relates to a parking assistance device and a parking assistance method. [Background technology]

[0003] Conventionally, a parking assistance device that automatically parks a vehicle at a predetermined planned parking position is known (see, for example, Patent Document 1). When the driver operates the parking assistance device to park the vehicle at the planned parking position, the parking assistance device described in Patent Document 1 learns the vehicle's travel route from a reference start position to the parking position, and automatically parks the vehicle at the planned parking position using the learning result. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2013-530867 Summary of the Invention

[0005] However, for example, if the distance between the vehicle and the planned parking position is large or there are obstacles around the planned parking position, the user cannot confirm the planned parking position before starting automatic parking of the vehicle at the planned parking position. This is undesirable because it reduces usability. An object of the present disclosure is to provide a parking assistance device and a parking assistance method that are capable of improving usability.

[0006] According to one aspect of the disclosure, The parking assistance device is a route generating unit that generates a target route that the vehicle should take when parking the vehicle based on route information including a travel route of the vehicle when a user performs a parking operation of the vehicle and information about the surroundings of the vehicle on the travel route; A following control unit that performs a following control process to automatically move the vehicle to a planned parking position along a target route; an information providing unit for providing information to a user, The information providing unit provides the information on the planned parking position included in the route information to the user in a visual form before the start of the tracking control process.

[0007] According to another aspect of the present disclosure, Parking assistance methods include: generating a target route that the vehicle should take when parking the vehicle based on route information including a travel route of the vehicle when a user performs a parking operation of the vehicle and information about the surroundings of the vehicle on the travel route; performing a tracking control process for automatically moving the vehicle along the target route to a planned parking position; Providing information to users, including Providing information to the user includes providing information on the planned parking position, which is included in the route information, to the user in a visual form before the start of the tracking control process.

[0008] According to these, the user can start automatic parking of the vehicle at the planned parking position after clearly understanding the planned parking position. Therefore, according to the parking assistance device and parking assistance method disclosed herein, the usability of automatic parking can be improved. Note that "usability" refers to the effectiveness, efficiency, and degree of user satisfaction when a product is used by a specific user in a specific usage situation to achieve a specified goal.

[0009] The reference symbols in parentheses attached to each component indicate an example of the correspondence between the component and specific components described in the embodiments described below. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of an automatic parking system according to an embodiment of the present disclosure. FIG. [Diagram 2] FIG. 1 is an explanatory diagram for explaining a parking lot including vehicle parking spaces. [Diagram 3] 5 is a flowchart showing an example of a learning process executed by a parking control unit of the parking assistance device. [Figure 4] 5 is a flowchart showing an example of an assistance process executed by a parking control unit. [Diagram 5] 11 is an explanatory diagram for explaining an example of display content on a touch panel display unit before the start of the support process; FIG. [Figure 6] 10 is a flowchart showing an example of a target route generation process executed by a parking control unit. [Figure 7] FIG. 2 is an explanatory diagram for explaining a target route. [Figure 8] FIG. 11 is an explanatory diagram for explaining an object avoidance path. [Figure 9] 10 is a flowchart showing an example of a planned parking position display process executed by a parking control unit. [Figure 10] 11 is an explanatory diagram for explaining an example of the display content on a touch panel display unit before the start of a tracking control process; FIG. [Figure 11] 10 is an explanatory diagram for explaining an example of a display mode of a planned parking position on a touch panel display unit. FIG. [Figure 12] 11 is an explanatory diagram for explaining an example of the display content on the display unit when there are multiple candidates for the planned parking position. FIG. [Figure 13] 5 is a flowchart showing an example of a following control process executed by a parking control unit. [Figure 14] 11 is an explanatory diagram illustrating an example of the display content on a touch panel display unit at the start of a tracking control process; FIG. [Figure 15] 11 is an explanatory diagram illustrating an example of display content on a touch panel display unit after the start of a tracking control process; FIG. [Figure 16]11 is an explanatory diagram for explaining an example of automatic adjustment of the angle of the virtual viewpoint of the virtual viewpoint image in response to a change in the positions of the target and the vehicle. FIG. [Figure 17] 13 is an explanatory diagram for explaining another example of the display content on the touch panel display unit after the start of the tracking control process. FIG. [Figure 18] 11 is an explanatory diagram for explaining an example of the display content on a touch panel display unit during a search for an avoidance route; FIG. [Figure 19] FIG. 11 is an explanatory diagram for explaining an example of an avoidance route. [Figure 20] 11 is an explanatory diagram for explaining an example of a display mode of an avoidance route or the like on a touch panel display unit. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of the present disclosure will be described with reference to Figs. 1 to 20. In this embodiment, an example will be described in which a parking assistance device 5 and a parking assistance method of the present disclosure are applied to an automatic parking system 1. As shown in Fig. 1, the automatic parking system 1 has a perimeter monitoring sensor 3, various ECUs 4, and a parking assistance device 5. The parking assistance device 5 is communicatively connected to the perimeter monitoring sensor 3 and the various ECUs 4 directly or via an in-vehicle LAN (Local Area Network).

[0012] The perimeter monitoring sensor 3 is an autonomous sensor that monitors the surrounding environment of the vehicle V. For example, the perimeter monitoring sensor 3 detects obstacles OB consisting of three-dimensional objects around the vehicle, such as moving dynamic targets such as pedestrians and other vehicles, stationary static targets such as road structures, and parking assistance marks showing parking information, which is information about a parking lot PL, as detection targets. The vehicle V is equipped with the perimeter monitoring sensor 3, which includes a perimeter monitoring camera 31 that captures an image of a predetermined range around the vehicle, a sonar 32 that transmits a search wave to a predetermined range around the vehicle, a millimeter wave radar 33, and a search wave sensor such as a LiDAR (abbreviation for Light Detection and Ranging) 34.

[0013] The periphery monitoring camera 31 corresponds to an imaging device, captures an image of the periphery of the vehicle, and outputs the captured image data as sensing information to the parking assistance device 5. Here, the periphery monitoring camera 31 is exemplified by a front camera 31a, a rear camera 31b, a left side camera 31c, and a right side camera 31d, which capture images in front of, behind, and to the left and right sides of the vehicle, but is not limited to this.

[0014] The exploration wave sensor outputs exploration waves and sequentially outputs measurement results such as the relative speed and distance to the target and the azimuth angle at which the target is located, obtained by acquiring the reflected waves, to the parking assistance device 5 as sensing information. The sonar 32 performs measurements using ultrasonic waves as exploration waves, and is provided at multiple locations on the vehicle V, for example, multiple units are arranged in the left and right directions of the vehicle on the front and rear bumpers, and performs measurements by outputting exploration waves around the vehicle. The millimeter wave radar 33 performs measurements using millimeter waves as exploration waves. The LiDAR 34 performs measurements using laser light as exploration waves. Both the millimeter wave radar 33 and the LiDAR 34 output exploration waves within a predetermined range, for example, in front of the vehicle V, and perform measurements within the output range.

[0015] In this embodiment, the perimeter monitoring sensor 3 is exemplified by a perimeter monitoring camera 31, a sonar 32, a millimeter wave radar 33, and a LiDAR 34, but it is sufficient if perimeter monitoring can be performed using one or a combination of more of these, and it is not necessary to include all of them.

[0016] The parking assistance device 5 constitutes an ECU (i.e., an electronic control device) for performing various controls to realize the parking assistance method in the automatic parking system 1, and is composed of a microcomputer equipped with a CPU, a memory unit 50, I / O, etc.

[0017] The storage unit 50 includes a ROM, a RAM, an EEPROM, etc. That is, the storage unit 50 has a volatile memory such as a RAM, and a non-volatile memory such as an EEPROM. The storage unit 50 is configured as a non-transferable tangible recording medium.

[0018] The parking assistance device 5 generates a target route TP that the vehicle V should take when parking the vehicle V, based on the driving route of the vehicle V when the user performs a parking operation of the vehicle V and information about the surroundings of the vehicle V on the driving route. The "information about the surroundings of the vehicle V" is, for example, information about dynamic objects such as people and other vehicles around the vehicle V, static objects such as curbs and buildings around the vehicle V, various signs, road markings such as guide lines, etc. Then, during parking assistance, the parking assistance device 5 automatically moves the vehicle V from the assistance start position STP to the planned parking position SEP along the target route TP. The planned parking position SEP is a position that is an end point of the target route TP. The planned parking position SEP is registered in advance by the user as the parking space SP of the vehicle.

[0019] Specifically, when the user parks the vehicle V, the parking assistance device 5 stores sensing information, which is the detection result of the perimeter monitoring sensor 3, in the non-volatile memory of the storage unit 50. Then, the parking assistance device 5 performs various controls for generating the target route TP and parking assistance based on the sensing information stored in the storage unit 50 and the sensing information of the perimeter monitoring sensor 3 during parking assistance.

[0020] The learning process for storing information about the driving route and the surroundings of the vehicle V during manual driving by the user is executed when an instruction to execute the learning process is given, for example, when a learning switch (not shown) is operated by the user. The parking assistance is executed when an instruction to execute parking assistance is given by the user, for example, when a parking assistance start switch 35 is operated by the user.

[0021] When the parking assistance device 5 executes the learning process, it recognizes targets, free spaces where parking is possible, parking positions, and the like on the driving route of the vehicle V based on the sensing information of the periphery monitoring sensor 3. These recognition results are sequentially stored in the non-volatile memory of the storage unit 50 and are used during parking assistance.

[0022] In addition, when a parking assistance instruction is issued by the user, the parking assistance device 5 generates a target route TP based on the sensing information stored in the storage unit 50 and the sensing information of the surrounding monitoring sensor 3 during parking assistance, and performs route tracking control according to the route. Specifically, the parking assistance device 5 is configured to have a recognition processing unit 51, a vehicle information acquisition unit 52, and a parking control unit 53 as functional units that execute various controls.

[0023] The recognition processing unit 51 receives sensing information from the surroundings monitoring sensor 3, and recognizes the surrounding environment of the vehicle to be parked, the scene of the parking scene, and objects present around the vehicle based on the sensing information. Here, the recognition processing unit 51 is composed of an image recognition unit 51a, a space recognition unit 51b, and a free space recognition unit 51c.

[0024] The image recognition unit 51a performs scene recognition, three-dimensional object recognition, etc. Various types of recognition by the image recognition unit 51a are realized by performing image analysis on the captured image data from the surrounding monitoring camera 31 input as sensing information.

[0025] In scene recognition, it is recognized what kind of parking scene it is. For example, it is recognized whether it is a normal parking scene in which there is no obstacle OB near the intended parking position SEP and parking of the vehicle V is not particularly restricted, or whether it is a special parking scene in which parking of the vehicle V is restricted by an obstacle OB.

[0026] Since the image data input from the periphery monitoring camera 31 shows the surroundings, by analyzing the image, it is possible to determine whether the parking scene is a normal parking scene or a special parking scene. For example, if an object around the intended parking position SEP is detected from the image data and the object becomes an obstacle to parking at the intended parking position SEP, it can be determined that the parking scene is a special parking scene. Note that the scene recognition may be performed based not only on the sensing information of the periphery monitoring camera 31 but also on the sensing information of the search wave sensor.

[0027] In the three-dimensional object recognition, obstacles OB, which are three-dimensional objects around the vehicle, such as dynamic targets and static targets, are recognized as detection objects. Based on the detection objects recognized by this three-dimensional object recognition, preferably the shapes of static targets, the above-mentioned scene recognition and the generation of a parking assistance map including the obstacles OB are performed.

[0028] The space recognition unit 51b performs three-dimensional object recognition and the like. The space recognition unit 51b recognizes three-dimensional objects in the space around the vehicle based on sensing information from at least one of the sonar 32, the millimeter wave radar 33, and the LiDAR 34. The three-dimensional object recognition here is similar to the three-dimensional object recognition performed by the image recognition unit 51a. Therefore, three-dimensional object recognition can be performed if either the image recognition unit 51a or the space recognition unit 51b is provided. In addition, in the case of this embodiment, the space recognition unit 51b does not perform scene recognition, but the space recognition unit 51b can also perform scene recognition based on sensing information from at least one of the sonar 32, the millimeter wave radar 33, and the LIDAR 34.

[0029] Although three-dimensional object recognition and scene recognition can be performed by either the image recognition unit 51a or the space recognition unit 51b, more accurate three-dimensional object recognition and scene recognition can be performed by using both of them. For example, three-dimensional object recognition and scene recognition by the image recognition unit 51a can be complemented by three-dimensional object recognition and scene recognition by the space recognition unit 51b, thereby making it possible to perform three-dimensional object recognition and scene recognition with higher accuracy.

[0030] The free space recognition unit 51c performs free space recognition to recognize a place that is a free space in the parking lot PL. A free space means, for example, a space in the parking lot PL that has a size and shape that allows a vehicle V to stop. There may be only one free space in the parking lot PL, not necessarily a plurality of free spaces.

[0031] The free space recognition unit 51c recognizes free spaces in the parking lot PL based on the results of scene recognition and three-dimensional object recognition by the image recognition unit 51a and the space recognition unit 51b. For example, the results of scene recognition and three-dimensional object recognition can be used to understand the shape of the parking lot PL and the presence or absence of other vehicles parked therein, and the free space recognition unit 51c recognizes free spaces in the parking lot PL based on this. In addition, the free space recognition unit 51c identifies free spaces in the image by using, for example, semantic segmentation, which categorizes each pixel in the image based on the surrounding information of each pixel.

[0032] The vehicle information acquisition unit 52 acquires information on the operation amount of the vehicle V from other ECUs 4, etc. Specifically, the vehicle information acquisition unit 52 acquires detection signals output from each sensor mounted on the vehicle V, such as an accelerator position sensor, a brake depression force sensor, a steering angle sensor, a wheel speed sensor, a shift position sensor, etc.

[0033] The parking control unit 53 executes various controls required for parking assistance. Specifically, the parking control unit 53 includes a route storage unit 54, a route generation unit 55, a position estimation unit 56, a following control unit 57, an information provision unit 58, and an image generation unit 59 as functional units that execute various controls.

[0034] The route storage unit 54 stores in the storage unit 50 sensing information of the perimeter monitoring sensor 3 when the user parks the vehicle V. For example, when the learning process is started, the route storage unit 54 stores in the storage unit 50 as route information the landmarks, free spaces in which parking is possible, parking positions, and the like on the travel route of the vehicle V successively acquired by the recognition processing unit 51.

[0035] In addition, when the learning process is started, the route storage unit 54 stores the image data and the like sequentially input from the surrounding monitoring camera 31 in the storage unit 50 as route information. Note that, when the image data of each of the front camera 31a, the rear camera 31b, the left side camera 31c, and the right side camera 31d is sequentially stored in the storage unit 50, the amount of information of the route information increases, which may cause the capacity of the storage unit 50 to be insufficient. For this reason, the route storage unit 54 may be configured to store, for example, composite image data obtained by synthesizing the image data of the front camera 31a, the rear camera 31b, the left side camera 31c, and the right side camera 31d in the storage unit 50.

[0036] The route generating unit 55 generates a route based on the results of the scene recognition, the three-dimensional object recognition, and the free space recognition. The route generating unit 55 generates a target route TP that the vehicle V should take when parking the vehicle V, based on the travel route of the vehicle V during the learning process and information about the periphery of the vehicle V on the travel route. For example, the route generating unit 55 uses the travel route of the vehicle V as a reference route, and when there is a section in the reference route where the distance between the vehicle V and an obstacle OB is equal to or less than a predetermined value, the route generating unit 55 replaces the section with a route where the distance between the vehicle V and the obstacle OB exceeds the predetermined value, and generates the target route TP. Note that the obstacle OB is composed of a three-dimensional object recognized by the three-dimensional object recognition.

[0037] The position estimation unit 56 estimates the current position of the vehicle V based on the sensing information stored in the memory unit 50 and the sensing information successively acquired by the periphery monitoring sensor 3 during parking assistance. For example, the position estimation unit 56 compares the sensing information stored in the memory unit 50 with the sensing information acquired during parking assistance, and estimates the current position based on the difference therebetween.

[0038] The following control unit 57 automatically moves the vehicle V from the assistance start position STP to the intended parking position SEP along the target route TP by performing vehicle motion control such as acceleration / deceleration control and steering control of the vehicle V. Specifically, the following control unit 57 outputs control signals to the various ECUs 4 so that the current position of the vehicle V estimated by the position estimation unit 56 reaches the intended parking position SEP along the target route TP.

[0039] The various ECUs 4 include a steering ECU 41 that performs steering control, a brake ECU 42 that performs acceleration / deceleration control, a power management ECU 43, and a body ECU 44 that controls various electrical components such as lights and door mirrors.

[0040] Specifically, the following control unit 57 acquires detection signals output from various sensors mounted on the vehicle V, such as an accelerator position sensor, a brake depression force sensor, a steering angle sensor, a wheel speed sensor, and a shift position sensor, via the vehicle information acquisition unit 52. Then, the following control unit 57 detects the state of each part from the acquired detection signals, and outputs control signals to various ECUs 4 to move the vehicle V by following the target route TP.

[0041] The information providing unit 58 provides information to the user by using the HMI (short for Human Machine Interface) 45. The HMI 45 is a device for providing various types of support to the user. The HMI 45 has a touch panel display unit 46 and a speaker 47. The touch panel display unit 46 is a touch panel type display used in a navigation system or a meter system.

[0042] Before the start of the tracking control process, the information providing unit 58 provides the user with information about the planned parking position SEP, which is included in the route information stored in the storage unit 50, in a visual form. For example, the information providing unit 58 displays an image showing the surroundings of the planned parking position SEP on the touch panel display unit 46 before the start of the tracking control process.

[0043] The information providing unit 58 displays various buttons to encourage the user to perform touch operations on the touch panel display unit 46. The various buttons are operation buttons that are touch-operated by the user. The information providing unit 58 displays, for example, a start button STB for the tracking control process, a selection button SLB for selecting the planned parking position SEP, and the like on the touch panel display unit 46. The touch panel display unit 46 of this embodiment not only displays information, but also serves as an "operation unit" that is operated by the user.

[0044] The information providing unit 58 changes the display content of the touch panel display unit 46 in response to an operation signal of a touch operation on the touch panel display unit 46. For example, the information providing unit 58 changes the viewpoint of a three-dimensional display (i.e., a 3D view) displayed on the touch panel display unit 46 in response to an operation signal of the touch panel display unit 46 by the user.

[0045] The image generating unit 59 generates imaging data to be displayed on the touch panel display unit 46 using imaging data captured by the periphery monitoring camera 31. Note that in this embodiment, the image generating unit 59 and the image recognizing unit 51a are separate, but this is not limiting, and for example, the image generating unit 59 may be included in the image recognizing unit 51a.

[0046] The image generating unit 59 periodically or irregularly generates surrounding image data (hereinafter also referred to as surrounding image) using, for example, image data from the front camera 31a, the rear camera 31b, the left side camera 31c, and the right side camera 31d. The surrounding image is an image corresponding to at least a part of the range of the area around the vehicle V, and includes a camera viewpoint image Gc and a composite image. The camera viewpoint image Gc is an image having the viewpoint of the position where the lens of each of the surrounding monitoring cameras 31 is disposed. One of the composite images is an image (hereinafter also referred to as a virtual viewpoint image) of the surroundings of the vehicle V viewed from a virtual viewpoint set at an arbitrary position around the vehicle V. A method of generating a virtual viewpoint image will be described below.

[0047] The image generating unit 59 projects information of each pixel included in the imaging data of the front camera 31a, the rear camera 31b, the left side camera 31c, and the right side camera 31d onto a predetermined projection curved surface (for example, a bowl-shaped curved surface) in a virtual three-dimensional space. Specifically, the image generating unit 59 projects information of each pixel included in the imaging data of the front camera 31a, the rear camera 31b, the left side camera 31c, and the right side camera 31d onto a part other than the center of the projection curved surface. The center of the projection curved surface is defined as the position of the vehicle V. Then, the image generating unit 59 sets a virtual viewpoint in the virtual three-dimensional space, and generates a virtual viewpoint image by cutting out a predetermined area of ​​the projection curved surface included in a predetermined viewing angle as image data when viewed from the virtual viewpoint. The virtual viewpoint image obtained in this manner is a three-dimensional display of an image showing the periphery of the vehicle V.

[0048] Here, the image generator 59 generates a virtual vehicle image Gv showing the vehicle V and an image in which lines, frames, marks, etc. for supporting the parking operation are superimposed on each of the camera viewpoint image Gc and the virtual viewpoint image. The virtual vehicle image Gv is composed of, for example, opaque or semi-transparent polygons showing the shape of the vehicle V.

[0049] The automatic parking system 1 according to the present embodiment is configured as described above. Next, the operation of the automatic parking system 1 configured as described above will be described. In this embodiment, the case where the vehicle V is parked in the parking lot PL shown in FIG. 2 will be described as an example. Four parking spaces SP for the vehicle V are set in the parking lot PL shown in FIG. 2. In the parking lot PL, a first parking space SP1 and a second parking space SP2 are provided vertically in a row along the passage PS extending in a straight line from the vehicle entrance / exit B. In addition, in the parking lot PL, a third parking space SP3 and a fourth parking space SP4 are provided adjacent to each other so as to intersect with the passage PS. The third parking space SP3 and the fourth parking space SP4 are provided between the building BL and the house HM. In the third parking space SP3, the vehicle V can be parked facing forward by moving forward and backward (i.e., turning around) after passing in front of the third parking space SP3. The same applies to the fourth parking space SP4. In FIG. 2, the third parking space SP3 is set as the planned parking position SEP, and the vehicle V is parked facing forward at the planned parking position SEP.

[0050] First, an example of the learning process will be described with reference to the flowchart shown in Fig. 3. The learning process shown in Fig. 3 is executed by the parking control unit 53 at every predetermined control period when an instruction to perform the learning process is issued, such as when a learning switch (not shown) is operated by a user. Note that each process shown in this flowchart is realized by each functional unit of the parking assistance device 5. Also, each step that realizes this process can be understood as each step that realizes a parking assistance method.

[0051] 3, the parking control unit 53 starts the recognition process in step S100. In this recognition process, the recognition processing unit 51 starts scene recognition, three-dimensional object recognition, and free space recognition based on the sensing information of the perimeter monitoring sensor 3.

[0052] Next, in step S110, the parking control unit 53 determines whether or not a learning start condition is satisfied. The learning start condition is satisfied, for example, when the vehicle V enters a learning start area around the parking lot PL that is designated in advance by the user. The learning start condition may be satisfied when a learning switch (not shown) is turned on.

[0053] The parking control unit 53 waits until the learning start condition is satisfied, and when the learning start condition is satisfied, in step S120, starts storing various information necessary for parking assistance. For example, the parking control unit 53 stores in the storage unit 50, as route information, targets on the travel route of the vehicle V, free spaces where parking is possible, parking positions, and the like, which are sequentially acquired by the recognition processing unit 51. The parking control unit 53 of this embodiment stores in the storage unit 50 peripheral images when the vehicle V is traveling and when it is parked at a parking position. Specifically, the parking control unit 53 stores in the storage unit 50 images captured by the front camera 31a, the rear camera 31b, the left side camera 31c, and the right side camera 31d as peripheral images when parking. Note that, if the storage capacity of the storage unit 50 is insufficient, it is desirable to store in the storage unit 50 a composite image obtained by combining images captured by the front camera 31a, the rear camera 31b, the left side camera 31c, and the right side camera 31d as a peripheral image when parking.

[0054] Next, in step S130, the parking control unit 53 determines whether or not a learning stop condition is satisfied. The learning stop condition is satisfied when the vehicle V stops at or near the planned parking position SEP designated in advance by the user. The learning stop condition may also be satisfied when the shift position is switched to a position indicating parking (for example, the P position).

[0055] Until the learning stop condition is satisfied, the parking control unit 53 continues to store the various information in the storage unit 50. On the other hand, when the learning stop condition is satisfied, the parking control unit 53 stops storing the various information in step S140.

[0056] Next, in step S150, the parking control unit 53 notifies the user via the HMI 45 that storage of various information has been completed, and ends the learning process. In the process of step S150, for example, the driving route of the vehicle V at the time of the learning process and the surrounding conditions on the driving route may be notified. The learning process from step S100 to step S150 is performed by the route storage unit 54 of the parking control unit 53.

[0057] Next, an example of an assistance process for automatically moving the vehicle V from the assistance start position STP to the intended parking position SEP along the target route TP will be described with reference to the flowchart shown in Fig. 4. The assistance process shown in Fig. 4 is executed by the parking control unit 53 at each predetermined control period under a condition in which the learning process has been performed at least once. Note that each process shown in this flowchart is realized by each functional unit of the parking assistance device 5. Also, each step for realizing this process can be understood as each step for realizing a parking assistance method.

[0058] As shown in Fig. 4, in step S200, the parking control unit 53 uses sensing information from the perimeter monitoring sensor 3 and a GPS and a map database (not shown) to determine whether the current position of the vehicle V is near the assistance start position STP. The assistance start position STP is set near the vehicle entrance / exit B of the parking lot PL. The vehicle entrance / exit B is the boundary between the public road OL and the parking lot PL. The assistance start position STP may be set on the public road OL side instead of on the parking lot PL side.

[0059] As a result of the determination process in step S200, if the current position of the vehicle V is near the assistance start position STP, the parking control unit 53 proceeds to the process in step S210. In step S210, the parking control unit 53 notifies the user that the current position of the vehicle V is near the assistance start position STP via the HMI 45. For example, as shown in FIG. 5, the parking control unit 53 notifies the user that the current position of the vehicle V is near the assistance start position STP by displaying a camera viewpoint image Gc in the left area of ​​the touch panel display unit 46 and displaying an overhead image Gh in the right area. The notification to the user may be realized by displaying a message on the touch panel display unit 46 indicating that the vehicle V is near the assistance start position STP, outputting a voice from the speaker 47 indicating that the vehicle V is near the assistance start position STP, or the like.

[0060] Here, the camera viewpoint image Gc shown in FIG. 5 is an image with the viewpoint being the lens installation position of a camera (front camera 31a in this example) that captures the scenery in the planned moving direction of the vehicle V. Also, the overhead image Gh shown in FIG. 5 is an image in which a virtual vehicle image Gv is superimposed on a virtual viewpoint image obtained by cutting out an area on the projection curved surface included in a predetermined viewing angle as an image when the projection curved surface is viewed from a virtual viewpoint set directly above the vehicle V. The camera viewpoint image Gc and the overhead image Gh are generated by the image generating unit 59 based on an image captured by the surrounding monitoring camera 2 during the execution of the assistance process. In order to facilitate understanding of the present disclosure, the parking space SP, objects, etc. shown in the camera viewpoint image Gc and the overhead image Gh are given the same reference numerals as the real objects. This also applies to images other than the camera viewpoint image Gc and the overhead image Gh.

[0061] Next, in step S220, the parking control unit 53 determines whether or not there is an instruction from the user to perform parking assistance via the operation of the parking assistance start switch 35. If the parking assistance start switch 35 has not been turned on by the user, the parking control unit 53 skips the subsequent processes and exits from this process. On the other hand, if the parking assistance start switch 35 has been turned on by the user, the parking control unit 53 performs a process of generating a target route TP in step S230. Details of the process of step S230 will be described below with reference to the flowchart shown in FIG.

[0062] 6, in step S300, the parking control unit 53 reads the route information stored in the memory unit 50 during the learning process. When multiple pieces of route information with different parking positions are stored in the memory unit 50, the parking control unit 53 reads the multiple pieces of route information.

[0063] Next, the parking control unit 53 starts the recognition process in step S310. In this recognition process, the recognition processing unit 51 starts scene recognition, three-dimensional object recognition, and free space recognition based on the sensing information of the perimeter monitoring sensor 3.

[0064] Next, in step S320, the parking control unit 53 generates a target route TP based on the route information. Specifically, the parking control unit 53 generates a target route TP that the vehicle V should take when parking the vehicle V based on the travel route of the vehicle V during the learning process and information about the periphery of the vehicle V on the travel route. As shown in FIG. 7, this target route TP is a route for parking the vehicle V facing forward into the third parking space SP3 by turning the vehicle V after passing in front of the third parking space SP3, similar to the learning process.

[0065] Here, for example, if the route information obtained when the vehicle V is parked in the third parking space SP3 is stored in the memory unit 50, the parking control unit 53 generates a target route TP with the third parking space SP3 as the planned parking position SEP.

[0066] On the other hand, if the route information obtained when the vehicle V is parked in the third parking space SP3 and the fourth parking space SP4 is stored in the storage unit 50, the parking control unit 53 sets the third parking space SP3 and the fourth parking space SP4 as candidate positions for the planned parking position SEP. Then, the parking control unit 53 generates a target route TP for each candidate position for the planned parking position SEP.

[0067] Next, in step S330, the parking control unit 53 determines whether there is a new obstacle OB on the target route TP that was not present at the time of the learning process. Specifically, the parking control unit 53 determines whether there is a new obstacle OB based on the recognition result of the three-dimensional object recognition at the assistance start position STP and the recognition result of the three-dimensional object recognition included in the route information. Note that the obstacle OB is composed of a three-dimensional object recognized by the three-dimensional object recognition.

[0068] If no new obstacle OB is found on the target route TP, the current situation is considered to be similar to the situation during the learning process, so the parking control unit 53 skips the subsequent processes and exits from this process.

[0069] On the other hand, if there is a new obstacle OB on the target route TP, it becomes difficult to move the vehicle V to the planned parking position SEP along the target route TP generated in the process of step S320. For this reason, in step S340, the parking control unit 53 searches for an object avoidance route that avoids the obstacle OB on the target route TP and reaches the planned parking position SEP, and attempts to generate the object avoidance route. Specifically, the parking control unit 53 searches for a section in the travel route of the vehicle V included in the route information where the distance between the vehicle V and the obstacle OB is equal to or less than a predetermined value, and replaces the section with a route where the distance between the vehicle V and the obstacle OB exceeds the predetermined value, thereby generating the object avoidance route. The object avoidance route generated in this way is, for example, a route in which a collision between the vehicle V and the obstacle OB is avoided, as shown in FIG. 8. Note that dynamic targets among the obstacles OB move. For this reason, it is desirable that the object avoidance route is a route that avoids only static targets.

[0070] Next, in step S350, the parking control unit 53 determines whether the object avoidance path has been generated. If the object avoidance path has been generated, in step S360, the parking control unit 53 sets the object avoidance path as the target path TP and exits this process. By setting the object avoidance path as the target path TP, the object avoidance path is provided to the user in a visual form during the display process of the planned parking position SEP, which will be described later.

[0071] On the other hand, if the object avoidance path cannot be generated, the parking control unit 53 turns on a parking impossible flag, which indicates that parking at the intended parking position SEP is impossible, in step S370, and ends this process. The processes of steps S320 to S370 are performed by the path generation unit 55 of the parking control unit 53.

[0072] When the generation process of the target route TP in step S230 is completed, the parking control unit 53 proceeds to step S240 shown in Fig. 4. In step S240, the parking control unit 53 determines whether parking is possible at the planned parking position SEP. In this determination process, for example, if the parking impossible flag is off, it is determined that parking is possible at the planned parking position SEP, and if the parking impossible flag is on, it is determined that parking is not possible at the planned parking position SEP.

[0073] When parking at the planned parking position SEP is possible, the parking control unit 53 performs a process of displaying the planned parking position SEP in step S250. This process of displaying the planned parking position SEP will be described with reference to the flowchart shown in FIG.

[0074] 9, in step S400, the parking control unit 53 determines whether there are multiple candidate positions for the planned parking position SEP. For example, the parking control unit 53 determines whether route information obtained when the vehicle V was parked in a different parking space SP is stored in the storage unit 50.

[0075] If there are not multiple candidate positions for the planned parking position SEP, the parking control unit 53 provides the information on the planned parking position SEP included in the route information to the user in a visual form in step S410. The process of step S410 is performed by the information providing unit 58 of the parking control unit 53.

[0076] The parking control unit 53 provides the user with a virtual parking image Gp obtained as information on the periphery of the planned parking position SEP from the route information before the start of the following control process. For example, as shown in Fig. 10, the parking control unit 53 displays the virtual parking image Gp in the upper right area of ​​the touch panel display unit 46. The virtual parking image Gp is ​​generated by the image generation unit 59 based on an image stored in the storage unit 50 as route information during the learning process.

[0077] Specifically, the parking control unit 53 superimposes a virtual vehicle image Gv and a parking space image Gf indicating the planned parking position SEP onto a virtual viewpoint image showing the area around the planned parking position SEP, and displays this as a virtual parking image Gp in the upper right area of ​​the touch panel display unit 46.

[0078] 11, the virtual vehicle image Gv is an image (polygon in this example) showing the vehicle V in a semi-transparent manner so that the user can distinguish between the image showing the current position of the vehicle V and the virtual parking image Gp. Also, the parking space image Gf is a thick-line image colored in blue or red so that the parking space image Gf can be distinguished from the parking space shown in the virtual viewpoint image.

[0079] The virtual parking image Gp is ​​a three-dimensional display of an image showing the periphery of the planned parking position SEP. The parking control unit 53 changes the viewpoint of the virtual parking image Gp in response to an operation signal of a touch operation on the touch panel display unit 46 by the user.

[0080] The parking control unit 53 acquires an operation signal corresponding to a touch operation such as a flick or drag on the touch panel display unit 46 in the direction indicated by the up-down and left-right rotation arrows R shown on the virtual parking image Gp, and changes the viewpoint of the virtual parking image Gp according to the operation signal. In addition, the parking control unit 53 enlarges and reduces the virtual parking image Gp by, for example, a touch operation on a zoom-in icon ZI and a zoom-out icon ZO shown on the virtual parking image Gp. Here, the enlargement and reduction of the virtual parking image Gp may be realized by an operation other than the icon operation. For example, the enlargement and reduction of the virtual parking image Gp is ​​preferably realized by pinching out, which spreads two fingers apart to widen the gap, and pinching in, which pinches two fingers together to narrow the gap, on the surface of the touch panel display unit 46. If the enlargement and reduction of the virtual parking image Gp is ​​realized by such a screen operation, it is possible to avoid a reduction in the display size of the virtual parking image Gp due to the icon display and a part of the image becoming difficult to see due to the superimposition of the icon. That is, the display size of the image and the visibility of the image can be ensured. These are not limited to the enlargement and reduction of the image, but also apply to a change in the viewpoint of the image, etc.

[0081] In addition to these, the parking control unit 53 displays a vehicle surroundings image Ga including a surrounding image showing the surroundings of the vehicle V in the left area of ​​the touch panel display unit 46, and displays an illustration image Gi in the lower right area of ​​the touch panel display unit 46.

[0082] The vehicle surroundings image Ga is an image in which a virtual vehicle image Gv, a target route image Gt, and an icon P are superimposed on a virtual viewpoint image obtained by cutting out an area on the projection curved surface included in a predetermined viewing angle when viewing the projection curved surface from a virtual viewpoint set behind the vehicle V. This vehicle surroundings image Ga is generated by the image generation unit 59 based on the image captured by the periphery monitoring camera 2 during the execution of the assistance process and information related to the target route TP.

[0083] The target route image Gt is an image showing a target route TP from the current position of the vehicle V to the planned parking position SEP. The parking control unit 53 specifies a route in front of an object shown in the vehicle surroundings image Ga on the target route TP as a forward route, and specifies a route behind the object on the target route TP as a rearward route. For example, the parking control unit 53 uses semantic segmentation to specify an object shown in the vehicle surroundings image Ga and specifies the front-rear positional relationship between the object and the target route TP. Then, the image generation unit 59 of the parking control unit 53 superimposes a portion Gtb corresponding to the rear route and a portion Gtf corresponding to the front route of the target route image Gt on the vehicle surroundings image Ga in different modes. For example, the image generation unit 59 superimposes an image in which the portion Gtf corresponding to the front route is shown as a solid line and the portion Gtb corresponding to the rear route is shown as a dashed line on the vehicle surroundings image Ga as the target route image Gt.

[0084] The virtual vehicle image Gv is displayed in an opaque manner on the vehicle surroundings image Ga. A front portion of the virtual vehicle image Gv is superimposed on the vehicle surroundings image Ga so that the viewpoint can be easily grasped. Note that the virtual vehicle image Gv showing the entire vehicle V may be superimposed on the vehicle surroundings image Ga.

[0085] The icon P indicates the planned parking position SEP. The icon P is superimposed near the planned parking position SEP. Note that, since the planned parking position SEP cannot be visually recognized in the vehicle surroundings image Ga shown in FIG. 10, the icon P is superimposed in a semi-transparent manner.

[0086] The illustration image Gi is an image showing the relationship between the current position of the vehicle V, the target route TP, and the planned parking position SEP. The illustration image Gi of this embodiment is composed of an overall bird's-eye view image including all of the current position of the vehicle V, the target route TP, and the planned parking position SEP. Unlike the virtual parking image Gp and the vehicle surroundings image Ga, the illustration image Gi is composed of pictures and drawings. The illustration image Gi is generated by the image generating unit 59 based on information about the target route TP and icons indicating the vehicle V and the planned parking position SEP that are prepared in advance. Note that the illustration image Gi shown in FIG. 10 shows only the current position of the vehicle V, the target route TP, and the planned parking position SEP, but other information such as an obstacle OB may also be shown.

[0087] Furthermore, the parking control unit 53 displays a start button STB in the area below the illustration image Gi on the touch panel display unit 46. This start button STB is a button that is touched by the user when instructing the start of the tracking control process.

[0088] In step S420, the parking control unit 53 determines whether or not the user has touched the start button STB. Then, the parking control unit 53 waits until the start button STB is touched, and when the start button STB is touched, the parking control unit 53 exits this process and proceeds to the follow-up control process in step S260 shown in FIG.

[0089] On the other hand, if there are multiple candidate positions for the planned parking position SEP, the parking control unit 53 provides the user with information on the multiple candidate positions and information prompting the user to move to the planned parking position SEP in step S430. This step S430 process is performed by the information providing unit 58 of the parking control unit 53.

[0090] For example, as shown in FIG. 12, the parking control unit 53 displays on the touch panel display unit 46 icons P1 and P2 indicating candidate positions superimposed on portions of the virtual parking image Gp, the vehicle surroundings image Ga, and the illustrated image Gi that correspond to the candidate positions of the planned parking position SEP.

[0091] In addition, the parking control unit 53 displays selection buttons SLB1, SLB2 corresponding to each candidate position in the lower area of ​​the vehicle surroundings image Ga on the touch panel display unit 46, and displays a decision button DB for determining the planned parking position SEP in the lower area of ​​the illustration image Gi.

[0092] After that, in step S440, the parking control unit 53 judges whether or not the user has selected a candidate position. For example, when one of the selection buttons SLB1, SLB2 is selected and the decision button DB is touched, the parking control unit 53 judges that the user has selected a candidate position. Also, when one of the selection buttons SLB1, SLB2 and the decision button DB is not touched, the parking control unit 53 judges that the user has not selected a candidate position.

[0093] The parking control unit 53 waits until the user selects a candidate position, and when the user selects a candidate position, the process proceeds to step S450. In step S450, the parking control unit 53 sets the candidate position selected by the user as the planned parking position SEP, and then proceeds to step S410. Here, when multiple candidate positions are far apart, the parking control unit 53 displays an image showing all of the multiple candidate positions on the touch panel display unit 46 before the user selects a candidate position. Then, when multiple candidate positions are far apart, the parking control unit 53 displays an image focused on the selected candidate position on the touch panel display unit 46 after the user selects a candidate position. This allows the user to clearly grasp the planned parking position SEP.

[0094] When the display process of the planned parking position SEP is completed in this manner, the parking control unit 53 proceeds to step S250 shown in Fig. 4. In step S250, the parking control unit 53 starts a following control process. The following control process is a process for automatically moving the vehicle V to the planned parking position SEP along the target route TP. The following control process will be described with reference to the flowchart shown in Fig. 13.

[0095] 13, the parking control unit 53 starts the recognition process in step S500. In this recognition process, the recognition processing unit 51 starts scene recognition, three-dimensional object recognition, and free space recognition based on the sensing information of the perimeter monitoring sensor 3.

[0096] Next, in step S510, the parking control unit 53 estimates the current position of the vehicle V based on the sensing information stored in the storage unit 50 and the sensing information successively acquired by the periphery monitoring sensor 3 during parking assistance. The process of step S510 is performed by the position estimation unit 56 of the parking control unit 53.

[0097] Next, in step S520, the parking control unit 53 starts automatic parking of the vehicle V at the intended parking position SEP by performing vehicle motion control such as acceleration / deceleration control and steering control of the vehicle V. The process of step S520 is performed by the following control unit 57 of the parking control unit 53.

[0098] Next, in step S530, the parking control unit 53 determines whether there is a new obstacle OB that is not present on the target route TP and at the planned parking position SEP at the time of the learning process. Specifically, the parking control unit 53 determines whether there is a new obstacle OB based on the recognition result of the three-dimensional object recognition after the start of the following control process and the recognition result of the three-dimensional object recognition included in the route information. Note that the obstacle OB is composed of a three-dimensional object recognized by the three-dimensional object recognition.

[0099] If there is no new obstacle OB on the target route TP or at the planned parking position SEP, the parking control unit 53 provides information about the target route TP to the user in step S540. This step S540 process is performed by the information providing unit 58 of the parking control unit 53.

[0100] For example, as shown in FIG. 14 and FIG. 15, the parking control unit 53 displays an icon P superimposed on a portion of the vehicle surroundings image Ga corresponding to the planned parking position SEP in the left area of ​​the touch panel display unit 46.

[0101] Here, Fig. 14 illustrates an example of the display content on the touch panel display unit 46 at the start of the tracking control process. Also, Fig. 15 illustrates an example of the display content on the touch panel display unit 46 after the tracking control process has started.

[0102] In the vehicle surroundings image Ga shown in FIG. 14, most of the planned parking position SEP is hidden by a building BL, so the icon P is superimposed in a semi-transparent manner. Also, in the vehicle surroundings image Ga shown in FIG. 15, the planned parking position SEP is visible, so the icon P is superimposed in an opaque manner. This makes it possible to grasp the location of the planned parking position SEP. Note that in the vehicle surroundings image Ga shown in FIG. 15, part of the planned parking position SEP is cut off. For this reason, the icon P is superimposed on the right edge of the vehicle surroundings image Ga.

[0103] Moreover, the parking control unit 53 displays the target path image Gt superimposed on the overhead image Gh in the right region of the touch panel display unit 46. Furthermore, the parking control unit 53 displays a progress bar PB indicating the progress of the automatic parking of the vehicle V in the lower region of the vehicle surroundings image Ga on the touch panel display unit 46. This progress bar PB has a horizontally long bar shape, and the colored part inside the bar increases as the distance from the assistance start position STP to the current position of the vehicle V increases. This allows the user to visually grasp the progress of the automatic parking. Note that, for example, the remaining distance to the planned parking position SEP may be displayed in the lower region of the vehicle surroundings image Ga instead of the progress bar PB.

[0104] As shown in Fig. 16, the parking control unit 53 changes the display mode of the virtual viewpoint image so that the angle of the virtual viewpoint of the virtual viewpoint image becomes larger as the vehicle approaches the planned parking position SEP, and the angle of the virtual viewpoint of the virtual viewpoint image becomes smaller as the vehicle moves away from the planned parking position SEP. That is, as with the user's field of view, the range to be displayed in the image becomes narrower as the target approaches. This makes it easier for the user to grasp the distance to the planned parking position SEP, since the display mode of the virtual viewpoint image is changed as with the user's field of view.

[0105] Here, the image displayed in the right region of the touch panel display unit 46 is not limited to the overhead image Gh. For example, as shown in FIG. 17, instead of the overhead image Gh, an illustrated image Gi may be displayed in the right region of the touch panel display unit 46. When the illustrated image Gi is displayed on the touch panel display unit 46 in this manner, it is desirable to superimpose marks indicating the current position of the vehicle V, the planned parking position SEP, and the assistance start position STP on the illustrated image Gi. In this case, the user can visually grasp the progress of the automatic parking.

[0106] After that, in step S550, the parking control unit 53 determines whether or not the vehicle V has reached the planned parking position SEP. If the vehicle V has not reached the planned parking position SEP, the parking control unit 53 returns to the process of step S510, and if the vehicle V reaches the planned parking position SEP, the parking control unit 53 exits the following control process.

[0107] On the other hand, if there is a new obstacle OB on the target route TP and at the planned parking position SEP, it becomes difficult to move the vehicle V to the planned parking position SEP along the target route TP. Therefore, in step S560, the parking control unit 53 searches for an avoidance route that avoids the obstacle OB on the target route TP and reaches the planned parking position SEP, and attempts to generate the avoidance route. This search for the avoidance route is the same as the process in step S340, so a description thereof will be omitted.

[0108] Here, searching for an avoidance route may take time. For this reason, when starting to search for an avoidance route, the parking control unit 53 displays a message image Gm indicating that an avoidance route is being searched for on the touch panel display unit 46, as shown in Fig. 18, for example. In this way, by notifying the user of the internal state of the system, the user can be prepared for route changes during automatic parking.

[0109] The notification that an avoidance route is being searched for is not limited to displaying a message image Gm on the touch panel display unit 46. For example, the user may be notified that an avoidance route is being searched for by changing the color of the target route TP shown in the vehicle surroundings image Ga or by blinking the line indicating the target route TP. Also, the user may be notified that an avoidance route is being searched for by voice.

[0110] Next, in step S570, the parking control unit 53 determines whether an avoidance route has been generated. If an avoidance route has been generated, in step S580, the parking control unit 53 replaces the avoidance route with the target route TP and displays information about the avoidance route on the touch panel display unit 46. For example, the parking control unit 53 displays an image showing the avoidance route superimposed on the vehicle surroundings image Ga and the bird's-eye view image Gh on the touch panel display unit 46. In addition, the parking control unit 53 uses the speaker 47 to announce to the user that the avoidance route will be replaced with the target route TP. Note that a message to the effect that the target route will be changed to the avoidance route may be displayed on the touch panel display unit 46. Changing the target route to the avoidance route is not intended by the user, so it is desirable to combine the display of the avoidance route on the touch panel display unit 46 with the display of a message about the route change and / or a voice notification.

[0111] After that, in step S550, the parking control unit 53 determines whether or not the vehicle V has reached the planned parking position SEP. If the vehicle V has not reached the planned parking position SEP, the parking control unit 53 returns to the process of step S510, and if the vehicle V reaches the planned parking position SEP, the parking control unit 53 exits the following control process.

[0112] On the other hand, if the avoidance route cannot be generated, the parking control unit 53, in step S590, identifies a stopping position TSP where the vehicle V can be stopped in the parking lot PL. Specifically, the parking control unit 53 identifies the stopping position TSP by using the recognition result of the free space recognition. For example, as shown in FIG. 19, if the second parking space SP2 is vacant, the parking control unit 53 identifies the second parking space SP2 as the stopping position TSP. This stopping position TSP is a possible stopping position different from the planned parking position SEP. Note that the parking control unit 53 may be configured to identify a free space other than the second parking space SP2 as the stopping position TSP.

[0113] Next, in step S600, the parking control unit 53 provides the user with information on the stop position TSP and the route to the stop position TSP, and recommends stopping at a position different from the planned parking position SEP. For example, when the second parking space SP2 is specified as the stop position TSP, the parking control unit 53 displays an icon P superimposed on a portion of the vehicle surroundings image Ga corresponding to the stop position TSP in the left area of ​​the touch panel display unit 46, as shown in FIG. 20. In addition, the parking control unit 53 displays an image showing the route to the stop position TSP superimposed on the overhead image Gh in the right area of ​​the touch panel display unit 46. Furthermore, the parking control unit 53 uses the speaker 47 to announce that the vehicle cannot be parked at the planned parking position SEP and will stop at a position different from the planned parking position SEP. The processing of step S600 is performed by the information providing unit 58 of the parking control unit 53. Furthermore, the parking control unit 53 displays a start button STB in the area below the vehicle surroundings image Ga on the touch panel display unit 46, and when the start button STB is touched, the process proceeds to another position stop process in step S610.

[0114] Next, in step S610, the parking control unit 53 executes another position stopping process to move and stop the vehicle V at the stopping position TSP. The parking control unit 53 starts automatic parking of the vehicle V at the stopping position TSP by performing vehicle motion control such as acceleration / deceleration control and steering control of the vehicle V. The process of step S610 is performed by the following control unit 57 of the parking control unit 53. When the vehicle V reaches the stopping position TSP, the parking control unit 53 stops the vehicle V at the stopping position TSP and exits this process.

[0115] Returning to FIG. 4, if the determination result of step S240 is that parking at the planned parking position SEP is not possible, the parking control unit 53 identifies a possible stopping position where the vehicle V can be stopped in step S270. In other words, if the object avoidance path cannot be generated when generating the target path TP, the parking control unit 53 identifies the possible stopping position. Specifically, the parking control unit 53 identifies the possible stopping position by using the recognition result of the free space recognition. For example, as shown in FIG. 19, if the second parking space SP2 is vacant, the parking control unit 53 identifies the second parking space SP2 as the possible stopping position. This possible stopping position is a possible stopping position different from the planned parking position SEP. Note that the parking control unit 53 may be configured to identify a free space other than the second parking space SP2 as the possible stopping position.

[0116] Next, in step S280, the parking control unit 53 provides the user with information on possible parking positions and routes to the possible parking positions, and recommends parking at a position different from the planned parking position SEP. In this process, for example, an icon P is superimposed on the possible parking positions shown in the vehicle surroundings image Ga and the overhead image Gh and displayed on the touch panel display unit 46. In addition, the parking control unit 53 uses the speaker 47 to announce that the vehicle cannot be parked at the planned parking position SEP and will be parked at a position different from the planned parking position SEP. The process of step S280 is performed by the information providing unit 58 of the parking control unit 53. In addition, the parking control unit 53 displays a start button STB on the touch panel display unit 46, and when the start button STB is touched, the process proceeds to another position stop process of step S290.

[0117] Next, in step S290, the parking control unit 53 executes another position stopping process to move the vehicle V to a possible stopping position and stop the vehicle V. The parking control unit 53 starts automatic parking of the vehicle V at the possible stopping position by performing vehicle motion control such as acceleration / deceleration control and steering control of the vehicle V. This process of step S290 is performed by the following control unit 57 of the parking control unit 53. When the vehicle V reaches the possible stopping position, the parking control unit 53 stops the vehicle V at the possible stopping position and exits this process.

[0118] The parking assistance device 5 and parking assistance method described above generate a target route TP that the vehicle V should take when parking the vehicle V, based on route information including information on the driving route of the vehicle V and the surroundings of the vehicle V on the driving route when the user parks the vehicle V. The parking assistance device 5 and parking assistance method perform a following control process that automatically moves the vehicle V to the planned parking position SEP along the target route TP. The parking assistance device 5 and parking assistance method provide information on the planned parking position SEP, which is included in the above-mentioned route information, to the user in a visual form before the following control process begins.

[0119] According to this, the user can start automatic parking of the vehicle V at the planned parking position SEP after clearly understanding the planned parking position SEP. Therefore, according to the parking assistance device 5 and the parking assistance method of the present disclosure, the usability of automatic parking can be improved.

[0120] Moreover, according to this embodiment, the following effects can be obtained.

[0121] (1) The information providing unit 58 provides the user with an image obtained as information about the vicinity of the planned parking position SEP from the route information, in which a virtual vehicle image Gv showing the vehicle V is superimposed on the planned parking position SEP shown in the image, before the start of the following control process. This allows the user to visually grasp the parking state of the vehicle V at the planned parking position SEP before the start of the following control process. In other words, the user can easily imagine the parking position of the vehicle V by automatic parking before the start of the following control process. This is a factor that increases user satisfaction and greatly contributes to improving the usability of automatic parking.

[0122] (2) The virtual vehicle image Gv is an image that shows the vehicle V in a semi-transparent manner. This allows the user to recognize that the image showing the parking state of the vehicle V at the planned parking position SEP does not show the current position of the vehicle V. In other words, it is possible to prevent the user from mistaking the image showing the parking state of the vehicle V at the planned parking position SEP for the current position of the vehicle V.

[0123] (3) The information providing unit 58 provides the user with an image obtained as information on the periphery of the planned parking position SEP from the route information, with a parking space image Gf showing the planned parking position SEP superimposed thereon, before the start of the tracking control process. By doing so, the planned parking position SEP is highlighted, making it easier for the user to visually grasp the planned parking position SEP before the start of the tracking control process.

[0124] (4) The information providing unit 58 provides the user with a three-dimensional display of an image showing the periphery of the planned parking position SEP, while changing the viewpoint of the three-dimensional display in response to an operation signal of the touch panel display unit 46 by the user. This makes it possible to provide the user with detailed information about the planned parking position SEP. In particular, being able to change the viewpoint of the three-dimensional display in response to a touch operation on the touch panel display unit 46 makes it possible to provide information in line with the user's wishes.

[0125] (5) When the route information includes a plurality of candidate positions for the planned parking position SEP, the information providing unit 58 visually provides the user with information on the plurality of candidate positions included in the route information. The information providing unit 58 then provides information for prompting the user to select the planned parking position SEP from among the plurality of candidate positions. In this way, if the user is able to select the planned parking position SEP, parking assistance that appropriately reflects the user's intentions can be realized.

[0126] (6) When a situation occurs in which the vehicle V cannot be parked at the planned parking position SEP after the start of the following control process, the following control unit 57 identifies a possible stopping position other than the planned parking position SEP based on information about the surroundings of the vehicle V obtained after the start of the following control process. Then, when a situation occurs in which the vehicle V cannot be parked at the planned parking position SEP after the start of the following control process, the information providing unit 58 provides information for recommending stopping at a possible stopping position. In this way, even if a situation occurs in which the vehicle V cannot be parked at the planned parking position SEP after the start of automatic parking, it is desirable to recommend to the user that the vehicle be stopped at a possible stopping position other than the planned parking position SEP. Note that, for example, a situation in which the vehicle V cannot be parked at the planned parking position SEP may be a case in which another vehicle is parked at the planned parking position SEP or an obstacle OB that prevents the vehicle V from parking is installed at the planned parking position SEP.

[0127] (7) After the start of the following control process, the information providing unit 58 provides the user with a target route image Gt indicating the target route TP superimposed on a surrounding image showing the surroundings of the vehicle V obtained during the following control process. In this manner, it is desirable that the route along which the vehicle V is scheduled to travel during automatic parking is provided to the user in a visual manner. This allows the user to clearly understand the travel route of the vehicle V to the planned parking position SEP before automatic parking of the vehicle V at the planned parking position SEP is provided.

[0128] (8) The information providing unit 58 identifies a route in front of an object shown in the peripheral image on the target route TP as a forward route, and identifies a route behind the object on the target route TP as a rearward route. The information providing unit 58 superimposes a portion Gtb of the target route image Gt corresponding to the rearward route and a portion Gtf corresponding to the forward route on the peripheral image in different modes. In this way, it is desirable that the target route TP of the vehicle V is provided in a manner in which a route that is actually visible to the user and a route that is actually invisible to the user are distinguished from each other.

[0129] (9) The information providing unit 58 provides the user with an illustrated image Gi showing the relationship between the current position of the vehicle V, the target route TP, and the planned parking position SEP. The illustrated image Gi contains less unnecessary information than a captured image, etc., so the current position of the vehicle V, the target route TP, and the planned parking position SEP stand out. For this reason, by providing the user with the illustrated image Gi showing the current position of the vehicle V, the target route TP, and the planned parking position SEP, it becomes easier to convey an overview of automatic parking to the user.

[0130] (10) When an obstacle OB is found on the target route TP, the following control unit 57 attempts to generate an avoidance route that avoids the obstacle OB and reaches the intended parking position SEP. Then, when an avoidance route is generated by the following control unit 57, the information providing unit 58 provides information about the avoidance route to the user in a visual manner. In this way, when an obstacle OB is found on the target route TP, it is desirable that the information about the avoidance route be provided to the user in a visual manner.

[0131] (11) When an avoidance route cannot be generated, the following control unit 57 identifies a stop position TSP where the vehicle V can be stopped. The information providing unit 58 provides the user with information on the stop position TSP and the route to the stop position TSP. In this way, when an avoidance route cannot be generated, it is desirable that the configuration be such that information on the stop position TSP of the vehicle V, etc., is provided to the user as an alternative to the avoidance route.

[0132] (12) When the generation of an avoidance route by the following control unit 57 starts, the information providing unit 58 provides the user with information indicating that the avoidance route is being generated by the following control unit 57. In this manner, it is desirable to inform the user that the avoidance route is being generated before providing the user with information, etc., related to the avoidance route. By informing the user of the route change in stages, rather than abruptly, it is expected that the psychological burden on the user associated with the route change can be reduced.

[0133] (13) If an obstacle OB is found on the target route TP before the start of the following control process, the route generation unit 55 attempts to generate an object avoidance route that avoids the obstacle OB and reaches the planned parking position before the start of the following control process. If an object avoidance route is generated by the route generation unit 55, the information provision unit 58 provides information about the object avoidance route to the user in a visual form before the start of the following control process. In this way, if it is found that an obstacle OB is on the target route TP before the start of automatic parking, it is desirable that the information about the object avoidance route be provided to the user in a visual form before the start of automatic parking.

[0134] (14) If the path generation unit 55 is unable to generate an object avoidance path, the following control unit 57 identifies a possible stopping position where the vehicle V can be stopped. Then, the information provision unit 58 provides the user with information regarding the possible stopping position and the path to the possible stopping position. In this way, if it is determined before the start of automatic parking that parking at the intended parking position SEP is not possible, it is desirable that the configuration be such that information regarding the possible stopping position of the vehicle V, etc. is provided to the user as an alternative to the object avoidance path.

[0135] (Other embodiments) Representative embodiments of the present disclosure have been described above, but the present disclosure is not limited to the above-described embodiments and can be modified in various ways, for example, as described below.

[0136] In the above embodiment, the detailed configuration of the parking assistance device 5, the detailed contents of the learning process, and the detailed contents of the following control process have been described, but the present invention is not limited to these, and some of these may be different.

[0137] The parking assistance device 5 described above displays the virtual parking image Gp on the touch panel display unit 46 as information related to the intended parking position SEP before the start of the following control process, but is not limited to this. For example, the parking assistance device 5 may be configured to provide the user with an image of the detection result of the search wave sensor at the intended parking position SEP.

[0138] The above-mentioned parking assistance device 5 displays not only the virtual parking image Gp but also the vehicle surroundings image Ga and the illustration image Gi on the touch panel display unit 46 before starting the following control process, but is not limited to this and may be configured to display, for example, only the virtual parking image Gp.

[0139] In the above embodiment, the vehicle surroundings image Ga is displayed in the left region of the touch panel display unit 46, and the overhead image Gh, the virtual parking image Gp, and the illustration image Gi are displayed in the right region of the touch panel display unit 46, but the layout of the images is not limited to this. The layout and image size of the images displayed on the touch panel display unit 46 may be different from those described above.

[0140] In the above embodiment, the HMI 45 has the touch panel display unit 46, but the HMI 45 is not limited to this. The HMI 45 may be configured to have a display operated by an operating device such as a remote control, instead of the touch panel display unit 46. The HMI 45 may be realized by using a part of a navigation system.

[0141] In addition, the touch panel display unit 46 also functions as the operation unit, but this is not limited thereto, and the operation unit and the display unit may be configured separately. The operation unit is not limited to touch operation, and may be configured to be operated by, for example, voice operation by the user.

[0142] In the above embodiment, the parking assistance device 5 is preferably configured to provide the user with an image obtained as information about the periphery of the planned parking position SEP, on which the virtual vehicle image Gv and the parking space image Gf are superimposed, but is not limited thereto. For example, the parking assistance device 5 may provide the user with an image obtained as information about the periphery of the planned parking position SEP itself, or an image obtained by superimposing one of the virtual vehicle image Gv and the parking space image Gf on the image. The virtual vehicle image Gv is not limited to an image showing the vehicle V in a semi-transparent manner, and may be an image showing the vehicle V in an opaque manner.

[0143] As in the above embodiment, the parking assistance device 5 is preferably configured to provide the user with a three-dimensional display of an image showing the periphery of the intended parking position SEP, while changing the viewpoint of the three-dimensional display in response to the user's operation of the operation unit, but is not limited thereto. For example, the parking assistance device 5 may be configured to provide the user with a two-dimensional display of an image showing the periphery of the intended parking position SEP.

[0144] As in the above embodiment, when the route information includes a plurality of candidate positions for the planned parking position SEP, the parking assistance device 5 preferably provides the user with information on the plurality of candidate positions included in the route information in a visual manner, but is not limited thereto. For example, when the route information includes a plurality of candidate positions for the planned parking position SEP, the parking assistance device 5 may automatically set one of the plurality of candidate positions as the planned parking position SEP based on a predetermined judgment criterion.

[0145] As in the above embodiment, the parking assistance device 5 is preferably configured to allow the user to visually grasp the progress of automatic parking by using a progress bar PB or the like, but is not limited to this. For example, the parking assistance device 5 may be configured to allow the user to audibly grasp the progress of automatic parking.

[0146] As in the above embodiment, the parking assistance device 5 is preferably configured to change the angle of the virtual viewpoint of the virtual viewpoint image according to the distance to the target, but is not limited to this. The parking assistance device 5 may have a constant angle of the virtual viewpoint image regardless of the distance to the target.

[0147] As in the above embodiment, when a situation occurs in which the vehicle cannot be parked at the intended parking position SEP after the start of automatic parking, the parking assistance device 5 desirably recommends the user to park at a possible parking position other than the intended parking position SEP, but is not limited to this. For example, when a situation occurs in which the vehicle cannot be parked at the intended parking position SEP, the parking assistance device 5 may inform the user of this, stop the vehicle V on the spot, and forcibly end the automatic parking.

[0148] As in the above embodiment, it is preferable that the parking assistance device 5 provides the user with a target route image Gt showing the target route TP superimposed on the surrounding image obtained during the execution of the following control process after the start of the following control process, but is not limited to this. For example, the parking assistance device 5 may display the surrounding image obtained during the execution of the following control process as it is after the start of the following control process.

[0149] As in the above embodiment, the parking assistant 5 is preferably configured to display the part behind the object and the part in front of the object in the target route image Gt in different manners, but is not limited thereto. For example, the parking assistant 5 may be configured to display the part behind the object and the part in front of the object in the same manner in the target route image Gt.

[0150] As in the above-described embodiment, it is desirable for the parking assistance device 5 to provide the user with an illustrated image Gi showing the relationship between the current position of the vehicle V, the target route TP, and the planned parking position SEP during automatic parking; however, this is not limited to this, and the parking assistance device 5 may not provide the illustrated image Gi.

[0151] As in the above embodiment, when an obstacle OB is present on the target route TP, the parking assistance device 5 is preferably configured to notify the user of the fact when searching for a route to avoid the obstacle OB, but is not limited to this and may be configured not to notify the user of anything.

[0152] As in the above embodiment, the parking assistance device 5 is preferably configured to search for a route to avoid the obstacle OB when the obstacle OB is on the target route TP, but is not limited to this. The parking assistance device 5 may be configured to, for example, urge the driver to stop the vehicle on the spot or request the driver to specify a parking position, without searching for a route to avoid the obstacle OB.

[0153] In the above embodiment, an example has been described in which the parking assistance device 5 of the present disclosure is applied to parking assistance in a parking lot PL having a plurality of parking spaces SP, but the application of the parking assistance device 5 is not limited to this. For example, the parking assistance device 5 can also be applied to parking assistance in a plot of land where one parking space SP is provided, such as in front of a house.

[0154] In the above-described embodiments, it goes without saying that the elements constituting the embodiments are not necessarily essential, except in cases where it is particularly expressly stated that they are essential or where they are clearly considered to be essential in principle.

[0155] In the above-described embodiments, when numerical values ​​such as the number, values, amounts, ranges, etc. of components of the embodiments are mentioned, they are not limited to the specific numbers unless it is specifically stated as required or unless it is clearly limited to a specific number in principle.

[0156] In the above-described embodiments, when referring to the shapes, positional relationships, etc. of components, etc., there is no limitation to those shapes, positional relationships, etc., unless specifically stated or in principle limited to a specific shape, positional relationship, etc.

[0157] The control unit and the method of the present disclosure may be implemented in a special-purpose computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. The control unit and the method of the present disclosure may be implemented in a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits. The control unit and the method of the present disclosure may be implemented in one or more special-purpose computers configured by a combination of a processor and a memory programmed to execute one or more functions and a processor configured with one or more hardware logic circuits. The computer program may also be stored in a computer-readable non-transitory tangible recording medium as instructions executed by a computer.

[0158] [Features of this disclosure] The present disclosure discloses the following technical features.

[0159] [Disclosure 1] A parking assistance device, a route generating unit (55) that generates a target route (TP) that the vehicle should take when parking the vehicle based on route information including a travel route of the vehicle when a user performs a parking operation of the vehicle and information about the periphery of the vehicle on the travel route; a following control unit (57) that performs a following control process to automatically move the vehicle along the target route to the intended parking position (SEP); an information providing unit (58) for providing information to the user, The information providing unit provides the information regarding the planned parking position, which is included in the route information, to the user in a visual form before the tracking control process is started.

[0160] [Disclosure 2] The parking assistance device described in Disclosure 1, wherein the information providing unit provides to the user, before the start of the following control process, an image obtained as information about the vicinity of the planned parking position from the route information, with a virtual vehicle image (Gv) showing the vehicle superimposed on the planned parking position shown in the image.

[0161] [Disclosure 3] The parking assistance device according to Disclosure 2, wherein the virtual vehicle image is an image showing the vehicle in a semi-transparent manner.

[0162] [Disclosure 4] The parking assistance device according to any one of Disclosures 1 to 3, wherein the information providing unit provides the user with an image obtained as information about the periphery of the planned parking position from the route information, superimposed with a parking space image (Gf) indicating the planned parking position, before the start of the following control process.

[0163] [Disclosure 5] The parking assistance device according to any one of Disclosures 1 to 4, wherein the information providing unit provides the user with a three-dimensional display of an image showing the surroundings of the intended parking position, and changes the viewpoint of the three-dimensional display in response to an operation signal from an operation unit (46) by the user.

[0164] [Disclosure 6] The parking assistance device according to any one of Disclosures 1 to 5, wherein, when the route information includes a plurality of candidate positions that are candidates for the planned parking position, the information providing unit provides the user with information regarding the plurality of candidate positions included in the route information in a visual manner, and provides information for prompting the user to select the planned parking position from among the plurality of candidate positions.

[0165] [Disclosure 7] When a situation occurs in which the vehicle cannot be parked at the planned parking position after the start of the following control process, the following control unit specifies a possible stopping position other than the planned parking position based on information about the surroundings of the vehicle obtained after the start of the following control process, The parking assistance device according to any one of Disclosures 1 to 6, wherein the information providing unit provides information to recommend stopping the vehicle at the possible stopping position when a situation occurs in which the vehicle cannot be parked at the intended parking position after the start of the following control process.

[0166] [Disclosure 8] The parking assistance device according to any one of Disclosures 1 to 7, wherein, after the start of the following control process, the information providing unit provides the user with a target route image (Gt) indicating the target route superimposed on a surrounding image showing the surroundings of the vehicle obtained during execution of the following control process.

[0167] [Disclosure 9] The information providing unit, in the target route image, A route in front of an object shown in the peripheral image is identified as a forward route on the target route, and a route behind the object is identified as a rearward route on the target route; The parking assistance device according to Disclosure 8, wherein a portion (Gtb) of the target route image corresponding to the rear route and a portion (Gtf) corresponding to the front route are superimposed on the peripheral image in different modes.

[0168] [Disclosure 10] The parking assistance device according to any one of Disclosures 1 to 9, wherein the information providing unit provides the user with an illustrated image (Gi) showing a relationship between the current position of the vehicle, the target route, and the planned parking position.

[0169] [Disclosure 11] when an obstacle is found on the target route, the following control unit attempts to generate an avoidance route that avoids the obstacle and reaches the planned parking position; The parking assistance device according to any one of Disclosures 1 to 9, wherein the information providing unit provides the user with information regarding the avoidance route in a visual manner when the avoidance route is generated by the following control unit.

[0170] [Disclosure 12] When the avoidance path cannot be generated, the following control unit identifies a stopping position where the vehicle can be stopped, The parking assistance device according to Disclosure 11, wherein the information providing unit provides the user with information regarding the stopping position and a route to the stopping position.

[0171] [Disclosure 13] The parking assistance device according to Disclosure 11 or 12, wherein, when the generation of the avoidance path by the following control unit starts, the information providing unit provides the user with information indicating that the generation of the avoidance path by the following control unit is in progress.

[0172] [Disclosure 14] the path generation unit, when an obstacle is found on the target path before the following control process is started, attempts to generate an object avoidance path that avoids the obstacle and reaches the planned parking position before the following control process is started, The parking assistance device according to any one of Disclosures 1 to 13, wherein, when the object avoidance path is generated by the path generation unit, the information providing unit provides information about the object avoidance path to the user in a visual form at a stage before the tracking control process is started.

[0173] [Disclosure 15] the following control unit, when the path generating unit is unable to generate the object avoidance path, identifies a possible stopping position at which the vehicle can be stopped, The parking assistance device according to Disclosure 14, wherein the information providing unit provides the user with information regarding the possible stopping position and a route to the possible stopping position.

[0174] [Disclosure 16] A parking assistance method, comprising: generating a target route (TP) that a vehicle (V) should take when parking the vehicle based on route information including a travel route of the vehicle and information about the periphery of the vehicle on the travel route when a parking operation of the vehicle (V) is performed by a user; performing a following control process for automatically moving the vehicle to a planned parking position (SEP) along the target route; providing information to said user; The parking assistance method, wherein providing information to the user includes providing information regarding the planned parking position included in the route information in a visual manner to the user before the following control process is started.

Claims

1. A parking assistance device, a route generating unit (55) that generates a target route (TP) that the vehicle should take when parking the vehicle based on route information including a travel route of the vehicle when a user has previously parked the vehicle (V) and information about the periphery of the vehicle on the travel route; a following control unit (57) that performs a following control process to automatically move the vehicle along the target route to a planned parking position (SEP); an information providing unit (58) for providing information to the user, The information providing unit is providing information regarding the planned parking position included in the route information to the user in a visual form before the start of the following control process; When the route information includes a plurality of candidate positions that are candidates for the planned parking position, information regarding the plurality of candidate positions included in the route information is provided to the user in a visual form, and information is provided to prompt the user to select the planned parking position from among the plurality of candidate positions; A parking assistance device that, when a plurality of candidate positions are located at distant locations, when the intended parking position is selected from the plurality of candidate positions, provides the user with an image focused on the selected candidate position.

2. 2. The parking assistance device according to claim 1, wherein the information providing unit provides the user with an image obtained as information about the vicinity of the planned parking position from the route information, the image having a virtual vehicle image (Gv) showing the vehicle at the planned parking position shown in the image superimposed thereon before the start of the following control process.

3. The parking assistance device according to claim 2 , wherein the virtual vehicle image is an image showing the vehicle in a semi-transparent manner.

4. 4. The parking assistance device according to claim 1, wherein the information providing unit provides the user with a parking space image (Gf) indicating the planned parking position superimposed on an image obtained as information about the vicinity of the planned parking position from the route information before the following control process is started.

5. 4. The parking assistance device according to claim 1, wherein the information providing unit provides the user with a three-dimensional display of an image showing the surroundings of the intended parking position, and changes the viewpoint of the three-dimensional display in response to an operation signal from an operation unit (46) by the user.

6. When a situation occurs in which the vehicle cannot be parked at the planned parking position after the start of the following control process, the following control unit specifies a possible stopping position other than the planned parking position based on information about the surroundings of the vehicle obtained after the start of the following control process, 4. The parking assistance device according to claim 1, wherein the information providing unit provides information to recommend stopping the vehicle at the possible stopping position when a situation occurs in which the vehicle cannot be parked at the intended parking position after the start of the following control process.

7. 4. The parking assistance device according to claim 1, wherein the information providing unit provides the user with a target route image (Gt) indicating the target route superimposed on a surrounding image showing the surroundings of the vehicle obtained during execution of the following control process after the following control process is started.

8. The information providing unit, in the target route image, A route in front of an object shown in the peripheral image is identified as a forward route on the target route, and a route behind the object is identified as a rearward route on the target route; The parking assistance device according to claim 7, wherein a portion (Gtb) of the target route image corresponding to the rear route and a portion (Gtf) of the target route image corresponding to the front route are superimposed on the surrounding image in different modes.

9. 4. The parking assistance device according to claim 1, wherein the information providing unit provides the user with an illustrated image (Gi) showing a relationship between the current position of the vehicle, the target route, and the planned parking position.

10. when an obstacle is found on the target route, the following control unit attempts to generate an avoidance route that avoids the obstacle and reaches the planned parking position; 4. The parking assistance device according to claim 1, wherein the information providing unit provides the user with information about the avoidance route in a visual form when the avoidance route is generated by the following control unit.

11. When the avoidance path cannot be generated, the following control unit identifies a stopping position where the vehicle can be stopped, The parking assistance device according to claim 10 , wherein the information providing unit provides the user with information regarding the stopping position and a route to the stopping position.

12. The parking assistance device according to claim 10 , wherein the information providing unit, when the generation of the avoidance path by the following control unit is started, provides the user with information indicating that the generation of the avoidance path by the following control unit is in progress.

13. the path generation unit, when an obstacle is found on the target path before the following control process is started, attempts to generate an object avoidance path that avoids the obstacle and reaches the planned parking position before the following control process is started, 4. The parking assistance device according to claim 1, wherein, when the object avoidance path is generated by the path generation unit, the information providing unit provides information about the object avoidance path to the user in a visual form at a stage before the tracking control process is started.

14. the following control unit, when the path generating unit is unable to generate the object avoidance path, identifies a possible stopping position at which the vehicle can be stopped, The parking assistance device according to claim 13 , wherein the information providing unit provides the user with information regarding the possible stop position and a route to the possible stop position.

15. A parking assistance method, comprising: generating a target route (TP) that the vehicle should take when parking the vehicle based on route information including a travel route of the vehicle when a user has previously parked the vehicle and information about the periphery of the vehicle on the travel route; performing a following control process for automatically moving the vehicle along the target route to a planned parking position (SEP); providing information to said user; By providing information to the user, providing information regarding the planned parking position included in the route information to the user in a visual form before the start of the following control process; When the route information includes a plurality of candidate positions that are candidates for the planned parking position, information regarding the plurality of candidate positions included in the route information is provided to the user in a visual form, and information is provided to prompt the user to select the planned parking position from among the plurality of candidate positions. and when the plurality of candidate positions are located at distant locations, when the intended parking position is selected from the plurality of candidate positions, providing the user with an image focused on the selected candidate position.

Images