Parking assistance device

Abstract

Parking assistance device, with: an information reference device (81, 82, 83) comprising at least an image acquisition device (83) configured to obtain image data of a vehicle's environments, wherein the information reference device is configured to obtain information about vehicle environments, including information about an object located in the vehicle's environments and information about a boundary line on a road surface located in the vehicle's environments; a path determination module (10, 10X) programmed to determine, based on information about the vehicle environments, a target area which is an area which the vehicle must occupy when parking the vehicle is complete, and to determine a path along which the vehicle is to move from a position of the vehicle at the present time to the target area as a target path; a parking assistance module (10, 10Y) programmed to perform a parking assistance control comprising an automatic steering angle control with respect to the vehicle to move the vehicle along the specified target path; and a display device (73, 51) configured to display an image to an occupant of the vehicle, wherein the parking assistance module is programmed to display on the display device an image (920, 1220, 1310, 1410, 1510, 1720) generated based on image data obtained by the image acquisition device, wherein the image is an image of the vehicle and its surroundings viewed from a position spaced from the vehicle in a direction different from a direction directly above the vehicle at the time when it is determined that the vehicle has reached the target area, characterized by the fact that the parking assistance module is programmed to: to set a multitude of viewing angle positions spaced from the vehicle in a multitude of directions different from the direction directly above the vehicle at a time when it is determined that the vehicle has reached the target area; to generate a multitude of images of the vehicle and its surroundings from the perspective of multiple viewpoint positions based on the image data obtained by the image acquisition device (83); and to display the multitude of images sequentially on the display device (73, 51).

Description

Background 1. Technical area

[0001] The present disclosure relates to a parking assist device configured to execute parking assist control for parking a vehicle in a predetermined space. 2. Description of the state of the art

[0002] Heretofore, a parking assist device has been proposed according to which a top view image, which is an image of a vehicle and a peripheral region of the vehicle as viewed from a position above the vehicle, is displayed on a screen of a display during execution of a parking assist control.

[0003] A prior art parking assist device (third embodiment of Japanese Patent Application Publication No. 2007-168560; hereinafter referred to as "prior art device") superimposes and displays on a screen a first top view image (reduced image), which is an image obtained by reducing a top view image by a preset reduction ratio, and a second top view image (enlarged image), which is an image obtained by enlarging the top view image by a preset magnification, and which includes a rear end portion of the vehicle and a boundary line in the vicinity of the rear end portion.

[0004] A driver may desire to confirm a final parking state of the vehicle (e.g., a positional relationship between the vehicle and a boundary line, an object, and the like present in the surroundings of the vehicle) when the parking assist control is completed (i.e., a time point at which the vehicle has reached a target area when the parking of the vehicle is completed). However, in the prior art device, visibility presented when the driver confirms the final parking state of the vehicle is not considered.

[0005] For example, similarly to the execution of parking assist control, a top-view image may also be displayed on the screen even after the parking assist control is completed. However, the top-view image is an image of the vehicle and a peripheral region of the vehicle viewed from a position directly above the vehicle, and therefore the following problems may occur. In particular, when the vehicle is parked in a parking area divided by dividing lines, part of a dividing line and / or part of an object (e.g., wheel stoppers) may be hidden below the vehicle body. Therefore, it may be difficult for the driver to accurately grasp the "positional relationship between the vehicle and a dividing line, an object, and the like" from only the top-view image.

[0006] Furthermore, DE 10 2015 215 918 A1 discloses a method and a device for determining a parking trajectory for a driver assistance system for maneuvering a vehicle into a target parking position. A perspective view of the vehicle's surroundings is displayed to a user on a display device, and a previously determined parking trajectory is output in the display of the vehicle's surroundings. The previously determined parking trajectory can be influenced by the user by means of at least one manual input, and an adapted parking trajectory is determined depending on the manual input. Furthermore, DE 10 2009 051 526 A1 discloses a system and a method for displaying vehicle surroundings with a configurable or adjustable viewing angle. The sensor device is configured to measure distances to objects in the vehicle's surroundings.Furthermore, the processor is configured to generate a three-dimensional environment map based on the measured distances of the sensor device. Further prior art is known from CN 107 176 101 A, DE 10 2006 040 605 A1, and DE 10 2011 017 233 A1. Summary

[0007] It is an object of the present disclosure to provide a parking assistance device capable of improving visibility presented when a driver confirms a final parking state of a vehicle. This object is achieved according to the invention by a parking assistance device according to claim 1. Further features and advantageous developments are shown in the subclaims.

[0008] A parking assistance device according to an embodiment (hereinafter occasionally referred to as a "device of an embodiment") comprises: an information acquisition device (81, 82, 83) having at least one image pickup device (83) configured to acquire image data of surroundings of a vehicle, wherein the information acquisition device is configured to acquire vehicle surroundings information comprising information about an object located in the surroundings of the vehicle and information about a boundary line on a road surface in the surroundings of a vehicle; a path determination module (10, 10X) programmed to determine, based on the vehicle surroundings information, a target area, which is an area that the vehicle is to occupy when the parking of the vehicle is completed, and to determine a path,along which the vehicle is movable from a position of the vehicle at the current time to the target area, as a target path; a parking assistance module (10, 10Y) programmed to execute parking assistance control comprising automatic steering angle control on the vehicle to move the vehicle along the determined target path; and a display device (73, 51) configured to display an image to an occupant of the vehicle. The parking assistance module is programmed to display, on the display device, a screen (920, 1220, 1310, 1410, 1510, 1720) generated based on the image data acquired by the image acquisition device, wherein the image is an image of the vehicle and surroundings of the vehicle as viewed from a position away from the vehicle in a direction different from a direction directly above the vehicle at a time when it is determined that the vehicle has reached the target area.The parking assistance module is further programmed to set a plurality of view angle positions spaced from the vehicle in a plurality of directions different from the direction directly above the vehicle at a time when it is determined that the vehicle has reached the target area, generate a plurality of images of the vehicle and a surrounding area of ​​the vehicle from the viewpoint of the plurality of view angle positions based on the image data acquired by the image pickup device (83), and sequentially display the plurality of images on the display device (73, 51).

[0009] The device of an embodiment having such a configuration displays, on the display device, an image of the vehicle and a peripheral region of the vehicle as viewed from a position different from the position directly above the vehicle at the time when it is determined that the vehicle has reached a target area. As a result, the driver can confirm the position of, for example, a boundary line and / or an object that are difficult to detect in an image viewed from above by viewing the above-mentioned image at and after the time when the parking assist control is completed (that is, at and after the time when it is determined that the vehicle has reached the target area). Therefore, the driver can more accurately grasp the "positional relationship between the vehicle and a boundary line, an object, and the like that are present in the surroundings of the vehicle."

[0010] According to one aspect of the device of an embodiment, the display device is configured to display an image comprising a first display area (301) and a second display area (302). Further, the parking assistance module is programmed to: sequentially display the plurality of images in the second display area; based on the image data acquired by the image acquisition device, acquire a first viewpoint image (810, 910, 1110, 1210, 1710) of the vehicle and surroundings of the vehicle as viewed from a position away from the vehicle in a direction directly above the vehicle at the time when it is determined that the vehicle has reached the target area; and display the first viewpoint image in the first display area.

[0011] The parking assistance module according to this aspect displays the first view angle image in the first display area of ​​the screen and displays the second view angle image in the second display area of ​​the screen. The driver can confirm the final parking state of the own vehicle by viewing a plurality of view angle images (first view angle image and second view angle image) of the own vehicle and the peripheral position of the own vehicle from two different view angle positions. Therefore, the driver can more accurately grasp the "positional relationship between the own vehicle and a boundary line, an object, and the like."

[0012] According to one aspect of the apparatus of an embodiment, the parking assist module is programmed to: generate the first view angle image during a period from a time at which the parking assist control starts to a time at which it is determined that the vehicle has reached the target area, and display the first view angle image in the first display area at a first display magnification that enables the entire vehicle around at least a part of the target area to be displayed, and to, at and after a time at which it is determined that the vehicle has reached the target area, display the first view angle image in the first display area at a second display magnification that enables the entire vehicle and at least a part of a peripheral region of the vehicle to be displayed, and which is larger than the first display magnification.

[0013] The parking assist module according to this aspect displays the first view angle image in the first display area at the second display magnification that is larger than the first display magnification used during the execution of the parking assist control, at and after the time point when it is determined that the vehicle has reached the target area. The entire vehicle and a peripheral region of the vehicle are displayed in an enlarged manner in the first display area at and after the time point when it is determined that the vehicle has reached the target area, and therefore, the visibility presented when the driver performs final confirmation of the parking state of the vehicle is further improved.

[0014] The parking assistance module according to the invention sequentially displays, in the second display area, a plurality of viewpoint images viewed from the plurality of viewpoint positions other than the viewpoint position away from the vehicle in the direction directly above the vehicle. The driver can confirm the final parking state of the vehicle by viewing the sequential display of the plurality of viewpoint images. Therefore, the driver can easily and accurately grasp the positional relationship between the entire vehicle and a boundary line of another vehicle, etc., located in the surroundings of the vehicle.

[0015] Further features related to the present disclosure will become apparent from the description and the accompanying drawings. Problems, configurations, and effects different from those described above will become apparent from the following description of embodiments of the present disclosure.

[0016] In the above description, to facilitate understanding of the present disclosure, a name and / or reference symbol used in the later-described embodiments of the present disclosure have been assigned to each of the components according to the embodiments. However, none of the components is limited to the embodiments defined by the name and / or reference symbols. Short description of the drawings Fig. 1 is a schematic configuration diagram of a parking assist device according to an embodiment. Fig. 2 is a plan view of a vehicle illustrating an arrangement of first ultrasonic sensors, second ultrasonic sensors, and cameras. Fig. 3 is a diagram illustrating a state in which a screen (display screen) of a Fig. 1 illustrated touch panels were divided into three regions. Fig. 4 is a flowchart illustrating a “perpendicular parking assist start routine” to be executed by a CPU of a parking assist ECU in the embodiment. Fig. 5 is a flowchart illustrating a “parking assist control execution routine” to be executed by the CPU of the parking assist ECU in the embodiment. Fig. 6 is a flowchart illustrating a “parking assist control end routine” to be executed by the CPU of the parking assist ECU in the embodiment. Fig. 7 is a plan view illustrating a state in which the parking assist ECU of the embodiment has calculated a target path. Fig. 8 is a diagram illustrating an image displayed on the touch panel screen during execution of the perpendicular parking assist control. Fig. 9 is a diagram illustrating an image displayed on the touch panel screen at and after completion of a perpendicular parking assist control. Fig. 10 is a flowchart illustrating a “parallel parking assist start routine” to be executed by the CPU of the parking assist ECU in the embodiment. Fig. 11 is a diagram illustrating an image displayed on the touch panel screen during execution of a parallel parking assist control. Fig. 12 is a diagram illustrating an image displayed on the touch panel screen at and after completion of parallel parking assist control. Fig. 13 is a diagram for illustrating a modification example of an image displayed on the touch panel screen at and after completion of a perpendicular parking assist control. Fig. 14 is a diagram for illustrating a modification example of an image displayed on the touch panel screen at and after completion of a perpendicular parking assist control. Fig. 15 is a diagram illustrating a modification example of an image displayed on the touch panel screen at and after completion of a perpendicular parking assist control. Fig. 16 is a diagram for illustrating a modification example of an image displayed on the touch panel screen at and after completion of a perpendicular parking assist control. Fig. 17 is a diagram for illustrating a modification example of an image displayed on the touch panel screen at and after completion of a perpendicular parking assist control. Description of the embodiments

[0017] A description of one or more embodiments of the present disclosure will now be provided with reference to the accompanying drawings. The accompanying drawings illustrate specific embodiments; however, these illustrations are examples used to understand the embodiments and are not intended to limit the interpretation of the present disclosure.

[0018] A parking assist device according to an embodiment (hereinafter occasionally referred to as "device of this embodiment") is applied to a vehicle. Hereinafter, a vehicle equipped with the parking assist device may be referred to as "own vehicle" to distinguish the vehicle from other vehicles.

[0019] As in Fig. As illustrated in Figure 1, the parking assist device includes a parking assist ECU 10. The parking assist ECU 10 includes a microcomputer with a central processing unit (CPU) 10a, a random access memory (RAM) 10b, a read-only memory (ROM) 10c, an interface (I / F) 10d, and other components. ECU stands for "electronic control unit." The ECU includes a microcomputer with a CPU, a RAM, a ROM, an interface, and other components. The CPU is configured to execute instructions stored in the ROM to implement various functions.

[0020] The vehicle further includes an engine ECU 20, a brake ECU 30, an electric power steering ECU (hereinafter referred to as "EPS ECU") 40, a meter ECU 50, a shift-by-wire (SBW) ECU 60, and a navigation ECU 70. The parking assist ECU 10 and those other ECUs are interconnected such that information can be sent and received to and from each other via a control unit network (CAN) 90. A detection signal from a sensor connected to a specific ECU is also sent to the other ECUs.

[0021] The engine ECU 20 is connected to an engine actuator 21. The engine actuator 21 includes a throttle valve actuator configured to change the opening degree of a throttle valve of the internal combustion engine 22. The engine ECU 20 is capable of changing the torque to be generated by the internal combustion engine 22 by driving the engine actuator 21. Therefore, the engine ECU 20 is capable of controlling the driving force of the vehicle by controlling the engine actuator 21. When the vehicle is a hybrid vehicle, the engine ECU 20 is capable of controlling the driving force of the vehicle generated by one or both of an internal combustion engine and / or a motor serving as driving force sources. Further, when the vehicle is an electric vehicle, the engine ECU 20 is capable of controlling the driving force of the vehicle generated by a motor serving as a driving force source.

[0022] The brake ECU 30 is connected to a brake actuator 31. A braking force (braking torque) to be applied to the wheels of the vehicle is controlled via the brake actuator 31. The brake actuator 31 adjusts a hydraulic pressure of a fluid to be supplied to the wheel cylinders integrated with the brake calipers 32b according to an instruction from the brake ECU to use the hydraulic pressure to press brake pads against brake discs 33a, thereby generating frictional braking forces. Therefore, the brake ECU 30 is capable of controlling a braking force of the vehicle by controlling the brake actuator 31.

[0023] The EPS-ECU 40 is connected to an auxiliary motor (M) 41. The auxiliary motor 41 is integrated into a "steering mechanism" comprising a steering wheel, a steering shaft coupled to the steering wheel, and a gear mechanism for steering (not shown) of the vehicle. The EPS-ECU 40 uses a steering torque sensor (not shown) provided in the steering shaft to detect a steering torque input to the steering wheel by the driver, thereby driving the auxiliary motor 41 based on the steering torque. The EPS-ECU 40 applies steering torque (steering assist torque) to the steering mechanism via the drive of the auxiliary motor 41, thereby being able to assist the driver's steering operation. The meter ECU 50 is connected to a display 41 and a vehicle speed sensor 52. The display 51 is a multi-information display provided in front of a driver's seat.The display 51 displays various types of information in addition to measured values ​​such as vehicle speed and engine speed. A head-up display can be used as the display 51. The vehicle speed sensor 52 detects the speed of the vehicle (vehicle speed) and outputs a signal indicating the vehicle speed to the meter ECU 50. The vehicle speed detected by the vehicle speed sensor 52 is also transmitted to the parking assist ECU 10.

[0024] The SPW-ECU 60 is connected to a shift position sensor 61. The shift position sensor 61 detects a position of a shift lever serving as a movable portion of a shift operating portion. In this example, positions of the shift lever include a park position (P), a drive position (D), and a reverse position (R). The SPW-ECU 60 is configured to receive the position of the shift lever from the shift position sensor 61 to control a transmission and / or a direction-changing mechanism (not shown) of the vehicle based on the shift lever position (i.e., performs shift control of the vehicle). Specifically, when the position of the shift lever is "P," the SPW-ECU 60 controls the transmission and / or the direction-changing mechanism so that the driving force is not transmitted to the drive wheels, and the vehicle is therefore mechanically locked to a stop position.When the shift lever position is "D," the SPW-ECU 60 controls the transmission and / or the direction-changing mechanism so that the driving force for moving the vehicle forward is transmitted to the drive wheels. Furthermore, when the shift lever position is "R," the SPW-ECU 60 controls the transmission and / or the direction-changing mechanism so that the driving force for moving the vehicle rearward is transmitted to the drive wheels. The SPW-ECU 60 is configured to output to the steering assist ECU 10 a signal received from the shift position sensor 61 indicating the position of the shift lever.

[0025] The navigation ECU 70 includes a GPS receiver 71 for receiving a GPS signal to acquire the longitude and latitude of the vehicle's current position, a map database 72 that stores map information, and a touch panel (a touch-sensitive display) 73. The navigation ECU 70 performs various arithmetic processing based on the longitude and latitude of the vehicle's current position, the map information, and the like, and displays the vehicle's position on the map on the touch panel 73. The display mode at the time the map and the vehicle's position on the map are displayed on the touch panel 73 is hereinafter referred to as the "navigation mode." The touch panel 73 is a touch-sensitive display capable of displaying maps, images, and the like.Therefore, for simplicity, the touch panel 73 is also referred to as a “display device” or “display”.

[0026] The display mode of the touch panel 73 includes, in addition to the navigation mode, a parking assist mode. The parking assist mode is a display mode when parking assist control for parking is performed. A home button (not shown) is arranged near the touch panel 73. In a case where the display mode is the parking assist mode, the display mode switches to the navigation mode when the home button is pressed. In a case where the display mode is the navigation mode, the display mode switches to the parking assist mode when the home button is pressed.

[0027] The parking assist ECU 10 is connected to a plurality of first ultrasonic sensors 81a to 81d, a plurality of second ultrasonic sensors 82a to 82h, a plurality of cameras 83a to 83d, a parking assist switch 84, and a speaker 85. The plurality of first ultrasonic sensors 81a to 81d are collectively referred to as "first ultrasonic sensors 81." The plurality of second ultrasonic sensors 82a to 82d are collectively referred to as "second ultrasonic sensors 82." The plurality of cameras 83a to 83d are collectively referred to as "cameras 83."

[0028] Each of the first ultrasonic sensors 81 and the second ultrasonic sensors 82 (hereinafter collectively referred to as an "ultrasonic sensor" if it is not necessary to distinguish between the sensors) transmits ultrasonic waves in a pulsed manner within a predetermined range and receives reflected waves reflected by an object. The ultrasonic sensor is capable of detecting a distance (reflection point distance) between the ultrasonic sensor and a "reflection point, which is a point on the object from which the transmitted ultrasonic waves were reflected," based on the time from the transmission of the ultrasonic waves to the reception thereof.

[0029] The first ultrasonic sensors 81 are used to detect objects located relatively far away from the vehicle, as compared with the second ultrasonic sensors 82. As shown in Fig. As illustrated in Fig. 2, the first ultrasonic sensor 81a is arranged at a position on the right side of a front portion of a vehicle body 200 (e.g., a right end portion of a front bumper 201), and detects the reflection point distance of an object on the right side in a front portion of the vehicle. The first ultrasonic sensor 81b is arranged at a position on the left side of the front portion of the vehicle body 200 (e.g., a left-side end portion of the front bumper 201), and detects the reflection point distance of an object on the left side in the front portion of the vehicle. The first ultrasonic sensor 81c is arranged at a position on the right side at a corner portion of the vehicle body 200 (e.g., a right-side end portion of a rear bumper 202), and detects the reflection point distance of an object on the right side at a rear portion of the vehicle.The first ultrasonic sensor 81d is arranged at a position on the left side of the rear bumper of the vehicle body 200 (e.g., a left-side end portion of the rear bumper 202), and detects the reflection point distance of an object on the left side at the rear portion of the vehicle.

[0030] The second ultrasonic sensors 82 are used to detect objects located relatively close to the vehicle. As shown in Fig. As illustrated in Figure 2, four second ultrasonic sensors 82a to 82d are arranged on the front bumper 201 at intervals in a vehicle width direction. The second ultrasonic sensors 82a to 82d detect the reflection point distance of objects in front of the vehicle. Four second ultrasonic sensors 82e to 82h are arranged on the rear bumper 202 at intervals in the vehicle width direction. The second ultrasonic sensors 82e to 82h detect the reflection point distance of objects behind the vehicle.

[0031] Each of the cameras 83 is, for example, a digital camera having an image pickup element of a charge-coupled device (CCD) or a CMOS image sensor (CIS). The cameras 83 output image data at a predetermined frame rate (i.e., every time a predetermined time elapses). An optical axis of each camera is set obliquely downward from the vehicle body of the vehicle. Therefore, the cameras 83 are configured to photograph a peripheral state (including the position, shape, etc., boundary lines, objects, parking areas, etc.) of the vehicle to be confirmed when the vehicle is parked, and output image data to the parking assist ECU 10.

[0032] As in Fig. As illustrated in Fig. 2, the camera 83a is arranged at a substantially central portion of the front bumper 201 in the vehicle width direction, and photographs the area in front of the vehicle. The camera 83b is arranged at a wall portion of a rear trunk 203 at the rear portion of the vehicle body 200, and photographs the area behind the vehicle. The camera 83c is arranged at a right-side door mirror 204, and photographs the area on the right side of the vehicle. The camera 83d is arranged at a left-side door mirror 205, and photographs the area to the left of the vehicle. Hereinafter, items of image data obtained by the cameras 83a, 83b, 83c, and 83d photographed may be referred to as "front image data," "rear image data," "right-side image data," and "left-side image data," respectively.

[0033] The parking assist ECU 10 receives a detection signal from each of the first ultrasonic sensors 81 and the second ultrasonic sensors 82 every time a predetermined period of time (hereinafter also referred to as "first predetermined time" for convenience) elapses. The parking assist ECU 10 records information (i.e., reflection points and reflection point distances) included in the detection signals on a two-dimensional map. This two-dimensional map is a plan view in which a vehicle position is set as the origin, a traveling direction of the vehicle is set as an X-axis, and a leftward direction of the vehicle is set as a Y-axis. The position of the vehicle is the central position in a plan view of a left front wheel and a right front wheel. The position of the vehicle may also be another specific position on the vehicle (e.g.,a central position of a left rear wheel and a right rear wheel in plan view, a position of the center of gravity of the vehicle in plan view, or a geometric center position of the vehicle in plan view.

[0034] Each time the first predetermined time elapses, the parking assist ECU 10 acquires image data from each of the cameras 83. The parking assist ECU 10 detects an object in the surroundings of the vehicle by analyzing the image data from each camera 83, and identifies the position (distance and direction) and shape of the object relative to the vehicle. The parking assist ECU 10 also acquires, in the image data from each of the cameras 83, boundary lines (including boundary lines defining lanes and boundary lines defining parking areas) drawn on the road surface in the surroundings of the vehicle, and identifies the position (distance and direction) and shape of the boundary lines relative to the vehicle. The parking assist ECU 10 plots the objects and boundary lines specified (acquired) based on the image data on the above-mentioned two-dimensional map.

[0035] The parking assist ECU 10 detects objects located in the surroundings of the vehicle (within a predetermined distance range from the vehicle's position) based on the information displayed on the two-dimensional map, and also detects an "object-free area" in the surroundings of the vehicle. If the object-free area is an area with a size and shape that allows the vehicle to park with a surplus, the parking assist ECU 10 determines that this area is a "candidate area." The candidate area has, for example, a rectangular shape that is not crossed by detected boundary lines, with a long side that is longer than the length in the longitudinal direction of the vehicle by a first margin, and a short side that is longer than the length in the transverse direction of the vehicle by a second margin.

[0036] The "first ultrasonic sensors 81, second ultrasonic sensors 82, and cameras 83" are collectively referred to as vehicle peripheral sensors (or information acquisition devices). The cameras 83 may also be referred to as an image acquisition device configured to acquire image data of the surroundings of the vehicle. The "information (e.g., position and shape) of an object located in the surroundings of the vehicle and information (e.g., position and shape) of a boundary line located on the road surface in the surroundings of the vehicle" obtained based on the signals from the vehicle peripheral sensors are also referred to as "vehicle surroundings information."

[0037] The parking assist switch 84 is a switch to be operated (pressed / depressed) when the driver issues a parking assist instruction to the parking assist ECU 10 (when a parking assist instruction described later is issued). Parking assist control is a well-known control for assisting a driver's driving operation when the driver parks the vehicle. Parking assist control is also referred to as an "intelligent parking assist (IPA)."

[0038] The speaker 85 generates a sound when an instruction to output a sound is received from the parking assist ECU 10. (Details of the parking assistance control)

[0039] Each time the parking assist switch 84 is pressed, the parking assist ECU 10 sequentially switches the switching mode from, in order, a perpendicular parking mode, a parallel parking mode, and an unset mode. Therefore, for example, if the parking assist switch 84 is pressed once when the switching mode is in the unset mode, the switching mode is changed to the perpendicular parking mode, and if the parking assist switch 84 is pressed twice when the switching mode is in the unset mode, the switching mode is changed to the parallel parking mode. If the parking assist switch 84 is pressed twice when the switching mode is in the perpendicular parking mode, the switching mode is changed to the unset mode.The parking assist switch 84 may be a rotary type switch, and in this case, the switching mode is switched to the perpendicular parking mode, the parallel parking mode, and the unadjusted mode according to a position to which the parking assist switch 84 is rotated.

[0040] Perpendicular parking mode is a mode in which parking assistance is performed when the host vehicle is parked in a direction perpendicular to the traveling direction of a road along which driving is performed. Perpendicular parking refers to moving the host vehicle to park the host vehicle parallel to other parked vehicles. Specifically, perpendicular parking refers to parking the host vehicle such that one side of the host vehicle faces one side of another vehicle (first other vehicle), and the other side of the other vehicle faces one side of another vehicle (second other vehicle), and a longitudinal axis passing through the center of the host vehicle in the vehicle width direction and a longitudinal axis passing through the center of each of the first and second other vehicles in the vehicle width direction are parallel to each other.Perpendicular parking involves parking the vehicle so that the vehicle faces the roadway along which it is traveling in a direction at right angles to the direction of travel, with at least one of the left or right sides of the vehicle parallel to "a white line, a wall, a fence, a guardrail, or the like."

[0041] Parallel parking mode is the mode in which parking assistance is performed when the vehicle is parked in a direction parallel to the direction of travel on the road. Parallel parking is synonymous with parking the vehicle in line with other vehicles parked along the road.

[0042] Specifically, in the parallel parking, the own vehicle is parked such that the front end portion of the own vehicle faces the rear end portion (or front end portion) of the first other vehicle, and the rear end portion of the own vehicle faces the front end portion (or rear end portion) of the second other vehicle, and the longitudinal direction axis passing through the center of the own vehicle in the vehicle width direction and the longitudinal direction axis passing through the center of each of the first and second other vehicles in the vehicle width direction are substantially on the same line.

[0043] As described later, the parking assist ECU 10 monitors operations related to the parking assist switch 84, the position of the shift lever, and the state of the host vehicle to determine whether or not a parking assist request has been issued. Examples of the parking assist request include a perpendicular parking assist request and a parallel parking assist request. < <querparkassistenzanforderung>>

[0044] The parking assist ECU 10 determines that a perpendicular parking assist request has been issued when all of the conditions described below are met. (Condition A1) Neither a perpendicular parking assistance request nor a parallel parking assistance request was issued. (Condition A2) The perpendicular parking mode has been selected by a predetermined operation on the parking assist switch 84 (e.g., pressed once). (Condition A3) The position of the shift lever at the time when condition A2 is satisfied is the drive position (D). (Condition A4) At the time when condition A2 is met, the driver has pressed a brake pedal to stop the own vehicle (ie the speed is 0 [km / h]). (Condition A5) An area candidate (perpendicular parking area candidate) located adjacent to the traveling road, having a shortest distance from the position of the own vehicle less than or equal to a predetermined distance, and having a size and shape that allows parking of the own vehicle via the perpendicular parking mode, has been detected.

[0045] Condition A4 may be a condition that the vehicle speed is less than or equal to a predetermined lower speed threshold (e.g. 30 [km / h]). < <parallelparkassistenzanforderung>>

[0046] The parking assist ECU 10 determines that a parallel parking assist request has been issued when all of the conditions described below are met. (Condition B1) Neither a perpendicular parking assistance request nor a parallel parking request was issued. (Condition B2) The parallel parking mode has been selected via a predetermined operation to the parking assist switch 84 (e.g., pressed twice consecutively). (Condition B3) The position of the shift lever at the time when condition B2 is satisfied is the drive position (D). (Condition B4) At the time when condition B2 is met, the driver has depressed the brake pedal to stop the own vehicle (ie the vehicle speed is 0 [km / h]). (Condition B5) An area candidate (parallel parking area candidate) located adjacent to the traveling road, having a shortest distance from the position of the own vehicle less than or equal to a predetermined distance, and having a size and shape that allows parking via the parallel parking mode of the own vehicle, has been detected.

[0047] Condition B4 may be a condition that the vehicle speed is less than or equal to a predetermined lower speed threshold (e.g., 30 [km / h]).

[0048] When a perpendicular parking assist request has been issued, the parking assist ECU 10 executes parking assist control for causing the own vehicle to park itself in a predetermined area (target area described later) within the predetermined parking area candidate.

[0049] When a parallel parking assist request is issued, the parking assist ECU 10 executes parking assist control for causing the host vehicle to park in a predetermined area (target area) within the parallel parking area candidate.

[0050] When it is determined that the above-mentioned parking assistance request has been issued, the parking assistance ECU 10 sets the position of the own vehicle at the time when the own vehicle is assumed to be parked in the area candidate (e.g., a center position of a plan view of the left front wheel and the right front wheel of the own vehicle) as the target position. The parking assistance ECU 10 determines / sets, as a target path, a path along which the position of the own vehicle is to move from the current position of the own vehicle (current position) to the target position. The parking assistance ECU 10 executes parking assistance control such that the vehicle moves along the target path.

[0051] The parking assist ECU 10 is configured to perform shift control, automatic steering angle control, driving force control, and braking force control as parking assist control. Therefore, once the target path has been determined, the parking assist ECU 10 determines the direction for moving the host vehicle (specifically, the position of the shift lever), a steering angle pattern, and a speed pattern for moving the host vehicle along the target path.

[0052] The parking assist ECU 10 sends a shift control instruction to the SBW-ECU 60 via the CAN 90 according to the determined position of the shift lever. Upon receiving a shift control instruction from the parking assist ECU 10, the SBW-ECU 60 changes the position of the shift lever to the position specified by the shift control instruction (i.e., executes a shift control).

[0053] The steering angle pattern includes data that links the position of the host vehicle on the target path and the steering angle. The parking assist ECU 10 sends a steering command (including a target steering angle) to the EPS ECU 40 via the CAN 90 according to the determined steering angle pattern. Upon receiving the steering command from the parking assist ECU 10, the EPS ECU 40 controls the operating motor 41 based on the steering torque specified by the steering command to make the assist steering angle consistent with the target steering angle (i.e., executes driving force control).

[0054] The speed pattern includes data linking the position of the host vehicle on the target path and a traveling speed of the host vehicle. The speed pattern represents changes in the traveling speed that occur when the vehicle travels along the target path. The parking assist ECU 10 sends a driving force control command to the engine ECU 20 via the CAN 90 according to the predetermined speed pattern. Upon receiving the driving force control command from the parking assist ECU 10, the engine ECU 20 controls the engine actuator 21 according to the driving force control command (i.e., executes driving force control). The parking assist ECU 10 also sends a braking force control command to the braking ECU 30 via the CAN 90 according to the predetermined speed pattern.When the braking force control instruction is received from the parking assist ECU 10, the brake ECU 30 controls the brake actuator 31 according to the braking force control instruction (ie, executes braking force control).

[0055] As described above, the parking assist ECU 10 has functions as a "path determination module 10X" programmed to determine a target path and a "parking assist module 10Y programmed to execute parking assist control", which are implemented by the CPU 10a. (Screen display)

[0056] Next, the screen of the touch panel 73 (hereinafter simply referred to as "screen") at the time when the display mode is the parking assist mode will be described. As shown in Fig. 3, the screen includes a first display area 301, a second display area 302, and a third display area 303. The first display area 301 is a left side area obtained when the screen is divided into two areas of the left and right areas, and has a vertically elongated rectangular shape. The second display area 302 is a part of the right side area obtained when the screen is divided into two areas of the left and right areas as described above, and is an upper side area obtained when the right side area is divided into two areas of an upper and a lower area. The second display area 302 has a horizontally elongated rectangular shape.The third display area 303 is a lower-side area obtained by dividing the above-mentioned right-side area into two areas, an upper area and a lower area, and has a horizontally elongated rectangular shape. The size of each of the first display area 301 to the third display area 303 is a fixed predetermined size. The first display area 301 to the third display area 303 are divided such that the area of ​​the second display area 302 is the largest and the area of ​​the third display area 303 is the smallest. (Image generation)

[0057] When the display mode is the parking assist mode, the parking assist ECU 10 displays a "traveling direction image and view angle image" described below on the screen. A method of generating each of the traveling direction image and the view angle image will be briefly described below.

[0058] The parking assist ECU 10 generates a traveling direction image indicating the area of ​​the traveling direction of the host vehicle based on the front image data and the rear image data. While the host vehicle is traveling backward (i.e., the position of the shift lever detected by the shift position sensor 61 is "R"), the parking assist ECU 10 generates a traveling direction image indicating the area behind the host vehicle based on the rear image data. Meanwhile, when the host vehicle is traveling in the forward direction (i.e., when the position of the shift lever detected by the shift position sensor 61 is "D"), the parking assist ECU 10 generates a traveling direction image indicating the area in front of the host vehicle based on the front image data.

[0059] The parking assist ECU 10 also generates an image of the own vehicle and the peripheral region of the own vehicle from a set view angle position (hereinafter referred to as a "view angle image") based on the front image data, the rear image data, the right-side image data, and the left-side image data. A method of generating such a view angle image is known (see, for example, Japanese Patent Application Publication No. 2012-217000 and Japanese Patent Application Publication No. 2013-021468). Therefore, an example of the method of generating a view angle image will be briefly described below.

[0060] The parking assist ECU 10 generates three-dimensional data of the peripheral region of the own vehicle by combining at least the "image data acquired from each of the cameras 83 at each time point when the first predetermined time has elapsed during the period from the start time point of the parking assist control to the completion time point thereof." The three-dimensional data includes data on "boundary lines and objects (e.g., wheel stops)" and the like located below the own vehicle at the time point when the own vehicle reaches the target position. In other words, the generated three-dimensional data includes data on "objects and / or boundary lines" or the like that are not included in the images acquired by each of the cameras 83 at the time point when the own vehicle reaches the target position.

[0061] The parking assistance ECU 10 projects the three-dimensional data generated in the above-described manner onto a three-dimensional curved surface in a virtual three-dimensional space covering the host vehicle. The three-dimensional curved surface has, for example, a substantially hemispherical shape. The relationship between the coordinates of the three-dimensional data and the coordinates of the three-dimensional curved surface is defined in advance. The host vehicle is located at the center (lower part of the hemisphere) of the three-dimensional curved surface. The host vehicle in the three-dimensional curved surface is generated, for example, as a polygon model representing a three-dimensional shape based on data, the shape and size of the vehicle body being stored in advance in the ROM 10c.

[0062] The parking assistance ECU 10 sets virtual view angles (first virtual view angle and second virtual view angle, as described later) in the virtual three-dimensional space. Each of the virtual view angles is defined by a view angle position and a viewing direction. The parking assistance ECU 10 extracts images from the virtual three-dimensional space based on the set virtual view angles.

[0063] Specifically, the parking assist ECU 10 generates a "first view angle image, which is an image of the own vehicle and the peripheral region of the own vehicle based on a first virtual view angle," during the period from the time the parking assist control starts until before the own vehicle reaches the target area. When it is determined that the own vehicle has reached the target area, the parking assist ECU 10 further generates the "first view angle image, which is an image of the own vehicle and the peripheral region of the own vehicle based on the first virtual view angle" and a "second view angle image, which is an image of the own vehicle and the peripheral region of the own vehicle based on a second virtual view angle different from the first virtual view angle."

[0064] The viewing angle position of the first virtual viewing angle (hereinafter, occasionally referred to as the "first viewing angle position") is a position spaced from the center position in the plan view of the vehicle body of the own vehicle by a first predetermined distance in a direction directly upward. The viewing direction of the first virtual viewing direction is a direction directly downward from the first viewing angle position toward the own vehicle. Therefore, the first viewing angle image is an image looking down at the own vehicle from a position directly above the own vehicle. Such a viewing angle image is also referred to as a "top-view image."

[0065] The viewing angle position of the second virtual viewpoint (hereinafter occasionally referred to as the "second viewing angle position") is a position spaced from the host vehicle in a direction different from the direction directly above the host vehicle. In this example, the viewing angle position of the second virtual viewpoint is a position spaced from the center position in the vehicle width direction of the front end portion of the vehicle body of the host vehicle by a predetermined second distance in front of the host vehicle. The viewing direction of the second virtual viewpoint is the direction from the second viewing angle position toward the host vehicle. Therefore, the second viewing angle image is an image as if the host vehicle is viewed from the front. Such a viewing angle image is also referred to as a "front image."

[0066] As described above, the parking assist ECU 10 is capable of generating view angle images to show how the own vehicle and the peripheral region of the own vehicle look from the set virtual view angles. (Overview of the operation)

[0067] There may be cases where the driver wishes to confirm the final parking state of the own vehicle (e.g., the positional relationship between the own vehicle and a boundary line, an object, and the like located in the surroundings of the own vehicle) even after the parking assist control is completed (i.e., a time when the own vehicle reaches the target area). However, when the own vehicle is parked in a parking area delimited by boundary lines, part of the boundary lines and / or part of the object (e.g., wheel stops) below the vehicle body may be obscured.Under such circumstances, it is difficult for the driver to accurately recognize the "positional relationship between the own vehicle and a boundary line, an object, and the like" based only on the top view image, which is an image of the vehicle and the peripheral region of the own vehicle viewed from a position directly above the own vehicle.

[0068] In view of this, the device of this embodiment displays on the screen an image (second view angle image) of the own vehicle and the surroundings of the own vehicle as viewed from a position (second view angle position) that is spaced from the own vehicle in a direction different from the direction directly above the own vehicle at the time when it is determined that the vehicle has reached the target area. In the example described below, the second view angle image is a front image. The device of this embodiment can also display on the screen box for changing the position of the second virtual view angle. The driver can change the position of the second virtual view angle by operating the buttons on the screen.According to the above configuration, the driver can confirm the position of boundary lines and / or objects, etc., that are difficult to detect in an image viewed from above (first viewpoint image) by viewing the second viewpoint image. Therefore, the driver can more accurately grasp the positional relationship between the vehicle and a boundary line, an object, etc., located in the surroundings of the vehicle.

[0069] The device of this embodiment also displays a plurality of viewpoint images on the screen at and after the time when it is determined that the vehicle has reached the target position. Specifically, the device of this embodiment displays, in the first display area 301, a first viewpoint image (top view image) that is an image of the own vehicle and the peripheral region of the own vehicle based on the aforementioned first virtual viewpoint. The device of this embodiment also displays, in the second display area 302, an image (second viewpoint image) of the own vehicle and the peripheral region of the own vehicle based on the second virtual viewpoint different from the first virtual viewpoint.According to this configuration, the driver can confirm the final parking status of the own vehicle based on a variety of different viewing angles. Therefore, the driver can more accurately grasp the positional relationship between the own vehicle, a boundary line, an object, and the like.

[0070] The device of this embodiment also generates a first viewpoint image (top view image) during the period from the time the parking assist control starts until before the host vehicle reaches the target area, and displays the first viewpoint image in the first display area 301 with a first display delay that allows the entire host vehicle and at least a part of the target region to be displayed. The device of this embodiment also displays the first viewpoint image (top view image) in the first display area 301 at and after the time it is determined that the vehicle has reached the target area, at a second display magnification that allows the entire host vehicle and at least a part of a peripheral region of the host vehicle to be displayed, and at a larger magnification than the first display magnification.According to this configuration, at and after the time when it is determined that the own vehicle has reached the target area, the "size of the own vehicle and the size of a boundary line, an object, and the like" are enlarged on the screen. Therefore, visibility presented when the driver confirms the "positional relationship between the own vehicle, a boundary line, an object, and the like" is improved. (Perpendicular parking assistance operations)

[0071] Next, operations to be performed when the parking assist control is executed in response to a perpendicular parking assist request will be described. The CPU 10 of the parking assist ECU 10 (hereinafter simply referred to as "CPU") is configured to execute each of the Fig. 4 to 6 each time a "second predetermined time longer than the first predetermined time" elapses. The CPU further acquires the information about vehicle surroundings from the vehicle peripheral sensors by executing a routine (not shown) each time the first predetermined time elapses. The CPU also updates the above-mentioned two-dimensional map based on the information about the vehicle surroundings by executing a routine (not shown) each time the first predetermined time elapses.

[0072] In addition, when an ignition key switch (start button) (not shown) of the vehicle is changed from an off position to an on position, the CPU executes an initialization routine (not shown) to set values ​​of various flags, which will be described later, to "0".

[0073] When a predetermined time is reached, the CPU starts processing from step 400 of Fig. 4, and proceeds to the processing at step 405. In step 405, the CPU determines whether a value of a parking assist request flag (hereinafter simply referred to as a "request flag") FHS is "0" or not. If the value of the request flag FHS is "0," it means that a parking assist request (any of the perpendicular parking assist request or the parallel parking assist request) has not been issued. If the value of the request flag FHS is "1," it means that a parking assist request has been issued. Therefore, in step 405, the CPU determines whether or not the condition A1 is satisfied. If the value of the request flag FHS is not "0," the CPU makes a "NO" determination in step 405, directly proceeds to the processing of step 405, and temporarily terminates this routine.

[0074] Assuming that the value of the request flag FHS is "0," the CPU makes a "YES" determination in step 405, and the processing proceeds to step 410. In step 410, the CPU determines whether or not the perpendicular parking mode has been selected by the predetermined operation of the parking assist switch 84 (whether or not the condition A2 is satisfied). If it is determined that the perpendicular parking mode is not selected, the CPU makes a "NO" determination in step 410, and the processing proceeds directly to step 495 and temporarily terminates this routine.

[0075] Assuming that the perpendicular parking mode has been selected, the CPU makes a "YES" determination in step 410, and processing proceeds to step 415. In step 415, the CPU determines whether or not all of the above-mentioned "Condition A3, Condition A4, and Condition A5" are satisfied. The condition that all of Conditions A3, A4, and A5 are satisfied is also referred to as a "perpendicular parking assist execution condition." If the perpendicular parking assist execution condition is not satisfied, the CPU makes a "NO" determination in step 415, and the process proceeds directly to step 415, temporarily ending this routine.

[0076] Assuming that the perpendicular parking assist execution condition is satisfied, the CPU makes a "YES" determination in step 415, executes the processing of step 420 and step 425 (described below) sequentially, and the processing proceeds to step 430.

[0077] Step 420: The CPU sets the value of the request flag FHS to "1".

[0078] Step 425: The CPU sets, as a provisional target area, an area occupied by the vehicle body of the host vehicle when the host vehicle is assumed to have parked in each of the detected perpendicular parking area candidates. The CPU also sets, as a provisional target position, the position of the host vehicle at the time the host vehicle was parked in this provisional target area.

[0079] In step 425, the CPU also calculates a path along which the position of the host vehicle is to move from the current position of the host vehicle (present position) to the provisional target area as a provisional target path. The target path is a path along which the host vehicle body can move from its current position to the target position while maintaining a gap greater than or equal to a predetermined distance from an object (e.g., another vehicle, a curb, or a guardrail). Therefore, the CPU determines, as the provisional target path, a path that allows the host vehicle to move to the provisional target area "while maintaining the distance between the host vehicle and an object at a predetermined distance (distance margin) or greater."Therefore, in some situations, it may not be possible to calculate a provisional target path (a provisional target path does not exist). The target path can also be calculated using one of several known calculation methods (e.g., a method proposed in Japanese Patent Application Publication No. 2015-3565). The CPU then determines a target path with the shortest distance among the provisional target paths as the final target path.

[0080] For example, in the Fig. 7, a plurality of parking areas 701 are provided in the vicinity of a host vehicle 100 located at a current position Pnow. The plurality of parking areas 701 are separated by a first boundary line 702 and a plurality of second boundary lines 703. The CPU has detected another vehicle Vot as an object. Therefore, the CPU recognizes that a perpendicular parking area candidate As1 and a perpendicular parking area candidate As2 exist in the vicinity of the host vehicle 100.

[0081] In this situation, when the CPU returns to step 425 of the Fig. 4, the CPU sets a provisional target area Fp1 for the perpendicular parking area candidate As1, and determines, as a provisional target position Ptgt1, a position of the own vehicle 100 at the time when the own vehicle 100 is assumed to have parked in the provisional target area Fp1. Subsequently, the CPU calculates, as a provisional target path Ltgt1, a path for moving the position of the own vehicle 100 from the current position Pnow of the own vehicle 100 to the provisional target position Ptgt1. Similarly, the CPU sets a provisional target area Fp2 for the perpendicular parking area candidate As2, and determines, as a provisional target position Ptgt2, a position of the own vehicle 100 at the time when the own vehicle 100 is assumed to have parked in the provisional target area Fp2.Subsequently, the CPU calculates, as a provisional target path Ltgt2, a path for moving the position of the host vehicle 100 from the current position Pnow of the host vehicle 100 to the provisional target position Ptgt2. Subsequently, the CPU determines the target path Ltgt1, which has the shortest distance of the provisional target paths (Ltgt1 and Ltgt2), as a final target path Ltgt. Therefore, the perpendicular parking area candidate As1 is determined as the final perpendicular parking area, the provisional target area Fp1 is determined as the final target area, and the provisional target position Ptgt1 is determined as the final target position Ptgt.

[0082] Next, the CPU continues processing to step 430 of the Fig. 4 and determines whether or not a final target path exists (whether or not a final target path has been successfully calculated). If no final target path exists, the CPU makes a "NO" determination in step 430, processing proceeds directly to step 495, and temporarily terminates this routine. In this case, the CPU may display a "Please move vehicle to another location" message on the screen and then return to step 425. The CPU may also cause the speaker 85 to output a message displayed on the screen.

[0083] On the other hand, if there is a final target path, the CPU makes a "YES" determination in step 430, and processing proceeds to step 435. In step 435, the CPU determines "the direction for moving the own vehicle (specifically, the position of the shift lever), the steering angle pattern, and the speed pattern for moving the own vehicle" for moving the own vehicle along the final target path Ltgt.

[0084] Next, in step 440, the CPU determines whether or not the current shift lever position matches the "position assigned in step 435." If the current shift lever position matches the assigned position, the CPU makes a "YES" determination in step 440, and processing proceeds to step 450. In step 450, the CPU issues a request to the driver to release the brake pedal operation. Specifically, the CPU displays a message "Please remove your foot from the brake pedal" on the screen and causes the speaker 85 to output this message.

[0085] Next, in step 455, the CPU determines whether the driver is depressing the brake pedal or not. If the driver is not depressing the brake pedal (the driver has removed his or her foot from the brake pedal), the CPU makes a "NO" determination in step 455, and processing proceeds to step 460. In step 460, the CPU sets a value of a parking assist control execution flag (hereinafter referred to simply as "execution flag") FHE to "1." Then, the CPU proceeds to step 495 and temporarily terminates this routine. As a result, a parking assist control is started (see "YES" determination in step 510 of the routine of Fig. 5, which will be described later). On the other hand, if the driver is still depressing the brake pedal at the time the CPU proceeds to step 455, the CPU returns to the processing of step 450.

[0086] On the other hand, if the shift lever position does not agree with the "position assigned in step 435" at the time the CPU proceeds to step 440, the CPU makes a "NO" determination in step 440, and processing proceeds to step 445. In step 445, the CPU sends a shift control instruction to the SPW-ECU 60 according to the assigned shift lever position. The SPW-ECU 60 changes the shift lever position to the position specified by the shift control instruction. Subsequently, the CPU executes the processing from steps 450 to 460 sequentially as described above. Then, the CPU proceeds to the processing in step 495 and temporarily terminates this routine.

[0087] When a predetermined time is reached, the CPU starts processing from step 500 of Fig. 5, and proceeds to the processing in step 510. In step 510, the CPU determines whether a value of the execution flag FHE is "1" or not. If the value of the execution flag FHE is not "1", the CPU makes a "NO" determination in step 510, directly proceeds to the processing in step 595, and temporarily terminates this routine. On the other hand, if the value of the execution flag FHE is "1", the CPU makes a "YES" determination in step 510, and sequentially executes the processes of step 520 and step 530 described below. Then, the CPU proceeds to the processing in step 595 and temporarily terminates this routine.

[0088] Step 520: The CPU automatically changes the display mode of the touch panel 23 to the parking assistance mode. As shown in Fig. 8, the CPU generates a first top view image 810 based on the front image data, the rear image data, the right side image data, and the left side image data, and displays the first top view image 810 at the first display magnification in the first display area 301.

[0089] The first top view image 810 is an image of the own vehicle and the peripheral region of the own vehicle based on the aforementioned first virtual view angle. Therefore, the first top view image 810 corresponds to an example of the "first view angle image." The view angle position of the first virtual view angle is a position (first view angle position) spaced from the top view center position of the vehicle body by a predetermined first distance in the direction directly upward. The viewing direction of the first virtual view angle is the direction directly downward from the first view angle position toward the own vehicle. This first view angle position is set to a position sufficiently spaced upward from the own vehicle so that the entire own vehicle and the target area are included in the view angle image (first top view image 810).

[0090] The CPU sets the first display magnification to a magnification that allows the entire host vehicle 811 and at least part of the target area 813 to be displayed. Therefore, the first top view image 810 includes the entire host vehicle 811 and a peripheral region 812 thereof. The peripheral region 812 includes the boundary lines and objects located in the surroundings of the host vehicle 811, as well as the target area 813 corresponding to the target position.

[0091] As in Fig. As illustrated in Figure 8, the CPU displays a traveling direction image 820 in the second display area 302. The traveling direction image 820 is an image from the camera corresponding to the traveling direction of the host vehicle. The host vehicle is currently moving backward, and therefore, the traveling direction image 820 is the rear image data acquired by the camera 83b.

[0092] The CPU also displays a traveling direction message 831 and a vehicle marker 832 in the third display area 303. The traveling direction message 831 is a message including the direction in which the host vehicle is to travel (ie, the position of the gearshift lever). The vehicle marker 832 is a marker indicating in which direction, of the forward direction or the reverse direction of the host vehicle, the traveling direction image 820 is displayed.

[0093] Step 530: The CPU executes parking assist control. Specifically, the CPU executes automatic steering angle control by sending a steering command (target steering angle) to the EPS ECU 40 according to the steering angle pattern. The CPU also executes driving force control by sending a driving force control command to the engine ECU 20 according to the speed pattern. The CPU also executes braking force control by sending a braking force control command to the brake ECU 30 according to the speed pattern. As a result, the driver can move the vehicle to the target area (move the vehicle's position to the target position) without manually operating the steering wheel, accelerator pedal, or brake pedal.If the driver requests a large braking force by depressing the brake pedal at a time when step 520 is executed, the brake actuator 31 is controlled to generate a braking force corresponding to this request. In this case, the driving force of the vehicle is set to "0" by controlling the engine actuator 21.

[0094] When a predetermined time is reached, the CPU starts processing from step 600 of Fig. 4, and processing proceeds to step 610. In step 610, the CPU determines whether the value of the execution flag FHE is "1" or not. If the value of the execution flag FHE is not "1", the CPU makes a "NO" determination in step 610, proceeds directly to the processing of step 695, and temporarily terminates this routine.

[0095] If the value of the execution flag FHE is "1," the CPU makes a "YES" determination in step 610 and proceeds to the processing in step 620. In step 620, the CPU determines whether the position of the own vehicle has reached the final target position or not (whether the own vehicle has reached the final target area or not). If the position of the own vehicle has not reached the target position, the CPU makes a "NO" determination in step 620, proceeds directly to the processing in step 695, and temporarily terminates this routine.

[0096] On the other hand, when the position of the own vehicle has reached the target position, the CPU makes a "YES" determination in step 620, and executes the processing from step 630 to step 650 (described below) sequentially. Then, the CPU proceeds to the processing in step 695, and temporarily terminates this routine.

[0097] Step 630: The CPU executes processing to complete the parking assist control. Specifically, the CPU sends a braking force control instruction to the brake ECU 30 to generate braking force and stop the host vehicle. The CPU also sends the shift control instruction to the SPW ECU 60 to change the position of the shift lever to a parking (P) position.

[0098] Step 640: as in Fig. 9, the CPU displays a second top view image 910 in the first display area 301. The second top view image 910 corresponds to an example of the "first view angle image." In this example, the second top view image 910 is an enlarged image of the first top view image 810. Specifically, the CPU displays the first top view image 810 in the first display area 301 at a second display magnification that is greater than the first display delay.

[0099] If boundary lines are present that delimit the perpendicular parking area, the CPU sets the second display delay to a magnification that allows display of the entire own vehicle 911 and boundary lines (first boundary line 913 and second boundary lines 914) that separate a perpendicular parking area in which the own vehicle 911 is parked. Therefore, the second top view image 910 includes the entire own vehicle 911 and a peripheral region 912 of the own vehicle 911. The peripheral region 912 includes the first boundary line 913 and the second boundary lines 914 on the left and right sides of the own vehicle 911.

[0100] The CPU generates a second view angle image 920 based on the front image data, the rear image data, the right-side image data, and the left-side image data, and displays the second view angle image 920 in the second display area 302. The second view angle image 920 is an image (front image) of the own vehicle and the peripheral region of the own vehicle based on the above-mentioned second virtual view angle. The view angle position of the second virtual view angle is a position (second view angle position) spaced from the center position in the vehicle width direction of the front end portion of the vehicle body toward the front of the own vehicle by a predetermined second distance. The viewing direction of the second virtual view angle is the direction from the second view angle position toward the own vehicle.The CPU sets the second view angle position such that the entire own vehicle 911 and the boundary lines (first boundary line 913 and second boundary lines 914) that separate the perpendicular parking area in which the own vehicle 911 is parked are included in the view angle image (front image).

[0101] The CPU also displays, in the third display area 303, a message 930 for notifying that the parking of the vehicle has been completed, and causes the speaker 85 to output this message. When a predetermined period of time has elapsed since the start of the screen display of this step, the CPU terminates the screen display in the parking assist mode and switches to a screen display in the navigation mode. Even after the predetermined period of time has elapsed, if a switch (not shown) is operated (in the case where the host vehicle has not moved from the position at the time of completion of parking), the CPU may display the second top view image 910 in the first display area 301 and the second view angle image 920 in the second display area 302.

[0102] Step 650: The CPU sets the value of the request flag FHS and the value of the execution flag FHE to "0".

[0103] As described above, when it is determined that the own vehicle has reached the target position, the device of this embodiment displays, in the second display area 302, the second view angle image 920, which is an image of the own vehicle and the peripheral region of the own vehicle, based on the second virtual view angle different from the first virtual view angle. For example, in the case of the second top view image 910, because part of the first boundary line 913 is hidden under the front end portion of the own vehicle 911, it may be difficult for the driver to grasp the positional relationship between the front end portion of the own vehicle 911 and the first boundary line 913 in a vicinity thereof.In contrast, with the device of this embodiment, the driver can more accurately grasp the positional relationship between the front end portion of the own vehicle 911 and the first boundary line 913 in the vicinity of the front end portion by observing the second view angle image 920.

[0104] The device of this embodiment displays, at and after the time point when it is determined that the host vehicle has reached the target position, the second top-view image 910, which is the aforementioned image of the host vehicle and the peripheral region of the host vehicle based on the first virtual view angle, in the first display area 301. In addition, the device of this embodiment displays the second view angle image 920, which is an image of the host vehicle and the peripheral region of the host vehicle based on the second virtual view angle, in the second display area 302. Therefore, the driver can confirm the final parking state of the host vehicle from a variety of view angle positions. This allows the driver to more accurately grasp the positional relationship between the host vehicle and a boundary line, an object, and the like.

[0105] Regarding the second top view image 910, the size of the host vehicle 911 and the size of each of the first boundary line 913 and the second boundary lines 914 are larger than those of the first top view image 810 displayed during execution of the parking assist control. Therefore, visibility presented when confirming the "positional relationship between the vehicle and a boundary line, an object, and the like" is enhanced. (Operations with a parallel parking assistance system)

[0106] Next, the operations to be performed when parking assistance control is executed in response to a parallel parking assistance request are described. Parallel parking assistance control is the same as perpendicular parking assistance control, except that the area (target area) within which the host vehicle is to be ultimately moved and the viewing angle image displayed on the screen are different. The following description focuses primarily on these differences.

[0107] The CPU is configured to Fig. 10 each time the second predetermined time elapses. When a predetermined time is reached, the CPU starts processing from step 1000 of Fig. 10, and proceeds to the processing in step 1005. In step 1005, the CPU determines whether the value of the request flag FHS is "0" or not. Therefore, the CPU determines whether the above-mentioned condition B1 is satisfied or not. If the value of the request flag FHS is not "0", the CPU makes a "NO" determination in step 1005 and directly proceeds to the processing in step 1005, temporarily terminating this routine.

[0108] Assuming that the value of the request flag FHS is "0," the CPU makes a "YES" determination in step 1005 and proceeds to the processing in step 1010. In step 1010, the CPU determines whether or not the parallel parking mode has been selected by a predetermined operation of the parking assist switch 84 (whether or not the condition B2 is satisfied). If it is determined that the parallel parking mode is not selected, the CPU makes a "NO" determination in step 1010, proceeds directly to the processing in step 1095, and temporarily terminates this routine.

[0109] Assuming that the parallel parking mode has been selected, the CPU makes a "YES" determination in step 1010 and proceeds to the processing in step 1015. In step 1015, the CPU determines whether or not all of the above-mentioned "Condition B3, Condition B4, and Condition B5" are satisfied. The condition that all of Conditions B3, B4, and B5 are satisfied is also referred to as a "parallel parking assist execution condition." If the parallel parking assist execution condition is not satisfied, the CPU makes a "NO" determination in step 1015, proceeds directly to the processing in step 1095, and temporarily terminates this routine.

[0110] Assuming that the parallel parking assist execution condition is satisfied, the CPU makes a "YES" determination in step 1015, and executes the processing from steps 1020 to 1060 sequentially. The CPU then proceeds to the processing in step 1095, and temporarily terminates this routine. The details of steps 1020 to 1060 are the same as the details of steps 420 to 460 of the routine of Fig. 4, and therefore a description of it is omitted here.

[0111] The Fig. The routine illustrated in Figure 5 can be applied to the parallel parking assist control. Therefore, when the predetermined time is reached, the CPU starts processing starting from step 500 in Fig. 5. When the processing by the CPU has proceeded to step 520 as shown in Fig. As illustrated in Figure 11, the CPU displays a first top view image 1110 in the first display area 301 at the first display magnification, and displays a traveling direction image 1120 in the second display area 302. The first top view image 1110 corresponds to an example of the "first view angle image." The first top view image 1110 includes a self-vehicle 1111 and a peripheral region 1112 of the self-vehicle 1111. The peripheral region 1112 includes an object located in the vicinity of the self-vehicle 1111 and at least a part of a target area 1113 corresponding to the target position. The CPU also displays a traveling direction message 1131 and a vehicle marker 1132 in the third display area 303.

[0112] The Fig. The routine illustrated in Figure 6 can be applied to the parallel parking assist control. Therefore, when the predetermined time is reached, the CPU starts processing starting from step 600 in Fig. 6. When the processing by the CPU has proceeded to step 640 as shown in Fig. 12, the CPU displays a third top view image 1210 in the first display area 301, and displays a third view angle image 1220 in the second display area 302.

[0113] In this example, the third top view image 1210 is an enlarged image of the first top view image 1110, and corresponds to an example of the "first view angle image." The CPU displays the first top view image 1110 in the first display area 301 at a third display magnification that is greater than the first display delay. For example, when a first other vehicle 1221 is in front of the own vehicle 1211, and a second other vehicle 1222 is behind the own vehicle 1211, the CPU sets the third display magnification to a magnification that enables display of the entire own vehicle 1211, a part of the rear end portion of the first other vehicle 1221, and a part of the front end portion of the second other vehicle 1222. Therefore, the third image from the top view 1210 includes the entire own vehicle 1211 and a peripheral region 1212 of the own vehicle 1211.The peripheral region 1212 includes a part of the rear end portion of the first other vehicle 1221 located in front of the own vehicle 1211 and a part of the front end portion of the second other vehicle 1222 located behind the own vehicle 1211.

[0114] The third viewpoint image 1220 is a viewpoint image of the own vehicle and the peripheral region of the own vehicle based on a third virtual viewpoint, which is different from the first virtual viewpoint and the second virtual viewpoint described above. The viewpoint position of the third virtual viewpoint is a position on the roadway along which the own vehicle has been traveling, and is a position (third viewpoint position) spaced from the own vehicle by a predetermined third distance in an oblique front direction to the right of the own vehicle. The viewpoint direction of the third virtual viewpoint is the direction from the third viewpoint position toward the own vehicle. The third viewpoint image 1220 may also be referred to as an "oblique viewpoint image."The CPU sets the third view angle position such that the right side of the own vehicle 1211, a part of the right side of the first other vehicle 1221, and a part of the right side of the second other vehicle 1222 are included in the view angle image (oblique view angle image).

[0115] As described above, the device of this embodiment displays, in the second display area 302, the third view angle image 1220, which is an image of the own vehicle and the peripheral region of the own vehicle at the time when it is determined that the own vehicle has reached the target position, and is based on a third virtual view angle different from the first virtual view angle. Thus, the virtual view angle (third virtual view angle) set at the time of completion of the parallel parking assist control (the time when the own vehicle reaches the target position) is different from the virtual view angle (second virtual view angle) set at the time of completion of the perpendicular parking assist control.The driver can confirm the final parking state of the own vehicle at the time the vehicle is parked using the parallel parking mode from an appropriate virtual viewing angle. By viewing the third viewing angle image 1220, the driver can more accurately grasp whether the right side of the own vehicle 1211 (the side facing the road on which the own vehicle is traveling) does not protrude toward the side of the road compared to the right side of the first other vehicle 1221 and the right side of the second other vehicle 1222.

[0116] The driver can also more accurately grasp the distance between the own vehicle 1211 and the rear end portion of the first other vehicle 1221 and the distance between the own vehicle 1211 and the front end portion of the second other vehicle 1222 by viewing the third top view image 1210, which is an enlarged image of the first top view image 1210.

[0117] The present disclosure is not limited to the embodiments described above, and various modification examples can be applied within the scope of the present disclosure.

[0118] The view angle image (second view angle image) displayed at and after the time when it is determined that the own vehicle has reached the target position is not limited to the example (front image) described above. It is only required that the second view angle image be an image of the own vehicle and the surroundings of the own vehicle from a position spaced from the own vehicle in a direction different from the direction directly above the own vehicle. As shown in Fig. As illustrated in FIG. 13, the parking assist ECU 10 may display, in the second display area 302, a side image 1310 as the second viewpoint image at and after the time point at which it is determined that the own vehicle has reached the target position. The side image 1310 is a viewpoint image of the own vehicle and the peripheral region of the own vehicle based on a fourth virtual viewpoint. The viewpoint position of the fourth virtual viewpoint is a position (fourth viewpoint position) spaced from the center position in a vehicle longitudinal direction on the right side of the vehicle body by a predetermined fourth distance in the right direction. The viewing direction of the fourth virtual viewpoint is the direction from the fourth viewpoint position toward the own vehicle.As described above, when the parking of the own vehicle is completed, part of the wheel stopper may be hidden under the own vehicle. In the case of the top view image (second top view image 910), the driver cannot accurately confirm the position of a wheel stopper 1311. According to this configuration, the driver can accurately grasp the distance between the rear wheels and the wheel stopper 1311 by viewing the side image 1310. As another example, when an obstacle (e.g., a fence or wall) is located behind the own vehicle, the driver can accurately grasp the distance between the own vehicle and the obstacle by viewing the side image 1310.

[0119] As in Fig. 14, the parking assist ECU 10 may also display, in the second display area 302, a rear view image 1410 as the second view angle image at and after the time when it is determined that the own vehicle has reached the target position. The rear view image 1410 is a view angle image of the own vehicle and the peripheral region of the own vehicle based on a fifth virtual view angle. The view angle position of the fifth virtual view angle is a position (fifth view angle position) spaced from the center position in the vehicle width direction of the rear end portion of the vehicle body by a predetermined fifth distance toward the rear of the own vehicle. The viewing direction of the fifth virtual view angle is the direction from the fifth view angle position toward the own vehicle.According to this configuration, the driver can accurately grasp the positional relationship between the rear wheels and the wheel stopper 1411 by viewing the rear image 1410.

[0120] As in Fig. As illustrated in FIG. 15, the parking assist ECU 10 may also display, in the second display area 302, an oblique top view image 1510 as the second view angle image at and after the time point at which it is determined that the vehicle has reached the target position. The oblique top view image 1510 is a view angle image of the own vehicle and the peripheral region of the own vehicle based on a sixth virtual view angle. The view angle position of the sixth virtual view angle is the position (sixth view angle position) spaced from the top view center position of the own vehicle body by a predetermined sixth distance in an oblique upward direction. The viewing direction of the sixth virtual view angle is the direction from the sixth view angle position toward the own vehicle.

[0121] The parking assist ECU 10 may set a plurality of viewing direction positions spaced from the host vehicle in a plurality of directions different from the direction directly above the host vehicle at the time the host vehicle is determined to have reached the target area. The parking assist ECU 10 may also generate a plurality of view angle images of the host vehicle and the surroundings of the host vehicle viewed from the plurality of view angle positions and sequentially display the plurality of view angle images in the second display area 302. For example, at and after the time the host vehicle is determined to have reached the target position, the parking assist ECU 10 may sequentially display two or more of the aforementioned "front image, side image, rear image, and oblique image viewed from above" in the second display area 302.

[0122] The parking assistance ECU 10 can also generate a view angle image while rotating the position of the virtual view angle around the own vehicle by 360° in plan view, and display the generated view angle image in the second display area 302. This allows the driver to confirm the final parking state of the own vehicle with a smoothly moving image with continuity. For example, the parking assistance ECU 10 generates a view angle image each time the position of the virtual view angle is rotated by a predetermined angle around the own vehicle from the second view angle position to and after the time point at which it is determined that the own vehicle has reached the target area, and displays the generated view angle image in the second display area 302. The driver can confirm the final parking state of the own vehicle while rotating the position of the view angle by 360°.

[0123] As in Fig. As illustrated in Fig. 16, the parking assist ECU 10 may display on the screen the following numerical values ​​1 and 2 representing the positional relationship between the subject vehicle and a boundary line at the time when it is determined that the subject vehicle has reached the target position. (Numerical value 1) Inclination of a "longitudinal axis 1610 passing through the center of the subject vehicle in the vehicle width direction" with respect to the second boundary line 914. (Numerical value 2) Difference between a distance "a" from the right side of the subject vehicle 911 to the second boundary line 914 on the right side and a distance "b" from the left side of the subject vehicle 911 to the second boundary line 914 on the left side.

[0124] If there is no boundary line demarcating the perpendicular parking area and your vehicle is to be parked in a space between two other vehicles (first other vehicle and second other vehicle), as in Fig. As illustrated in Figure 17, the parking assist ECU 10 may generate a second top view image 1710 and a second view angle image 1720. The parking assist ECU 10 sets the second display magnification to a magnification that enables display of the entire own vehicle 1711, a part of the right side of the first other vehicle 1712, and a part of the left side of a second other vehicle 1713. The CPU sets the second view angle position such that the view angle image (front image) includes the entire own vehicle 1711, the part of the right side of the first other vehicle 1712, and the part of the left side of the second other vehicle 1713. According to this configuration, the driver can more accurately grasp the positional relationship between the own vehicle 1711 and the first other vehicle 1712 and the second other vehicle 1713 at and after the parking assist control regarding the perpendicular parking is completed.

[0125] The view angle image displayed at and after the time point at which the parallel parking assist control is completed (at and after the time point at which it is determined that the vehicle has reached the target position) is not limited to the above-described example (oblique view angle image). The parking assist ECU 10 may display one or more of the above-mentioned "front image, side image, rear image, and oblique top view image" in the second display area 302 instead of or in addition to the third view angle image (oblique view angle image) 1220 at and after the time point at which it is determined that the vehicle has reached the target position.As described above, the parking assist ECU 10 may also generate a view angle image at and after the time point at which it is determined that the own vehicle has reached the target position while rotating the position of the virtual view angle by 360° in plan view around the own vehicle, and may display the generated view angle image in the second display area 302.

[0126] The parking assist ECU 10 may, at and after the time point at which it is determined that the host vehicle has reached the target position, display the first image as viewed from above in the first display area 301 at the display magnification (ie, first display magnification) used during the execution of the parking assist control.

[0127] The CPU may, when the host vehicle cannot be moved to the provisional target position merely by moving the host vehicle to the target position merely by moving the host vehicle with a single reset, calculate as the provisional target path a path according to which the host vehicle is moved forward and then backward, or backward, then forward, and then backward again (i.e., a route involving a change in the traveling direction of the host vehicle). For example, the CPU calculates a first path for moving the host vehicle forward to a traveling direction change position (i.e.,a position at which the host vehicle is temporarily stopped to change the position of the shift lever from the drive position (D) to the reverse position (R) from the current position, and a second path for moving the host vehicle backward from the direction change position to the target position, and sets the first path and the second path as the provisional target path. According to this configuration, when the host vehicle moves to the direction change position during parking assist control (perpendicular parking assist control or parallel parking assist control), the CPU sends a braking force control instruction to the brake ECU 30 to generate braking force and causes the host vehicle to stop. The CPU also sends a shift control instruction to the SBW-ECU 60 to change the position of the shift lever to the position specified by the shift control instruction.

[0128] It is only required that the parking assist switch 84 be a switch to be operated when the driver requests parking assistance, for generating a signal representing the request. The parking assist switch may be a device configured to recognize a driver's request for parking assistance using a voice recognition device. Such a device is equivalent to a switch operated by voice and may serve as an operation switch (operating device) in one embodiment. The parking assist ECU may include a request monitoring function for determining whether or not a parking assist request has been issued from a switching operation by the driver and / or the driver's voice.

[0129] Images related to parking assistance (the aforementioned viewing angle image and driving direction image) may be displayed on the display 51 instead of or in addition to the touch panel 73. The meter ECU 50 may display an image related to parking assistance according to a display instruction sent from the parking assistance ECU 10. The display 51 may include a display dedicated to parking assistance.

[0130] The parking assist ECU 10 may also be configured to further execute "forward perpendicular parking assist control" in which parking assistance is performed when the subject vehicle is moved forward so that the longitudinal direction of the subject vehicle and the longitudinal direction of another vehicle are parallel to each other. In this case, each time the parking assist switch 84 is pressed, the switching mode is sequentially switched among the reverse perpendicular parking mode, the forward perpendicular parking mode, the parallel parking mode, and the unset mode.

[0131] The parking assist ECU 10 may be configured to perform only automatic steering angle control as the parking assist control. In this case, when the final target position and the target path are determined, the parking assist ECU 10 displays a parking assist guidance (gear lever position) on the touch panel 73. The driver changes the gear lever position according to the parking assist guidance. When the driver changes the gear lever position to the guided position, the parking assist ECU 10 starts automatic steering angle control. The driver moves the host vehicle by operating the brake pedal and the accelerator pedal. When the host vehicle reaches the target position, the parking assist ECU 10 displays a message on the screen notifying the driver to stop the vehicle and causes the speaker 85 to output this message.When the driver depresses the brake pedal to stop the vehicle, the parking assist ECU 10 terminates the parking assist control (steering angle automatic control) (ie, parking assist control not completed). Subsequently, the parking assist ECU 10 displays the second top view image (or the third top view image) in the first display area 301, and displays the second view angle image (or the third view angle image) in the second display area 302.

[0132] A parking assistance device performs parking assistance control to move a vehicle along a predetermined target path toward a target area. The parking assistance device displays, on a display device, an image of the vehicle and surroundings of the vehicle as viewed from a position spaced apart from the vehicle in a direction different from a direction directly above the vehicle at a time when it is determined that the host vehicle has reached the target area.< / parallelparkassistenzanforderung> < / querparkassistenzanforderung>

Claims

[1] Parking assistance device, with: an information retrieval device (81, 82, 83) comprising at least one image recording device (83) configured to obtain image data from surroundings of a vehicle, wherein the information retrieval device is configured to obtain information about vehicle surroundings comprising information about an object located in the surroundings of the vehicle and information about a boundary line on a road surface located in the surroundings of the vehicle; a path determination module (10, 10X) programmed to determine, based on the information about the vehicle surroundings, a target area, which is an area that the vehicle is to occupy when parking of the vehicle is completed, and to determine a path along which the vehicle is to move from a position of the vehicle at the current time to the target area as a target path; a parking assistance module (10, 10Y) programmed to execute a parking assistance control comprising automatic steering angle control with respect to the vehicle for moving the vehicle along the determined target path; and a display device (73, 51) configured to display an image to an occupant of the vehicle, wherein the parking assistance module is programmed to display on the display device an image (920, 1220, 1310, 1410, 1510, 1720) generated based on the image data acquired by the image acquisition device, wherein the image is an image of the vehicle and a surrounding area of the vehicle viewed from a position spaced from the vehicle in a direction different from a direction from directly above the vehicle at the time when it is determined that the vehicle has reached the target area, characterized by that the parking assistance module is programmed to: setting a plurality of view angle positions spaced from the vehicle in a plurality of directions different from the direction directly above the vehicle at a time when it is determined that the vehicle has reached the target area; to generate a plurality of images of the vehicle and a surrounding of the vehicle from the viewpoint of the plurality of viewing angle positions based on the image data acquired by the image pickup device (83); and to display the plurality of images on the display device (73, 51) one after the other. [2] Parking assistance device according to claim 1, wherein the display device (73, 51) is configured to display an image having a first display area (301) and a second display area (302), and where the parking assistance module is programmed to: display the plurality of images sequentially in the second display area; based on the image data acquired by the image pickup device, a first viewpoint image (810, 910, 1110, 1210, 1710) of the vehicle and a surrounding area of the vehicle viewed from a position spaced from the vehicle in a direction directly above the vehicle at the time when it is determined that the vehicle has reached the target area; and to display the first viewing angle image (810, 910, 1110, 1210, 1710) in the first display area (301). [3] Parking assistance device according to claim 2, wherein the parking assistance module is programmed to: generate the first view angle image during a period from a time at which the parking assistance control starts to a time at which it is determined that the vehicle has reached the target area, and display the first view angle image (810, 910, 1110, 1210, 1710) in the first display area (301) at a first display magnification that enables the entire vehicle and at least a part of the target area to be displayed; and at and after a time at which it is determined that the vehicle has reached the target area, display the first viewpoint image (810, 910, 1110, 1210, 1710) in the first display area (301) with a second display delay that is greater than the first display delay that allows the entire vehicle and at least a part of a peripheral region of the vehicle to be displayed. [4] Parking assistance device according to claim 3, wherein the parking assistance module is programmed to: based on the image data acquired by the image pickup device (83), generate a travel direction image (820, 1120) of a range of a travel direction of the vehicle during the period from the time at which the parking assistance control starts to the time at which it is determined that the vehicle has reached the target area; and display the traveling direction image (820, 1120) in the second display area (302) during the period from the time the parking assist control starts to the time it is determined that the vehicle has reached the target area. [5] The parking assist device according to claim 4, wherein the plurality of images of the vehicle and a surrounding of the vehicle as viewed from the plurality of view angle positions sequentially displayed in the second display area (302) includes at least two of a front image of the vehicle, a side image of the vehicle, and a rear image of the vehicle.

Images