Parking Assistance Device

Abstract

To provide a parking support device that enables an image of an underfloor of an own vehicle to be displayed on a display device, even if the volume of image data that can be utilized in generating the image of the underfloor of the own vehicle is small.SOLUTION: A parking support device 10 comprises: a control device 90 that executes parking support control of making an own vehicle 100 automatically travel to park at a designated parking section 200D, and executes underfloor-image display control of making a display device 80 display an image of an underfloor of the vehicle; and an image sensor 72 that provides image data for generating the image of the underfloor. The display device has a region with a predetermined area as an underfloor-image display region for displaying the image of the underfloor by the underfloor image display control. The control device generates the image of the underfloor that can be generated using provided image data and makes the display device display the image of the underfloor when image data that can generate the image of the underfloor satisfying the whole range of the underfloor-image display region are not provided during the execution of the underfloor-image display control.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] The present invention relates to a parking assistance device. [Background technology]

[0002] When an image of the underfloor of the vehicle is displayed on a display inside the vehicle, the driver can drive the vehicle while checking the condition of the underfloor of the vehicle by looking at the image when parking the vehicle. Therefore, a parking assistance device that displays an image of the underfloor of the vehicle on a display in response to a request from the driver is known (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-6921 A Summary of the Invention

[0004] The conventional parking assistance device described above captures images of the area around the vehicle using a camera mounted on the vehicle, and when the driver requests that an image of the underfloor of the vehicle (underfloor image) be displayed on the display, it generates an underfloor image using past image data captured by the camera and displays the underfloor image on the display.

[0005] Since this parking assistance device is premised on displaying an underfloor image in a predetermined area of ​​the display, if the vehicle does not travel a certain distance or more in a certain direction, sufficient image data cannot be obtained to generate an underfloor image that fills the entire predetermined area of ​​the display, and the underfloor image cannot be displayed on the display. For example, when parking the vehicle, the driver may turn the vehicle by moving the vehicle forward and backward for a short distance, but when the vehicle turns in this way, a situation occurs in which the vehicle does not travel a certain distance or more in a certain direction, and therefore conventional parking assistance devices cannot obtain sufficient image data to generate an underfloor image that fills the entire predetermined area of ​​the display, and the underfloor image cannot be displayed on the display.

[0006] An object of the present invention is to provide a parking assistance device that can display an underfloor image of the vehicle on a display device such as a display even if the amount of image data available for generating the underfloor image of the vehicle is small.

[0007] The parking assistance device of the present invention comprises a control device that performs parking assistance control to automatically drive the vehicle and park it in a designated parking space, and also performs underfloor image display control to display an underfloor image representing the underfloor of the vehicle on a display device of the vehicle, and an image sensor that captures an image of the area around the vehicle and provides image data for generating the underfloor image.

[0008] The control device is configured to start the underfloor image display control when the vehicle reaches a point a predetermined distance from a position where parking of the vehicle in the designated parking space by the parking assistance control is completed.

[0009] The display device has an area of ​​a predetermined size as an under-floor image display area for displaying the under-floor image by the under-floor image display control.

[0010] The control device is configured to, when the image data capable of generating an under-floor image that fills the entire under-floor image display area is provided during execution of the under-floor image display control, generate the under-floor image that fills the entire under-floor image display area and display the under-floor image on the display device. On the other hand, when the image data capable of generating an under-floor image that fills the entire under-floor image display area is not provided during execution of the under-floor image display control, generate the under-floor image that can be generated from the provided image data and display the under-floor image on the display device.

[0011] This makes it possible to display the under-floor image on the display device even if the amount of image data available for generating the under-floor image is small.

[0012] The components of the present invention are not limited to the embodiments of the present invention described below with reference to the drawings. Other objects, other features and associated advantages of the present invention will be easily understood from the description of the embodiments of the present invention. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing a parking assistance device according to an embodiment of the present invention and a vehicle (host vehicle) equipped with the parking assistance device. [Diagram 2] FIG. 2 is a diagram showing a scene when the host vehicle is stopped next to a parking space in which the host vehicle is to be parked. [Diagram 3] FIG. 3 is a diagram showing a route (target driving route) along which the vehicle should travel in order to park the vehicle in a designated parking space. [Figure 4] FIG. 4 is a diagram showing a scene in which the host vehicle moves forward while turning right due to parking assist control. [Diagram 5] FIG. 5 is a diagram showing a scene in which the host vehicle moves backward while turning left due to parking assist control. [Figure 6] FIG. 6 is a diagram showing a scene in which the host vehicle is moving backwards straight due to parking assist control. [Figure 7]FIG. 7 is a diagram showing a scene where parking of the host vehicle is completed by the parking assist control. [Figure 8] FIG. 8 is a diagram showing a scene in which the host vehicle is moving backwards straight due to parking assist control. [Figure 9] FIG. 9 is a diagram showing a scene in which the host vehicle moves forward while turning right in order to turn around due to parking assist control. [Figure 10] FIG. 10 is a diagram showing a scene in which the host vehicle is turned around by the parking assist control and moves backward while making a left turn. [Figure 11] FIG. 11 is a diagram showing a scene where parking of the host vehicle has been completed through parking assist control. [Figure 12] (A) of Figure 12 is a diagram showing the underfloor image, etc. displayed on the display device when the display of the underfloor image, etc. on the display device is started when a sufficient amount of image information has been obtained to generate the underfloor image; (B) of Figure 12 is a diagram showing the underfloor image, etc. displayed on the display device when a predetermined time has elapsed from the time point of (A) of Figure 12 when a sufficient amount of image information has been obtained to generate the underfloor image; (C) of Figure 12 is a diagram showing the underfloor image, etc. displayed on the display device when a predetermined time has elapsed from the time point of (B) of Figure 12 when a sufficient amount of image information has been obtained to generate the underfloor image; and (D) of Figure 12 is a diagram showing the underfloor image, etc. displayed on the display device when a predetermined time has elapsed from the time point of (C) of Figure 12 when a sufficient amount of image information has been obtained to generate the underfloor image. [Figure 13](A) of Figure 13 is a diagram showing the under-floor image, etc. displayed on the display device when the display of the under-floor image, etc. on the display device is started when a sufficient amount of image information has not been obtained to generate the under-floor image, (B) of Figure 13 is a diagram showing the under-floor image, etc. displayed on the display device when a predetermined time has elapsed from the time point of (A) of Figure 13 when a sufficient amount of image information has not been obtained to generate the under-floor image, (C) of Figure 13 is a diagram showing the under-floor image, etc. displayed on the display device when a predetermined time has elapsed from the time point of (B) of Figure 13 when a sufficient amount of image information has not been obtained to generate the under-floor image, and (D) of Figure 13 is a diagram showing the under-floor image, etc. displayed on the display device when a predetermined time has elapsed from the time point of (C) of Figure 13 when a sufficient amount of image information has been obtained to generate the under-floor image. [Figure 14] FIG. 14 is a flowchart showing a routine executed by the parking assistance device according to the embodiment of the present invention. [Figure 15] FIG. 15 is a flowchart showing a routine executed by the parking assistance device according to the embodiment of the present invention. [Figure 16] FIG. 16 is a flowchart showing a routine executed by the parking assistance device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, a parking assistance device according to an embodiment of the present invention will be described with reference to the drawings. As shown in Fig. 1, a parking assistance device 10 according to an embodiment of the present invention is mounted on a vehicle (host vehicle) 100.

[0015] The parking assistance device 10 is a device that executes a parking assistance system in which the vehicle 100 is automatically driven and automatically stopped at a designated location (designated parking spot), thereby parking the vehicle 100 in the designated parking spot.

[0016] As shown in Fig. 1, the parking assistance device 10 includes an ECU 90 (controller). The ECU 90 includes a microcomputer as a main part. The ECU 90 includes a CPU, a ROM, a RAM, a non-volatile memory, and an interface. The CPU realizes various functions by executing instructions (programs, routines) stored in the ROM.

[0017] <Vehicle driving device> The host vehicle 100 is equipped with a vehicle driving device 20. The vehicle driving device 20 is a device that drives, brakes, steers, and changes gears of the host vehicle 100, and in this example, includes a drive device 21, a braking device 22, a steering device 23, and a transmission device 24.

[0018] <Drive unit> The drive device 21 is a device that outputs a driving force provided to the host vehicle 100 to make the host vehicle 100 run, and is, for example, an internal combustion engine and / or a motor. The drive device 21 is electrically connected to the ECU 90. The ECU 90 can control the driving force output from the drive device 21 by controlling the operation of the drive device 21.

[0019] <Brake device> The braking device 22 is a device that outputs a braking force to be applied to the host vehicle 100 in order to brake the host vehicle 100, and is, for example, a hydraulic brake device. The braking device 22 is electrically connected to the ECU 90. The ECU 90 can control the braking force output from the braking device 22 by controlling the operation of the braking device 22.

[0020] <Steering system> The steering device 23 is a device that outputs a steering force applied to the host vehicle 100 in order to steer the host vehicle 100, and is, for example, a power steering device. The steering device 23 is electrically connected to the ECU 90. The ECU 90 can control the steering force output from the steering device 23 by controlling the operation of the steering device 23.

[0021] <Transmission device> The transmission device 24 is a device that switches whether or not the driving force output from the drive device 21 is transmitted to the driving wheels of the host vehicle 100, and switches whether the driving force is transmitted to the driving wheels so as to move the host vehicle 100 forward or to the driving wheels so as to move the host vehicle 100 backward. Furthermore, the transmission device 24 is also a device that holds the host vehicle 100 in a stopped state by locking the gear so as not to rotate by engaging a claw-shaped part (parking lock pole) on the gear of the transmission device 24. Therefore, the transmission device 24 also functions as a stop holding device that holds the host vehicle 100 in a stopped state.

[0022] The transmission device 24 operates in one of the following states: a state in which the driving force is transmitted to the drive wheels to move the vehicle 100 forward (drive range state SD), a state in which the driving force is transmitted to the drive wheels to move the vehicle 100 backward (rearrangement state SR), a state in which the driving force is not transmitted to the drive wheels of the vehicle 100 (neutral range state SN), and a state in which the vehicle 100 is kept stopped (parking range state SP).

[0023] The transmission device 24 is electrically connected to the ECU 90. The ECU 90 controls the operation of the transmission device 24 to set the transmission device 24 to any one of a drive range state SD, a rearrange state SR, a neutral range state SN, and a parking range state SP.

[0024] <Stopping holding device> The host vehicle 100 is also equipped with a vehicle stop maintaining device 30. The vehicle stop maintaining device 30 is a device that maintains the host vehicle 100 in a stopped state, and in this example, is a parking brake device 31. The parking brake device 31 may be an electric parking brake device or a manual parking brake device. The parking brake device 31 is a device that maintains the host vehicle 100 in a stopped state by applying a braking force to the wheels of the host vehicle 100 that is stopped. In particular, the parking brake device 31 is a device that maintains the host vehicle 100 in a stopped state by applying a braking force to the wheels by pressing brake pads against brake discs provided on the wheels of the host vehicle 100 that is stopped. In this example, the parking brake device 31 is an electric parking brake device, and is therefore electrically connected to the ECU 90. The ECU 90 can maintain the host vehicle 100 in a stopped state by operating the parking brake device 31.

[0025] <Sensors, etc.> Furthermore, the vehicle 100 is equipped with an accelerator pedal 41, an accelerator pedal operation amount sensor 42, a brake pedal 43, a brake pedal operation amount sensor 44, a steering wheel 45, a steering shaft 46, a steering angle sensor 47, a steering torque sensor 48, a parking assistance request operator 49, a shift lever 51, a shift sensor 52, a vehicle speed detection device 60 and a surrounding information detection device 70.

[0026] <Accelerator pedal operation amount sensor> Accelerator pedal operation amount sensor 42 is a sensor that detects the amount of operation of accelerator pedal 41. Accelerator pedal operation amount sensor 42 is electrically connected to ECU 90. Accelerator pedal operation amount sensor 42 transmits information on the detected amount of operation of accelerator pedal 41 to ECU 90. Based on that information, ECU 90 obtains the amount of operation of accelerator pedal 41 as accelerator pedal operation amount AP.

[0027] Except when executing parking assist control described later, the ECU 90 calculates a required driving force (required driving torque) based on the accelerator pedal operation amount AP and the traveling speed (host vehicle speed) of the host vehicle 100. The ECU 90 controls the operation of the drive device 21 so that the required driving force is output. Furthermore, when executing parking assist control described later, the ECU 90 determines a driving force required to drive the host vehicle 100 as desired by the parking assist control, and controls the operation of the drive device 21 so that the driving force is output.

[0028] <Brake pedal operation amount sensor> The brake pedal operation amount sensor 44 is a sensor that detects the amount of operation of the brake pedal 43. The brake pedal operation amount sensor 44 is electrically connected to the ECU 90. The brake pedal operation amount sensor 44 transmits information on the detected amount of operation of the brake pedal 43 to the ECU 90. The ECU 90 obtains the amount of operation of the brake pedal 43 as a brake pedal operation amount BP based on the information.

[0029] Except when executing parking assist control described later, the ECU 90 obtains a required braking force (required braking torque) by calculation from the brake pedal operation amount BP. The ECU 90 controls the operation of the braking device 22 so that the required braking force is output. When executing parking assist control described later, the ECU 90 determines a braking force required to brake the host vehicle 100 as desired by the parking assist control, and controls the operation of the braking device 22 so that the braking force is output.

[0030] <Steering angle sensor> The steering angle sensor 47 is a sensor that detects the rotation angle of the steering shaft 46 with respect to the neutral position, and is electrically connected to the ECU 90. The steering angle sensor 47 transmits information on the detected rotation angle of the steering shaft 46 to the ECU 90. The ECU 90 obtains the rotation angle of the steering shaft 46 as the steering angle θ based on the information.

[0031] <Steering torque sensor> The steering torque sensor 48 is a sensor that detects the torque input to the steering shaft 46 by the driver DR of the vehicle 100 via the steering wheel 45, and is electrically connected to the ECU 90. The steering torque sensor 48 transmits information on the detected torque to the ECU 90. The ECU 90 acquires the torque input to the steering shaft 46 by the driver DR via the steering wheel 45 (driver input torque) based on the information.

[0032] Except when executing parking assist control described later, the ECU 90 obtains a required steering force (required steering torque) based on the steering angle θ, the driver input torque, and the traveling speed (host vehicle speed) of the host vehicle 100, and controls the operation of the steering device 23 so that the required steering torque is output from the steering device 23. Furthermore, when executing parking assist control described later, the ECU 90 determines a steering force required for driving the host vehicle 100 as desired by the parking assist control, and controls the operation of the steering device 23 so that the steering force is output.

[0033] <Parking assistance request controller> The parking assist request operation device 49 is a device, such as a switch, operated by the driver DR to request the ECU 90 to execute parking assist control, which will be described later. The parking assist request operation device 49 is electrically connected to the ECU 90. When the driver DR performs a predetermined operation to request the ECU 90 to execute parking assist control, the parking assist request operation device 49 transmits a predetermined signal to the ECU 90. When the ECU 90 receives the predetermined signal, it determines that execution of parking assist control has been requested.

[0034] <Shift sensor> The shift sensor 52 is a sensor that detects the setting position of the shift lever 51. The shift lever 51 is a device that is operated by the driver DR of the vehicle 100, and the setting positions of the shift lever 51 that the driver DR can set are a forward position (drive range D), a reverse position (rearrange R), a neutral position (neutral range N), and a parking position (parking range P). The shift sensor 52 is electrically connected to the ECU 90. The shift sensor 52 transmits a signal indicating the detected setting position of the shift lever 51 to the ECU 90.

[0035] When the shift lever 51 is set to drive range D, the shift sensor 52 transmits a signal to the ECU 90 indicating that the set position of the shift lever 51 is drive range D. Upon receiving the signal, the ECU 90 controls the operation of the transmission device 24 so that the transmission device 24 is in the drive range state SD (performs a shift change).

[0036] Furthermore, when the shift lever 51 is set to rearrange R, the shift sensor 52 transmits a signal to the ECU 90 indicating that the set position of the shift lever 51 is rearrange R. Upon receiving this signal, the ECU 90 controls the operation of the transmission device 24 so that the transmission device 24 is in the rearrange state SR (performs a shift change).

[0037] Furthermore, when the shift lever 51 is set to the neutral range N, the shift sensor 52 transmits a signal indicating that the set position of the shift lever 51 is the neutral range N to the ECU 90. Upon receiving this signal, the ECU 90 controls the operation of the transmission device 24 so that the transmission device 24 is in the neutral range state SN (performs a shift change).

[0038] Furthermore, when the shift lever 51 is set to the parking range P, the shift sensor 52 transmits a signal indicating that the set position of the shift lever 51 is the parking range P to the ECU 90. Upon receiving this signal, the ECU 90 controls the operation of the transmission device 24 so that the transmission device 24 is in the parking range state SP (performs a shift change).

[0039] When executing parking assist control, which will be described later, the ECU 90 controls the operation of the transmission device 24 (performs a shift change) as necessary to make the host vehicle 100 run as desired through the parking assist control.

[0040] <Vehicle speed detection device> The vehicle speed detection device 60 is a device that detects the traveling speed of the host vehicle 100, and is, for example, a wheel speed sensor. The vehicle speed detection device 60 is electrically connected to the ECU 90. The vehicle speed detection device 60 transmits information on the detected traveling speed of the host vehicle 100 to the ECU 90. The ECU 90 acquires the traveling speed of the host vehicle 100 (host vehicle speed V) based on the information.

[0041] <Peripheral information detection device> The surrounding information detection device 70 is a device that detects information about the surroundings of the vehicle 100, and in this example, includes a radio wave sensor 71 and an image sensor 72.

[0042] <Radio wave sensor> The radio wave sensor 71 is a sensor that detects information about an object present in the vicinity of the vehicle 100 using radio waves, and is, for example, at least one of a radar sensor (such as a millimeter wave radar), a sonic sensor such as an ultrasonic sensor (clearance sonar), and an optical sensor such as a laser radar (LiDAR). The radio wave sensor 71 is electrically connected to the ECU 90. The radio wave sensor 71 transmits radio waves and receives radio waves (reflected waves) reflected by an object. The radio wave sensor 71 transmits information related to the transmitted radio waves and the received radio waves (reflected waves) to the ECU 90. In other words, the radio wave sensor 71 detects an object present in the vicinity of the vehicle 100 and transmits information related to the detected object to the ECU 90. The ECU 90 can obtain information (periphery detection information IS) related to an object present in the vicinity of the vehicle 100 based on the information (radio wave information IR or radio wave data). Objects detected using the radio wave sensor 71 are, for example, vehicles, walls, bicycles, people, and the like.

[0043] <Image sensor> The image sensor 72 is a sensor, such as a camera, that captures an image of the periphery of the vehicle 100. The image sensor 72 is electrically connected to the ECU 90. The image sensor 72 captures an image of the periphery of the vehicle 100 and transmits information related to the captured image to the ECU 90. The ECU 90 can acquire information (periphery detection information IS) related to the periphery of the vehicle 100 based on the information (image information IC or image data).

[0044] <Display device> The vehicle 100 is also equipped with a display device 80. The display device 80 is a device that displays images, such as a display. The display device 80 is disposed in a position in the vehicle 100 where the driver DR can view the image displayed by the display device 80. The display device 80 is electrically connected to the ECU 90. The ECU 90 can display various images using the display device 80.

[0045] <Overview of parking assistance system operation> Next, there will be described an outline of the operation of the parking assistance device 10. The parking assistance device 10 is configured to be capable of executing parking assistance control for automatically driving the host vehicle 100 and parking it in a designated parking space.

[0046] For example, as shown in FIG. 2, when the driver DR stops the vehicle 100 so that the parking space 200 in which the vehicle 100 is to be parked is located to the left of the vehicle 100, and operates the parking assistance request operator 49 to request the execution of parking assistance control, the parking assistance device 10 starts parking assistance control.

[0047] When the parking assistance device 10 starts parking assistance control, it recognizes objects such as adjacent parked vehicles 300 and walls 301 around the vehicle 100, as well as the partition lines 201 on the ground around them, based on the periphery detection information IS. In this example, the adjacent parked vehicles 300 are other vehicles parked in parking spaces on the right or left side of the parking space 200C. Furthermore, the parking assistance device 10 acquires a parking space 200 (parking space 200C) in which the vehicle 100 can be parked, based on the recognized objects and partition lines 201 and the periphery detection information IS.

[0048] When the parking assistance device 10 acquires the available parking section 200C, it causes the display device 80 to display an image (available parking section image) representing the acquired available parking section 200C.

[0049] When the driver DR performs an operation (parking section setting operation) to designate or set the parking section 200C (or any one of the multiple parking sections 200C) shown in the parking section image displayed on the display device 80 as a section in which the vehicle 100 is to be parked, the parking assistance device 10 sets the designated parking section 200C as a section in which the vehicle 100 is to be parked (designated parking section 200D). In this example, the display device 80 is configured to detect a touch operation (contact operation) on the display device 80 by the driver DR and transmit a signal indicating that the driver DR has performed a touch operation on the display device 80 to the ECU 90, and the operation to designate or set the section in which the vehicle 100 is to be parked is a contact operation on the display device 80 by the driver DR.

[0050] When the parking assistance device 10 sets the designated parking section 200D, it sets a position (target parking position Ptgt) for parking the vehicle 100 in the designated parking section 200D based on the positions of the designated parking section 200D, the object, the partition line 201, etc., as shown in Fig. 3. The target parking position Ptgt is a target position for a specific portion (reference portion 100P) of the vehicle 100 to reach by parking assistance control. The reference portion 100P of the vehicle 100 is a predetermined portion, and in this example, it is the central portion in the front-rear direction (longitudinal direction) and the left-right direction (width direction) of the vehicle 100.

[0051] 3, the reference numeral 202 indicates a wheel stopper arranged in the designated parking section 200D. In this manner, when the wheel stopper 202 is arranged in the designated parking section 200D, the parking assistance device 10 sets the target parking position Ptgt so that the rear wheels of the host vehicle 100 are near the wheel stopper 202 when parking of the host vehicle 100 is completed by moving the host vehicle 100 backward through parking assistance control, and sets the target parking position Ptgt so that the front wheels of the host vehicle 100 are near the wheel stopper 202 when parking of the host vehicle 100 is completed by moving the host vehicle 100 forward through parking assistance control.

[0052] When the parking assistance device 10 sets the target parking position Ptgt, as shown in FIG. 3, it sets a route (target driving route Rtgt) along which the vehicle 100 will travel in order to cause the reference portion 100P of the vehicle 100 to reach the target parking position Ptgt.

[0053] When the parking assistant 10 sets the target travel route Rtgt, it executes a process (vehicle travel process) of making the host vehicle 100 travel along the target travel route Rtgt. Specifically, the parking assistant 10 first makes the host vehicle 100 move forward a predetermined distance while turning right, and stops it, as shown in FIG. 4. Next, the parking assistant 10 makes the host vehicle 100 move backward while turning left, as shown in FIG. 5, gradually reducing the steering angle θ, and steers the host vehicle 100 so that the steering angle θ becomes zero when the front-rear direction of the host vehicle 100 becomes parallel to the longitudinal direction of the designated parking section 200D. After that, the parking assistant 10 makes the host vehicle 100 move backward straight, as shown in FIG. 6.

[0054] Then, as shown in FIG. 7, when the reference portion 100P of the host vehicle 100 reaches the target parking position Ptgt, the parking assistance device 10 controls the operation of the braking device 22 to stop the host vehicle 100, activates the parking brake device 31 to hold the host vehicle 100 in a stopped state, and ends the parking assistance control.

[0055] During the execution of the parking assist control, the parking assist device 10 monitors the host vehicle speed V, and controls the operation of the drive device 21 and the brake device 22 so that the host vehicle speed V does not exceed a predetermined vehicle speed Vlimit.

[0056] However, for example, when the vehicle 100 is being reversed by parking assist control as shown in FIG. 8, it may be necessary to change the target driving route Rtgt due to the detection of an obstacle that impedes the driving of the vehicle 100.

[0057] When it becomes necessary to change the target driving path Rtgt in this way, the parking assistance device 10 performs an emergency stop process to temporarily stop the host vehicle 100 by controlling the operation of the braking device 22, and then changes (resets) the target driving path Rtgt. For example, as shown in FIG. 9, the host vehicle 100 is temporarily turned right and moved forward a predetermined distance and stopped, and then, as shown in FIG. 10, the host vehicle 100 is moved backward while gradually decreasing the steering angle θ, and when the fore-and-aft direction of the host vehicle 100 becomes parallel to the longitudinal direction of the designated parking space 200D, the steering angle θ is set to zero, and the host vehicle 100 is moved backward straight as shown in FIG. 11. When the reference portion 100P of the host vehicle 100 reaches the target parking position Ptgt, the operation of the braking device 22 is controlled to stop the host vehicle 100, and the parking brake device 31 is operated to hold the host vehicle 100 in a stopped state, and the parking assistance control is terminated.

[0058] In this manner, the parking assistance device 10 automatically drives the host vehicle 100 and parks it in the designated parking section 200D.

[0059] The parking assist control executed by the parking assist device 10 has been described above using an example in which the host vehicle 100 is reversed and parked in the designated parking section 200D located to the left of the host vehicle 100, but the parking assist control can also reverse the host vehicle 100 to park in the designated parking section 200D located to the right of the host vehicle 100, or can move the host vehicle 100 forward to park in the designated parking section 200D located to the left of the host vehicle 100, or can move the host vehicle 100 forward to park in the designated parking section 200D located to the right of the host vehicle 100. Furthermore, the parking assist control can also reverse the host vehicle 100 to park in the designated parking section 200D located behind the host vehicle 100, or can move the host vehicle 100 forward to park in the designated parking section 200D located in front of the host vehicle 100. Furthermore, the parking assistance control can move the vehicle 100 forward to park in a designated parking space 200D located behind the vehicle 100, or can move the vehicle 100 backward to park in a designated parking space 200D located in front of the vehicle 100.

[0060] <Display of underfloor images, etc.> Meanwhile, while the parking assistance control is being executed, the parking assistance device 10 acquires a distance (remaining travel distance DIS) that is predicted to be traveled by the vehicle 100 before it reaches the target parking position Ptgt based on the surrounding detection information IS, and monitors whether the remaining travel distance DIS has been shortened to a predetermined distance DISth.

[0061] When the parking assistance device 10 determines that the remaining travel distance DIS has been shortened to a predetermined distance DISth, it starts image display control (underfloor image display control) for displaying a composite image IM showing the surroundings of the vehicle 100 on the display device 80 as shown in Fig. 12(A). Fig. 12 shows an example in which the remaining travel distance DIS has been shortened to the predetermined distance DISth when the vehicle 100 is backing up straight by the parking assistance control as shown in Fig. 6. In Figs. 12 and 13, the symbol IMD is an image showing a line indicating the designated parking section 200D.

[0062] The display device 80 is also configured to display the under-floor image IMF in an area (under-floor image display area ARF) having a predetermined area.

[0063] The composite image IM is an image generated by combining the underfloor image IMF, the surroundings image IMS, and the vehicle image IMV.

[0064] The surrounding image IMS is an image showing the state of the underfloor of the vehicle 100 and its surroundings, and the parking assistance device 10 generates the surrounding image IMS using image information IC acquired in real time and image information IC acquired in the past. The parking assistance device 10 stores image information IC acquired at a predetermined cycle for a certain period of time, and uses the stored past image information IC to generate the surrounding image IMS.

[0065] The vehicle image IMV is an image representing the outer shape of the body of the vehicle 100 and the wheels (tires) of the vehicle 100, and the parking assistance device 10 stores the vehicle image IMV in advance. The image representing the wheels (tires) of the vehicle 100 displayed as the vehicle image IMV is adjusted to form an angle corresponding to the steering angle θ at that time with respect to the image representing the outer shape of the body of the vehicle 100.

[0066] Moreover, the underfloor image IMF is an image showing the state of the underfloor of the host vehicle 100. The parking assistance device 10 generates the underfloor image IMF by using the stored past image information IC.

[0067] If the vehicle 100 has traveled a certain distance or more in a certain direction at the time when the parking assistance device 10 generates the under-floor image IMF, the amount of past image information IC stored by the parking assistance device 10 at that time is sufficient to generate an under-floor image IMF that fills the entire under-floor image display area ARF, so in this case, the parking assistance device 10 uses the past image information IC to generate an under-floor image IMF that fills the entire under-floor image display area ARF, and displays the under-floor image IMF on the display device 80. In this case, as shown in Fig. 12(A), the under-floor image IMF is displayed in the entire under-floor image display area ARF.

[0068] Then, thereafter, until the reference portion 100P of the vehicle 100 reaches the target parking position Ptgt, the parking assistance device 10 generates an under-floor image IMF that fills the entire under-floor image display area ARF using past image information IC at a predetermined period, and displays the under-floor image IMF on the display device 80. As a result, the under-floor images IMF are displayed in the under-floor image display area ARF in the order of (B) to (D) in FIG.

[0069] On the other hand, as described with reference to Figures 8 to 11, when the vehicle 100 has not traveled a certain distance in a certain direction at the time when the parking assistance device 10 generates the under-floor image IMF because the vehicle 100 has turned around, the amount of past image information IC stored by the parking assistance device 10 at that time is not sufficient to generate an under-floor image IMF that fills the entire under-floor image display area ARF, so the parking assistance device 10 cannot generate an under-floor image IMF that fills the entire under-floor image display area ARF, but in that case, the parking assistance device 10 generates an under-floor image IMF that can be generated using the past image information IC stored by the parking assistance device 10 at that time, and displays the under-floor image IMF on the display device 80. In this case, as shown in Figure 13 (A), the under-floor image IMF is displayed only in a part of the under-floor image display area ARF.

[0070] Then, the parking assistance device 10 generates an under-floor image IMF at a predetermined period until the reference portion 100P of the vehicle 100 reaches the target parking position Ptgt, and at this time, if the amount of past image information IC is sufficient to generate an under-floor image IMF that fills the entire under-floor image display area ARF, the parking assistance device 10 generates an under-floor image IMF that fills the entire under-floor image display area ARF, and if the amount of past image information IC is not sufficient to generate an under-floor image IMF that fills the entire under-floor image display area ARF, the parking assistance device 10 generates an under-floor image IMF that can be generated using the past image information IC, and displays the generated under-floor image IMF on the display device 80. As a result, the under-floor images IMF are displayed in the under-floor image display area ARF in the order of (B) to (D) in FIG.

[0071] According to this, even after the remaining traveling distance DIS becomes short to the predetermined distance DISth, if the vehicle 100 continues to travel in a certain direction, the amount of past image information IC increases, and the area of ​​the underfloor image IMF that the parking assistance device 10 can generate also becomes larger, and as a result, the area of ​​the underfloor image IMF displayed in the underfloor image display area ARF also becomes larger.

[0072] Incidentally, Figure 13 shows the under-floor image IMF displayed on the display device 80 when the amount of past image information IC is not sufficient to generate an under-floor image IMF that fills the entire under-floor image display area ARF.

[0073] <Effects> According to this, even when the vehicle 100 turns around and does not travel in a fixed direction for a fixed distance or more, and as a result the amount of past image information IC available for generating the under-floor image IMF is not sufficient to generate an under-floor image IMF that fills the entire under-floor image display area ARF, an under-floor image IMF that can be generated using the past image information IC available for generating the under-floor image IMF at that time is generated and displayed on the display device 80. Therefore, even if the amount of past image information IC available for generating the under-floor image IMF is small, the under-floor image IMF can be displayed on the display device 80.

[0074] <Specific operation of the parking assistance system> Next, a specific operation of the parking assistance device 10 will be described. The CPU of the ECU 90 of the parking assistance device 10 executes the routine shown in FIG. 14 at a predetermined calculation cycle. Therefore, at a predetermined timing, the CPU starts the process from step 1400 in FIG. 14, advances the process to step 1405, and determines whether the value of the vehicle running start flag Xstart is "0". The vehicle running start flag Xstart is a flag that indicates whether the vehicle running process by the parking assistance control is being executed or not, and its value is set to "1" when the vehicle running process is being executed, and is set to "0" when the vehicle running process is not being executed.

[0075] If the CPU determines "Yes" in step 1405, it proceeds to step 1410 and determines whether the value of the parking assist request flag Xrequest is "1." The parking assist request flag Xrequest is a flag indicating whether the execution of parking assist control has been requested by the parking assist request operation unit 49, and its value is set to "1" when the execution of parking assist control has been requested, and is set to "0" when the parking assist control has been terminated.

[0076] If the CPU determines "Yes" in step 1410, it proceeds to step 1415 and determines whether the value of the parking space display flag Xdisplay is "0." The parking space display flag Xdisplay is a flag that indicates whether an image (parking space image) representing the parking space 200C is displayed on the display device 80, and its value is set to "1" when the parking space image is displayed on the display device 80, and is set to "0" when the parking space image is not displayed on the display device 80.

[0077] If the CPU determines "Yes" in step 1415, the process proceeds to step 1420, where it displays the available parking space image on the display device 80. Next, the CPU proceeds to step 1425, where it sets the value of the available parking space display flag Xdisplay to "1." Next, the CPU proceeds to step 1430.

[0078] On the other hand, if the CPU determines “No” in step 1415 , it proceeds directly to step 1430 .

[0079] When the CPU proceeds to step 1430, it determines whether or not the driver DR has performed an operation (parking space setting operation) to designate or set the parking space 200C (or any of the multiple parking spaces 200C) indicated by the parking space image displayed on the display device 80 as the space in which to park the vehicle 100.

[0080] If the CPU determines "Yes" in step 1430, the process proceeds to step 1435, where the CPU sets the section designated or set by the driver DR as the designated parking section 200D. Next, the CPU proceeds to step 1440, where the CPU sets the target driving route Rtgt. Next, the CPU proceeds to step 1445, where the CPU ends the display of the available parking section image on the display device 80. Next, the CPU proceeds to step 1450, where the CPU sets the value of the available parking section display flag Xdisplay to "0". Next, the CPU proceeds to step 1455, where the CPU sets the value of the vehicle driving start flag Xstart to "1". Next, the CPU proceeds to step 1495, where the CPU temporarily ends this routine.

[0081] On the other hand, if the CPU determines "No" in step 1430, it proceeds directly to step 1495 and temporarily ends this routine.

[0082] Also, if the CPU makes a "No" determination in step 1405 or step 1410, the CPU proceeds directly to step 1495 and temporarily ends this routine.

[0083] Furthermore, the CPU executes the routine shown in Fig. 15 at a predetermined calculation cycle. Therefore, at a predetermined timing, the CPU starts the process from step 1500 in Fig. 15, and proceeds to step 1505 to determine whether the value of the vehicle travel start flag Xstart is "1".

[0084] If the CPU judges "Yes" in step 1505, the process proceeds to step 1510, where it judges whether or not an obstacle that would be an impediment to the traveling of the host vehicle 100 has been detected.

[0085] If the CPU determines "Yes" in step 1510, the process proceeds to step 1515, where it executes emergency stop processing to temporarily stop the host vehicle 100 by controlling the operation of the braking device 22. Next, the CPU proceeds to step 1520, where it resets the target driving route Rtgt based on the surrounding detection information IS. Next, the CPU proceeds to step 1525.

[0086] On the other hand, if the CPU determines “No” in step 1510 , it proceeds directly to step 1525 .

[0087] The CPU executes vehicle driving processing when the process proceeds to step 1525. Next, the CPU proceeds to step 1530 and determines whether or not the reference portion 100P of the host vehicle 100 has reached the target parking position Ptgt.

[0088] If the CPU determines "Yes" in step 1530, the process proceeds to step 1535, where it controls the operation of the braking device 22 to stop the host vehicle 100, and ends the vehicle travel process. Next, the CPU proceeds to step 1540, where it executes a stop process that operates the parking brake device 31 to hold the host vehicle 100 in a stopped state. Next, the CPU proceeds to step 1545, and ends the parking assistance control. Next, the CPU proceeds to step 1550, where it sets the values ​​of the vehicle travel start flag Xstart and the parking assistance request flag Xrequest to "0". Next, the CPU proceeds to step 1595, and ends this routine.

[0089] On the other hand, if the CPU determines "No" in step 1530, it proceeds directly to step 1595 and temporarily ends this routine.

[0090] Also, if the CPU determines "No" in step 1505, it proceeds directly to step 1595 and temporarily ends this routine.

[0091] Furthermore, the CPU executes the routine shown in Fig. 16 at a predetermined calculation cycle. Therefore, at a predetermined timing, the CPU starts the process from step 1600 in Fig. 16, and proceeds to step 1605 to determine whether the value of the vehicle travel start flag Xstart is "1".

[0092] If the CPU judges "Yes" in step 1605, it judges whether the remaining travel distance DIS is equal to or less than a predetermined distance DISth.

[0093] If the CPU determines "Yes" in step 1610, it generates a composite image IM including the underfloor image IMF as described above, and displays the composite image IM on the display device 80. Next, the CPU proceeds to step 1620, where it determines whether or not the reference portion 100P of the host vehicle 100 has reached the target parking position Ptgt.

[0094] If the CPU determines "Yes" in step 1620, the process proceeds to step 1625, where it ends the display of the composite image IM on the display device 80. Next, the CPU proceeds to step 1695, where it temporarily ends this routine.

[0095] On the other hand, if the CPU determines "No" in step 1620, it proceeds directly to step 1695 and temporarily ends this routine.

[0096] Also, if the CPU determines "No" in step 1605 or step 1610, it advances the process directly to step 1605 and temporarily ends this routine.

[0097] The specific operation of the parking assistance device 10 has been described above.

[0098] The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the present invention. [Explanation of symbols]

[0099] 10... parking assistance device, 20... vehicle driving device, 21... drive device, 22... braking device, 23... steering device, 24... transmission, 70... surrounding information detection device, 71... radio wave sensor, 72... image sensor, 80... display device, 90... ECU, 100... host vehicle, 200D... designated parking space, IMF... underfloor image

Claims

[Claim 1] a control device that executes parking assistance control for automatically driving the host vehicle and parking the vehicle in a designated parking space, and executes underfloor image display control for displaying an underfloor image representing the underfloor of the host vehicle on a display device of the host vehicle; an image sensor that captures an image of the surroundings of the host vehicle and provides image data for generating the underfloor image; Equipped with The control device is configured to start the underfloor image display control when the host vehicle reaches a point that is a predetermined distance from a position where parking of the host vehicle in the designated parking space by the parking assist control is completed. In the parking assistance device, the display device has an area of ​​a predetermined size as an under-floor image display area for displaying the under-floor image by the under-floor image display control, The control device includes: During execution of the under-floor image display control, when the image data capable of generating the under-floor image that fills the entire under-floor image display area is provided, the under-floor image that fills the entire under-floor image display area is generated, and the under-floor image is displayed on the display device; During execution of the under-floor image display control, if the image data capable of generating the under-floor image that fills the entire under-floor image display area is not provided, the under-floor image that can be generated from the provided image data is generated, and the under-floor image is displayed on the display device. It is configured as follows: Parking assistance device.

Images