Parking support device

Abstract

To provide a parking support apparatus capable of reducing a possibility of being unable to park a vehicle at a registered parking position which has been registered in advance, through automatic parking control.SOLUTION: When an automatic parking registration method is selected, a vehicle control ECU 10 executes automatic parking control and stores, as a registered parking position in a non-volatile memory, a position determined on the basis of a position of a vehicle at a point in time when the vehicle is set to a parking state by the automatic parking control. When an execution condition of post-registration automatic parking control is satisfied, the vehicle control ECU 10 executes the automatic parking control after setting, as a target parking position, the stored registered parking position.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a parking assistance device that controls a vehicle so that the vehicle automatically moves to a pre-registered parking position and stops.

Background Art

[0002] The parking assistance device disclosed in Patent Document 1 (hereinafter referred to as the "conventional device") can register (store) the position of a parking space not demarcated by parking frame lines as a pre-registered parking position. According to the conventional device, in order to register the position of the parking space, the driver himself / herself drives the vehicle to the parking space where registration is planned. Further, when the position of the parking space is registered as a pre-registered parking position, the conventional device can automatically move the vehicle to the pre-registered parking place and park the vehicle by executing automatic parking control. At this time, the conventional device determines a target route for moving the vehicle to the pre-registered parking position and moves the vehicle along the target route.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] As described above, when registering the position of the parking space in the conventional device, the driver himself / herself needs to drive the vehicle to the parking space. Therefore, when there is a place where the vehicle is difficult to travel (for example, a sidewalk step, a curb, a pole, etc.) between the position of the vehicle at the time of starting the driving for parking and the parking space to be registered, the driver moves the vehicle so as to avoid that place.

[0005] However, when a parking space where the vehicle is moved by the driver himself / herself is registered as a registered parking position, when the vehicle is parked at the registered parking place by automatic parking control, the generated target path may include "a place where it is difficult for the vehicle to travel". Such a situation may occur, for example, when the imaging device cannot capture "a place where it is difficult for the vehicle to travel" at the point where the automatic parking control is started, or when it is difficult to determine whether the place is a place where the vehicle has difficulty traveling. In this case, the conventional device cannot move the vehicle along the target path. That is, according to the conventional device, there is a possibility that the vehicle cannot be moved to the position of the parking space (registered parking position) by automatic parking control even though the position of the parking space has been registered.

[0006] The present invention has been made to address the above-described problems. That is, one of the objects of the present invention is to provide a parking support device (hereinafter, also referred to as "the device of the present invention") that can reduce the possibility that the vehicle cannot be parked at a pre-registered registered parking position by automatic parking control.

[0007] The device of the present invention One aspect of 、 an imaging device (21) that acquires image data by imaging the surroundings of a vehicle (SV); an instruction device (22) configured to be able to generate an instruction by a driver of the vehicle; a control device (10) capable of executing automatic parking control to automatically move the vehicle to a predetermined target parking position based on the image data and then set it to a parked state; a storage device (10) capable of storing a predetermined parking position as a registered parking position in advance; and includes The control device when receiving a first instruction from the instruction device (determined as "Yes" in step 1110), executes the automatic parking control (step 1115) and stores, in the storage device, as the registered parking position, a position determined based on the position of the vehicle at the time when the vehicle is set to the parked state by the automatic parking control (step 1135); When receiving the second instruction from the instruction device (determined as "Yes" in step 1325), it is configured to execute the automatic parking control (step 1330) after setting the stored registered parking position as the target parking position.

[0008] In the above aspect According to this, when storing the registered parking position in the storage device, while executing the automatic parking control, the position determined based on the position of the vehicle at the time when the vehicle is set to the parked state by the automatic parking control is stored in the storage device as the registered parking position. Thereby, when executing the automatic parking control after setting the stored registered parking position as the target parking position, the vehicle is highly likely to be able to move to the registered parking position. As a result, the present embodiment device can reduce the possibility that the vehicle cannot be parked at the pre-registered registered parking position.

[0009] In one aspect of the device of the present invention, the control device When receiving the third instruction from the instruction device (determined as "Yes" in step 1210), it is configured to recognize the position of the vehicle at the time when the vehicle is set to the parked state after being moved to a predetermined position by the driving of the driver as a reference position, and store the position determined based on the reference position in the storage device as the registered parking position (step 1230).

[0010] According to one aspect of the device of the present invention, the registered parking position can also be stored in the storage device by the movement of the vehicle by the driving of the driver.

[0011] In one aspect of the device of the present invention, the control device When receiving the third instruction, it is configured to recognize the position of the vehicle at the time when the position of the shift lever (72) of the vehicle is changed from a position other than the parking position to the parking position as the reference position (step 1220).

[0012] According to one aspect of the device of the present invention, when the parking position is registered in the storage device by the movement of the vehicle caused by the driver's driving, the parking position can be easily registered without requiring the driver to perform complicated operations.

[0013] In one aspect of the device of the present invention, the control device when receiving the first instruction, acquires feature point information including position information of a three-dimensional object existing on the road surface around the vehicle based on the image data acquired during the period from the start time to the end time of the automatic parking control, and stores the feature point information in the storage device in association with the registered parking position (step 1135), when receiving the second instruction, acquires the feature point information as actual feature point information based on the image data, and is configured to execute the automatic parking control based on the feature point information stored in the storage device and the actual feature point information (step 1330).

[0014] According to one aspect of the device of the present invention, when receiving the second instruction, even if the registered parking position is a parking space without a frame line, the automatic parking control can be executed.

[0015] One aspect of the device of the present invention is the control device is configured to receive the second instruction when it is determined based on the image data that the vehicle is located around the registered parking position stored in the storage device (step 1320).

[0016] According to one aspect of the device of the present invention, the driver can be made aware that the vehicle is in a position where it can park at the registered parking position.

[0017] In one aspect of the present invention, the indicating device includes a touch panel (22) capable of displaying an image, the control device After receiving the first instruction, an image (G5) including a surrounding image representing the periphery of the vehicle at the time when the vehicle is set to the parking state by the automatic parking control and a planned registration position display frame representing the parking position of the vehicle, a registration execution button (G5a3), and a position operation button (G5a2) are displayed on the touch panel (step 1125). When the position operation button is operated, the position in the surrounding image of the planned registration position display frame is changed according to the operation (steps 1125 and 1130). When the registration execution button is operated (determined as "Yes" in step 1130), the parking position corresponding to the position of the planned registration position display frame in the surrounding image is stored in the storage device as the registered parking position (step 1135). It is configured as such.

[0018] According to one aspect of the device of the present invention, an operation by the user for finely adjusting the position to be stored as the registered parking position of the vehicle in the storage device can be made an intuitive operation.

[0019] In the above description, in order to assist the understanding of the present invention, the names and / or symbols used in the embodiments are added in parentheses to the configuration of the invention corresponding to the embodiments described later. However, each component of the present invention is not limited to the embodiments defined by the above names and / or symbols.

Brief Description of the Drawings

[0020]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

[0021] <Configuration> The parking support device according to an embodiment of the present invention (hereinafter referred to as "the present device") is applied to a vehicle SV (see FIG. 2). As shown in FIG. 1, the present device includes a vehicle control ECU 10, a PVM (Panoramic View Monitor)-ECU 20, an engine ECU 30, a brake ECU 40, an EPS·ECU 50, a meter ECU 60, and an SBW (Shift-by-Wire)·ECU 70. In the following, the vehicle control ECU 10 is also simply referred to as "VC (Vehicle Control) ECU".

[0022] Each ECU includes a microcomputer. The microcomputer includes a CPU, a ROM, a RAM, a readable and writable non-volatile memory, an interface, etc. The CPU realizes various functions by executing instructions (programs, routines) stored in the ROM. Furthermore, these ECUs are connected to each other so that data can be exchanged (communicated) via a CAN (Controller Area Network). Therefore, the detection values of sensors (including switches) connected to a specific ECU are also transmitted to other ECUs.

[0023] A plurality of radar sensors 11a to 11e, a plurality of first ultrasonic sensors 12a to 12d, a plurality of second ultrasonic sensors 13a to 13h, a parking assist switch 14, and a vehicle speed sensor 15 are connected to the VC ECU.

[0024] Note that when it is not necessary to distinguish the plurality of radar sensors 11a to 11e, they are referred to as "radar sensor 11". When it is not necessary to distinguish the plurality of first ultrasonic sensors 12a to 12d, they are referred to as "first ultrasonic sensor 12". When it is not necessary to distinguish the plurality of second ultrasonic sensors 13a to 13h, they are referred to as "second ultrasonic sensor 13".

[0025] The radar sensor 11 is a well-known sensor that uses millimeter-wave radio waves (hereinafter referred to as "millimeter waves"). The radar sensor 11 acquires target information such as the distance between the vehicle SV and a three-dimensional object, the relative speed between the vehicle SV and the three-dimensional object, and the relative position (direction) of the three-dimensional object with respect to the vehicle SV, and outputs the target information to the VC ECU.

[0026] The radar sensor 11 (11a to 11e) is disposed at a predetermined position of the vehicle SV shown in FIG. 2, and acquires target information of three-dimensional objects existing in a predetermined region described below.

[0027] The radar sensor 11a acquires target information of three-dimensional objects existing in the right front region of the vehicle SV. The radar sensor 11b acquires target information of a solid object existing in the front area of the vehicle SV. The radar sensor 11c acquires target information of a solid object existing in the left front area of the vehicle SV. The radar sensor 11d acquires target information of a solid object existing in the right rear area of the vehicle SV. The radar sensor 11e acquires target information of a solid object existing in the left rear area of the vehicle SV.

[0028] Each of the first ultrasonic sensor 12 and the second ultrasonic sensor 13 is a well-known sensor that uses ultrasonic waves. When it is not necessary to distinguish between the first ultrasonic sensor 12 and the second ultrasonic sensor 13, they are collectively referred to as the "ultrasonic sensor".

[0029] The ultrasonic sensor transmits ultrasonic waves to a predetermined range, receives the reflected waves reflected by the solid object, and detects the presence or absence of the solid object and the distance to the solid object based on the time from the transmission to the reception of the ultrasonic waves. The first ultrasonic sensor 12 is used to detect a solid object at a relatively far position from the vehicle SV compared to the second ultrasonic sensor 13. The ultrasonic sensor is disposed at a predetermined position of the vehicle body 200 shown in FIG. 2.

[0030] The first ultrasonic sensor 12 (the first ultrasonic sensors 12a to 12c) acquires the distance between the solid object existing in the predetermined area (detection area) described below and the first ultrasonic sensor 12, and transmits information regarding the distance to the VCECU.

[0031] The detection area of the first ultrasonic sensor 12a is the area in the front part and on the right side of the vehicle SV. The detection area of the first ultrasonic sensor 12b is the area in the front part and on the left side of the vehicle SV. The detection area of the first ultrasonic sensor 12c is the area in the rear part and on the right side of the vehicle SV. The detection area of the first ultrasonic sensor 12d is the area in the rear part and on the left side of the vehicle SV.

[0032] The second ultrasonic sensor 13 (second ultrasonic sensors 13a to 13h) acquires the distance between a three-dimensional object existing in a predetermined region (detection region) described below and the second ultrasonic sensor 13, and transmits information regarding the distance to the VCECU.

[0033] The detection regions of the second ultrasonic sensors 13a to 13d are regions in front of the vehicle SV. The detection regions of the second ultrasonic sensors 13e to 13h are regions behind the vehicle SV.

[0034] The parking assist switch 14 is a switch operated (pressed) by the user.

[0035] The vehicle speed sensor 15 detects the vehicle speed of the vehicle SV and outputs a signal representing the vehicle speed. Strictly speaking, the vehicle speed sensor 15 is a wheel speed sensor provided for each of the four wheels of the vehicle SV. The VCECU acquires the vehicle speed indicating the speed of the vehicle SV based on the wheel speeds of the respective wheels detected by the vehicle speed sensor 15 (wheel speed sensor).

[0036] A front camera 21a, a back camera 21b, a right side camera 21c, and a left side camera 21d are connected to the PVM-ECU 20. Hereinafter, when it is not necessary to distinguish between the front camera 21a, the back camera 21b, the right side camera 21c, and the left side camera 21d, they are collectively referred to as "camera 21 (also referred to as an "imaging device")."

[0037] As shown in FIG. 2, the front camera 21a is provided at a substantially central portion in the vehicle width direction of the front bumper FB. The optical axis of the front camera 21a faces forward of the vehicle SV. The back camera 21b is provided on the wall portion of the rear trunk RT at the rear of the vehicle SV. The optical axis of the back camera 21b faces rearward of the vehicle SV. The right side camera 21c is provided on the right side door mirror DMR. The optical axis of the right side camera 21c faces right sideward of the vehicle SV. The left side camera 21d is provided on the left side door mirror DML. The optical axis of the left side camera 21d is directed toward the left side of the vehicle SV.

[0038] The angle of view of the camera 21 is wide. Therefore, the imaging range of the camera 21 includes the ranges of "diagonally right, diagonally left, diagonally downward, and diagonally upward" with respect to the optical axis. The entire periphery of the vehicle SV is included in the imaging ranges of the four cameras 21.

[0039] Every time a predetermined time elapses, the camera 21 acquires image information (image data) by imaging the area around the vehicle SV corresponding to the imaging range. The camera 21 transmits the acquired image data to the PVM-ECU 20 and the VC ECU.

[0040] Specifically, the front camera 21a images the "peripheral area in front of the vehicle SV" corresponding to its imaging range. The front camera 21a acquires the image data (hereinafter referred to as "front image data") obtained by the imaging, and transmits the acquired image data to the PVM-ECU 20. The back camera 21b images the "peripheral area behind the vehicle SV" corresponding to its imaging range. The back camera 21b transmits the image data (hereinafter referred to as "back image data") obtained by the imaging to the PVM-ECU 20. The right side camera 21c images the "peripheral area on the right side of the vehicle SV" corresponding to its imaging range. The right side camera 21c transmits the image data (hereinafter referred to as "right side image data") obtained by the imaging to the PVM-ECU 20. The left side camera 21d images the "peripheral area on the left side of the vehicle SV" corresponding to its imaging range. The left side camera 21d transmits the image data (hereinafter referred to as "left side image data") obtained by the imaging to the PVM-ECU 20.

[0041] Every time a predetermined time elapses, the PVM-ECU 20 generates peripheral image data using the front image data, rear image data, right-side image data, and left-side image data. An image displayed (generated) based on the peripheral image data is called a peripheral image. The peripheral image is an image corresponding to at least a partial range of the area around the vehicle SV, and includes a camera viewpoint image, a composite image, and the like. The camera viewpoint image is an image with the arrangement position of each lens of the camera 21 as the viewpoint. One of the composite images is an image of looking around the vehicle SV from a virtual viewpoint set at an arbitrary position around the vehicle SV (also called a "virtual viewpoint image").

[0042] The method for generating this virtual viewpoint image is well known (see, for example, Japanese Patent Application Laid-Open Nos. 2012-217000, 2016-192772, and 2018-107754, etc.). Note that the PVM-ECU 20 may further generate an image in which a vehicle image (for example, a polygon indicating the shape of the vehicle) and a line supporting the parking operation are synthesized (superimposed) for each of the camera viewpoint image and the virtual viewpoint image. Such an image is also called a peripheral image.

[0043] To briefly explain the outline of the method for generating virtual viewpoint image data that forms the basis of the virtual viewpoint image, the PVM-ECU 20 projects the pixels included in the front image data, rear image data, right-side image data, and left-side image data onto a predetermined projection surface (for example, a bowl-shaped surface) in a virtual three-dimensional space.

[0044] The center of the projection surface is defined as the position of the vehicle SV. The portion other than the center of the projection surface corresponds to the front image data, rear image data, right-side image data, and left-side image data. The PVM-ECU 20 projects the information of the pixels included in the front image data, rear image data, right-side image data, and left-side image data onto the portion other than the center of the projection surface.

[0045] The PVM-ECU 20 arranges a polygon representing the shape of the vehicle SV at the center of the projection surface. Then, the PVM-ECU 20 sets a virtual viewpoint in a virtual three-dimensional space, and cuts out, as image data, a predetermined area of the projection surface included within a predetermined viewing angle as seen from the virtual viewpoint. Further, the "polygon representing the shape of the vehicle" included within the predetermined viewing angle as seen from the virtual viewpoint is superimposed on the cut-out image data. Thereby, virtual viewpoint image data is generated.

[0046] A touch panel display unit 22 is further connected to the PVM-ECU 20. The touch panel display unit 22 is a touch panel type display included in a navigation device (not shown). The PVM-ECU 20 displays a peripheral image on the touch panel display unit 22 in response to a command transmitted from the VCECU.

[0047] When the VCECU executes parking support control and parking position registration, the PVM-ECU 20 displays a parking support image (operation image) including a peripheral image on the touch panel display unit 22 in response to a command transmitted from the VCECU.

[0048] The PVM-ECU 20 can detect (recognize) feature points included in the peripheral image and store, in a non-volatile memory, information including the position information (three-dimensional position) etc. of the feature points (also referred to as "feature point information") in association with ID information for identifying the feature points, by performing image analysis processing on the peripheral image generated every time a predetermined time elapses. Further, the PVM-ECU 20 transmits the feature point information to the VCECU every time a predetermined time elapses.

[0049] The engine ECU 30 is connected to the engine actuator 31. The engine actuator 31 includes a throttle valve actuator that changes the opening degree of the throttle valve of the engine (spark ignition, fuel injection type, internal combustion engine) 32. By driving the engine actuator 31, the engine ECU 30 can change the torque generated by the engine 32. The torque generated by the engine 32 is transmitted to the drive wheels via a transmission (not shown).

[0050] Therefore, the engine ECU 30 can control the driving force of the vehicle SV by controlling the engine actuator 31. The VC ECU can send a driving command to the engine ECU 30. When the engine ECU 30 receives the driving command, it controls the engine actuator 31 according to the driving command. Therefore, the VC ECU can execute the "automatic driving force control" described later via the engine ECU 30. When the vehicle SV is a hybrid vehicle, the engine ECU 30 can control the driving force of the vehicle SV generated by either one or both of the "engine and electric motor" as the vehicle drive source. Further, when the vehicle SV is an electric vehicle, the engine ECU 30 can control the driving force of the vehicle SV generated by the electric motor as the vehicle drive source.

[0051] The brake ECU 40 is connected to the brake actuator 41. The brake actuator 41 is provided in a hydraulic circuit between a master cylinder (not shown) that pressurizes hydraulic oil by the depressing force of the brake pedal and a friction brake mechanism 42 provided on the left and right front and rear wheels. The friction brake mechanism 42 includes a brake disk 42a fixed to the wheel and a brake caliper 42b fixed to the vehicle body.

[0052] The brake actuator 41 adjusts the hydraulic pressure supplied to the wheel cylinder built in the brake caliper 42b according to an instruction from the brake ECU 40, and generates frictional braking force by operating the wheel cylinder with the hydraulic pressure to press the brake pad against the brake disk 42a. Therefore, the brake ECU 40 can control the braking force of the vehicle SV by controlling the brake actuator 41. The VC ECU can transmit a braking command to the brake ECU 40. When the brake ECU 40 receives the braking command, it controls the brake actuator 41 according to the braking command. Therefore, the VC ECU can execute "automatic braking force control" described later via the brake ECU 40.

[0053] The EPS·ECU 50 is a control device for a well-known electric power steering system and is connected to the motor driver 51. The motor driver 51 is connected to the steering motor 52. The steering motor 52 is incorporated in a "steering mechanism including a steering handle SW, a steering shaft US, and a steering gear mechanism (not shown)". The steering motor 52 can generate torque by the electric power supplied from the motor driver 51, and can generate steering assist torque or steer the left and right steering wheels with this torque. That is, the steering motor 52 can change the steering angle of the vehicle SV (also referred to as "steering angle" or "rudder angle").

[0054] Furthermore, the EPS·ECU 50 is connected to a steering angle sensor 53 and a steering torque sensor 54. The steering angle sensor 53 detects the steering angle of the steering handle SW of the vehicle SV and outputs a signal representing the steering angle. The steering torque sensor 54 detects the steering torque applied to the steering shaft SF of the vehicle SV by operating the steering handle SW and outputs a signal representing the steering torque.

[0055] The EPS·ECU 50 detects the steering torque input by the driver to the steering wheel SW by the steering torque sensor 54, and drives the steering motor 52 based on this steering torque. The EPS·ECU 50 applies a steering torque (steering assist torque) to the steering mechanism by driving this steering motor 52, and thereby can assist the driver's steering operation.

[0056] The VC ECU can transmit a steering command to the EPS·ECU 50. When the EPS·ECU 50 receives the steering command, it drives the steering motor 52 based on the received steering command. Therefore, the VC ECU can automatically change the steering angle of the steered wheels of the vehicle SV (i.e., without requiring a steering operation by the driver) via the EPS·ECU 50. That is, the VC ECU can execute the "automatic steering angle control" described later via the EPS·ECU 50.

[0057] The meter ECU 60 is connected to the display 61. The display 61 is a multi-information display provided in front of the driver's seat. In addition to displaying measured values such as vehicle speed and engine rotation speed, the display 61 also displays various types of information.

[0058] The SBW·ECU 70 is connected to the shift position sensor 71. The shift position sensor 71 detects the position of the shift lever 72 as a movable part of the shift operation unit. In this example, the positions of the shift lever 72 are the parking position (P), the forward position (D), and the reverse position (R). The SBW·ECU 70 receives the position of the shift lever 72 from the shift position sensor 71, and controls the transmission and / or the drive direction switching mechanism (not shown) of the vehicle SV based on this position (i.e., performs the shift control of the vehicle SV).

[0059] More specifically, when the position of the shift lever 72 is "P", the SBW·ECU 70 controls the transmission and / or the drive direction switching mechanism so that no driving force is transmitted to the drive wheels and the vehicle SV is mechanically locked in the stop position. When the position of the shift lever 72 is "D", the SBW·ECU 70 controls the transmission and / or the drive direction switching mechanism so that a driving force for moving the vehicle SV forward is transmitted to the drive wheels. Further, when the position of the shift lever 72 is "R", the SBW·ECU 70 controls the transmission and / or the drive direction switching mechanism so that a driving force for moving the vehicle SV backward is transmitted to the drive wheels.

[0060] The VC ECU can send a shift command to the SBW·ECU 70. When receiving the shift command, the SBW·ECU 70 controls the transmission and / or the drive direction switching mechanism according to the shift command, without relying on the operation of the driver's shift lever 72, and can switch the position of the shift lever 72. The control of the transmission and the drive direction switching mechanism based on the shift command sent from this VC ECU is called shift position automatic control.

[0061] <Operation> (Registration of Parking Position) The user of the vehicle SV can register in advance with the VC ECU the "position where the user plans to park the vehicle SV (i.e., the planned parking position)" as the registered parking position. As shown in the "upper part above the dashed line Ln" in FIG. 3, the registered parking position may be a parking space PS without a border line, such as a parking lot at the home MH, or a parking space with a border line.

[0062] In FIGS. 3 to 9, the figures shown in the portion above the dashed line Ln are referred to as "upper figures", and the figures shown in the portion below the dashed line Ln are referred to as "lower figures". Each upper figure in FIGS. 3 to 9 shows the positional relationship among the vehicle SV, the home MH, and the parking space PS. Each lower figure in FIGS. 3 to 9 shows the image displayed on the touch panel display unit 22 when the vehicle SV, the home MH, and the parking space PS are in the positional relationship shown in the upper figure of each figure. Further, within the frame line W1 of each upper figure in FIGS. 3 to 9, there is a description of the operation of the vehicle SV or the present implementation device. Within the frame line W2 of each lower figure in FIGS. 3 to 9, there is a description of the operation (operation) of the user U1.

[0063] In the present implementation device, it is premised that the vehicle SV is moved while reversing to the parking position. Therefore, when the user parks the vehicle SV, the user can stop the vehicle SV at a position where it can be moved while reversing it to the "position where parking is desired", and in that state, operate the parking assistance switch 14.

[0064] For example, as shown in the upper figure of FIG. 3, when the user parks the vehicle SV in the parking space PS of the home MH, the user stops the vehicle SV at position P1. Next, the user operates the parking assistance switch 14 in that state. As a result, the VCECU displays two buttons (not shown) on the touch panel display unit 22. One of these two buttons is a switch (parking request switch) that is operated when parking is desired, and the other one is a switch (parking rejection switch) that is operated when parking is not desired. Note that the buttons displayed on the touch panel display unit 22 are operated by the user touching the buttons. Hereinafter, this operation is referred to as a touch operation. By the touch operation of the buttons displayed on the touch panel display unit 22, an operation signal corresponding to the touch operation is generated, and the VCECU receives the generated operation signal.

[0065] When the user touches and operates the aforementioned parking request switch (not shown), the VCECU displays the display image G1 shown in the lower figure of FIG. 3 on the touch panel display unit 22. The display image G1 is divided into a left region (the region to the left of the imaginary line In) and a right region (the region to the right of the imaginary line In). The left region of the display image G1 includes the camera viewpoint image G1c and the display button G1a1 for registering the home parking position. The right region of the display image G1 includes the bird's-eye view image G1h.

[0066] The camera viewpoint image G1c is a camera viewpoint image with the arrangement position of the lens of the camera 21 that captures the scenery in the planned moving direction of the vehicle SV as the viewpoint. The camera viewpoint image G1c in FIG. 3 is the camera viewpoint image of the back camera 21b. When the registered parking position is not registered, a frame indicating the candidate parking position (i.e., the candidate parking space) PS based on the current position of the vehicle SV is superimposed and displayed in this camera viewpoint image. The display button G1a1 for registering the home parking position is a button that is touched and operated by the user U1 to cause the VCECU to start the registration operation of the parking position.

[0067] In addition, when the planned parking position has already been registered as the registered parking position and the registered parking position is detected in the available parking space, a frame indicating the registered parking position PS is superimposed and displayed in this camera viewpoint image. Further, an "execution button for automatic parking control after registration" is additionally displayed within the display image G1.

[0068] The bird's-eye view image G1h is a peripheral image in which a polygon SP corresponding to the vehicle SV is superimposed on a virtual viewpoint image obtained by cutting out, as an image, the region on the projection surface included in a predetermined viewing angle when viewing the projection surface from a virtual viewpoint set directly above the vehicle SV.

[0069] When user U1 touches the display button G1a1 for registering the home parking position, the VCECU displays the display image G2 shown in the lower figure of Fig. 4 on the touch panel display unit 22. The display image G2 includes four parking method selection buttons (i.e., the first parking method selection button G2a1, the second parking method selection button G2a2, the third parking method selection button G2a3, and the fourth parking method selection button G2a4). These parking method selection buttons are buttons that user U1 touches to select a parking method.

[0070] When user U1 touches any one of the first to fourth parking method selection buttons (G2a1 to G2a4), the VCECU determines (identifies) the parking method corresponding to the touched parking method selection button as the parking method of vehicle SV.

[0071] Specifically, when the first parking method selection button G2a1 is touched, the VCECU determines the parking method by parallel parking on the left as the parking method of vehicle SV. The parking method by parallel parking on the left is a parking method in which vehicle SV is parked in parallel in the parking space PS existing on the left side of vehicle SV.

[0072] When the second parking method selection button G2a2 is touched, the VCECU determines the parking method by parallel parking on the right as the parking method of vehicle SV. The parking method by parallel parking on the right is a parking method in which vehicle SV is parked in parallel in the parking space PS existing on the right side of vehicle SV.

[0073] When the third parking method selection button G2a3 is touched, the VCECU determines the parking method by perpendicular parking on the left as the parking method of vehicle SV. The parking method by perpendicular parking on the left is a parking method in which vehicle SV is parked perpendicularly in the parking space PS existing on the left side of vehicle SV.

[0074] When the fourth parking method selection button G2a4 is touched, the VCECU determines the parking method by perpendicular parking on the right as the parking method of vehicle SV. The parking method by perpendicular parking on the right is a parking method in which vehicle SV is parked perpendicularly in the parking space PS existing on the right side of vehicle SV.

[0075] Now, assume that the first parking method selection button G2a1 is touched by the user U1. In this case, the VCECU determines the parking method by parallel parking on the left as the parking method of the vehicle SV. Further, the VCECU displays the display image G3 shown in the lower figure of FIG. 5 on the touch panel display unit 22.

[0076] The display image G3 includes a registration button G3a1 for the automatic parking position and a registration button G3a2 for the driver parking position. These registration buttons are buttons that the user U1 touches to select the "parking position to be registered in the VCECU" and are buttons for determining the registration method. In other words, these registration buttons are buttons that are touched to determine whether to register the position parked by any parking method (either the parking method by automatic parking control or the parking method by manual driving by the driver) in the VSECU. Hereinafter, the "parking position to be registered in the VCECU" may be referred to as the "parking position to be registered".

[0077] When the user U1 touches either the registration button G3a1 or the registration button G3a2, the VCECU determines the parking position determined by the parking method corresponding to the touched registration button as the parking position to be registered. Specifically, when the registration button G3a1 for the automatic parking position is touched, the VCECU determines the position where the vehicle SV is parked by automatic parking control as the "parking position to be registered". In other words, when the registration button G3a1 for the automatic parking position is touched, the VCECU selects the automatic parking registration method. When the registration button G3a2 for the driver parking position is touched, the VCECU determines the position where the vehicle SV is parked by manual driving as the parking position to be registered. In other words, when the registration button G3a2 for the driver parking position is touched, the VCECU selects the manual parking registration method.

[0078] Hereinafter, each case will be described in order: when the button G3a1 for registering the automatic parking position is touch-operated, and when the button G3a2 for registering the driver's parking position is touch-operated.

[0079] 1: When the button G3a1 for registering the automatic parking position is touch-operated In this case, the VCECU displays the display image G4 shown in the lower figure of Fig. 6 on the touch panel display unit 22. The display image G4 includes an aerial view image G4h in its left region. A planned registration position display frame G4a1 is superimposed on this aerial view image G4h. The display image G4 includes "position operation buttons G4a2 and parking start instruction button G4a3" in its right region. The position operation buttons G4a2 include six arrow buttons: an upward arrow, a downward arrow, a leftward arrow, a rightward arrow, a clockwise arrow, and a counterclockwise arrow.

[0080] The planned registration position display frame G4a1 indicates the "parking position to be registered". The position operation buttons G4a2 are operated by the user U1 to move the position in the aerial view image G4h of the planned registration position display frame G4a1.

[0081] When one of the upward arrow, downward arrow, leftward arrow, and rightward arrow is touch-operated only once, the planned registration position display frame G4a1 moves a predetermined distance in the direction of the touch-operated arrow in the aerial view image G4h. When one of the clockwise arrow and counterclockwise arrow is touch-operated, the planned registration position display frame G4a1 rotates around the center point of the planned registration position display frame G4a1 by a predetermined angle in the rotation direction of the touch-operated arrow in the aerial view image G4h.

[0082] The parking start instruction button G4a3 tentatively determines the position indicated by the planned registration position display frame G4a1 as the "parking position to be registered", and is a button that is touch-operated to automatically start moving the vehicle SV to the position indicated by the planned registration position display frame G4a1 (that is, to start executing automatic parking control).

[0083] When the parking start instruction button G4a3 is touch-operated, the VCECU executes automatic parking control to automatically move the vehicle SV toward the "planned parking position" temporarily determined for the vehicle SV. Since the automatic parking control in this case is performed before the planned parking position is actually registered in the VCECU, it may be referred to as "registration automatic parking control".

[0084] Specifically, as shown in the upper figure of Fig. 7, the VCECU determines the target path T1 for moving the vehicle SV from the current position (the position specified by point P1) to the planned parking position (the position specified by point P2) without the vehicle SV contacting an obstacle. That is, the VCECU specifies the positional relationship between the current position of the vehicle SV and the planned parking position, and calculates the target path T1 along which the vehicle SV can move from the current position of the vehicle SV to the planned parking position. The VCECU determines the "direction in which the vehicle SV should move (specifically, the position of the shift lever 72), the steering angle pattern, and the speed pattern" for moving the vehicle SV along the target path T1. The VCECU performs shift position automatic control to switch the position of the shift lever 72 (the state of the transmission and the drive direction switching mechanism) according to the determined position of the shift lever 72, and then executes steering angle automatic control, driving force automatic control, and braking force automatic control so that the vehicle SV travels according to the steering angle pattern and the speed pattern.

[0085] While the vehicle SV is moving toward the planned parking position, the VCECU acquires the ID of the feature point and the feature point information from the PVM-ECU20 every time a predetermined time elapses, and stores them in the non-volatile memory of the VCECU.

[0086] When the vehicle SV moves to and stops at the parking position (the position specified by point P2) to be registered, the VCECU switches the image displayed on the touch panel display unit 22 from the display image G4 shown in the lower figure of FIG. 6 to the display image G5 shown in the lower figure of FIG. 7. The display image G5 includes, in its left region, "an overhead image G5h in which a planned registration position display frame G5a1 is superimposed" and "a registration cancellation button G5a4". Further, the display image G5 includes, in its right region, "a position operation button G5a2 and a registration execution button G5a3".

[0087] The planned registration position display frame G5a1 indicates the final "parking position to be registered". The position operation button G5a2 is the same button as the position operation button G4a2, and is a button that is touch-operated for the user U1 to move the position in the overhead image G5h of the planned registration position display frame G5a1. The registration execution button G5a3 is a button that is touch-operated for the user U1 to cause the VCECU to register the "parking position to be registered" indicated by the planned registration position display frame G5a1 as the "registered parking position". The registration cancellation button G5a4 is a button that is touch-operated for the user U1 to cancel the registration of the "parking position to be registered" indicated by the planned registration position display frame G5a1 as the "registered parking position" in the VCECU.

[0088] The user U1 touches and operates the position operation button G5a2 as needed to move the position in the overhead image G5h of the planned registration position display frame G5a1. That is, fine adjustment of the parking position to be registered is performed. Then, the user U1 touches and operates the registration execution button G5a3.

[0089] When the registration execution button G5a3 is touch-operated, the VCECU determines the "parking position to be registered" indicated by the planned registration position display frame G5a1 at that time (i.e., the time when the registration execution button G5a3 is touch-operated) as the final registered parking position, and stores (registers) the determined registered parking position in the non-volatile memory.

[0090] Next, the VCECU switches the image displayed on the touch panel display unit 22 from the display image G5 to the display image G6 shown in the lower figure of FIG. 8. The display image G6 includes a registration completion message G6m and an image G6f indicating the registered parking position. The registration completion message G6m is a message for notifying the user U1 that the registration of the planned parking position has been completed. The image G6f is a camera viewpoint image of the back camera 21b when viewing the registered parking position from the position of the vehicle SV at the time when the parking start instruction button G4a3 shown in the lower figure of FIG. 6 is touched, and a frame indicating the registered parking position is superimposed and displayed on the camera viewpoint image. As described above, the registration of the planned parking position is completed when the automatic parking position registration button G3a1 is touched.

[0091] 2: When the driver parking position registration button G3a2 (see FIG. 5) is touched In this case, the VCECU switches the image displayed on the touch panel display unit 22 from the display image G3 shown in the lower figure of FIG. 5 to the display image G7 shown in the lower figure of FIG. 9. The display image G7 includes an aerial view image G7h in its left region. This aerial view image G7h includes a registrable area frame G7a1. The display image G7 includes "message G7m and driver parking position registration process start button G7a2" in its right region.

[0092] The registrable area frame G7a1 indicates a range including all parking positions that can be registered. The message G7m is an instruction message to the user U1. The driver parking position registration process start button G7a2 is a button that is touched by the user U1 when the user U1 decides to start a series of processes (manual parking registration method) for registering the driver parking position.

[0093] After user U1 touches the registration process start button G7a2 for the driver's parking position, the user drives the vehicle SV himself / herself and performs manual driving parking in which the vehicle SV is moved towards the "parking position (the position specified by point P2) that is desired to be registered" and stopped. Even in this case, while the vehicle SV is moving towards the parking position that is desired to be registered, the VCECU acquires the ID of the feature point and the feature point information from the PVM-ECU20 every time a predetermined time elapses and stores them in the non-volatile memory.

[0094] When the vehicle SV stops at the parking position that is desired to be registered, the user U1 changes the position of the shift lever 72 from the reverse position (R) to the parking position (P). Thereby, the VCECU determines that the parking of the vehicle SV is completed. At this time, the VCECU switches the image displayed on the touch panel display unit 22 from the display image G7 to the display image G5 shown in the lower figure of FIG. 7 described above. In this case, the planned registration position display frame 5Ga1 is displayed at a position corresponding to the position where the vehicle SV is actually stopped. Thereafter, as described above, the registration of the parking position is performed.

[0095] That is, the position in the bird's-eye view image G5h of the planned registration position display frame 5Ga1 is moved (corrected) by the touch operation on the position operation button G5a2 of the user U1. When the registration execution button G5a3 is touched by the user U1, the VCECU determines the "parking position to be registered" indicated by the planned registration position display frame G5a1 at that point in time (i.e., the point in time when the registration execution button G5a3 is touched) as the final registered parking position and stores (registers) the determined registered parking position in the non-volatile memory. As described above, the registration of the planned parking position is completed when the registration button G3a2 for the driver's parking position is touched.

[0096] (Automatic parking control after registration of the planned parking position) When the planned parking position is registered as the registered parking position, thereafter, the user U1 can automatically park the vehicle SV at the registered parking position by automatic parking control (i.e., perform automatic parking). For convenience, the automatic parking control at this time is referred to as "post-registration automatic parking control".

[0097] More specifically, the VCECU causes the driver (user U1) to stop the vehicle SV at a predetermined position near the registered parking position, operate the parking assist switch 14, and touch-operate the aforementioned parking request switch. At this time, if it is determined that the registered parking position exists in the vehicle SV, the "process of automatically parking the vehicle SV at the registered parking position by automatic parking control (post-registration automatic parking control)" is started.

[0098] Also in this case, while the vehicle SV is moving toward the registered parking position, the VCECU acquires the ID of the feature point and the feature point information from the PVM-ECU20 every time a predetermined time elapses. Hereinafter, the feature points acquired at this stage are referred to as "actual feature points". Then, based on the feature point information about the "feature points stored in the non-volatile memory when registering the planned parking position" and the feature point information of the actual feature points, the VCECU specifies the current position of the vehicle SV and the registered parking position, and calculates a path (i.e., the target path) along which the vehicle SV can move from the current position of the vehicle SV to the registered parking position without contacting an obstacle.

[0099] And, similar to the above, the VCECU executes shift position automatic control, steering angle automatic control, driving force automatic control, and braking force automatic control according to the target path. The above is the outline of the post-registration automatic parking control. Note that the automatic parking control using feature points itself is well known (see, for example, Japanese Patent Application Laid-Open Nos. 2017-21427, 2017-138664, and 2018-127065).

[0100] When the automatic parking control is executed after registration, the movement path when the vehicle SV moves is likely to substantially match all or part of the path (hereinafter referred to as the "movement path at the time of registration") along which the vehicle SV moved when the registration automatic parking control was performed when registering the registered parking position. Since the movement path at the time of registration is the path along which the vehicle SV could actually move by the registration automatic parking control, it is less likely to include a place where the vehicle SV is difficult to drive. Therefore, when the automatic parking control is performed after registration, the vehicle SV is likely to be able to move to the registered parking position. As a result, the VCECU can reduce the possibility that the vehicle SV cannot be parked at the previously registered registered parking position by the automatic parking control after registration.

[0101] <Specific operation> The CPU of the VCECU (simply referred to as the "CPU") is configured to execute the routine shown by the flowchart in FIG. 10 every time a predetermined time elapses.

[0102] Therefore, at a predetermined timing, the CPU starts processing from step 1000 in FIG. 10 and proceeds to step 1005 to determine whether the values of the automatic parking position registration flag Xap and the driver parking position registration flag Xmp are both "0".

[0103] The automatic parking position registration flag Xap indicates that the automatic parking registration method has been selected by a touch operation of the automatic parking position registration button G3a1 by the user U1 when its value is "1". The driver parking position registration flag Xmp indicates that the manual parking registration method has been selected by a touch operation of the driver parking position registration button G3a2 by the user U1 when its value is "1". Note that the values of these flags (Xap and Xmp) are set to "0" in an initial routine (not shown) that is executed when the ignition key switch of the vehicle SV is changed from off to on.

[0104] If any of the values of these flags (Xap and Xmp) is not "0" (i.e., if at least one of the values is "1"), the CPU determines "No" in step 1005 and proceeds to step 1095, ending this routine once.

[0105] On the contrary, if all of the values of these flags (Xap and Xmp) are "0", the CPU determines "Yes" in step 1005 and proceeds to step 1010 to recognize the surrounding environment. Specifically, the CPU generates and acquires, as vehicle surrounding information, data representing the "position" of "three-dimensional objects, spaces (spaces), and lane lines" existing around the vehicle SV and their shapes based on the information transmitted from the radar sensor 11, the first ultrasonic sensor 12, the second ultrasonic sensor, the camera 21, and the PVM-ECU 20.

[0106] After that, the CPU proceeds to step 1015 to determine whether the display condition of the display image G1 is satisfied. The display condition of the display image G1 is a condition that is satisfied when all of the following conditions A1 to A3 are satisfied. Condition A1: The vehicle SV is stopped. Condition A2: A parkable space has been detected based on the vehicle surrounding information. Condition A3: The aforementioned parking wish switch (not shown) has been touched.

[0107] If the display condition of the display image G1 is not satisfied, the CPU determines "No" in step 1015 and proceeds to step 1095, ending this routine once.

[0108] On the contrary, if the display condition of the display image G1 is satisfied, the CPU determines "Yes" in step 1015 and proceeds to step 1020 to display the display image G1. Next, the CPU proceeds to step 1025 to determine whether the display button G1a1 for registering the home parking position has been touched. That is, the CPU determines whether the registration start condition of the home parking position is satisfied.

[0109] If the condition for starting the registration of the home parking position is not satisfied, the CPU determines "No" in step 1025, proceeds to step 1095, and temporarily ends this routine.

[0110] On the other hand, when the display button G1a1 is touch-operated and the condition for starting the registration of the home parking position is satisfied, the CPU determines "Yes" in step 1025, proceeds to step 1030, and displays the display image G2. Next, the CPU proceeds to step 1035 and determines whether the user U1 has touch-operated any one of the first to fourth parking method selection buttons G2a1 to G2a4. In other words, the CPU determines whether the parking method to be executed has been selected from among parallel parking on the left, parallel parking on the right, perpendicular parking on the left, and perpendicular parking on the right.

[0111] If no parking method has been selected, the CPU determines "No" in step 1035, proceeds to step 1095, and temporarily ends this routine.

[0112] On the other hand, when the parking method to be executed has been selected, the CPU determines "Yes" in step 1035, proceeds to step 1040, and displays the display image G3. Next, the CPU proceeds to step 1045 and determines whether the user U1 has touch-operated the automatic parking position registration button G3a1. That is, the CPU determines whether the automatic parking registration method has been selected.

[0113] When the automatic parking registration method has been selected, the CPU determines "Yes" in step 1045, sequentially performs the processes of step 1050 and step 1055 described below, proceeds to step 1095, and temporarily ends this routine.

[0114] Step 1050: The CPU displays the display image G4. Step 1055: The CPU sets the value of the automatic parking position registration flag Xap to "1".

[0115] On the other hand, when the automatic parking registration method is not selected, the CPU determines "No" in step 1045 and proceeds to step 1060 to determine whether the user U1 has touched the registration button G3a2 for the driver's parking position. That is, the CPU determines whether the manual parking registration method has been selected.

[0116] When the manual parking registration method is selected, the CPU determines "Yes" in step 1060, performs the processes of step 1065 and step 1070 described below in order, proceeds to step 1095, and temporarily ends this routine.

[0117] Step 1065: The CPU displays the display image G7. Step 1070: The CPU sets the value of the driver's parking position registration flag Xmp to "1".

[0118] On the other hand, when the manual parking registration method is not selected, the CPU determines "No" in step 1060 and proceeds to step 1095 to temporarily end this routine.

[0119] Furthermore, the CPU is configured to execute the routine shown by the flowchart in FIG. 11 every time a predetermined time elapses.

[0120] Therefore, at a predetermined timing, the CPU starts processing from step 1100 in FIG. 11 and proceeds to step 1105 to determine whether the value of the automatic parking position registration flag Xap is "1" and the value of the driver's parking position registration flag Xmp is "0".

[0121] If either the value of the automatic parking position registration flag Xap is "0" or the value of the driver's parking position registration flag Xmp is "1", the CPU determines "No" in step 1105 and proceeds to step 1195 to temporarily end this routine.

[0122] On the other hand, when the value of the automatic parking position registration flag Xap is "1" and the value of the driver parking position registration flag Xmp is "0", the CPU determines "Yes" in step 1105 and proceeds to step 1110. The CPU proceeds to step 1110 and determines whether the user U1 has touched the parking start instruction button G4a3. That is, the CPU determines whether the parking position to be registered has been tentatively determined. If the user U1 touches the position operation button G4a2 between the time when the display image G4 starts to be displayed and the time when the parking start instruction button G4a3 is touched, the CPU moves the planned registration position display frame G4a1 in response to the touch operation.

[0123] If the parking position to be registered has not been tentatively determined, the CPU determines "No" in step 1110 and proceeds to step 1195, temporarily ending this routine.

[0124] On the other hand, if the parking position to be registered has been tentatively determined, the CPU determines "Yes" in step 1110 and proceeds to step 1115, and executes automatic parking control for registration according to the parking method selected in step 1035. When the processing of step 1105 and step 1110 is executed, if the automatic parking control for registration is in progress, the CPU proceeds to step 1115 and continues to execute the automatic parking control for registration.

[0125] After that, the CPU proceeds to step 1120 and determines whether the automatic parking control for registration has ended (whether the vehicle SV has stopped at the tentatively determined parking position to be registered).

[0126] If the automatic parking control for registration has not ended at the current time, the CPU determines "No" in step 1120 and proceeds to step 1195, temporarily ending this routine.

[0127] If the registration automatic parking control has been completed at the current time, the CPU determines "Yes" in step 1120 and proceeds to step 1125 to display the display image G5. Next, the CPU proceeds to step 1130 to determine whether the user U1 has touched the registration execution button G5a3. That is, the CPU determines whether the "parking position scheduled for registration" indicated by the scheduled registration position display frame G5a1 has been determined as the final registered parking position. If the user U1 touches the position operation button G5a2 between the time when the display image G5 starts to be displayed and the time when the registration execution button G5a3 is touched, the CPU moves the scheduled registration position display frame G5a1 in response to the touch operation.

[0128] When the "parking position scheduled for registration" is determined as the final registered parking position, the CPU determines "Yes" in step 1130, sequentially executes the processes of steps 1135 to 1145 described below, proceeds to step 1195, and temporarily ends this routine.

[0129] Step 1135: The CPU stores the final registered parking position in the non-volatile memory to complete the registration of the scheduled parking position. This final registered parking position is stored in the non-volatile memory in association with the feature point information obtained during the execution of the registration automatic parking control. Step 1140: The CPU displays a predetermined home screen (not shown). Step 1145: The CPU sets the value of the automatic parking position registration flag Xap to "0".

[0130] On the other hand, when the CPU executes the process of step 1130, if the "parking position scheduled for registration" has not been determined as the final registered parking position, the CPU determines "No" in step 1130 and proceeds to step 1150. In step 1150, the CPU determines whether the registration cancellation button G5a4 has been touched.

[0131] When the registration cancellation button G5a4 is touch-operated, the CPU determines "Yes" in step 1150, performs the processes of "step 1140 and step 1145" described above, proceeds to step 1195, and temporarily ends this routine. In this case, the final registered parking position is not stored (registered). On the other hand, when the registration cancellation button G5a4 is not touch-operated, the CPU determines "No" in step 1150, proceeds to step 1195, and temporarily ends this routine.

[0132] Furthermore, the CPU is configured to execute the routine shown in the flowchart of FIG. 12 every time a predetermined time elapses.

[0133] Therefore, at a predetermined timing, the CPU starts processing from step 1200 in FIG. 12, proceeds to step 1205, and determines whether the value of the driver parking position registration flag Xmp is "1" and the value of the automatic parking position registration flag Xap is "0".

[0134] When the determination condition in step 1205 is not satisfied, the CPU determines "No" at that step 1205, proceeds to step 1295, and temporarily ends this routine.

[0135] On the other hand, when the value of the driver parking position registration flag Xmp is "1" and the value of the automatic parking position registration flag Xap is "0", the CPU determines "Yes" in step 1205 and proceeds to step 1210 to determine whether the user U1 has touch-operated the registration process start button G7a2 (see the lower figure in FIG. 9). That is, the CPU determines whether it has been decided to start the process for registering the driver parking position.

[0136] When the registration process start button G7a2 is not touch-operated, the CPU determines "No" in step 1210, proceeds to step 1295, and temporarily ends this routine.

[0137] On the other hand, when the registration process start button G7a2 is touched, the CPU determines "Yes" in step 1210 and proceeds to step 1215 to determine whether the parking of the vehicle SV by the driver (user U1) is completed at the current time (manual driving parking). Specifically, when the position of the vehicle SV is within the range indicated by the registrable area frame G7a1 and the position of the shift lever 72 is changed to the "P range", the CPU determines that the parking of the vehicle by the driver is completed.

[0138] If the manual driving parking is not completed, the CPU determines "No" in step 1215 and proceeds to step 1295 to temporarily end this routine.

[0139] On the other hand, if the manual driving parking is completed, the CPU determines "Yes" in step 1215 and proceeds to step 1220 to display the display image G5. Next, the CPU proceeds to step 1225 to determine whether the user U1 has touched the registration execution button G5a3. That is, the CPU determines whether the "parking position to be registered" indicated by the planned registration position display frame G5a1 has been determined as the final registered parking position. Also in this case, if the user U1 touches the position operation button G5a2 between the time when the display image G5 starts to be displayed and the time when the registration execution button G5a3 is touched, the CPU moves the planned registration position display frame G5a1 in response to the touch operation.

[0140] When the "parking position to be registered" is determined as the final registered parking position, the CPU determines "Yes" in step 1225, sequentially executes the processes of steps 1230 to 1240 described below, proceeds to step 1295, and temporarily ends this routine.

[0141] Step 1230: The CPU stores the final registered parking position in the non-volatile memory and completes the registration of the planned parking position. Note that this final registered parking position is stored in the non-volatile memory in association with the feature point information acquired during the execution of the manual driving parking. Step 1235: The CPU displays a predetermined home screen (not shown). Step 1240: The CPU sets the value of the driver parking position registration flag Xmp to "0".

[0142] On the other hand, when the CPU executes the process of Step 1225, if the "parking position to be registered" has not been determined as the final registered parking position, the CPU determines "No" in Step 1225 and proceeds to Step 1245. The CPU determines whether or not the registration cancellation button G5a4 has been touched in Step 1245.

[0143] If the registration cancellation button G5a4 has been touched, the CPU determines "Yes" in Step 1245, performs the processes of "Step 1235 and Step 1240" described above, proceeds to Step 1295, and temporarily ends this routine. On the contrary, if the registration cancellation button G5a4 has not been touched, the CPU determines "No" in Step 1245, proceeds to Step 1295, and temporarily ends this routine.

[0144] Furthermore, the CPU is configured to execute the routine shown by the flowchart in FIG. 13 every time a predetermined time elapses.

[0145] Therefore, at a predetermined timing, the CPU starts processing from Step 1300 in FIG. 13, proceeds to Step 1305, and recognizes the surrounding environment by executing the same process as the process of Step 1010 in FIG. 10. That is, the CPU acquires vehicle surrounding information.

[0146] Thereafter, the CPU proceeds to Step 1310 and determines whether or not the value of the automatic parking execution flag Xauto after registration is "0". The value of this flag Xauto is also set to "0" in the initial routine described above.

[0147] When the value of flag Xauto is "0", the CPU determines "Yes" in step 1310 and proceeds to step 1315 to determine whether the display condition of the above-described display image G1 is satisfied. If the display condition of the display image G1 is not satisfied, the CPU determines "No" in step 1315 and proceeds to step 1395 to temporarily end this routine.

[0148] On the other hand, when the display condition of the display image G1 is satisfied, the CPU determines "Yes" in step 1315 and proceeds to 1320 to display the display image G1. In this case, if a registered parking position is detected in the parkable space based on the vehicle surrounding information, the CPU additionally displays an "execution button for automatic parking control after registration" in the display image G1.

[0149] Next, the CPU proceeds to step 1325 to determine whether the execution condition of the automatic parking control after registration is satisfied. The execution condition of the automatic parking control after registration is a condition that is satisfied when both of the following conditions B1 and B2 are satisfied.

[0150] Condition B1: The parking assistance switch 14 has been operated. Condition B2: The execution button for the automatic parking control after registration has been touch-operated.

[0151] If the execution condition of the automatic parking control after registration is not satisfied, the CPU determines "No" in step 1325 and proceeds to step 1395 to temporarily end this routine.

[0152] On the other hand, when the execution condition of the automatic parking control after registration is satisfied, the CPU determines "Yes" in step 1325 and proceeds to step 1330 to start the above-described automatic parking control after registration for automatically moving the vehicle SV to the registered parking position registered (stored) in the non-volatile memory. Next, the CPU proceeds to step 1335 to set the value of the flag Xauto to "1". Further, the CPU proceeds to step 1340 to determine whether the vehicle SV has reached the registered parking position.

[0153] If the vehicle SV has not reached the registered parking position, the CPU determines "No" in step 1340, proceeds to step 1395, and temporarily ends this routine.

[0154] On the other hand, if the vehicle SV has reached the registered parking position, the CPU determines "Yes" in step 1340, proceeds to step 1345, and sets the value of flag Xauto to "1". Then, the CPU proceeds to step 1395 and temporarily ends this routine.

[0155] Note that when the CPU executes the process of step 1310, if the value of flag Xauto is not "0" (that is, if it is "1"), the CPU determines "No" in step 1310 and directly proceeds to step 1340. Thereby, the automatic parking control after registration is continued.

[0156] As described above, the embodiments of the present invention have been specifically described. However, the present invention is not limited to the above-described embodiments, and various modifications based on the technical idea of the present invention can be adopted. For example, the embodiment of the present invention may be configured to include a voice recognition device (not shown), and part or all of the operations performed by touch panel operation may be configured to be performed by the user U1's voice operation.

Explanation of Reference Numerals

[0157] 10... Vehicle control ECU, 11... Radar sensor, 12... First ultrasonic sensor, 13... Second ultrasonic sensor, 14... Parking assist switch, 15... Vehicle speed sensor, 20... PVM-ECU, 21a... Front camera, 21b... Rear camera, 21c... Right side camera, 21d... Left side camera, 22... Touch panel display unit, 30... Engine ECU, 40... Brake EUU, 50... EPS·ECU, 70... SBW·ECU, 72... Shift lever

Claims

1. An imaging device that captures an image of the surroundings of the vehicle to obtain image data; a display device capable of displaying an image based on the image data; a control device capable of executing automatic parking control for automatically moving the vehicle to a predetermined target parking position based on the image data; A storage device capable of storing a predetermined parking position as a registered parking position in advance; Equipped with The control device is configured to, after the automatic parking control is completed, display on the display device an image including a surrounding image showing the surroundings of the vehicle parked by the automatic parking control and a display frame to be registered superimposed on the surrounding image, and to store in the storage device a position of the display frame moved by a user of the vehicle as the registered parking position. Parking assistance device.

2. 2. The parking assistance device according to claim 1, The control device includes: The position of the display frame moved by the user of the vehicle is stored in the storage device as the registered parking position, and then the registered parking position stored in the storage device is set as the target parking position, and the automatic parking control is executed. Parking assistance device.

Images