Vehicle parking aids

Abstract

A vehicle parking assistance device is provided, which can reliably grasp the positional relationship between the vehicle and the parking lot when the vehicle is automatically driven in order to park the vehicle in the parking lot. When the vehicle is parked next to the parking lot, the vehicle parking assistance device acquires information related to the parking lot from a camera image as first information and registers the first information as the parking lot information. When the parking of the vehicle in the parking lot is completed by parking assistance control, the vehicle parking assistance device acquires information related to the parking lot from the camera image as second information and registers the second information as the parking lot information. After the parking lot information is registered, when the vehicle is parked next to the parking lot, the vehicle parking assistance device acquires information related to the parking lot from the camera image acquired this time as the current parking lot information, and performs parking assistance control while grasping the positional relationship between the vehicle and the parking lot by comparing the current parking lot information with the registered parking lot information.

Description

Technical Field

[0001] The invention relates to a vehicle parking assistance device. Background Art

[0002] A vehicle parking assistance device is known that enables a vehicle to be automatically parked in a parking lot (e.g., a parking lot of a private residence) that is not provided with partitioning lines such as white lines that divide parking areas. As such a vehicle parking assistance device, a vehicle parking assistance device having the following structure is known (e.g., refer to Japanese Patent Publication No. 2017-138664): when a vehicle is parked in a parking lot, information related to the parking lot (hereinafter referred to as "parking lot information") is registered, and when the vehicle is automatically parked in the parking lot next time, the parking lot information obtained at that time is compared with the registered parking lot information, and the vehicle is automatically parked in the parking lot while grasping the positional relationship between the vehicle and the parking lot. The vehicle parking assistance device is configured to obtain the parking lot information based on an image of the parking lot captured by a camera (hereinafter referred to as "camera image"), for example. Summary of the invention

[0003] While the vehicle is being automatically parked in the parking lot, the positional relationship between the vehicle and the parking lot is constantly changing. Furthermore, if the positional relationship between the vehicle and the parking lot (hereinafter referred to as the "relative positional relationship") is different, even if the same object is captured in the camera image, the shape of the image of the object captured by the camera image is different. Therefore, in the case where parking lot information acquired by a camera image at a time point when the relative positional relationship is in a certain specific positional relationship is registered, when the vehicle is automatically driven in the parking lot, the parking lot information acquired by the camera image at a time point when the relative positional relationship has greatly deviated from the above-mentioned specific positional relationship may not be able to reliably grasp the relative positional relationship at that time even if it is compared with the registered parking lot information. In this case, there is a possibility that the vehicle cannot be reliably parked at the specified position of the parking lot.

[0004] The present invention has been made to address the above-mentioned problems. Specifically, one object of the present invention is to provide a vehicle parking assistance device that can reliably grasp the positional relationship between a vehicle and a parking space when a vehicle is automatically driven to park the vehicle in the parking space.

[0005] The vehicle parking assistance device involved in the present invention comprises: a camera installed on the vehicle in a manner capable of photographing the surroundings of the vehicle; and a control unit configured to be able to perform the following parking assistance control, namely, obtaining an image photographed by the camera as a camera image, and automatically parking the vehicle in the parking space while using information related to the parking space obtained from the camera image.

[0006] When the vehicle parks next to the parking lot, the control unit obtains information related to the parking lot from the camera image as first information, and registers the first information as parking lot information. When the parking of the vehicle in the parking lot is completed through the parking assist control, the control unit obtains information related to the parking lot from the camera image as second information, and registers the second information as the parking lot information.

[0007] In addition, after the control unit has registered the parking lot information, when the vehicle parks next to the parking lot, the control unit obtains information related to the parking lot from the camera image obtained this time as the parking lot information this time, and performs the parking assist control while grasping the positional relationship between the vehicle and the parking lot by comparing the parking lot information this time with the registered parking lot information.

[0008] Accordingly, the parking lot information is acquired twice: when the vehicle stops near the entrance of the parking lot and when the vehicle completes parking in the parking lot. As a result, the deviation between the positional relationship between the vehicle and the parking lot and the positional relationship between the vehicle and the parking lot when the parking lot information is acquired converges to a smaller range until the vehicle is automatically parked in the parking lot. Therefore, by comparing the parking lot information acquired until the vehicle is automatically parked in the parking lot with the registered parking lot information, the positional relationship between the vehicle and the parking lot can be more reliably grasped. As a result, the vehicle can be more reliably parked at a specified position in the parking lot.

[0009] In the vehicle parking assistance device involved in the present invention, the control unit can also be configured to: after the vehicle starts traveling through the parking assistance control until the vehicle is parked in the parking lot through the parking assistance control, obtain information related to the parking lot from the camera image as the third information at least once, and register the third information as the parking lot information.

[0010] According to this, the parking lot information is acquired three times: when the vehicle stops near the entrance of the parking lot, when the vehicle completes parking in the parking lot, and until the vehicle completes parking in the parking lot. As a result, the deviation between the positional relationship between the vehicle and the parking lot until the vehicle is automatically parked in the parking lot and the positional relationship between the vehicle and the parking lot when the parking lot information is acquired converges to a smaller range. Therefore, by comparing the parking lot information acquired until the vehicle is automatically parked in the parking lot with the registered parking lot information, the positional relationship between the vehicle and the parking lot can be more reliably grasped.

[0011] In addition, in the vehicle parking assistance device involved in the present invention, the control unit can also be configured to: after the vehicle starts traveling by the parking assistance control, until the parking of the vehicle in the parking lot is completed by the parking assistance control, after it is predicted that the vehicle will go straight, until the parking of the vehicle in the parking lot is completed by the parking assistance control, obtain information related to the parking lot from the camera image as the third information at least once, and register the third information as the parking lot information.

[0012] In addition, in the vehicle parking assistance device involved in the present invention, the control unit can also be configured as: after the vehicle starts traveling by the parking assistance control, until the parking of the vehicle in the parking lot is completed by the parking assistance control, at a time point when the vehicle is predicted to go straight, information related to the parking lot is obtained from the camera image as the third information, and the third information is registered as the parking lot information.

[0013] In addition, in the vehicle parking assistance device involved in the present invention, the control unit can also be configured as: after the vehicle starts traveling by the parking assistance control, until the parking of the vehicle in the parking lot is completed by the parking assistance control, from the time point when the vehicle is predicted to travel a specified distance after the vehicle is predicted to go straight, information related to the parking lot is obtained from the camera image as the fourth information, and the fourth information is registered as the parking lot information.

[0014] In addition, in the vehicle parking assistance device involved in the present invention, the control unit can also be configured as follows: after the vehicle starts traveling by the parking assistance control, until the parking of the vehicle in the parking lot is completed by the parking assistance control, from the time when the vehicle is predicted to go straight, each time the vehicle travels a specified distance, information related to the parking lot is obtained from the camera image as the fourth information, and the fourth information is registered as the parking lot information.

[0015] According to this, the parking lot information when the vehicle is not turning but moving straight is registered. Regardless of the positional relationship of the vehicle relative to the parking lot, fixed parking lot information can be stably acquired from the camera image when the vehicle is moving straight. Therefore, by comparing the parking lot information acquired when the vehicle is moving straight with the parking lot information acquired and registered when the vehicle is moving straight, the positional relationship between the vehicle and the parking lot can be more reliably grasped.

[0016] Furthermore, in the vehicle parking assistance device according to the present invention, the information related to the parking space includes, for example, information related to feature points in the camera image.

[0017] According to this, information on the characteristic points of the parking lot in the camera image is registered as the parking lot information.

[0018] Furthermore, in the vehicle parking assistance device according to the present invention, the information related to the parking space includes, for example, information related to the position of a feature point in the camera image with reference to a predetermined position.

[0019] According to this, the position of the feature point based on the predetermined position is registered as the parking space information.

[0020] In addition, in the vehicle parking assistance device involved in the present invention, the camera includes, for example, a front camera for photographing the front of the vehicle, a rear camera for photographing the rear of the vehicle, a left camera for photographing the left side of the vehicle, and a right camera for photographing the right side of the vehicle.

[0021] The components of the present invention are not limited to the embodiments described with reference to the following drawings. Other objects, other features, and advantages of the present invention can be easily understood through the description of the embodiments of the present invention described with reference to the following drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a diagram showing a vehicle parking assistance device according to an embodiment of the present invention and a vehicle to which the vehicle parking assistance device is applied.

[0023] Figure 2 It is a diagram showing the arrangement and detection range of the sonar sensor device.

[0024] Figure 3 It is a diagram showing the arrangement and imaging range of a camera sensor device.

[0025] Figure 4 This is a diagram showing an example of a parking lot.

[0026] Figure 5 It is a diagram showing the front range and the rear range.

[0027] Figure 6 It is a diagram showing the left range and the right range.

[0028] Figure 7 is a diagram showing feature points.

[0029] Figure 8 is a diagram showing parking zones.

[0030] Fig. 9 (A) to (D) are diagrams showing displays.

[0031] Fig.10 (A) to (C) are diagrams showing displays.

[0032] Fig.11 It is used to illustrate Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0033] Fig.12 It is used to illustrate Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0034] Fig.13 It is used to illustrate Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0035] Fig.14 is a diagram showing entry feature points.

[0036] Fig.15 It is used to illustrate Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0037] Fig.16 It is used to illustrate Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0038] Fig.17 It is used to illustrate Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0039] Fig.18 It is used to illustrate Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0040] Fig.19 (A) to (C) are diagrams showing displays.

[0041] Fig. 20 It is shown Figure 1 A flowchart of a routine executed by the CPU of the ECU is shown.

[0042] Fig.21 It is shown Figure 1 A flowchart of a routine executed by the CPU of the ECU is shown.

[0043] Fig. 22 It is shown Figure 1 A flowchart of a routine executed by the CPU of the ECU is shown.

[0044] Fig.23 It is shown Figure 1 A flowchart of a routine executed by the CPU of the ECU is shown. DETAILED DESCRIPTION

[0045] Hereinafter, a vehicle parking assistance device according to an embodiment of the present invention will be described with reference to the drawings. Figure 1 A vehicle parking assistance device 10 according to an embodiment of the present invention and a vehicle 100 to which the vehicle parking assistance device 10 is applied are shown.

[0046] like Figure 1 As shown, the vehicle parking assistance device 10 includes an ECU 90. ECU is an abbreviation for an electronic control unit. The ECU 90 includes a microcomputer as a main component. The microcomputer includes a CPU, a ROM, a RAM, a nonvolatile memory, an interface, etc. The CPU is configured to implement various functions by executing instructions, programs, or routines stored in the ROM.

[0047] The vehicle 100 is equipped with a vehicle driving force generating device 11, a braking device 12, and a steering device 13. The vehicle driving force generating device 11 is a device for generating a driving force for driving the vehicle 100 and applying the driving force to the driving wheels of the vehicle 100. The vehicle driving force generating device 11 is, for example, an internal combustion engine, an electric engine (motor), etc. The braking device 12 is a device for applying a braking force for braking the vehicle 100 to the wheels of the vehicle 100. The steering device 13 is a device for applying a steering torque for steering the vehicle 100 to the steering wheels of the vehicle 100.

[0048] The vehicle driving force generating device 11, the braking device 12, and the steering device 13 are electrically connected to the ECU 90. The ECU 90 controls the driving force applied to the driving wheels of the vehicle 100 by controlling the operation of the vehicle driving force generating device 11. In addition, the ECU 90 controls the braking force applied to the wheels of the vehicle 100 by controlling the operation of the braking device 12. In addition, the ECU 90 controls the steering torque applied to the steering wheels of the vehicle 100 by controlling the operation of the steering device 13.

[0049] <Sensors, etc.>

[0050] The vehicle parking assist device 10 includes an accelerator pedal operation amount sensor 21, a brake pedal operation amount sensor 22, a steering angle sensor 23, a steering torque sensor 24, a vehicle speed sensor 25, a yaw rate sensor 26, a longitudinal acceleration sensor 27, a lateral acceleration sensor 28, a sonar sensor device 30, a camera sensor device 40 and a display 50.

[0051] The accelerator pedal operation amount sensor 21 is electrically connected to the ECU 90. The ECU 90 detects the operation amount AP of the accelerator pedal 14 via the accelerator pedal operation amount sensor 21 and acquires the accelerator pedal operation amount AP. The ECU 90 controls the operation of the vehicle drive force generation device 11 in such a manner that a driving force corresponding to the acquired accelerator pedal operation amount AP is given from the vehicle drive force generation device 11 to the driving wheels of the vehicle 100.

[0052] The brake pedal operation amount sensor 22 is electrically connected to the ECU 90. The ECU 90 detects the operation amount BP of the driver on the brake pedal 15 via the brake pedal operation amount sensor 22 and obtains it as the brake pedal operation amount BP. The ECU 90 controls the operation of the brake device 12 in such a manner that a braking force corresponding to the obtained brake pedal operation amount BP is applied from the brake device 12 to the wheels of the vehicle 100.

[0053] The steering angle sensor 23 is electrically connected to the ECU 90. The ECU 90 detects the rotation angle θst of the steering wheel 16 with respect to the neutral position via the steering angle sensor 23 and acquires the detected rotation angle θst as the steering angle θst.

[0054] The steering torque sensor 24 is electrically connected to the ECU 90. The ECU 90 detects the torque TQst input from the driver to the steering shaft 17 via the steering torque sensor 24 and acquires the torque TQst as the steering torque TQst.

[0055] The ECU 90 controls the operation of the steering device 13 so as to apply a steering torque corresponding to the acquired steering angle θst and steering torque TQst to the steered wheels of the vehicle 100 .

[0056] The vehicle speed sensor 25 is electrically connected to the ECU 90. The ECU 90 detects the rotation speed Vrot of each wheel of the vehicle 100 via the vehicle speed sensor 25 and acquires the rotation speed Vrot of each wheel. The ECU 90 acquires the running speed SPD of the vehicle 100 as the vehicle speed SPD based on the acquired rotation speed Vrot of each wheel.

[0057] The yaw rate sensor 26 is electrically connected to the ECU 90. The ECU 90 detects the yaw rate YR of the vehicle 100 via the yaw rate sensor 26 and acquires the yaw rate YR as the vehicle yaw rate YR.

[0058] The longitudinal acceleration sensor 27 is electrically connected to the ECU 90. The ECU 90 detects the longitudinal acceleration Gx of the vehicle 100 via the longitudinal acceleration sensor 27 and acquires the longitudinal acceleration Gx as the vehicle longitudinal acceleration Gx.

[0059] The lateral acceleration sensor 28 is electrically connected to the ECU 90. The ECU 90 detects the lateral acceleration Gy of the vehicle 100 via the lateral acceleration sensor 28, and acquires the lateral acceleration Gy as the vehicle lateral acceleration Gy.

[0060] The sonar sensor device 30 includes first to twelfth clearance sonars 301 to 312 .

[0061] like Figure 2 As shown, the first clearance sonar 301 is mounted on the vehicle 100 in such a manner as to emit sound waves from the front left end portion of the vehicle 100 to the left front. The second clearance sonar 302 is mounted on the vehicle 100 in such a manner as to emit sound waves from the left front end portion of the vehicle 100 to the front. The third clearance sonar 303 is mounted on the vehicle 100 in such a manner as to emit sound waves from the front right end portion of the vehicle 100 to the right front. The fourth clearance sonar 304 is mounted on the vehicle 100 in such a manner as to emit sound waves from the right front end portion of the vehicle 100 to the front.

[0062] Furthermore, the fifth clearance sonar 305 is mounted on the vehicle 100 in such a manner as to emit sound waves from the rear left end portion of the vehicle 100 to the left rear. The sixth clearance sonar 306 is mounted on the vehicle 100 in such a manner as to emit sound waves from the left rear end portion of the vehicle 100 to the rear. The seventh clearance sonar 307 is mounted on the vehicle 100 in such a manner as to emit sound waves from the rear right end portion of the vehicle 100 to the right rear. The eighth clearance sonar 308 is mounted on the vehicle 100 in such a manner as to emit sound waves from the right rear end portion of the vehicle 100 to the rear.

[0063] Furthermore, the ninth clearance sonar 309 is mounted on the vehicle 100 in such a manner as to emit sound waves from the front left side portion of the vehicle 100 to the left. The tenth clearance sonar 310 is mounted on the vehicle 100 in such a manner as to emit sound waves from the rear left side portion of the vehicle 100 to the left. The eleventh clearance sonar 311 is mounted on the vehicle 100 in such a manner as to emit sound waves from the front right side portion of the vehicle 100 to the right. The twelfth clearance sonar 312 is mounted on the vehicle 100 in such a manner as to emit sound waves from the rear right side portion of the vehicle 100 to the right.

[0064] The first clearance sonar 301 to the twelfth clearance sonar 312 receive the sound waves reflected by the object.

[0065] The sonar sensor device 30 is electrically connected to the ECU 90. The sonar sensor device 30 transmits information related to "sound waves emitted by the first clearance sonar 301 to the twelfth clearance sonar 312" and "sound waves received by the first clearance sonar 301 to the twelfth clearance sonar 312" to the ECU 90. The ECU 90 acquires information related to objects existing around the vehicle 100 as object information OBJ based on the information received from the sonar sensor device 30 (hereinafter referred to as "sonar information SON").

[0066] exist Figure 2 In the figure, the direction indicated by the figure mark Dx is the front-rear direction of the vehicle 100, and this direction is hereinafter referred to as the "vehicle front-rear direction Dx". The direction indicated by the figure mark Dy is the width direction of the vehicle 100, and this direction is hereinafter referred to as the "vehicle width direction Dy".

[0067] The camera sensor device 40 includes a front camera 41, a rear camera 42, a left camera 43, and a right camera 44. Hereinafter, the front camera 41, the rear camera 42, the left camera 43, and the right camera 44 are collectively referred to as "cameras 45" as needed.

[0068] like Figure 3 As shown, the front camera 41 is installed at the center of the front end of the vehicle 100 in such a manner as to photograph the scenery in front of the vehicle 100, and its viewing angle 41A is approximately 180°. The rear camera 42 is installed at the center of the rear end of the vehicle 100 in such a manner as to photograph the scenery behind the vehicle 100, and its viewing angle 42A is also approximately 180°. The left camera 43 is installed at the left side of the vehicle 100 in such a manner as to photograph the scenery on the left side of the vehicle 100, and its viewing angle 43A is also approximately 180°. The right camera 44 is installed at the right side of the vehicle 100 in such a manner as to photograph the scenery on the right side of the vehicle 100, and its viewing angle 44A is also approximately 180°.

[0069] The camera sensor device 40 is electrically connected to the ECU 90. The ECU 90 can acquire information on the image of the scenery captured by each camera 45 via the camera sensor device 40.

[0070] Hereinafter, as required, information related to the image of the scenery captured by the front camera 41 is referred to as "front image information IMG1", information related to the image of the scenery captured by the rear camera 42 is referred to as "rear image information IMG2", information related to the image of the scenery captured by the left camera 43 is referred to as "left image information IMG3", and information related to the image of the scenery captured by the right camera 44 is referred to as "right image information IMG4". In addition, as required, the front image information IMG1, the rear image information IMG2, the left image information IMG3, and the right image information IMG4 are collectively referred to as "image information IMG" below.

[0071] When a predetermined condition is satisfied, the vehicle parking assistance device 10 acquires a feature point F based on the image information IMG. The feature point F is an image within a predetermined range where the brightness of the image captured by each camera 45 changes greatly.

[0072] For example, when the camera 45 captures Figure 4In the case of the parking lot 62 shown, corner portions of concrete blocks 63B, corner portions of ground 63 consisting of lawn 63L, and boundary portions between ground 63 consisting of blocks 63B and ground 63 consisting of lawn 63L are acquired as feature points F.

[0073] Figure 4 The floor 63 of the parking lot 62 shown is composed of a floor 63 composed of concrete 63C and a floor 63 composed of lawn 63L. In addition, a plurality of concrete blocks 63B are arranged to block the side grooves at the entrance 62ent of the parking lot 62. Thus, the floor 63 of the entrance 62ent of the parking lot 62 is composed of the surface of the blocks 63B.

[0074] The vehicle parking assistance device 10 obtains a predetermined range 71 (see FIG. 1 ) in front of the vehicle 100 based on the front image information IMG1. Figure 5 ) on the ground 63 (hereinafter referred to as "front feature point F1"). In addition, the vehicle parking assistance device 10 obtains a predetermined range 72 (refer to Figure 5 ) on the ground 63 (hereinafter referred to as "rear feature point F2"). In addition, the vehicle parking assistance device 10 obtains a predetermined range 73 on the left side of the vehicle 100 based on the left image information IMG3 (refer to Figure 6 ) on the ground 63 (hereinafter referred to as "left feature point F3"). The vehicle parking assistance device 10 obtains a predetermined range 74 on the right side of the vehicle 100 based on the right image information IMG4 (refer to Figure 6 ) at the ground 63 (hereinafter referred to as “right feature point F4”).

[0075] like Figure 5 As shown, the prescribed range 71 is a range defined by lines L711, L712, L713, and L714. Line L711 is a line L711 extending in the vehicle width direction Dy from the front camera 41 at a prescribed distance Dset to the front of the vehicle 100. Line L712 is a line L712 passing through the front camera 41 and extending in the vehicle width direction Dy. Line L713 is a line extending in the vehicle front-rear direction Dx from the front camera 41 at a prescribed distance Dset to the left of the vehicle 100. Line L714 is a line extending in the vehicle front-rear direction Dx from the front camera 41 at a prescribed distance Dset to the right of the vehicle 100. Hereinafter, the prescribed range 71 is referred to as the "front range 71".

[0076] The front range 71 is divided into eight ranges 71D that are divided into four equal parts in the vehicle width direction Dy and two equal parts in the vehicle front-rear direction Dx. That is, the front range 71 is divided into a plurality of ranges 71D of equal area. Hereinafter, these ranges 71D are referred to as "front divided ranges 71D". In addition, among the front divided ranges 71D, the two front divided ranges 71D set at the left end in the vehicle width direction Dy are referred to as "left end divided ranges 71D3", the two front divided ranges 71D set at the right end in the vehicle width direction Dy are referred to as "right end divided ranges 71D4", and the four front divided ranges 71D set in the middle in the vehicle width direction Dy are referred to as "middle divided ranges 71D5".

[0077] In addition, if Figure 5 As shown, the prescribed range 72 is a range defined by lines L721, L722, L723, and L724. Line L721 is a line that passes through the rear camera 42 and extends in the vehicle width direction Dy. Line L722 is a line that is a prescribed distance Dset away from the rear camera 42 to the rear of the vehicle 100 and extends in the vehicle width direction Dy. Line L723 is a line that is a prescribed distance Dset away from the rear camera 42 to the left of the vehicle 100 and extends in the vehicle front-rear direction Dx. Line L724 is a line that is a prescribed distance Dset away from the rear camera 42 to the right of the vehicle 100 and extends in the vehicle front-rear direction Dx. Hereinafter, the prescribed range 72 will be referred to as the "rear range 72".

[0078] The rear range 72 is divided into eight ranges 72D that are divided into four equal parts in the vehicle width direction Dy and two equal parts in the vehicle front-rear direction. That is, the rear range 72 is divided into a plurality of ranges 72D of equal area. Hereinafter, these ranges 72D are referred to as "rear divided ranges 72D". In addition, among the rear divided ranges 72D, the two rear divided ranges 72D set at the left end in the vehicle width direction Dy are referred to as "left end divided ranges 72D3", the two rear divided ranges 72D set at the right end in the vehicle width direction Dy are referred to as "right end divided ranges 72D4", and the four rear divided ranges 72D set in the middle in the vehicle width direction Dy are referred to as "middle divided ranges 72D5".

[0079] In addition, if Figure 6As shown, the prescribed range 73 is a range defined by lines L731, L732, L734, and L733. Line L731 is a line extending in the vehicle width direction Dy from the left camera 43 at a prescribed distance Dset to the front of the vehicle 100. Line L732 is a line extending in the vehicle width direction Dy from the left camera 43 at a prescribed distance Dset to the rear of the vehicle 100. Line L733 is a line extending in the vehicle front-rear direction Dx from the left camera 43 at a prescribed distance Dset to the left of the vehicle 100. Line L734 is a line extending in the vehicle front-rear direction Dx through the left camera 43. Hereinafter, the prescribed range 73 will be referred to as the "left range 73".

[0080] The left side range 73 is divided into eight ranges 73D that are divided into four equal parts in the vehicle front-rear direction Dx and two equal parts in the vehicle width direction Dy. That is, the left side range 73 is divided into a plurality of ranges 73D of equal area. Hereinafter, these ranges 73D are referred to as "left side divided ranges 73D". In addition, among the left side divided ranges 73D, the two left side divided ranges 73D set at the front end in the vehicle front-rear direction Dx are referred to as "front end divided ranges 73D1", the two left side divided ranges 73D set at the rear end in the vehicle front-rear direction Dx are referred to as "rear end divided ranges 73D2", and the four left side divided ranges 73D set in the middle in the vehicle front-rear direction Dx are referred to as "middle divided ranges 73D5".

[0081] In addition, if Figure 6 As shown, the prescribed range 74 is a range defined by lines L741, L742, L743, and L744. Line L741 is a line extending in the vehicle width direction Dy from the right camera 44 at a prescribed distance Dset to the front of the vehicle 100. Line L742 is a line extending in the vehicle width direction Dy from the right camera 44 at a prescribed distance Dset to the rear of the vehicle 100. Line L743 is a line passing through the right camera 44 and extending in the vehicle front-rear direction Dx. Line L744 is a line extending in the vehicle front-rear direction Dx from the right camera 44 at a prescribed distance Dset to the right of the vehicle 100. Hereinafter, the prescribed range 74 will be referred to as the "right range 74".

[0082] The right side range 74 is divided into eight ranges 74D that are divided into four equal parts in the vehicle front-rear direction Dx and two equal parts in the vehicle width direction Dy. That is, the right side range 74 is divided into a plurality of ranges 74D of equal area. Hereinafter, these ranges 74D are referred to as "right side divided ranges 74D". In addition, among the right side divided ranges 74D, two right side divided ranges 74D set at the front end in the vehicle front-rear direction Dx are referred to as "front end divided ranges 74D1", two right side divided ranges 74D set at the rear end in the vehicle front-rear direction Dx are referred to as "rear end divided ranges 74D2", and four right side divided ranges 74D set in the middle in the vehicle front-rear direction Dx are referred to as "middle divided ranges 74D5".

[0083] When the image of the range corresponding to the feature point F captured by the camera 45 is converted into a bird's-eye view image, as shown in FIG. Figure 7 As shown, the transformed image of the range corresponding to the feature point F is an image of a range 75 of a square with a side of a specified length Lset. When the specified conditions are met, the vehicle parking assistance device 10 divides the feature point F into 25 identical square ranges 75D, and obtains the brightness LUM of each range 75D. In addition, the vehicle parking assistance device 10 respectively calculates the value ΔLUM (=LUM-LUMave) obtained by subtracting the average value LUMave of the acquired brightness LUM from each brightness LUM, and obtains the high and low tendency of the brightness LUM at the feature point F based on these values ​​ΔLUM as the light and dark information CT. That is, when the specified conditions are met, the vehicle parking assistance device 10 obtains the light and dark pattern in the image of the feature point F captured by the camera 45 as the light and dark information CT.

[0084] The display 50 is disposed at a position visible to the driver of the vehicle 100. In this example, the display 50 is a so-called display of a navigation device.

[0085] The display 50 is electrically connected to the ECU 90. The ECU 90 can display various images on the display 50. In this example, the ECU 90 can display a camera image 51C, a top view image 51P, a parking assist switch image 51M, a registration main switch image 52M, a parking partition selection image 52S, a parking partition line image 52, a setting button image 53, a registration start button image 54, a registration button image 55, a parking start button image 56, a movement button image 57, and an angle adjustment button image 58 on the display 50.

[0086] The camera image 51C is an image captured by the camera 45 .

[0087] The overhead image 51P includes a vehicle overhead image and a vehicle surrounding image. The vehicle overhead image is an image showing the vehicle 100 when viewed from above in the vertical direction. The vehicle surrounding image is an image showing the surroundings of the vehicle 100 when viewed from above in the vertical direction, and includes at least an image showing the parking lot 62. These vehicle overhead image and vehicle surrounding image are generated by the ECU 90 based on the image information IMG.

[0088] The parking assist switch image 51M is an image corresponding to a parking assist switch that is touched and operated by the driver in order to cause the vehicle parking assist device 10 to start parking assist control described later.

[0089] The registration main switch image 52M is an image corresponding to a registration main switch that is touched and operated by the driver in order to register (ie, store) the parking space information Ipark in the vehicle parking assistance device 10 by parking assistance control described later.

[0090] like Fig. 9 As shown in (C), the parking zone selection image 52S includes a left lateral parking zone selection image 52SLa, a left longitudinal parking zone selection image 52SLb, a right lateral parking zone selection image 52SRa, and a right longitudinal parking zone selection image 52SRb.

[0091] The left lateral parking zone selection image 52SLa is an image that is touched and operated by the driver in order to select a parking zone 61 of a parking lot 62 to park the vehicle 100 on the left side of the vehicle 100 by changing the orientation of the vehicle 100 by 90 degrees. The parking zone 61 is a zone (or range, area) in which the vehicle 100 is parked by the parking assist control described later. Figure 8 As shown, the parking zone 61 is set in the parking lot 62. The left longitudinal parking zone selection image 52SLb is an image that is touched and operated by the driver in order to select the parking zone 61 for parking in the parking lot 62 on the left side of the vehicle 100 without changing the orientation of the vehicle 100. The right lateral parking zone selection image 52SRa is an image that is touched and operated by the driver in order to select the parking zone 61 for parking in the parking lot 62 on the right side of the vehicle 100 by changing the orientation of the vehicle 100 by 90 degrees. The right longitudinal parking zone selection image 52SRb is an image that is touched and operated by the driver in order to select the parking zone 61 for parking in the parking lot 62 on the right side of the vehicle 100 without changing the orientation of the vehicle 100.

[0092] In addition, when there is no parking zone 61 for parking in the parking space 62 on the left side of the vehicle 100 by changing the orientation of the vehicle 100 by 90 degrees, the left transverse parking zone selection image 52SLa is not displayed on the display 50 or is displayed with reduced brightness (so-called dark tone) on the display 50. Similarly, when there is no parking zone 61 for parking in the parking space 62 on the left side of the vehicle 100 by changing the orientation of the vehicle 100 by 90 degrees, the left longitudinal parking zone selection image 52SLb is not displayed on the display 50 or is displayed with reduced brightness (so-called dark tone) on the display 50. Similarly, when there is no parking zone 61 for parking in the parking space 62 on the right side of the vehicle 100 by changing the orientation of the vehicle 100 by 90 degrees, the right transverse parking zone selection image 52SRa is not displayed on the display 50 or is displayed with reduced brightness (so-called dark tone) on the display 50. Similarly, when there is no parking zone 61 for parking in the parking lot 62 on the right side of the vehicle 100 without changing the orientation of the vehicle 100, the right longitudinal parking zone selection image 52SRb is not displayed on the display 50 or is displayed on the display 50 with reduced brightness (so-called dimming).

[0093] Furthermore, when the parking zone selection image 52S displayed on the display 50 in a darkened tone is touched by the driver, the ECU 90 does not determine that the parking zone 61 corresponding to the touched parking zone selection image 52S is selected.

[0094] The parking zone line image 52 is an image showing the parking zone 61 .

[0095] The setting button image 53 is an image corresponding to a setting button that is touched and operated by the driver in order to set (or confirm or determine) the parking zone 61 in which the vehicle 100 is to be parked in parking assist control described later.

[0096] The registration start button image 54 is an image corresponding to a registration start button that is touched and operated by the driver in order to start a first parking travel process of a parking assist control described later.

[0097] The registration button image 55 is an image of a registration button that is touched and operated by the driver to register the parking space information Ipark acquired by the parking assistance control described later in the vehicle parking assistance device 10 (particularly, the RAM of the ECU 90). The parking space information Ipark is information related to the parking space 62 that is used by the vehicle parking assistance device 10 to automatically park the vehicle 100 in the parking space 62.

[0098] The parking start button image 56 is an image corresponding to a parking start button that is touched and operated by the driver to start parking assist control, which will be described later, for parking the vehicle 100 in the parking zone 61 registered in the vehicle parking assist device 10 .

[0099] The move button image 57 includes an upper move button image 57U, a lower move button image 57D, a left move button image 57L, and a right move button image 57R. The upper move button image 57U is an image operated by the driver to move the parking partition line image 52 upward on the display 50. The lower move button image 57D is an image operated by the driver to move the parking partition line image 52 downward on the display 50. The left move button image 57L is an image operated by the driver to move the parking partition line image 52 to the left on the display 50. The right move button image 57R is an image operated by the driver to move the parking partition line image 52 to the right on the display 50.

[0100] The angle adjustment button image 58 includes a left turn angle adjustment button image 58A and a right turn angle adjustment button image 58B. The left turn angle adjustment button image 58A is an image operated by the driver to rotate the parking partition line image 52 in the left turn direction on the display 50. The right turn angle adjustment button image 58B is an image operated by the driver to rotate the parking partition line image 52 in the right turn direction on the display 50.

[0101] <Overview of the operation of the vehicle parking assist system>

[0102] Next, the operation of the vehicle parking assistance device 10 will be described in general. The vehicle parking assistance device 10 is configured to be able to perform parking assistance control. The parking assistance control is a control for parking the vehicle 100 in the parking section 61 without the driver having to operate the accelerator pedal 14, the brake pedal 15, or the steering wheel 16.

[0103] There are parking lots where parking sections are divided by white lines or the like (hereinafter referred to as "section lines"). When a vehicle is automatically parked in such a parking lot, the vehicle can be automatically driven and parked in the parking section using the section lines photographed by a camera as markers.

[0104] On the other hand, there are parking lots such as parking lots of private houses that do not have partition lines for dividing parking zones. When the vehicle is automatically parked in such a parking lot, there are no partition lines as markers. The parking assistance control performed by the vehicle parking assistance device 10 includes control for automatically parking the vehicle in the parking lot and registering parking lot information related to the parking lot, and control for automatically parking the vehicle in the parking lot where the parking lot information has been registered.

[0105] After the vehicle speed SPD becomes less than the prescribed vehicle speed SPDth, the vehicle parking assistance device 10 performs a search process of searching the camera image CMR for an image portion having a light and dark pattern that is respectively consistent with the light and dark patterns of a plurality of registered entrance feature points Fent_reg. The vehicle parking assistance device 10 uses the left image information IMG3 and the right image information IMG4 to perform the search process. The registered entrance feature point Fent_reg is an entrance feature point Fent of the registered entrance light and dark information CTent_reg registered (i.e., stored) in the vehicle parking assistance device 10 through the parking assistance control, which will be described in detail later. The entrance feature point Fent is a feature point F of the entrance 62ent of the parking lot 62 acquired through the parking assistance control. In addition, the registered entrance light and dark information CTent_reg is the light and dark information CT of the registered entrance feature point Fent_reg. In addition, the camera image CMR is an image captured and acquired by the camera 45.

[0106] When image portions having light and dark patterns respectively consistent with the light and dark patterns of multiple registered entry feature points Fent_reg are extracted from the camera image CMR, the vehicle parking assistance device 10 compares (or matches) the positional relationship between the multiple extracted image portions and the positional relationship between the multiple registered entry feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of the multiple image portions.

[0107] When the positional relationship between the extracted multiple image parts and the positional relationship between the multiple registered entry feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of the multiple image parts are consistent, the vehicle parking assistance device 10 determines that there is a registered parking lot 62 near the vehicle 100. The registered parking lot 62 is a parking lot whose parking lot information Ipark has been registered (i.e., stored) in the vehicle parking assistance device 10 through the parking assistance control.

[0108] More specifically, when the positional relationship between multiple image parts extracted based on the left image information IMG3 and the positional relationship between multiple registered entry feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of these multiple image parts are consistent, the vehicle parking assistance device 10 determines that there is a registered parking space 62 on the left side of the vehicle 100.

[0109] On the other hand, when the positional relationship between multiple image parts extracted based on the right image information IMG4 and the positional relationship between multiple registered entry feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of these multiple image parts are consistent, the vehicle parking assistance device 10 determines that there is a registered parking space 62 on the right side of the vehicle 100.

[0110] Furthermore, when an image portion having a light-dark pattern that matches the light-dark patterns of the plurality of registered entry feature points Fent_reg is not extracted from the camera image CMR, the vehicle parking assistance device 10 determines that there is no registered parking lot 62 near the vehicle 100. Alternatively, when the positional relationship between the plurality of extracted image portions and the positional relationship between the plurality of registered entry feature points Fent_reg having a light-dark pattern that matches the light-dark patterns of the plurality of image portions are inconsistent, the vehicle parking assistance device 10 also determines that there is no registered parking lot 62 near the vehicle 100.

[0111] <Parking lot registration>

[0112] After the vehicle parking assistance device 10 determines that the vehicle 100 stops near the entrance 62ent of the parking lot 62 and the parking assistance switch image 51M is touched, if the registration start button image 54 is touched and the registration of the parking lot information Ipark starts, the vehicle parking assistance device 10 acquires the temporary entrance information Ient_pre and the temporary intermediate information Imid_pre as described later. In addition, the vehicle parking assistance device 10 registers (i.e. stores) the registration entrance information Ient_reg, the registration in-parking information Iin_reg, and the registration area information Iarea_reg as the parking lot information Ipark as described later.

[0113] The vehicle parking assistance device 10 determines that the vehicle 100 is stopped near the entrance 62ent of the parking lot 62 and that the parking lot 62 is not a registered parking lot, and if Fig. 9 After the parking assist switch image 51M displayed on the display 50 as shown in (A) is touched by the driver, the parking assist switch image 51M is cleared from the display 50. Fig. 9 As shown in FIG. 5B , the camera image 51C, the bird's-eye view image 51P, and the registered main switch image 52M are displayed on the display 50 .

[0114] After the registered main switch image 52M is touched and operated by the driver, the vehicle parking assistance device 10 clears the camera image 51C, the top view image 51P, and the registered main switch image 52M from the display 50. Fig. 9As shown in FIG. 5C , the parking zone selection image 52S is displayed on the display 50 .

[0115] After the driver touches and operates either the parking zone selection image 52S displayed on the display 50 or the parking zone selection image 52S displayed on the display 50 without being dimmed, the vehicle parking assistance device 10 clears the parking zone selection image 52S from the display 50. Fig. 9 As shown in (D) of FIG. 1 , the overhead image 51P, the parking partition line image 52, the setting button image 53, the moving button image 57, and the angle adjustment button image 58 are displayed on the display 50. At this time, when there is an unregistered parking space 62 on the left side of the vehicle 100, the vehicle parking assistance device 10 displays the overhead image 51P on the display 50 in a manner of displaying the parking space image on the left side of the vehicle image, and when there is an unregistered parking space 62 on the right side of the vehicle 100, the vehicle parking assistance device 10 displays the overhead image 51P on the display 50 in a manner of displaying the parking space image on the right side of the vehicle image.

[0116] Furthermore, the vehicle parking assistance device 10 sets the range in which the vehicle 100 can be parked in the parking lot 62 as the parking zone 61 based on the image information IMG and the sonar information SON, and displays the parking zone line image 52 on the display 50 as an image representing the set parking zone 61. The vehicle parking assistance device 10 uses the sonar information SON, for example, to obtain the width of the entrance 62ent of the parking lot 62.

[0117] The driver can move the parking partition line image 52 on the display 50 by touching and operating the move button image 57 before touching and operating the setting button image 53. The driver can change the position of the parking partition 61 in which the vehicle 100 is parked by moving the parking partition line image 52 on the display 50. In addition, the driver can rotate the parking partition line image 52 on the display 50 by touching and operating the angle adjustment button image 58 before touching and operating the setting button image 53. The driver can change the position of the parking partition 61 in which the vehicle 100 is parked by rotating the parking partition line image 52 on the display 50.

[0118] When the setting button image 53 is touched by the driver, the vehicle parking assistance device 10 clears the setting button image 53, the moving button image 57, and the angle adjustment button image 58 from the display 50. Fig.10 As shown in (A) of FIG. 5 , a camera image 51C, a bird's-eye view image 51P, and a registration start button image 54 are displayed.

[0119] Furthermore, when the setting button image 53 is touched and operated by the driver, the vehicle parking assistance device 10 sets the parking zone 61 at a position corresponding to the parking zone line image 52 displayed on the display 50 as the registration target parking zone 61 set.

[0120] Furthermore, when the setting button image 53 is touched and operated by the driver, the vehicle parking assistance device 10 sets a target driving route Rtgt for the vehicle 100 to drive in order to park the vehicle 100 in the registration target parking zone 61set. Fig.11 As shown in FIG. 1 , when the vehicle 100 stops on the right side of the unregistered parking space 62, the vehicle parking assistance device 10 is configured as follows. Fig.12 The target travel route Rtgt is set as shown.

[0121] Furthermore, when the setting button image 53 is touched by the driver when the vehicle 100 stops at the right side of the parking lot 62, the vehicle parking assistance device 10 acquires one or more prescribed numbers of new left feature points F3new as entry feature points Fent for each of the middle division range 73D5, the front division range 73D1, and the rear division range 73D2 of the left range 73. On the other hand, when the setting button image 53 is touched by the driver when the vehicle 100 stops at the left side of the parking lot 62, the vehicle parking assistance device 10 acquires one or more prescribed numbers of new right feature points F4new as entry feature points Fent for each of the middle division range 74D5, the front division range 74D1, and the rear division range 74D2 of the right range 74.

[0122] In this example, when the setting button image 53 is touched and operated by the driver when the vehicle 100 stops on the right side of the parking lot 62, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent for each intermediate segmentation range 73D5 than for each front segmentation range 73D1 and rear segmentation range 73D2. That is, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent in the range 73D5 close to the center of the entrance 62ent of the parking lot 62 among the multiple ranges 73D5, 73D1 and 73D2 than in the ranges 73D1 and 73D2 far from the center of the entrance 62ent of the parking lot 62 among the multiple ranges.

[0123] On the other hand, when the setting button image 53 is touched and operated by the driver when the vehicle 100 stops on the left side of the parking lot 62, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent for each intermediate segmentation range 74D5 than for each front segmentation range 74D1 and rear segmentation range 74D2. That is, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent in the range 74D5 close to the center of the entrance 62ent of the parking lot 62 among the multiple ranges 74D5, 74D1 and 74D2 than in the ranges 74D1 and 74D2 far from the center of the entrance 62ent of the parking lot 62 among the multiple ranges.

[0124] For example, in the vehicle 100 Fig.11 When the vehicle stops at the right side of the parking lot 62 as shown in FIG. Fig.13 and Fig.14 As shown, the vehicle parking assistance device 10 obtains two new left feature points F3new as entry feature points Fent for each of the four middle division ranges 73D5 of the left range 73, obtains one new left feature point F3new as entry feature point Fent for each of the two front division ranges 73D1, and obtains one new left feature point F3new as entry feature point Fent for each of the two rear division ranges 73D2. In addition, when the vehicle 100 stops on the left side of the parking lot 62, the vehicle parking assistance device 10 obtains two new right feature points F4new as entry feature points Fent for each of the four middle division ranges 74D5 of the right range 74, obtains one new right feature point F4new as entry feature point Fent for each of the two front division ranges 74D1, and obtains one new right feature point F4new as entry feature point Fent for each of the two rear division ranges 74D2.

[0125] In addition, when the driver stops the vehicle 100 on the right side of the entrance 62ent of the parking lot 62, the vehicle parking assistance device 10 may be configured to obtain a larger number of entrance feature points Fent for the front end division range 73D1 and the two adjacent intermediate division ranges 73D5 than for the rear end division range 73D2 and the two adjacent intermediate division ranges 73D5. Similarly, when the driver stops the vehicle 100 on the left side of the entrance 62ent of the parking lot 62, the vehicle parking assistance device 10 may be configured to obtain a larger number of entrance feature points Fent for the front end division range 74D1 and the two adjacent intermediate division ranges 74D5 than for the rear end division range 74D2 and the two adjacent intermediate division ranges 74D.

[0126] Furthermore, when the prescribed number of new left feature points F3new cannot be obtained in the middle division range 73D5, the front division range 73D1, and a part of the rear division range 73D2 of the left range 73, the vehicle parking assistance device 10 obtains the same number of new left feature points F3new as the new left feature points F3new that cannot be obtained in the remaining ranges 73D5, 73D1, and 73D2 as the entry feature points Fent. Similarly, when the prescribed number of new right feature points F4new cannot be obtained in the middle division range 74D5, the front division range 74D1, and a part of the rear division range 74D2 of the right range 74, the vehicle parking assistance device 10 obtains the same number of new right feature points F4new as the new right feature points F4new that cannot be obtained in the remaining ranges 74D5, 74D1, and 74D2 as the entry feature points Fent.

[0127] After acquiring the entry feature point Fent, the vehicle parking assistance device 10 acquires and stores the coordinates XY of the acquired entry feature point Fent in the temporary coordinate system Cpre as temporary entry coordinates XYent_pre, and acquires and stores the light and dark information CT of the acquired entry feature point Fent as temporary entry light and dark information CTent_pre. The temporary coordinate system Cpre is a coordinate system with the specified position Ppre in the registration object parking partition 61set as the origin. Therefore, the temporary entry coordinates XYent_pre are the positions of the entry feature point Fent based on the specified position Ppre. In addition, the temporary entry information Ient_pre includes the temporary entry coordinates XYent_pre and the temporary entry light and dark information CTent_pre.

[0128] When the registration start button image 54 is touched by the driver, the vehicle parking assistance device 10 clears the registration start button image 54 from the display 50. Fig.10 As shown in (B) of FIG. 1 , the vehicle parking assistance device 10 displays the camera image 51C and the overhead image 51P on the display 50. At this time, when the unregistered parking lot 62 exists on the left side of the vehicle 100, the vehicle parking assistance device 10 displays the image including the unregistered parking lot 62 captured by the left camera 43 on the display 50 as the camera image 51C, and displays the overhead image 51P on the display 50 in a manner that the parking lot image is displayed on the left side of the vehicle image. On the other hand, when the unregistered parking lot 62 exists on the right side of the vehicle 100, the vehicle parking assistance device 10 displays the image including the unregistered parking lot 62 captured by the right camera 44 on the display 50 as the camera image 51C, and displays the overhead image 51P on the display 50 in a manner that the parking lot image is displayed on the right side of the vehicle image.

[0129] In addition, when the registration start button image 54 is touched by the driver, the vehicle parking assistance device 10 performs a first parking driving process for driving the vehicle 100 along the target driving route Rtgt to the registration target parking section 61set. The first parking driving process is a process for controlling the operation of the vehicle driving force generation device 11, the operation of the braking device 12, and the operation of the steering device 13 so that the vehicle 100 travels along the target driving route Rtgt based on the "image information IMG, object information OBJ, steering angle θst, steering torque TQst, vehicle speed SPD, vehicle yaw rate YR, vehicle longitudinal acceleration Gx, and vehicle lateral acceleration Gy".

[0130] For example, in the vehicle 100 Fig.10 When the vehicle stops at the right side of the unregistered parking space 62 as shown in FIG. 1 , the vehicle parking assistance device 10 starts the first parking driving process, first, as shown in FIG. Fig.14 As shown in FIG. 1 , the vehicle 100 is stopped while moving forward while turning right. Fig.15 As shown, the vehicle parking assistance device 10 causes the vehicle 100 to move backward while turning left.

[0131] During the period until the parking of the vehicle 100 in the parking lot 62 is completed, the vehicle parking assistance device 10 acquires the feature point F at least once, and acquires the coordinates XY of the feature point F in the temporary coordinate system Cpre and the light and dark information CT of the feature point F. In particular, after the vehicle 100 starts traveling in order to park the vehicle 100 in the parking lot 62, the vehicle parking assistance device 10 acquires the feature point F at least once, and acquires the coordinates XY of the feature point F in the temporary coordinate system Cpre and the light and dark information CT of the feature point F when it is predicted that the vehicle 100 will not turn but go straight thereafter.

[0132] In this example, the vehicle parking assistance device 10 is causing the vehicle 100 to move backward at a time point when it is predicted that the vehicle 100 will not turn but will move straight thereafter (see Fig.16 ), and obtain the rear feature point F2 as a new rear feature point F2new. In addition, the vehicle parking assistance device 10 may also obtain the rear feature point F2 when the vehicle 100 is reversed by a specified distance Dtravel_th, from the time when it is predicted that the vehicle 100 will not turn but go straight thereafter, while the vehicle 100 is being reversed. Alternatively, the vehicle parking assistance device 10 may also obtain the rear feature point F2 when the vehicle 100 is reversed by a specified distance Dtravel_th, from the time when it is predicted that the vehicle 100 will not turn but go straight thereafter, and thereafter. In addition, the vehicle parking assistance device 10 may also obtain the rear feature point F2 every time the vehicle 100 is reversed by a specified distance Dtravel_th, from the time when it is predicted that the vehicle 100 will not turn but go straight thereafter, and thereafter, while the vehicle 100 is being reversed. Furthermore, the vehicle parking assistance device 10 may acquire the rear feature point F2 every time the vehicle 100 moves backward a predetermined distance Dtravel_th after predicting that the vehicle 100 will not turn but move straight thereafter while moving the vehicle 100 backward. Furthermore, in addition to acquiring the rear feature point F2, the vehicle parking assistance device 10 may acquire any feature point among the front feature point F1, the left feature point F3, and the right feature point F4.

[0133] In this example, the predetermined distance Dtravel_th is set to a distance at which the rear range 72 when the rear feature point F2 was acquired last time does not overlap with the rear range 72 of the rear feature point F2 at the current medium speed.

[0134] Then, the vehicle parking assistance device 10 obtains at least one new rear feature point F2new existing in each rear divided range 72D as an intermediate feature point Fmid. The vehicle parking assistance device 10 obtains and stores the coordinates XY of the obtained intermediate feature point Fmid in the temporary coordinate system Cpre as temporary intermediate coordinates XYmid_pre, and obtains and stores the light and dark information CT of the obtained intermediate feature point Fmid as temporary intermediate light and dark information CTmid_pre. The temporary intermediate coordinates XYmid_pre are the positions of the intermediate feature points Fmid based on the specified position Ppre. The temporary intermediate information Imid_pre includes the temporary intermediate coordinates XYmid_pre and the temporary intermediate light and dark information CTmid_pre.

[0135] The vehicle parking assistance device 10 performs a safety determination process during the execution of the first parking driving process, that is, determines whether the vehicle 100 can be safely driven to the registered parking section 61set without contacting the solid objects existing in the parking lot 62 when the vehicle 100 is driven along the target driving route Rtgt. When the vehicle parking assistance device 10 determines that the vehicle 100 cannot be safely driven to the registered parking section 61set, the vehicle parking assistance device 10 corrects the target driving route Rtgt in a manner that enables the vehicle 100 to be safely driven to the registered parking section 61set. The vehicle parking assistance device 10 performs the above-mentioned safety determination process using the image information IMG and the object information OBJ acquired during the execution of the first parking driving process.

[0136] In addition, the vehicle parking assistance device 10 performs a route judgment process during the execution of the first parking driving process, that is, it judges whether the vehicle 100 can be parked in the registered object parking section 61set when the vehicle 100 is driven along the target driving route Rtgt. When the vehicle parking assistance device 10 determines that the vehicle 100 cannot be parked in the registered object parking section 61set, the vehicle parking assistance device 10 corrects the target driving route Rtgt in a manner that enables the vehicle 100 to be parked in the registered object parking section 61set. The vehicle parking assistance device 10 uses the image information IMG (especially the feature point F) acquired during the execution of the first parking driving process to perform the above-mentioned route judgment process.

[0137] When the entire vehicle 100 enters the registration target parking zone 61set (see Fig.18), the vehicle parking assistance device 10 stops the vehicle 100 and ends the first parking driving process. As a result, the parking of the vehicle 100 in the parking lot 62 is completed. At this time, the vehicle parking assistance device 10 obtains the front feature point F1, the left feature point F3 and the right feature point F4 as the new front feature point F1new, the new left feature point F3new and the new right feature point F4new. At this time, the vehicle parking assistance device 10 may also obtain the rear feature point F2 as the new rear feature point F2new.

[0138] Then, the vehicle parking assistance device 10 obtains at least one new front feature point F1new existing in the front segmentation range 71D as the final feature point Ffin, obtains at least one new left feature point F3new existing in the left segmentation range 73D as the final feature point Ffin, and obtains at least one new right feature point F4new existing in the right segmentation range 74D as the final feature point Ffin. At this time, when the vehicle parking assistance device 10 obtains a new rear feature point F2new, it can also obtain at least one new rear feature point F2new existing in the rear segmentation range 72D as the final feature point Ffin.

[0139] <Registration of parking lot information>

[0140] Furthermore, when the parking of the vehicle 100 in the parking lot 62 is completed, the vehicle parking assistance device 10 clears the camera image 51C from the display 50, as shown in FIG. Fig.10 As shown in FIG. 5C , the overhead view image 51P, the registration button image 55 , the move button image 57 , and the angle adjustment button image 58 are displayed on the display 50 .

[0141] The driver can move the parking zone line image 52 on the display 50 by touching the move button image 57 before touching the register button image 55. The driver can change the position of the parking zone 61 where the vehicle 100 is parked by moving the parking zone line image 52 on the display 50. In addition, the driver can rotate the parking zone line image 52 on the display 50 by touching the angle adjustment button image 58 before touching the register button image 55. The driver can change the position of the parking zone 61 where the vehicle 100 is parked by rotating the parking zone line image 52 on the display 50.

[0142] When the registration button image 55 is touched by the driver, the vehicle parking assistance device 10 obtains and registers (i.e. stores) the coordinates XY of the acquired final feature point Ffin in the registration coordinate system Creg as the registration field coordinates XYin_reg, and obtains and registers (i.e. stores) the light and dark information CT of the acquired final feature point Ffin as the registration field light and dark information CTin_reg. The registration coordinate system Creg is a coordinate system with a predetermined position Preg at the center of the vehicle width direction Dy on the axis connecting the left rear wheel and the right rear wheel of the vehicle 100 when the vehicle 100 is parked in the registration object parking partition 61set as the origin (refer to Fig.18 ). Thus, the registration field coordinate XYin_reg is the position of the final feature point Ffin based on the predetermined position Preg.

[0143] In addition, the vehicle parking assistance device 10 transforms the temporary intermediate coordinates XYmid_pre into the coordinates XY in the registration coordinate system Creg to obtain and register (i.e. store) as the registration field coordinates XYin_reg, and registers (i.e. stores) the temporary intermediate light and dark information CTmid_pre as the registration field light and dark information CTin_reg. Thus, the registration field coordinates XYin_reg are the positions of the intermediate feature points Fmid based on the specified position Preg.

[0144] The registration field information Iin_reg includes the registration field coordinates XYin_reg and the registration field brightness information CTin_reg.

[0145] The vehicle parking assistance device 10 registers (ie stores) the coordinates XY of the registration target parking section 61set in the registration coordinate system Creg as the registration section coordinates XYarea_reg. The registration section coordinates XYarea_reg are the positions of the parking section 61 based on the predetermined position Preg. The registration section information Iarea_reg includes the registration section coordinates XYarea_reg.

[0146] In addition, the vehicle parking assistance device 10 transforms the temporary entrance coordinates XYent_pre into the coordinates XY in the registration coordinate system Creg to obtain and register (i.e. store) as the registered entrance coordinates XYent_reg, and registers (i.e. stores) the temporary entrance light and dark information CTent_pre as the registered entrance light and dark information CTent_reg. Thus, the registered entrance coordinates XYent_reg are the positions of the entrance feature points Fent based on the specified position Preg. The registered entrance information Ient_reg includes the registered entrance coordinates XYent_reg and the registered entrance light and dark information CTent_reg.

[0147] In addition, as described above, the parking lot information Ipark includes the registered entrance information Ient_reg, the registered in-park information Iin_reg, and the registered area information Iarea_reg.

[0148] In this case, the vehicle parking assistance device 10 registers the registered entrance information Ient_reg as the parking lot information Ipark, and registers the registered in-park information Iin_reg related to the final feature point Ffin as the parking lot information Ipark.

[0149] <Parking your vehicle in a registered parking lot>

[0150] When the vehicle parking assistance device 10 determines that the vehicle 100 has stopped near the entrance 62ent of the parking lot 62 and that the parking lot 62 is a registered parking lot, the vehicle parking assistance device 10 performs the following steps: Fig.10 After the parking assist switch image 51M displayed on the display 50 as shown in (A) is touched and operated by the driver, the vehicle parking assist device 10 clears the parking assist switch image 51M from the display 50. Fig.19 As shown in (B) of FIG. 1 , the camera image 51C, the overhead image 51P, the registered main switch image 52M, and the parking start button image 56 are displayed on the display 50. At this time, when the registered parking lot 62 exists on the left side of the vehicle 100, the vehicle parking assistance device 10 displays the image including the registered parking lot 62 captured by the left camera 43 on the display 50 as the camera image 51C, and displays the overhead image 51P on the display 50 so that the parking lot image is displayed on the left side of the vehicle image. On the other hand, when the registered parking lot 62 exists on the right side of the vehicle 100, the vehicle parking assistance device 10 displays the image including the registered parking lot 62 captured by the right camera 44 on the display 50 as the camera image 51C, and displays the overhead image 51P on the display 50 so that the parking lot image is displayed on the right side of the vehicle image.

[0151] Furthermore, the vehicle parking assistance device 10 specifies the position of the parking zone 61 based on the registered zone coordinates XYarea_reg included in the parking space information Ipark regarding the registered parking space 62 where the vehicle 100 is parked this time.

[0152] When the parking start button image 56 is touched and operated by the driver, the vehicle parking assistance device 10 clears the registered main switch image 52M and the parking start button image 56 from the display 50. Fig.19 As shown in (C) of FIG. 5 , the camera image 51C and the overhead view image 51P continue to be displayed on the display 50 .

[0153] Furthermore, when the parking start button image 56 is touched and operated by the driver, the vehicle parking assistance device 10 sets the parking zone 61 registered in the parking space information Ipark as the target parking zone 61tgt.

[0154] Furthermore, when the parking start button image 56 is touched and operated by the driver, the vehicle parking assistance device 10 sets a target travel route Rtgt for the vehicle 100 to travel in order to park the vehicle 100 in the target parking zone 61tgt.

[0155] Then, the vehicle parking assistance device 10 performs a second parking driving process for driving the vehicle 100 along the target driving route Rtgt to the target parking zone 61tgt. The second parking driving process is a process for controlling the operation of the vehicle driving force generation device 11, the operation of the braking device 12, and the operation of the steering device 13 so that the vehicle 100 drives along the target driving route Rtgt based on the “image information IMG, object information OBJ, steering angle θst, steering torque TQst, vehicle speed SPD, vehicle yaw rate YR, vehicle longitudinal acceleration Gx, and vehicle lateral acceleration Gy”.

[0156] In addition, the vehicle parking assistance device 10 performs a safety determination process during the execution of the second parking driving process, that is, determines whether the vehicle 100 can be safely driven to the target parking section 61tgt without contacting the solid object existing in the parking lot 62 when the vehicle 100 is driven along the target driving route Rtgt. When the vehicle parking assistance device 10 determines that the vehicle 100 cannot be safely driven to the target parking section 61tgt, the target driving route Rtgt is corrected so that the vehicle 100 can be safely driven to the target parking section 61tgt. The vehicle parking assistance device 10 performs the above safety determination process using the image information IMG and the object information OBJ acquired during the execution of the second parking driving process.

[0157] The vehicle parking assistance device 10 performs a search process during the execution of the second parking driving process, that is, searches the camera image CMR for an image portion having a light and dark pattern that is consistent with the light and dark patterns of the plurality of registered feature points Freg. The vehicle parking assistance device 10 performs the search process using the rear image information IMG2, the left image information IMG3, and the right image information IMG4. The registered feature point Freg is a feature point F for which the registration field light and dark information CTin_reg has been registered by the parking assistance control.

[0158] When image portions having light and dark patterns respectively consistent with the light and dark patterns of multiple registered feature points Freg are extracted from the camera image CMR, the vehicle parking assistance device 10 compares (or matches) the positional relationship between the multiple extracted image portions and the positional relationship between the multiple registered feature points Freg having light and dark patterns consistent with the light and dark patterns of the multiple image portions.

[0159] When the positional relationship between the extracted image portions and the positional relationship between the registered feature points Freg having a light-dark pattern that matches the light-dark pattern of the image portions match, the vehicle parking assistance device 10 performs a parking position determination process, that is, based on the positional relationship between the coordinates XY of the registered feature point Freg having a light-dark pattern that matches the light-dark pattern of the extracted image portion and the registered section coordinates XYarea_reg, determines whether the position of the target parking section 61tgt in the parking lot 62 matches the position indicated by the registered section coordinates XYarea_reg. When it is determined that the position of the target parking section 61tgt in the parking lot 62 does not match the position indicated by the registered section coordinates XYarea_reg, the vehicle parking assistance device 10 corrects the position of the target parking section 61tgt in the parking lot 62 so that the position of the target parking section 61tgt in the parking lot 62 matches the position indicated by the registered section coordinates XYarea_reg, and corrects the target driving route Rtgt so that the vehicle 100 can park in the target parking section 61tgt whose position has been corrected.

[0160] When the entire vehicle 100 enters the target parking zone 61tgt, the vehicle parking assistance device 10 stops the vehicle 100 and ends the second parking travel process. As a result, the parking of the vehicle 100 in the parking space 62 is completed.

[0161] The above is an overview of the operation of the vehicle parking assistance device 10. Accordingly, feature points F are acquired three times, when the vehicle 100 stops near the entrance 62ent of the parking lot 62, when the vehicle 100 completes parking in the parking lot 62, and during the period until the vehicle 100 completes parking in the parking lot 62, and the information related to these feature points F is registered as the parking lot information Ipark. As a result, the deviation between "the positional relationship between the vehicle 100 and the parking lot 62 until the vehicle 100 is automatically parked in the parking lot 62" and "the positional relationship between the vehicle 100 and the parking lot 62 when each feature point F is acquired" converges to a smaller range. Therefore, by comparing the image information IMG acquired during the period until the vehicle 100 is automatically parked in the parking lot 62 with the information related to the feature points F registered as the parking lot information Ipark, the positional relationship between the vehicle 100 and the parking lot 62 can be more reliably grasped. As a result, the vehicle 100 can be parked at a predetermined position in the parking lot 62 more reliably.

[0162] Furthermore, according to the vehicle parking assistance device 10, when it is predicted that the vehicle 100 will not turn but go straight from now on, the characteristic points F are acquired, and the information related to these characteristic points F is registered as the parking lot information Ipark. Regardless of the positional relationship of the vehicle 100 relative to the parking lot 62, the fixed parking lot information Ipark can be stably acquired from the camera image CMR when the vehicle 100 is going straight. Therefore, by comparing the image information IMG acquired when the vehicle 100 is going straight and the parking lot information Ipark related to the characteristic points F acquired when the vehicle 100 is going straight, the positional relationship between the vehicle 100 and the parking lot 62 can be more reliably grasped.

[0163] <Specific operation of the vehicle parking assist system>

[0164] Next, the specific operation of the vehicle parking assistance device 10 will be described. The CPU of the ECU 90 of the vehicle parking assistance device 10 is configured to execute the following operation every time a predetermined time has passed. Fig. 20 The routine shown.

[0165] Thus, if it is at the predetermined timing, the CPU starts the processing from step 2000, and advances the processing to step 2005, where it is determined whether the value of the parking assist switch operation flag X1_auto is "1". When it is determined that the vehicle 100 has stopped near the entrance 62ent of the parking lot 62, it is determined that the parking lot 62 is not a registered parking lot, and the parking assist switch image 51M is touched and operated, the value of the parking assist switch operation flag X1_auto is set to "1", and when the parking of the vehicle 100 in the parking lot 62 is completed, the value of the parking assist switch operation flag X1_auto is set to "0".

[0166] If the determination result is "yes" in step 2005, the CPU advances the process to step 2010 to determine whether the value of the first parking travel processing flag X1_exe is "0". The value of the first parking travel processing flag X1_exe is set to "1" when the first parking travel processing starts, and the value of the first parking travel processing flag X1_exe is set to "0" when the first parking travel processing ends.

[0167] If the determination result in step 2010 is YES, the CPU advances the process to step 2015, and displays the camera image 51C, the bird's-eye view image 51P, and the registered main switch image 52M on the display 50 (see FIG. 2 ). Fig. 9 (B)). Next, the CPU advances the processing to step 2020 to determine whether the value of the registered main switch operation flag Xmain is "1". When the registered main switch image 52M is touched, the value of the registered main switch operation flag Xmain is set to "1", and when the parking of the vehicle 100 in the parking lot 62 is completed, the value of the registered main switch operation flag Xmain is set to "0".

[0168] If the determination result in step 2020 is YES, the CPU proceeds to step 2025, clears the camera image 51C, the bird's-eye view image 51P, and the registered main switch image 52M from the display 50, and displays the parking zone selection image 52S on the display 50 (see FIG. Fig. 9 (C)). Next, the CPU advances the processing to step 2030 to determine whether the value of the selection completion flag Xselect is "1". When any parking partition selection image 52S is touched, the value of the selection completion flag Xselect is set to "1", and when the parking of the vehicle 100 in the parking lot 62 is completed, the value of the selection completion flag Xselect is set to "0".

[0169] If the determination result in step 2030 is "yes", the CPU advances the process to step 2035 and executes Fig.21 Thus, after the CPU advances the processing to step 2035, Fig.21 The process starts at step 2100, and the process proceeds to step 2105, where the parking partition selection image 52S is cleared from the display 50, and the top view image 51P, the parking partition line image 52, the setting button image 53, the moving button image 57, and the angle adjustment button image 58 are displayed on the display 50 (see Fig. 9 (D)).

[0170] Next, the CPU advances the processing to step 2110 to determine whether the value of the setting completion flag Xset is "1". When the setting button image 53 is touched, the value of the setting completion flag Xset is set to "1", and when the first parking driving process starts, the value of the setting completion flag Xset is set to "0".

[0171] If the result of the determination in step 2110 is "yes", the CPU proceeds to step 2115, clears the setting button image 53, the moving button image 57, and the angle adjustment button image 58 from the display 50, and displays the camera image 51C, the bird's-eye view image 51P, and the registration start button image 54 on the display 50 (see FIG. Fig.10 Next, the CPU advances the processing to step 2120, and sets the parking partition 61 corresponding to the parking partition line image 52 as the registration object parking partition 61set. Next, the CPU advances the processing to step 2125, and sets the driving route of the vehicle 100 to the registration object parking partition 61set as the target driving route Rtgt. Next, the CPU advances the processing to step 2130, and obtains the temporary entrance information Ient_pre as described above and stores it in the RAM.

[0172] Next, the CPU advances the process to step 2135 to determine whether the value of the registration start flag Xreg_start is "1". When the registration start button image 54 is touched, the value of the registration start flag Xreg_start is set to "1", and when the first parking travel process is started, the value of the registration start flag Xreg_start is set to "0".

[0173] If the determination result at step 2135 is YES, the CPU proceeds to step 2140, clears the registration start button image 54 from the display 50, and displays the camera image 51C and the bird's-eye view image 51P on the display 50 (see FIG. 214). Fig.10 (B)). Next, the CPU advances the processing to step 2145 and starts the first parking driving processing of driving the vehicle 100 along the target driving route Rtgt to the registered parking section 61set. Thereafter, the CPU advances the processing to step 2195. Fig. 20 Step 2095 temporarily ends this routine.

[0174] On the other hand, if the determination result in step 2135 is "NO", the CPU advances the processing to step 2195. Fig. 20 Step 2095 temporarily ends this routine.

[0175] In addition, when the determination result is "No" in step 2120, the CPU also advances the processing to step 2195. Fig. 20 Step 2095 temporarily ends this routine.

[0176] exist Fig. 20 If the determination result at step 2030 is "NO", the CPU directly advances the processing to step 2095 and temporarily ends this routine.

[0177] If a "NO" determination is made in step 2020, the CPU directly advances the processing to step 2095 and temporarily ends this routine.

[0178] If the determination result in step 2010 is "No", the CPU proceeds to step 2060 to determine whether the value of the intermediate information acquisition flag Xmid is "1". If it is predicted that the vehicle 100 will not turn but go straight after that while the vehicle 100 is moving backward, the value of the intermediate information acquisition flag Xmid is set to "1", and if the processing in step 2065 is performed, the value of the intermediate information acquisition flag Xmid is set to "0".

[0179] When it is determined as “YES” in step 2060 , the CPU advances the process to step 2065 , acquires temporary intermediate information Imid_pre and stores it in RAM as described above. Next, the CPU advances the process to step 2070 .

[0180] On the other hand, when the determination is NO at step 2060 , the CPU advances the processing to step 2070 .

[0181] After the CPU advances the processing to step 2070, it continues to execute the first parking travel processing. Next, the CPU advances the processing to step 2075 to determine whether the value of the parking completion flag Xpark_fin is "1". When the vehicle 100 enters the registration target parking partition 61set, the value of the parking completion flag Xpark_fin is set to "1", and when the first parking travel processing is completed, the value of the parking completion flag Xpark_fin is set to "0".

[0182] When it is determined as YES in step 2075, the CPU advances the process to step 2080 to terminate the first parking travel process. Next, the CPU advances the process to step 2095 to temporarily terminate this routine.

[0183] On the other hand, when a “NO” determination is made at step 2075 , the CPU directly advances the processing to step 2095 and temporarily terminates this routine.

[0184] If a "NO" determination is made in step 2005, the CPU advances the process to step 2090 to terminate the display of the overhead image 51P and other images on the display 50. Next, the CPU advances the process to step 2095 to temporarily terminate this routine.

[0185] In addition, the CPU is configured to execute the Fig. 22 Thus, if the specified timing is reached, the CPU will Fig. 22 The process starts at step 2200, and the process proceeds to step 2205, where it is determined whether the value of the information registration request flag Xreg_req is "1". When the parking of the vehicle 100 in the parking lot 62 by the first parking travel process is completed, the value of the information registration request flag Xreg_req is set to "1", and when the parking lot information Ipark is registered in the RAM, the value of the information registration request flag Xreg_req is set to "0".

[0186] If the determination result in step 2205 is "yes", the CPU advances the processing to step 2210, clears the camera image 51C from the display 50, and displays the bird's-eye view image 51P, the move button image 57, the angle adjustment button image 58, and the registration button image 55 on the display 50 (see FIG. Fig.10 (C)). Next, the CPU advances the processing to step 2215 to determine whether the value of the registration determination flag Xreg_det is "1". When the registration button image 55 is touched, the value of the registration determination flag Xreg_det is set to "1", and when the processing of step 2220 is performed, the value of the registration determination flag Xreg_det is set to "0".

[0187] If the determination result is YES in step 2215, the CPU proceeds to step 2220 to register the registration entrance information Ient_reg, the registration in-park information Iin_reg, and the registration area information Iarea_reg as parking lot information Ipark in the RAM as described above. Next, the CPU proceeds to step 2295 to temporarily terminate this routine.

[0188] On the other hand, when a "NO" determination is made at step 2215, the CPU directly advances the processing to step 2195 and temporarily terminates this routine.

[0189] If a "NO" determination is made in step 2205, the CPU directly advances the processing to step 2195 and temporarily terminates this routine.

[0190] In addition, the CPU is configured to execute the Fig.23Thus, when the specified timing is reached, the CPU Fig.23 The process starts at step 2300, and the process proceeds to step 2305, where it is determined whether the value of the parking assist switch operation flag X2_auto is "1". When it is determined that the vehicle 100 has stopped near the parking lot 62, and it is determined that the parking lot 62 is a registered parking lot, and the parking assist switch image 51M is touched and operated, the value of the parking assist switch operation flag X2_auto is set to "1", and when the parking of the vehicle 100 in the parking lot 62 is completed, the value of the parking assist switch operation flag X2_auto is set to "0".

[0191] If the determination result is "yes" in step 2305, the CPU advances the process to step 2310 to determine whether the value of the second parking travel processing flag X2_exe is "0". When the second parking travel processing starts, the value of the second parking travel processing flag X2_exe is set to "1", and when the second parking travel processing ends, the value of the second parking travel processing flag X2_exe is set to "0".

[0192] If the determination result in step 2310 is "yes", the CPU advances the processing to step 2315, clears the parking assist switch image 51M from the display 50, and displays the camera image 51C, the top view image 51P, the registered main switch image 52M, and the parking start button image 56 on the display 50 (see FIG. Fig.19 (B)).

[0193] Next, the CPU advances the processing to step 2317 to determine whether the value of the registered main switch operation flag Xmain is "0". When the registered main switch image 52M is touched, the value of the registered main switch operation flag Xmain is set to "1", and when the parking of the vehicle 100 in the parking lot 62 is completed, the value of the registered main switch operation flag Xmain is set to "0".

[0194] If it is determined to be "yes" in step 2317, the CPU advances the processing to step 2320 to determine whether the value of the parking start flag Xpark_start is "1". When the parking start button image 56 is touched, the value of the parking start flag Xpark_start is set to "1", and when the second parking travel process starts, the value of the parking start flag Xpark_start is set to "0".

[0195] If the determination result in step 2320 is "yes", the CPU advances the process to step 2325, and clears the registered main switch image 52M and the parking start button image 56 from the display 50. At this time, the CPU continues to display the camera image 51C and the overhead image 51P on the display 50 (see FIG. 23). Fig.19 Next, the CPU advances the processing to step 2330, and sets the parking partition 61 registered as the parking lot information Ipark as the target parking partition 61tgt. Next, the CPU advances the processing to step 2335, and sets the driving route for the vehicle 100 to drive to the target parking partition 61tgt as the target driving route Rtgt. Next, the CPU advances the processing to step 2340, and starts the second parking driving processing. Thereafter, the CPU advances the processing to step 2395, and temporarily ends this routine.

[0196] On the other hand, when a "NO" determination is made in step 2320, the CPU directly advances the processing to step 2395 and temporarily ends this routine.

[0197] In addition, if the determination result in step 2317 is "No", the CPU advances the processing to step 2342 and sets the value of the registered main switch operation flag Xmain to "1". Next, the CPU advances the processing to step 2395 and temporarily ends this routine. Fig. 20 In step 2005, it is determined as "yes".

[0198] In addition, if the determination result is "No" in step 2310, the CPU advances the processing to step 2345 to continue the second parking travel processing. Thereafter, the CPU advances the processing to step 2350 to determine whether the value of the parking completion flag Xpark_fin is "1". When the vehicle 100 enters the target parking section 61tgt, the value of the parking completion flag Xpark_fin is set to "1", and when the second parking travel processing is completed, the value of the parking completion flag Xpark_fin is set to "0".

[0199] When it is determined to be YES in step 2350, the CPU advances the process to step 2355 and ends the second parking travel process. Next, the CPU advances the process to step 2395 and temporarily ends this routine.

[0200] On the other hand, when a "NO" determination is made in step 2350, the CPU directly advances the processing to step 2395 and temporarily terminates this routine.

[0201] If a "NO" determination is made in step 2305, the CPU advances the process to step 2360 to terminate the display of the overhead view image 51P and other images on the display 50. Next, the CPU advances the process to step 2395 to temporarily terminate this routine.

[0202] The above is a specific operation of the vehicle parking assistance device 10. According to this, the feature point F (refer to Fig. 20 Step 2065, Fig.21 Step 2130 and Fig. 22 Step 2220), the information related to these feature points F is registered as parking lot information Ipark (refer to Fig. 22 Therefore, by comparing the image information IMG acquired until the vehicle 100 is automatically parked in the parking lot 62 with the information related to the feature point F registered as the parking lot information Ipark, the positional relationship between the vehicle 100 and the parking lot 62 can be more reliably grasped. As a result, the vehicle 100 can be parked in a predetermined position in the parking lot 62 more reliably.

[0203] Furthermore, according to the vehicle parking assistance device 10, when it is predicted that the vehicle 100 will not turn but go straight thereafter, the characteristic point F is acquired (see Fig. 20 In step 2065), the information related to these feature points F is registered as parking lot information Ipark (refer to Fig. 22 Therefore, by comparing the image information IMG acquired when the vehicle 100 is traveling straight with the parking lot information Ipark associated with the feature point F acquired when the vehicle 100 is traveling straight, the positional relationship between the vehicle 100 and the parking lot 62 can be more reliably grasped.

[0204] In addition, the present invention is not limited to the above-described embodiment, and various modifications can be adopted within the scope of the present invention.

[0205] Description of the label

[0206] 10: Vehicle parking assistance device; 11: Vehicle driving force generating device; 12: Braking device; 13: Steering device; 30: Sonar sensor device; 40: Camera device; 51C: Camera image; 51P: Top view image; 52: Parking partition line image; 61: Parking partition; 62: Parking lot; 63: Ground; 73: Left range; 74: Right range; 90: ECU; 100: Vehicle; F: Feature point.

Claims

1. A vehicle parking assistance device, comprising a plurality of cameras mounted on a vehicle, and performing the following parking assistance control: using information on a parking space acquired from camera images of the cameras, while automatically parking the vehicle in the parking space; The vehicle parking assistance device includes: a first registration unit that, when the vehicle is parked near the entrance of the parking lot, acquires a feature point near the entrance of the parking lot from the camera image and registers it as a first feature point; a second registration unit that, when parking of the vehicle in the parking lot is completed by the parking assist control, acquires a feature point at a parking completion position of the parking lot from the camera image captured by the camera and registers it as a second feature point; a determination unit for determining whether the parking lot is a registered parking lot by comparing a feature point of the camera image near the entrance of the parking lot acquired when the vehicle is parked near the entrance of the parking lot with the registered first feature point; as well as The determination unit determines that the parking lot is a registered parking lot by comparing the feature point of the camera image acquired during the execution of the parking assist control with the registered second feature point, thereby determining the positional relationship between the vehicle and the parking lot.

2. The vehicle parking assistance device according to claim 1, wherein: The third registration unit is provided, and the third registration unit obtains the feature point of the parking lot from the camera image at least once during the period from when the vehicle stops next to the parking lot and when the vehicle starts to travel in order to park the vehicle in the parking lot until the parking of the vehicle in the parking lot is completed, and registers the feature point as the third feature point.

3. The vehicle parking assistance device according to claim 2, wherein: The third registration unit is composed of: After the vehicle starts traveling to park the vehicle in the parking lot, when it is predicted that the vehicle will go straight without turning thereafter, a feature point of the parking lot is acquired from the camera image at least once and registered as the third feature point.

4. The vehicle parking assistance device according to claim 3, wherein: The third registration unit is composed of: After the vehicle starts traveling to park the vehicle in the parking lot, at a time point when it is predicted that the vehicle will not turn but go straight thereafter, a feature point of the parking lot is acquired from the camera image and registered as the third feature point.

5. The vehicle parking assistance device according to any one of claims 2 to 4, wherein: The third registration unit is composed of: After the vehicle starts traveling in order to park the vehicle in the parking lot, from the time point when the vehicle is predicted to travel a predetermined distance without turning and moving straight thereafter, feature points of the parking lot are acquired from the camera image and registered as the third feature points.

6. The vehicle parking assistance device according to any one of claims 2 to 4, wherein: The third registration unit is composed of: After the vehicle starts traveling in order to park the vehicle in the parking lot, since it is predicted that the vehicle will not turn but go straight thereafter, each time the vehicle travels a specified distance, information feature points of the parking lot are acquired from the camera image and registered as the third feature points.

7. The vehicle parking assistance device according to claim 6, wherein: The first registration unit, the second registration unit, and the third registration unit are configured to register information related to feature points in the camera image as the first feature point, the second feature point, and the third feature point, respectively.

8. The vehicle parking assistance device according to claim 7, wherein: The first registration unit, the second registration unit, and the third registration unit are configured to register the positions of the feature points based on a predetermined position as the first feature point, the second feature point, and the third feature point, respectively.

Images