Vehicle parking assist device

Abstract

A vehicle parking assistance device is provided, which can reliably grasp the positional relationship between the vehicle and the parking lot when the vehicle automatically travels to park in the parking lot. When the vehicle parks beside the parking lot, the vehicle parking assistance device obtains information related to the parking lot from the camera image as the first information and registers the first information as the parking lot information. When the vehicle completes parking in the parking lot through parking assistance control, the vehicle parking assistance device obtains information related to the parking lot from the camera image as the second information and registers the second information as the parking lot information. After registering the parking lot information, when the vehicle parks beside the parking lot, the vehicle parking assistance device obtains information related to the parking lot from the currently acquired camera image as the current parking lot information. By comparing the current parking lot information with the registered parking lot information, the vehicle parking assistance device performs parking assistance control while grasping the positional relationship between the vehicle and the parking lot.

Description

[0001] Divisional Application Statement

[0002] This application is a divisional application of a Chinese patent application with the invention title of "Vehicle Parking Assistance Device", application number 202011057372.X, and the filing date of September 29, 2020. Technical Field

[0003] The present invention relates to a vehicle parking assistance device. Background Art

[0004] There is known a vehicle parking assistance device that automatically parks a vehicle in a parking lot (such as a parking lot of a private house) where no partition lines such as white lines for partitioning parking areas are provided. As such a vehicle parking assistance device, there is known a vehicle parking assistance device configured as follows (for example, refer to Japanese Unexamined Patent Application Publication No. 2017-138664): When parking a vehicle in a parking lot, information related to the parking lot (hereinafter referred to as "parking lot information") is registered, and when the vehicle automatically parks in the parking lot next time, the parking lot information obtained at that time is compared with the registered parking lot information, and the vehicle is automatically parked in the parking lot while grasping the positional relationship between the vehicle and the parking lot. This vehicle parking assistance device is configured to obtain parking lot information, for example, based on an image of the parking lot captured by a camera (hereinafter referred to as "camera image"). Summary of the Invention

[0005] During the period when a vehicle automatically parks in a parking lot, the positional relationship between the vehicle and the parking lot is constantly changing. And, if the positional relationship between the vehicle and the parking lot (hereinafter referred to as "relative positional relationship") is different, even if the same object is captured in the camera image, the shape of the image of the object captured in the camera image is different. Therefore, in the case where parking lot information obtained from a camera image at a time point when the relative positional relationship is in a certain specific positional relationship is registered, when the vehicle automatically travels to park in the parking lot, even if the parking lot information obtained from a camera image at a time point when the relative positional relationship deviates significantly from the above specific positional relationship is compared with the registered parking lot information, it may not be possible to reliably grasp the relative positional relationship at that time. In this case, there is a possibility that the vehicle cannot be reliably parked at a specified position in the parking lot.

[0006] The present invention is made to address the above problems. That is, one of the objects of the present invention is to provide a vehicle parking assistance device that can reliably grasp the positional relationship between a vehicle and a parking lot when the vehicle automatically travels to park in the parking lot.

[0007] The vehicle parking assistance device according to the present invention includes: a camera mounted on the vehicle so as to be able to photograph the surroundings of the vehicle; and a control unit configured to be able to execute the following parking assistance control, that is, acquire an image captured by the camera as a camera image, and while using information related to the parking lot obtained from the camera image, automatically park the vehicle in the parking lot.

[0008] When the vehicle stops beside the parking lot, the control unit acquires information related to the parking lot from the camera image as first information, and registers the first information as parking lot information. When the parking of the vehicle in the parking lot is completed through the parking assistance control, the control unit acquires information related to the parking lot from the camera image as second information, and registers the second information as the parking lot information.

[0009] In addition, after registering the parking lot information, when the vehicle stops beside the parking lot, the control unit acquires information related to the parking lot from the currently acquired camera image as the current parking lot information, and while grasping the positional relationship between the vehicle and the parking lot by comparing the current parking lot information with the registered parking lot information, executes the parking assistance control.

[0010] Accordingly, the parking lot information is acquired twice, when the vehicle stops near the entrance of the parking lot and when the parking of the vehicle in the parking lot is completed. Thus, during the period until the vehicle is automatically parked in the parking lot, the deviation between the positional relationship between the vehicle and the parking lot and the positional relationship between the vehicle and the parking lot when the parking lot information is acquired converges to a smaller range. Therefore, by comparing the parking lot information acquired during the period until the vehicle is automatically parked in the parking lot with the registered parking lot information, the positional relationship between the vehicle and the parking lot can be grasped more reliably. As a result, the vehicle can be parked more reliably at a specified position in the parking lot.

[0011] In the vehicle parking assistance device according to the present invention, the control unit may also be configured to: at least once during the period from the start of the vehicle's travel through the parking assistance control until the parking of the vehicle in the parking lot is completed through the parking assistance control, acquire information related to the parking lot from the camera image as third information, and register the third information as the parking lot information.

[0012] Accordingly, the parking lot information is acquired three times: when the vehicle stops near the entrance of the parking lot, when the vehicle completes parking in the parking lot, and during the period until the vehicle completes parking in the parking lot. Thereby, the deviation between the positional relationship between the vehicle and the parking lot during the period until the vehicle automatically parks in the parking lot and the positional relationship between the vehicle and the parking lot when the parking lot information is acquired converges to a smaller range. Therefore, it is possible to more reliably grasp the positional relationship between the vehicle and the parking lot by comparing the parking lot information acquired during the period until the vehicle automatically parks in the parking lot with the registered parking lot information.

[0013] In addition, in the vehicle parking assistance device according to the present invention, the control unit may also be configured to: after the vehicle starts to travel by the parking assistance control, until the vehicle completes parking in the parking lot by the parking assistance control, when it is predicted that the vehicle travels straight, during the period until the vehicle completes parking in the parking lot by the parking assistance control, at least once acquire information related to the parking lot from the camera image as the third information, and register the third information as the parking lot information.

[0014] In addition, in the vehicle parking assistance device according to the present invention, the control unit may also be configured to: after the vehicle starts to travel by the parking assistance control, until the vehicle completes parking in the parking lot by the parking assistance control, at the time point when it is predicted that the vehicle travels straight, acquire information related to the parking lot from the camera image as the third information, and register the third information as the parking lot information.

[0015] In addition, in the vehicle parking assistance device according to the present invention, the control unit may also be configured to: after the vehicle starts to travel by the parking assistance control, until the vehicle completes parking in the parking lot by the parking assistance control, at the time point when the vehicle has traveled a predetermined distance from the time point when it is predicted that the vehicle travels straight, acquire information related to the parking lot from the camera image as the fourth information, and register the fourth information as the parking lot information.

[0016] In addition, in the vehicle parking assistance device according to the present invention, the control unit may also be configured to: after the vehicle starts to travel by the parking assistance control, until the vehicle completes parking in the parking lot by the parking assistance control, from the time point when it is predicted that the vehicle travels straight, every time the vehicle travels a predetermined distance, acquire information related to the parking lot from the camera image as the fourth information, and register the fourth information as the parking lot information.

[0017] Accordingly, parking lot information when the vehicle goes straight instead of turning is registered. Regardless of the positional relationship of the vehicle with respect to the parking lot, fixed parking lot information can be stably obtained from the camera image when the vehicle is going straight. Therefore, the positional relationship between the vehicle and the parking lot can be grasped more reliably by comparing the parking lot information obtained when the vehicle is going straight with the parking lot information obtained and registered when the vehicle is going straight.

[0018] In addition, in the vehicle parking assistance device according to the present invention, the information related to the parking lot includes, for example, information related to the feature points in the camera image.

[0019] Accordingly, information related to the feature points of the parking lot in the camera image is registered as parking lot information.

[0020] In addition, in the vehicle parking assistance device according to the present invention, the information related to the parking lot includes, for example, information related to the positions of the feature points in the camera image based on a specified position.

[0021] Accordingly, the positions of the feature points based on the specified position are registered as parking lot information.

[0022] In addition, in the vehicle parking assistance device according to the present invention, the camera includes, for example, a front camera that captures the front of the vehicle, a rear camera that captures the rear of the vehicle, a left camera that captures the left side of the vehicle, and a right camera that captures the right side of the vehicle.

[0023] Each component of the present invention is not limited to the embodiments described with reference to the following drawings. By explaining the embodiments of the present invention described with reference to the following drawings, other objects, other features, and the advantages brought by them can be easily understood. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a diagram showing a vehicle parking assistance device according to an embodiment of the present invention and a vehicle to which the vehicle parking assistance device is applied.

[0025] Figure 2 is a diagram showing the configuration and detection range of a sonar sensor device.

[0026] Figure 3 is a diagram showing the configuration and shooting range of a camera sensor device.

[0027] Figure 4 is a diagram showing an example of a parking lot.

[0028] Figure 5 is a diagram showing a front range and a rear range.

[0029] Figure 6 It is a diagram showing the left range and the right range.

[0030] Figure 7 It is a diagram showing feature points.

[0031] Figure 8 It is a diagram showing parking zones.

[0032] Figure 9 (A) to (D) of is a diagram showing a display.

[0033] Figure 10 (A) to (C) of is a diagram showing a display.

[0034] Figure 11 It is for explaining Figure 1 the operation of the vehicle parking assist device shown.

[0035] Figure 12 It is for explaining Figure 1 the operation of the vehicle parking assist device shown.

[0036] Figure 13 It is for explaining Figure 1 the operation of the vehicle parking assist device shown.

[0037] Figure 14 It is a diagram showing entrance feature points.

[0038] Figure 15 It is for explaining Figure 1 the operation of the vehicle parking assist device shown.

[0039] Figure 16 It is for explaining Figure 1 the operation of the vehicle parking assist device shown.

[0040] Figure 17 It is for explaining Figure 1 the operation of the vehicle parking assist device shown.

[0041] Figure 18 It is for explaining Figure 1 the operation of the vehicle parking assist device shown.

[0042] Figure 19 (A) to (C) of is a diagram showing a display.

[0043] Figure 20 It is showing Figure 1 the flowchart of the routine executed by the CPU of the ECU shown.

[0044] Figure 21 It is showing Figure 1Flowchart of a routine executed by the CPU of the ECU shown

[0045] Figure 22 is a diagram showing Figure 1 Flowchart of a routine executed by the CPU of the ECU shown

[0046] Figure 23 is a diagram showing Figure 1 Flowchart of a routine executed by the CPU of the ECU shown Detailed implementation mode

[0047] Hereinafter, with reference to the drawings, a vehicle parking assist device according to an embodiment of the present invention will be described Figure 1 FIG. shows a vehicle parking assist device 10 according to an embodiment of the present invention and a vehicle 100 to which the vehicle parking assist device 10 is applied

[0048] As Figure 1 shown, the vehicle parking assist device 10 includes an ECU 90. The ECU is an abbreviation for an electronic control unit. The ECU 90 includes a microcomputer as a main component. The microcomputer includes a CPU, a ROM, a RAM, a non-volatile memory, an interface, etc. The CPU is configured to implement various functions by executing instructions, programs or routines stored in the ROM

[0049] The vehicle 100 is equipped with a vehicle driving force generating device 11, a braking device 12 and a steering device 13. The vehicle driving force generating device 11 is a device for generating a driving force for driving the vehicle 100 and applying the driving force to the drive wheels of the vehicle 100. The vehicle driving force generating device 11 is, for example, an internal combustion engine, an electric motor, etc. The braking device 12 is a device for applying a braking force for braking the vehicle 100 to the wheels of the vehicle 100. The steering device 13 is a device for applying a steering torque for steering the vehicle 100 to the steering wheels of the vehicle 100

[0050] The vehicle driving force generating device 11, the braking device 12 and the steering device 13 are electrically connected to the ECU 90. The ECU 90 controls the driving force applied to the drive wheels of the vehicle 100 by controlling the operation of the vehicle driving force generating device 11. In addition, the ECU 90 controls the braking force applied to the wheels of the vehicle 100 by controlling the operation of the braking device 12. In addition, the ECU 90 controls the steering torque applied to the steering wheels of the vehicle 100 by controlling the operation of the steering device 13

[0051] <Sensors, etc.>

[0052] The vehicle parking assist device 10 includes an accelerator pedal operation amount sensor 21, a brake pedal operation amount sensor 22, a steering angle sensor 23, a steering torque sensor 24, a vehicle speed sensor 25, a yaw angular velocity sensor 26, a longitudinal and lateral acceleration sensor 27, a lateral acceleration sensor 28, a sonar sensor device 30, a camera sensor device 40, and a display 50.

[0053] The accelerator pedal operation amount sensor 21 is electrically connected to the ECU 90. The ECU 90 detects the operation amount AP of the accelerator pedal 14 via the accelerator pedal operation amount sensor 21 and obtains the accelerator pedal operation amount AP. The ECU 90 controls the operation of the vehicle driving force generating device 11 such that a driving force corresponding to the obtained accelerator pedal operation amount AP is applied to the drive wheels of the vehicle from the vehicle driving force generating device 11.

[0054] The brake pedal operation amount sensor 22 is electrically connected to the ECU . The ECU 90 detects the operation amount BP of the brake pedal 15 by the driver via the brake pedal operation amount sensor 22 and obtains the brake pedal operation amount BP. The ECU 90 controls the operation of the brake device 12 such that a braking force corresponding to the obtained brake pedal operation amount BP is applied to the wheels of the vehicle from the brake device 12.

[0055] The steering angle sensor 23 is electrically connected to the ECU 90. The ECU 90 detects the rotation angle θst of the steering wheel relative to the neutral position via the steering angle sensor 23 and obtains the steering angle θst.

[0056] The steering torque sensor is electrically connected to the ECU 90. The ECU 90 detects the torque TQst input from the driver to the steering shaft 17 via the steering torque sensor 24 and obtains the steering torque TQst.

[0057] The ECU 90 controls the operation of the steering device 13 such that a steering torque corresponding to the obtained steering angle θst and steering torque TQst is applied to the steering wheels of the vehicle.

[0058] >The vehicle speed sensor 25 is electrically connected to the ECU 90. The ECU 90 detects the rotational speed Vrot of each wheel of the vehicle 100 via the vehicle speed sensor 25 and obtains the rotational speed Vrot of each wheel. The ECU 90 obtains the traveling speed SPD of the vehicle 100 as the vehicle speed SPD based on the obtained rotational speed Vrot of each wheel.

[0059] The yaw angular velocity sensor 26 is electrically connected to the ECU 90. The ECU 90 detects the yaw angular velocity YR of the vehicle 100 via the yaw angular velocity sensor 26 and obtains the vehicle yaw angular velocity YR.

[0060] The front and rear acceleration sensor 27 is electrically connected to the ECU 90. The ECU 90 detects the front and rear acceleration Gx of the vehicle 100 via the front and rear acceleration sensor 27 and obtains the front and rear acceleration Gx of the vehicle.

[0061] The lateral acceleration sensor 28 is electrically connected to the ECU 90. The ECU 90 detects the lateral acceleration Gy of the vehicle 100 via the lateral acceleration sensor 28 and obtains the lateral acceleration Gy of the vehicle.

[0062] The sonar sensor device 30 includes the first gap sonar 301 to the twelfth gap sonar 312.

[0063] As Figure 2 shown, the first gap sonar 301 is installed on the vehicle 100 in such a way that it emits sound waves from the front left end of the vehicle 100 to the left front. The second gap sonar 302 is installed on the vehicle 100 in such a way that it emits sound waves from the front left end of the vehicle 100 to the front. The third gap sonar 303 is installed on the vehicle 100 in such a way that it emits sound waves from the front right end of the vehicle 100 to the right front. The fourth gap sonar 304 is installed on the vehicle 100 in such a way that it emits sound waves from the front right end of the vehicle 100 to the front.

[0064] In addition, the fifth gap sonar 305 is installed on the vehicle 100 in such a way that it emits sound waves from the rear left end of the vehicle 100 to the left rear. The sixth gap sonar 306 is installed on the vehicle 100 in such a way that it emits sound waves from the rear left end of the vehicle 100 to the rear. The seventh gap sonar 307 is installed on the vehicle 100 in such a way that it emits sound waves from the rear right end of the vehicle 100 to the right rear. The eighth gap sonar 308 is installed on the vehicle 100 in such a way that it emits sound waves from the rear right end of the vehicle 100 to the rear.

[0065] In addition, the ninth gap sonar 309 is installed on the vehicle 100 in such a way that it emits sound waves from the front left side of the vehicle 100 to the left. The tenth gap sonar 310 is installed on the vehicle 100 in such a way that it emits sound waves from the rear left side of the vehicle 100 to the left. The eleventh gap sonar 311 is installed on the vehicle 100 in such a way that it emits sound waves from the front right side of the vehicle 100 to the right. The twelfth gap sonar 312 is installed on the vehicle 100 in such a way that it emits sound waves from the rear right side of the vehicle 100 to the right.

[0066] The first gap sonar 301 to the twelfth gap sonar 312 receive the sound waves reflected by the object.

[0067] The sonar sensor device 30 is electrically connected to the ECU 90. The sonar sensor device 30 transmits information related to "the sound waves emitted by the first to twelfth gap sonars 301 to 312" and "the sound waves received by the first to twelfth gap sonars 301 to 312" to the ECU 90. Based on the information received from the sonar sensor device 30 (hereinafter referred to as "sonar information SON"), the ECU 90 obtains information related to the objects existing around the vehicle 100 as object information OBJ.

[0068] In Figure 2 the direction indicated by the reference sign Dx is the longitudinal direction of the vehicle 100, which will hereinafter be referred to as the "vehicle longitudinal direction Dx", and the direction indicated by the reference sign Dy is the width direction of the vehicle 100, which will hereinafter be referred to as the "vehicle width direction Dy".

[0069] The camera sensor device 40 includes a front camera 41, a rear camera 42, a left camera 43, and a right camera 44. Hereinafter, as needed, the front camera 41, the rear camera 42, the left camera 43, and the right camera 44 are collectively referred to as "camera 45".

[0070] As Figure 3 shown, the front camera 41 is installed at the center of the front end portion of the vehicle 100 so as to photograph the scenery in front of the vehicle 100, and its viewing angle 41A is approximately 180°. The rear camera 42 is installed at the center of the rear end portion of the vehicle 100 so as to photograph the scenery behind the vehicle 100, and its viewing angle 42A is also approximately 180°. The left camera 43 is installed at the left side portion of the vehicle 100 so as to photograph the scenery on the left side of the vehicle 100, and its viewing angle 43A is also approximately 180°. The right camera 44 is installed at the right side portion of the vehicle 100 so as to photograph the scenery on the right side of the vehicle 100, and its viewing angle 44A is also approximately 180°.

[0071] The camera sensor device 40 is electrically connected to the ECU 90. The ECU 90 can obtain information related to the images of the scenery photographed by each camera 45 via the camera sensor device 40.

[0072] Hereinafter, as needed, information related to an image of a landscape captured by the front camera 41 is referred to as "front image information IMG1", information related to an image of a landscape captured by the rear camera 42 is referred to as "rear image information IMG2", information related to an image of a landscape captured by the left camera 43 is referred to as "left image information IMG3", and information related to an image of a landscape captured by the right camera 44 is referred to as "right image information IMG4". Further, hereinafter, as needed, the front image information IMG1, the rear image information IMG2, the left image information IMG3, and the right image information IMG4 are collectively referred to as "image information IMG".

[0073] When a specified condition is satisfied, the vehicle parking assist device 10 acquires a feature point F based on the image information IMG. The feature point F is an image within a specified range in which the brightness changes significantly in the images captured by each camera 45.

[0074] For example, when the parking lot 62 shown in Figure 4 is captured by the camera 45, the corner portion of the concrete block 63B, the corner portion of the ground 63 composed of the lawn 63L, and the boundary portion between the ground 63 composed of the block 63B and the ground 63 composed of the lawn 63L are acquired as the feature point F.

[0075] Figure 4 The ground 63 of the parking lot 62 shown in is composed of the ground 63 made of concrete 63C and the ground 63 made of the lawn 63L. Further, a plurality of concrete blocks 63B that block the side grooves are arranged at the entrance 62ent of the parking lot 62. Thus, the ground 63 at the entrance 62ent of the parking lot 62 is composed of the surface of the block 63B.

[0076] The vehicle parking assist device 10 acquires the feature point F (hereinafter referred to as "front feature point F1") on the ground 63 within a specified range 71 in front of the vehicle 100 (see Figure 5 ) based on the front image information IMG1. Further, the vehicle parking assist device 10 acquires the feature point F (hereinafter referred to as "rear feature point F2") on the ground 63 within a specified range 72 behind the vehicle 100 (see Figure 5 ) based on the rear image information IMG2. Further, the vehicle parking assist device 10 acquires the feature point F (hereinafter referred to as "left feature point F3") on the ground 63 within a specified range 73 to the left of the vehicle 100 (see Figure 6 ) based on the left image information IMG3. The vehicle parking assist device 10 acquires the feature point F (hereinafter referred to as "right feature point F4") on the ground 63 within a specified range 74 to the right of the vehicle 100 (see Figure 6 ) based on the right image information IMG4.

[0077] As Figure 5 shown, the specified range 71 is a range defined by line L711, line L712, line L713, and line L714. Line L711 is a line L711 that extends in the vehicle width direction Dy at a specified distance Dset away from the front camera 41 toward the front of the vehicle 100. Line L712 is a line L712 that passes through the front camera 41 and extends in the vehicle width direction Dy. Line L713 is a line that extends in the vehicle longitudinal direction Dx at a specified distance Dset away from the front camera 41 toward the left of the vehicle 100. Line L714 is a line that extends in the vehicle longitudinal direction Dx at a specified distance Dset away from the front camera 41 toward the right of the vehicle 100. Hereinafter, the specified range 71 will be referred to as the "front range 71".

[0078] The front range 71 is divided into eight ranges 71D obtained by equally dividing it into four parts in the vehicle width direction Dy and two parts in the vehicle longitudinal direction Dx. That is, the front range 71 is divided into a plurality of ranges 71D with equal areas. Hereinafter, these ranges 71D will be referred to as the "front divided ranges 71D". In addition, among the front divided ranges 71D, the two front divided ranges 71D set at the left end in the vehicle width direction Dy will be referred to as the "left end divided ranges 71D3", the two front divided ranges 71D set at the right end in the vehicle width direction Dy will be referred to as the "right end divided ranges 71D4", and the four front divided ranges 71D set in the middle in the vehicle width direction Dy will be referred to as the "middle divided ranges 71D5".

[0079] In addition, as Figure 5 shown, the specified range 72 is a range defined by line L721, line L722, line L723, and line L724. Line L721 is a line that passes through the rear camera 42 and extends in the vehicle width direction Dy. Line L722 is a line L722 that extends in the vehicle width direction Dy at a specified distance Dset away from the rear camera 42 toward the rear of the vehicle 100. Line L723 is a line that extends in the vehicle longitudinal direction Dx at a specified distance Dset away from the rear camera 42 toward the left of the vehicle 100. Line L724 is a line that extends in the vehicle longitudinal direction Dx at a specified distance Dset away from the rear camera 42 toward the right of the vehicle 100. Hereinafter, the specified range 72 will be referred to as the "rear range 72".

[0080] The rear range 72 is divided into eight ranges 72D obtained by equally dividing it into four parts in the vehicle width direction Dy and two parts in the vehicle front-rear direction. That is, the rear range 72 is divided into a plurality of ranges 72D with equal areas. Hereinafter, these ranges 72D are referred to as "rear divided ranges 72D". In addition, among the rear divided ranges 72D, the two rear divided ranges 72D set at the left end in the vehicle width direction Dy are referred to as "left end divided ranges 72D3", the two rear divided ranges 72D set at the right end in the vehicle width direction Dy are referred to as "right end divided ranges 72D4", and the four rear divided ranges 72D set in the middle in the vehicle width direction Dy are referred to as "middle divided ranges 72D5".

[0081] In addition, as Figure 6 shown, the specified range 73 is a range defined by the line L731, the line L732, the line L734, and the line L733. The line L731 is a line that extends in the vehicle width direction Dy leaving a specified distance Dset from the left camera 43 toward the front of the vehicle 100. The line L732 is a line that extends in the vehicle width direction Dy leaving a specified distance Dset from the left camera 43 toward the rear of the vehicle 100. The line L733 is a line that extends in the vehicle front-rear direction Dx leaving a specified distance Dset from the left camera 43 toward the left of the vehicle 100. The line L734 is a line that passes through the left camera 43 and extends in the vehicle front-rear direction Dx. Hereinafter, the specified range 73 is referred to as the "left range 73".

[0082] The left range 73 is divided into eight ranges 73D obtained by equally dividing it into four parts in the vehicle front-rear direction Dx and two parts in the vehicle width direction Dy. That is, the left range 73 is divided into a plurality of ranges 73D with equal areas. Hereinafter, these ranges 73D are referred to as "left divided ranges 73D". In addition, among the left divided ranges 73D, the two left divided ranges 73D set at the front end in the vehicle front-rear direction Dx are referred to as "front end divided ranges 73D1", the two left divided ranges 73D set at the rear end in the vehicle front-rear direction Dx are referred to as "rear end divided ranges 73D2", and the four left divided ranges 73D set in the middle in the vehicle front-rear direction Dx are referred to as "middle divided ranges 73D5".

[0083] In addition, as Figure 6As shown, the specified range 74 is the range delimited by line L741, line L742, line L743, and line L744. Line L741 is a line that departs from the right camera 44 by a specified distance Dset toward the front of the vehicle 100 and extends in the vehicle width direction Dy. Line L742 is a line that departs from the right camera 44 by a specified distance Dset toward the rear of the vehicle 100 and extends in the vehicle width direction Dy. Line L743 is a line that passes through the right camera 44 and extends in the vehicle longitudinal direction Dx. Line L744 is a line that departs from the right camera 44 by a specified distance Dset toward the right side of the vehicle 100 and extends in the vehicle longitudinal direction Dx. Hereinafter, the specified range 74 will be referred to as the "right range 74".

[0084] The right range 74 is divided into eight ranges 74D obtained by equally dividing it into four parts in the vehicle longitudinal direction Dx and two parts in the vehicle width direction Dy. That is, the right range 74 is divided into a plurality of ranges 74D with equal areas. Hereinafter, these ranges 74D will be referred to as the "right divided ranges 74D". In addition, among the right divided ranges 74D, the two right divided ranges 74D set at the front end in the vehicle longitudinal direction Dx will be referred to as the "front divided ranges 74D1", the two right divided ranges 74D set at the rear end in the vehicle longitudinal direction Dx will be referred to as the "rear divided ranges 74D2", and the four right divided ranges 74D set in the middle in the vehicle longitudinal direction Dx will be referred to as the "middle divided ranges 74D5".

[0085] When the image of the range corresponding to the feature point F captured by the camera 45 is transformed into a bird's-eye view image, as Figure 7 shown, the transformed image of the range corresponding to the feature point F is an image of a range 75 that is a square with a side of a specified length Lset. When specified conditions are satisfied, the vehicle parking assist device 10 divides the feature point F into 25 identical square ranges 75D, and obtains the brightness LUM of each range 75D. And the vehicle parking assist device 10 respectively calculates the value ΔLUM (= LUM - LUMave) obtained by subtracting the average value LUMave of these obtained brightnesses LUM from each brightness LUM, and obtains the tendency of the high and low of the brightness LUM at the feature point F based on these values ΔLUM as the light and dark information CT. That is, when specified conditions are satisfied, the vehicle parking assist device 10 obtains the light and dark pattern in the image of the feature point F captured by the camera 45 as the light and dark information CT.

[0086] The display 50 is disposed at a position on the vehicle 100 that can be visually recognized by the driver. In this example, the display 50 is the display of a so-called navigation device.

[0087] The display 50 is electrically connected to the ECU 90. The ECU 90 can display various images on the display 50. In this example, the ECU 90 can display the camera image 51C, the top view image 51P, the parking assist switch image 51M, the registration main switch image 52M, the parking zone selection image 52S, the parking zone line image 52, the setting button image 53, the registration start button image 54, the registration button image 55, the parking start button image 56, the movement button image 57, and the angle adjustment button image 58 on the display 50.

[0088] The camera image 51C is an image captured by the camera 45.

[0089] The top view image 51P is an image including the vehicle top view image and the vehicle surrounding image. The vehicle top view image is an image representing the vehicle 100 when viewed from above in the vertical direction. The vehicle surrounding image is an image representing the surrounding of the vehicle 100 when viewed from above in the vertical direction, and at least includes an image representing the parking lot 62. These vehicle top view image and vehicle surrounding image are generated by the ECU 90 based on the image information IMG.

[0090] The parking assist switch image 51M is an image corresponding to the parking assist switch that is touched and operated by the driver to start the parking assist control described later for the vehicle parking assist device 10.

[0091] The registration main switch image 52M is an image corresponding to the registration main switch that is touched and operated by the driver to register (i.e., store) the parking lot information Ipark in the vehicle parking assist device 10 through the parking assist control described later.

[0092] As Figure 9 shown in (C) of

[0093] The left lateral parking zone selection image 52SLa is an image that is touched and operated by the driver to select the parking zone 61 of the parking lot 62 on the left side of the vehicle 100 where the vehicle 100 parks with its orientation changed by 90 degrees. The parking zone 61 is the zone (or range, area) where the vehicle 100 parks through the parking assist control described later. As Figure 8As shown, a parking area 61 is set within a parking lot 62. The left longitudinal parking area selection image 52SLb is an image that is touched and operated by the driver to select a parking area 61 in the parking lot 62 on the left side of the vehicle 100 for parking without changing the orientation of the vehicle 100. The right horizontal parking area selection image 52SRa is an image that is touched and operated by the driver to select a parking area 61 in the parking lot 62 on the right side of the vehicle 100 for parking after changing the orientation of the vehicle 100 by 90 degrees. The right longitudinal parking area selection image 52SRb is an image that is touched and operated by the driver to select a parking area 61 in the parking lot 62 on the right side of the vehicle 100 for parking without changing the orientation of the vehicle 100.

[0094] In addition, when there is no parking area 61 in the parking lot 62 on the left side of the vehicle 100 for parking after changing the orientation of the vehicle 100 by 90 degrees, the left horizontal parking area selection image 52SLa is not displayed on the display 50 or is displayed on the display 50 with reduced brightness (so-called dimmed tone). Similarly, when there is no parking area 61 in the parking lot 62 on the left side of the vehicle 100 for parking without changing the orientation of the vehicle 100, the left longitudinal parking area selection image 52SLb is not displayed on the display 50 or is displayed on the display 50 with reduced brightness (so-called dimmed tone). Similarly, when there is no parking area 61 in the parking lot 62 on the right side of the vehicle 100 for parking after changing the orientation of the vehicle 100 by 90 degrees, the right horizontal parking area selection image 52SRa is not displayed on the display 50 or is displayed on the display 50 with reduced brightness (so-called dimmed tone). Similarly, when there is no parking area 61 in the parking lot 62 on the right side of the vehicle 100 for parking without changing the orientation of the vehicle 100, the right longitudinal parking area selection image 52SRb is not displayed on the display 50 or is displayed on the display 50 with reduced brightness (so-called dimmed tone).

[0095] Furthermore, when a parking area selection image 52S that is displayed on the display 50 with dimmed tone is touched and operated by the driver, the ECU 90 does not determine that the parking area 61 corresponding to the touched and operated parking area selection image 52S is selected.

[0096] The parking area line image 52 is an image representing the parking area 61.

[0097] The setting button image 53 is an image corresponding to a setting button that is touched and operated by the driver to set (or confirm, determine) the parking area 61 where the vehicle 100 parks in the parking assistance control described later.

[0098] The registration start button image 54 is an image corresponding to the registration start button that is touched and operated by the driver to start the first parking driving process of the parking assistance control described later.

[0099] The registration button image 55 is an image corresponding to the registration button that is touched and operated by the driver to register the parking lot information Ipark obtained through the parking assistance control described later in the vehicle parking assistance device 10 (especially the RAM of the ECU 90). The parking lot information Ipark is information related to the parking lot 62 for the vehicle parking assistance device 10 to automatically park the vehicle 100 in the parking lot 62.

[0100] The parking start button image 56 is an image corresponding to the parking start button that is touched and operated by the driver to start the parking assistance control described later for parking the vehicle 100 in the parking area 61 registered in the vehicle parking assistance device 10.

[0101] The movement button image 57 includes an upper movement button image 57U, a lower movement button image 57D, a left movement button image 57L, and a right movement button image 57R. The upper movement button image 57U is an image operated by the driver to move the parking area line image 52 upward on the display 50. The lower movement button image 57D is an image operated by the driver to move the parking area line image 52 downward on the display 50. The left movement button image 57L is an image operated by the driver to move the parking area line image 52 leftward on the display 50. The right movement button image 57R is an image operated by the driver to move the parking area line image 52 rightward on the display 50.

[0102] The angle adjustment button image 58 includes a left turn angle adjustment button image 58A and a right turn angle adjustment button image 58B. The left turn angle adjustment button image 58A is an image operated by the driver to rotate the parking area line image 52 in the left turn direction on the display 50. The right turn angle adjustment button image 58B is an image operated by the driver to rotate the parking area line image 52 in the right turn direction on the display 5,0.

[0103] <Overview of the operation of the vehicle parking assistance device>

[0104] Next, an overview of the operation of the vehicle parking assistance device 10 will be described. The vehicle parking assistance device 10 is configured to be able to execute parking assistance control. The parking assistance control is a control for parking the vehicle 100 in the parking area 61 without the driver operating the accelerator pedal 14, the brake pedal 15, and the steering wheel 16.

[0105] There are parking lots where parking areas are demarcated by lines such as white lines (hereinafter referred to as "partition lines"). When automatically parking a vehicle in such a parking lot, the partition lines captured by a camera can be used as markers to automatically drive the vehicle and park it in the parking area.

[0106] On the other hand, there are also parking lots such as those in private residences that do not have partition lines demarcating parking areas. When automatically parking a vehicle in such a parking lot, there are no partition lines as markers. The parking assistance control performed by the vehicle parking assistance device 10 includes control to automatically park the vehicle in this parking lot and register parking lot information related to this parking lot, and control to automatically park the vehicle in a parking lot where the parking lot information has been registered.

[0107] After the vehicle speed SPD becomes equal to or lower than a specified vehicle speed SPDth, the vehicle parking assistance device 10 performs a search process of searching for an image portion in the camera image CMR that has light and dark patterns respectively consistent with the light and dark patterns of a plurality of registered entrance feature points Fent_reg. The vehicle parking assistance device 10 uses the left image information IMG3 and the right image information IMG4 to perform this search process. The registered entrance feature point Fent_reg is the entrance feature point Fent of the registered entrance light and dark information CTent_reg that is registered (i.e., stored) in the vehicle parking assistance device 10 through parking assistance control, which will be described in detail later. The entrance feature point Fent is the feature point F of the entrance 62ent of the parking lot 62 obtained through parking assistance control. In addition, the registered entrance light and dark information CTent_reg is the light and dark information CT of the registered entrance feature point Fent_reg. In addition, the camera image CMR is an image captured and obtained by the camera 45.

[0108] When an image portion having light and dark patterns respectively consistent with the light and dark patterns of a plurality of registered entrance feature points Fent_reg is extracted from the camera image CMR, the vehicle parking assistance device 10 compares (or matches) the positional relationships between these extracted multiple image portions, and the positional relationships between a plurality of registered entrance feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of these multiple image portions.

[0109] When the positional relationships between the extracted multiple image portions and the positional relationships between a plurality of registered entrance feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of these multiple image portions are consistent, the vehicle parking assistance device 10 determines that a registered parking lot 62 exists near the vehicle 100. The registered parking lot 62 is a parking lot in which the parking lot information Ipark has been registered (i.e., stored) in the vehicle parking assistance device 10 through parking assistance control.

[0110] More specifically, when the positional relationships among the plurality of image portions extracted based on the left-side image information IMG3 are consistent with the positional relationships among the plurality of registered entrance feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of these plurality of image portions, the vehicle parking assistance device 10 determines that there is a registered parking lot 62 on the left side of the vehicle 100.

[0111] On the other hand, when the positional relationships among the plurality of image portions extracted based on the right-side image information IMG4 are consistent with the positional relationships among the plurality of registered entrance feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of these plurality of image portions, the vehicle parking assistance device 10 determines that there is a registered parking lot 62 on the right side of the vehicle 100.

[0112] In addition, when no image portion having a light and dark pattern consistent with the light and dark pattern of the plurality of registered entrance feature points Fent_reg is extracted from the camera image CMR, the vehicle parking assistance device 10 determines that there is no registered parking lot 62 near the vehicle 100. Or, when the positional relationships among the plurality of extracted image portions are inconsistent with the positional relationships among the plurality of registered entrance feature points Fent_reg having light and dark patterns consistent with the light and dark patterns of these plurality of image portions, the vehicle parking assistance device 10 also determines that there is no registered parking lot 62 near the vehicle 100.

[0113] <Registration of Parking Lot>

[0114] After the vehicle parking assistance device 10 determines that the vehicle 100 has stopped near the entrance 62ent of the parking lot 62 and the parking assistance switch image 51M has been touched and operated, if the registration start button image 54 is touched and operated to start the registration of the parking lot information Ipark, the vehicle parking assistance device 10 obtains temporary entrance information Ient_pre and temporary intermediate information Imid_pre as described later. In addition, the vehicle parking assistance device 10 registers (i.e., stores) the registered entrance information Ient_reg, the in-parking lot information Iin_reg, and the registered partition information Iarea_reg as the parking lot information Ipark as described later.

[0115] After the vehicle parking assistance device 10 determines that the vehicle 100 has stopped near the entrance 62ent of the parking lot 62 and determines that this parking lot 62 is not a registered parking lot, and after the parking assistance switch image 51M displayed on the display 50 as shown in (A) of Figure 9 is touched and operated by the driver, the parking assistance switch image 51M is cleared from the display 50, as shown in Figure 9As shown in (B) of, the camera image 51C, the top view image 51P, and the registration main switch image 52M are displayed on the display 50.

[0116] After the registration main switch image 52M is touched and operated by the driver, the vehicle parking assist device 10 clears the camera image 51C, the top view image 51P, and the registration main switch image 52M from the display 50, and as Figure 9 shown in (C) of, the parking area selection image 52S is displayed on the display 50.

[0117] After any one of the parking area selection image 52S displayed on the display 50 or the parking area selection image 52S that is not dimmed and displayed on the display 50 is touched and operated by the driver, the vehicle parking assist device 10 clears the parking area selection image 52S from the display 50, and as Figure 9 shown in (D) of, the top view image 51P, the parking area line image 52, the setting button image 53, the movement button image 57, and the angle adjustment button image 58 are displayed on the display 50. At this time, when there is an unregistered parking lot 62 on the left side of the vehicle 100, the vehicle parking assist device 10 displays the top view image 51P on the display 50 in such a way that the parking lot image is displayed on the left side of the vehicle image. When there is an unregistered parking lot 62 on the right side of the vehicle 100, the vehicle parking assist device 10 displays the top view image 51P on the display 50 in such a way that the parking lot image is displayed on the right side of the vehicle image.

[0118] In addition, the vehicle parking assist device 10 sets the range in which the vehicle 100 can park in the parking lot 62 as the parking area 61 based on the image information IMG and the sonar information SON, and displays the above-mentioned parking area line image 52 on the display 50 as an image representing the set parking area 61. The vehicle parking assist device 10 uses the sonar information SON, for example, to obtain the width of the entrance 62ent of the parking lot 62.

[0119] Before touching and operating the setting button image 53, the driver can move the parking area line image 52 on the display 50 by touching and operating the movement button image 57. The driver can change the position of the parking area 61 where the vehicle 100 parks by moving the parking area line image 52 on the display 50. In addition, before touching and operating the setting button image 53, the driver can rotate the parking area line image 52 on the display 50 by touching and operating the angle adjustment button image 58. The driver can change the position of the parking area 61 where the vehicle 100 parks by rotating the parking area line image 52 on the display 50.

[0120] When the setting button image 53 is touched and operated by the driver, the vehicle parking assistance device 10 clears the setting button image 53, the movement button image 57, and the angle adjustment button image 58 from the display 50, and as shown in (A) of Figure 10 , displays the camera image 51C, the top view image 51P, and the registration start button image 54.

[0121] In addition, when the setting button image 53 is touched and operated by the driver, the vehicle parking assistance device 10 sets the parking zone 61 corresponding to the position of the parking zone line image 52 displayed on the display 50 as the registration target parking zone 61set.

[0122] In addition, when the setting button image 53 is touched and operated by the driver, the vehicle parking assistance device 10 sets a target driving route Rtgt for the vehicle 100 to park in the registration target parking zone 61set. For example, as shown in Figure 11 , when the vehicle 100 stops on the right side of the unregistered parking lot 62, the vehicle parking assistance device 10 sets the target driving route Rtgt as shown in Figure 12 .

[0123] In addition, when the setting button image 53 is touched and operated by the driver when the vehicle 100 stops on the right side of the parking lot 62, the vehicle parking assistance device 10 respectively obtains one or more new left feature points F3new as the entrance feature points Fent for the middle divided range 73D5, the front end divided range 73D1, and the rear end divided range 73D2 of the left range 73. On the other hand, when the setting button image 53 is touched and operated by the driver when the vehicle 100 stops on the left side of the parking lot 62, the vehicle parking assistance device 10 respectively obtains one or more new right feature points F4new as the entrance feature points Fent for the middle divided range 74D5, the front end divided range 74D1, and the rear end divided range 74D2 of the right range 74.

[0124] In this example, when the setting button image 53 is touched and operated by the driver when the vehicle 100 stops on the right side of the parking lot 62, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent for each middle divided range 73D5 compared to each front end divided range 73D1 and rear end divided range 73D2. That is, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent in the range 73D5, which is closer to the center of the entrance 62ent of the parking lot 62 among the multiple ranges 73D5, 73D1, and 73D2, compared to the ranges 73D1 and 73D2, which are farther from the center of the entrance 62ent of the parking lot 62 among the multiple ranges.

[0125] On the other hand, when the setting button image 53 is touched and operated by the driver while the vehicle 100 is stopped on the left side of the parking lot 62, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent for each of the intermediate division ranges 74D5 compared to each of the front-end division ranges 74D1 and the rear-end division ranges 74D2. That is, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent in the range 74D5, which is closer to the center of the entrance 62ent of the parking lot 62 among the multiple ranges 74D5, 74D1, and 74D2, compared to the ranges 74D1 and 74D2, which are farther from the center of the entrance 62ent of the parking lot 62 among the multiple ranges.

[0126] For example, when the vehicle 100 is stopped on the right side of the parking lot 62 as Figure 11 shown, as Figure 13 and Figure 14 shown, the vehicle parking assistance device 10 obtains 2 new left feature points F3new as entrance feature points Fent for each of the 4 intermediate division ranges 73D5 of the left range 73, obtains 1 new left feature point F3new as an entrance feature point Fent for each of the 2 front-end division ranges 73D1, and obtains 1 new left feature point F3new as an entrance feature point Fent for each of the 2 rear-end division ranges 73D2. In addition, when the vehicle 100 is stopped on the left side of the parking lot 62, the vehicle parking assistance device 10 obtains 2 new right feature points F4new as entrance feature points Fent for each of the 4 intermediate division ranges 74D5 of the right range 74, obtains 1 new right feature point F4new as an entrance feature point Fent for each of the 2 front-end division ranges 74D1, and obtains 1 new right feature point F4new as an entrance feature point Fent for each of the 2 rear-end division ranges 74D2.

[0127] In addition, when the driver tends to stop the vehicle 100 at a position that is about to arrive compared to a position directly across the entrance 62ent of the parking lot 62 when the vehicle 100 is stopped on the right side of the entrance 62ent of the parking lot 62, the vehicle parking assistance device 10 is configured to obtain a larger number of entrance feature points Fent for the front-end segmentation range 73D1 and two adjacent middle segmentation ranges 73D5 respectively compared to the rear-end segmentation range 73D2 and two adjacent middle segmentation ranges 73D5. Similarly, when the driver tends to stop the vehicle 100 at a position that is about to arrive compared to a position directly across the entrance 62ent of the parking lot 62 when the vehicle 100 is stopped on the left side of the entrance 62ent of the parking lot 62, the vehicle parking assistance device 10 is configured to obtain a larger number of entrance feature points Fent for the front-end segmentation range 74D1 and two adjacent middle segmentation ranges 74D5 respectively compared to the rear-end segmentation range 74D2 and two adjacent middle segmentation ranges 74D.

[0128] Furthermore, when a specified number of new left feature points F3new cannot be obtained in a part of the middle segmentation range 73D5, the front-end segmentation range 73D1, and the rear-end segmentation range 73D2 in the left range 73, the vehicle parking assistance device 10 obtains new left feature points F3new equal in number to the unobtainable new left feature points F3new as entrance feature points Fent in the remaining ranges 73D5, 73D1, and 73D2. Similarly, when a specified number of new right feature points F4new cannot be obtained in a part of the middle segmentation range 74D5, the front-end segmentation range 74D1, and the rear-end segmentation range 74D2 in the right range 74, the vehicle parking assistance device 10 obtains new right feature points F4new equal in number to the unobtainable new right feature points F4new as entrance feature points Fent in the remaining ranges 74D5, 74D1, and 74D2.

[0129] After the vehicle parking assistance device 10 obtains the entrance feature points Fent, it obtains and stores the coordinates XY of the obtained entrance feature points Fent in the temporary coordinate system Cpre as the temporary entrance coordinates XYent_pre, and obtains and stores the brightness information CT of the obtained entrance feature points Fent as the temporary entrance brightness information CTent_pre. The temporary coordinate system Cpre is a coordinate system with a specified position Ppre within the registered target parking zone 61set as the origin. Thus, the temporary entrance coordinates XYent_pre are the positions of the entrance feature points Fent based on the specified position Ppre. In addition, the temporary entrance information Ient_pre includes the temporary entrance coordinates XYent_pre and the temporary entrance brightness information CTent_pre.

[0130] When the registration start button image 54 is touched and operated by the driver, the vehicle parking assistance device 10 clears the registration start button image 54 from the display 50. As shown in (B) of Figure 10 , the vehicle parking assistance device 10 displays the camera image 51C and the bird's-eye view image 51P on the display 50. At this time, when an unregistered parking lot 62 exists on the left side of the vehicle 100, the vehicle parking assistance device 10 displays, as the camera image 51C on the display 50, an image including the unregistered parking lot 62 captured by the left camera 43, and displays the bird's-eye view image 51P on the display 50 in such a manner that the parking lot image is displayed on the left side of the vehicle image. On the other hand, when an unregistered parking lot 62 exists on the right side of the vehicle 100, the vehicle parking assistance device 10 displays, as the camera image 51C on the display 50, an image including the unregistered parking lot 62 captured by the right camera 44, and displays the bird's-eye view image 51P on the display 50 in such a manner that the parking lot image is displayed on the right side of the vehicle image.

[0131] In addition, when the registration start button image 54 is touched and operated by the driver, the vehicle parking assistance device 10 performs a first parking driving process of driving the vehicle 100 along the target driving route Rtgt to the parking area 61set to be registered. The first parking driving process is a process of controlling the operations of the vehicle driving force generating device 11, the braking device 12, and the steering device 13 in such a manner that the vehicle 100 is driven along the target driving route Rtgt based on "image information IMG, object information OBJ, steering angle θst, steering torque TQst, vehicle speed SPD, vehicle yaw angular velocity YR, vehicle longitudinal acceleration Gx, and vehicle lateral acceleration Gy".

[0132] For example, when the vehicle 100 stops on the right side of the unregistered parking lot 62 as shown in Figure 10 , after starting the first parking driving process, the vehicle parking assistance device 10 first makes the vehicle 100 move forward while turning right and then stop, as shown in Figure 14 . Next, as shown in Figure 15 , the vehicle parking assistance device 10 makes the vehicle 100 move backward while turning left.

[0133] During the period until the vehicle 100 completes parking in the parking lot 62, the vehicle parking assist device 10 acquires the feature point F at least once, and acquires the coordinates XY of the feature point F in the temporary coordinate system Cpre and the brightness information CT of the feature point F. In particular, after the vehicle parking assist device 10 starts the travel of the vehicle 100 in order to park the vehicle 100 in the parking lot 62, when it is predicted that the vehicle 100 will go straight without turning hereafter, the vehicle parking assist device 10 acquires the feature point F at least once, and acquires the coordinates XY of the feature point F in the temporary coordinate system Cpre and the brightness information CT of the feature point F.

[0134] In this example, during the period when the vehicle parking assist device 10 reverses the vehicle 100, at the time point when it is predicted that the vehicle 100 will go straight without turning hereafter (refer to Figure 16 ), the rear feature point F2 is acquired as the new rear feature point F2new. In addition, the vehicle parking assist device 10 may also acquire the rear feature point F2 at the time point when the vehicle 100 has reversed only the specified distance Dtravel_th after it is predicted that the vehicle 100 will go straight without turning hereafter during the period when the vehicle parking assist device 10 reverses the vehicle 100. Alternatively, the vehicle parking assist device 10 may also acquire the rear feature point F2 at the time point when the vehicle 100 has reversed only the specified distance Dtravel_th after the time point when it is predicted that the vehicle 100 will go straight without turning hereafter and after that during the period when the vehicle parking assist device 10 reverses the vehicle 100. In addition, the vehicle parking assist device 10 may also acquire the rear feature point F2 every time the vehicle 100 reverses the specified distance Dtravel_th after the time point when it is predicted that the vehicle 100 will go straight without turning hereafter and after that during the period when the vehicle parking assist device 10 reverses the vehicle 100. In addition, the vehicle parking assist device 10 may also acquire the rear feature point F2 every time the vehicle 100 reverses the specified distance Dtravel_th after it is predicted that the vehicle 100 will go straight without turning hereafter during the period when the vehicle parking assist device 10 reverses the vehicle 100. In addition to acquiring the rear feature point F2, the vehicle parking assist device 10 may also acquire any one of the front feature point F1, the left feature point F3, and the right feature point F4.

[0135] In this example, the above-mentioned specified distance Dtravel_th is set to the distance at which the rear range 72 when the rear feature point F2 was acquired last time and the rear range 72 of the rear feature point F2 at medium speed this time do not overlap.

[0136] Then, the vehicle parking assistance device 10 acquires at least one or more new rear feature points F2new existing in each rear division range 72D as intermediate feature points Fmid. The vehicle parking assistance device 10 acquires and stores the coordinates XY of the acquired intermediate feature point Fmid in the temporary coordinate system Cpre as the temporary intermediate coordinate XYmid_pre, and acquires and stores the brightness information CT of the acquired intermediate feature point Fmid as the temporary intermediate brightness information CTmid_pre. The temporary intermediate coordinate XYmid_pre is the position of the intermediate feature point Fmid based on the specified position Ppre. The temporary intermediate information Imid_pre includes the temporary intermediate coordinate XYmid_pre and the temporary intermediate brightness information CTmid_pre.

[0137] During the execution of the first parking driving process, the vehicle parking assistance device 10 performs a safety judgment process, that is, it judges whether the vehicle 100 can travel safely to the registered target parking area 61set without contacting the three-dimensional objects existing in the parking lot 62 when the vehicle 100 travels along the target driving route Rtgt. When the vehicle parking assistance device 10 determines that the vehicle 100 cannot travel safely to the registered target parking area 61set, it corrects the target driving route Rtgt in a manner that enables the vehicle 100 to travel safely to the registered target parking area 61set. The vehicle parking assistance device 10 performs the above safety judgment process using the image information IMG and the object information OBJ acquired during the execution of the first parking driving process.

[0138] In addition, during the execution of the first parking driving process, the vehicle parking assistance device 10 performs a route judgment process, that is, it judges whether the vehicle 100 can park within the registered target parking area 61set when the vehicle 100 travels along the target driving route Rtgt. When the vehicle parking assistance device 10 determines that the vehicle 100 cannot park within the registered target parking area 61set, it corrects the target driving route Rtgt in a manner that enables the vehicle 100 to park within the registered target parking area 61set. The vehicle parking assistance device 10 performs the above route judgment process using the image information IMG (especially the feature point F) acquired during the execution of the first parking driving process.

[0139] When the entire vehicle 100 enters the registered target parking area 61set (refer to Figure 18) The vehicle parking assistance device 10 stops the vehicle 100 to end the first parking driving process. As a result, the vehicle 100 completes parking in the parking lot 62. At this time, the vehicle parking assistance device 10 respectively obtains the front feature point F1, the left feature point F3, and the right feature point F4 as the new front feature point F1new, the new left feature point F3new, and the new right feature point F4new. At this time, the vehicle parking assistance device 10 may also obtain the rear feature point F2 as the new rear feature point F2new.

[0140] Then, the vehicle parking assistance device 10 obtains at least one or more new front feature points F1new existing in the front division range 71D as the final feature point Ffin, obtains at least one or more new left feature points F3new existing in the left division range 73D as the final feature point Ffin, and obtains at least one or more new right feature points F4new existing in the right division range 74D as the final feature point Ffin. At this time, when the vehicle parking assistance device 10 obtains the new rear feature point F2new, it may also obtain at least one or more new rear feature points F2new existing in the rear division range 72D as the final feature point Ffin.

[0141] <Registration of parking lot information>

[0142] In addition, when the vehicle 100 completes parking in the parking lot 62, the vehicle parking assistance device 10 clears the camera image 51C from the display 50, and as shown in (C) of Figure 10 , displays the top view image 51P, the registration button image 55, the movement button image 57, and the angle adjustment button image 58 on the display 50.

[0143] Before the driver touches and operates the registration button image 55, the driver can move the parking partition line image 52 on the display 50 by touching and operating the movement button image 57. The driver can change the position of the parking partition 61 where the vehicle 100 parks by moving the parking partition line image 52 on the display 50. In addition, before the driver touches and operates the registration button image 55, the driver can rotate the parking partition line image 52 on the display 50 by touching and operating the angle adjustment button image 58. The driver can change the position of the parking partition 61 where the vehicle 100 parks by rotating the parking partition line image 52 on the display 50.

[0144] When the registration button image 55 is touched and operated by the driver, the vehicle parking assistance device 10 acquires and registers (i.e., stores) the coordinates XY of the acquired final feature point Ffin in the registration coordinate system Creg as the in-registration field coordinates XYin_reg, and acquires and registers (i.e., stores) the brightness information CT of the acquired final feature point Ffin as the in-registration field brightness information CTin_reg. The registration coordinate system Creg is a coordinate system with the specified position Preg at the center in the vehicle width direction Dy on the axis connecting the left and right rear wheels of the vehicle 100 when parking in the parking area 61set to be registered as the origin (refer to Figure 18 ). Thus, the in-registration field coordinates XYin_reg are the position of the final feature point Ffin based on the specified position Preg.

[0145] In addition, the vehicle parking assistance device 10 transforms the temporary intermediate coordinates XYmid_pre into the coordinates XY in the registration coordinate system Creg to acquire and register (i.e., store) them as the in-registration field coordinates XYin_reg, and registers (i.e., stores) the temporary intermediate brightness information CTmid_pre as the in-registration field brightness information CTin_reg. Thus, the in-registration field coordinates XYin_reg are the position of the intermediate feature point Fmid based on the specified position Preg.

[0146] The in-registration field information Iin_reg includes the in-registration field coordinates XYin_reg and the in-registration field brightness information CTin_reg.

[0147] In addition, the vehicle parking assistance device 10 registers (i.e., stores) the coordinates XY of the parking area 61set to be registered in the registration coordinate system Creg as the parking area coordinates XYarea_reg. The parking area coordinates XYarea_reg are the position of the parking area 61 based on the specified position Preg. The parking area information Iarea_reg includes the parking area coordinates XYarea_reg.

[0148] In addition, the vehicle parking assistance device 10 transforms the temporary entrance coordinates XYent_pre into the coordinates XY in the registration coordinate system Creg to acquire and register (i.e., store) them as the registration entrance coordinates XYent_reg, and registers (i.e., stores) the temporary entrance brightness information CTent_pre as the registration entrance brightness information CTent_reg. Thus, the registration entrance coordinates XYent_reg are the position of the entrance feature point Fent based on the specified position Preg. The registration entrance information Ient_reg includes the registration entrance coordinates XYent_reg and the registration entrance brightness information CTent_reg.

[0149] In addition, as described above, the parking lot information Ipark includes the registered entrance information Ient_reg, the registered in-parking lot information Iin_reg, and the registered zoning information Iarea_reg.

[0150] In addition, the vehicle parking assistance device 10 may also be configured not to acquire the temporary intermediate information Imid_pre. In this case, the vehicle parking assistance device 10 registers the registered entrance information Ient_reg as the parking lot information Ipark, and registers the registered in-parking lot information Iin_reg related to the final feature point Ffin as the parking lot information Ipark.

[0151] <Parking of the vehicle at the registered parking lot>

[0152] When the vehicle parking assistance device 10 determines that the vehicle 100 has stopped near the entrance 62ent of the parking lot 62 and determines that this parking lot 62 is a registered parking lot, after the parking assistance switch image 51M displayed on the display 50 as shown in Figure 10 (A) is touched and operated by the driver, the vehicle parking assistance device 10 clears the parking assistance switch image 51M from the display 50, and as shown in Figure 19 (B), displays the camera image 51C, the bird's-eye view image 51P, the registered main switch image 52M, and the parking start button image 56 on the display 50. At this time, when the registered parking lot 62 exists on the left side of the vehicle 100, the vehicle parking assistance device 10 displays the image including the registered parking lot 62 captured by the left camera 43 as the camera image 51C on the display 50, and displays the bird's-eye view image 51P on the display 50 in such a manner that the parking lot image is displayed on the left side of the vehicle image. On the other hand, when the registered parking lot 62 exists on the right side of the vehicle 100, the vehicle parking assistance device 10 displays the image including the registered parking lot 62 captured by the right camera 44 as the camera image 51C on the display 50, and displays the bird's-eye view image 51P on the display �0 in such a manner that the parking lot image is displayed on the right side of the vehicle image.

[0153] In addition, the vehicle parking assistance device 10 determines the position of the parking zone 61 based on the registered zoning coordinates XYarea_reg included in the parking lot information Ipark related to the registered parking lot 62 where the vehicle 100 is parked this time.

[0154] When the parking start button image 56 is touched and operated by the driver, the vehicle parking assistance device 10 clears the registered main switch image 52M and the parking start button image 56 from the display 50, and as shown in Figure 19 (C), continues to display the camera image 51C and the bird's-eye view image 51P on the display 50.

[0155] In addition, when the parking start button image 56 is touched and operated by the driver, the vehicle parking assist device 10 sets the parking zone 61 registered as the parking lot information Ipark as the target parking zone 61tgt.

[0156] In addition, when the parking start button image 56 is touched and operated by the driver, the vehicle parking assist device 10 sets a target driving route Rtgt for the vehicle 100 to travel in order to park the vehicle 100 in the target parking zone 61tgt.

[0157] Then, the vehicle parking assist device 10 performs a second parking travel process of making the vehicle 100 travel along the target driving route Rtgt to the target parking zone 61tgt. The second parking travel process is a process of controlling the operations of the vehicle driving force generating device 11, the braking device 12, and the steering device 13 in a manner of making the vehicle 100 travel along the target driving route Rtgt based on "image information IMG, object information OBJ, steering angle θst, steering torque TQst, vehicle speed SPD, vehicle yaw angular velocity YR, vehicle longitudinal acceleration Gx, and vehicle lateral acceleration Gy".

[0158] In addition, the vehicle parking assist device 10 performs a safety determination process during the execution of the second parking travel process, that is, determines whether the vehicle 100 can travel safely to the target parking zone 61tgt without contacting the three-dimensional objects existing in the parking lot 62 when making the vehicle 100 travel along the target driving route Rtgt. When the vehicle parking assist device 10 determines that the vehicle 100 cannot travel safely to the target parking zone 61tgt, it corrects the target driving route Rtgt in a manner that enables the vehicle 100 to travel safely to the target parking zone 61tgt. The vehicle parking assist device 10 performs the above safety determination process using the image information IMG and object information OBJ acquired during the execution of the second parking travel process.

[0159] The vehicle parking assist device 10 performs a search process during the execution of the second parking travel process, that is, searches for an image portion having a light and dark pattern that respectively matches the light and dark patterns of the plurality of registered feature points Freg in the camera image CMR. The vehicle parking assist device 10 performs this search process using the rear image information IMG2, the left image information IMG3, and the right image information IMG4. The registered feature points Freg are the feature points F for which the light and dark information CTin_reg in the parking lot has been registered through parking assist control.

[0160] When an image portion having a light and dark pattern that matches the light and dark patterns of a plurality of registered feature points Freg is extracted from the camera image CMR, the vehicle parking assist device 10 compares (or matches) the positional relationships between these extracted multiple image portions and the positional relationships between the multiple registered feature points Freg having light and dark patterns that match the light and dark patterns of these multiple image portions.

[0161] When the positional relationships between the extracted multiple image portions and the positional relationships between the multiple registered feature points Freg having light and dark patterns that match the light and dark patterns of these multiple image portions are consistent, the vehicle parking assist device 10 performs a parking position determination process, that is, based on the positional relationship between the coordinates XY of the registered feature point Freg having a light and dark pattern that matches the light and dark pattern of the extracted image portion and the registration partition coordinates XYarea_reg, it determines whether the position of the target parking partition 61tgt in the parking lot 62 is consistent with the position indicated by the registration partition coordinates XYarea_reg. When the vehicle parking assist device 10 determines that the position of the target parking partition 61tgt in the parking lot 62 is not consistent with the position indicated by the registration partition coordinates XYarea_reg, it corrects the position of the target parking partition 61tgt so that the position of the target parking partition 61tgt in the parking lot 62 is consistent with the position indicated by the registration partition coordinates XYarea_reg, and corrects the target driving route Rtgt so that the vehicle 100 can park in the target parking partition 61tgt whose position has been corrected.

[0162] When the entire vehicle 100 enters the target parking partition 61tgt, the vehicle parking assist device 10 stops the vehicle 100 and ends the second parking driving process. Thereby, the vehicle 100 completes parking in the parking lot 62.

[0163] The above is an overview of the operation of the vehicle parking assistance device 10. Accordingly, when the vehicle 100 stops near the entrance 62ent of the parking lot 62, when the vehicle 100 completes parking in the parking lot 62, and three times during the period until the vehicle 100 completes parking in the parking lot 62, the feature point F is acquired, and the information related to these feature points F is registered as the parking lot information Ipark. Thus, the deviation between "the positional relationship between the vehicle 100 and the parking lot 62 during the period until the vehicle 100 automatically parks in the parking lot 62" and "the positional relationship between the vehicle 100 and the parking lot 62 when each feature point F is acquired" converges to a smaller range. Therefore, by comparing the image information IMG acquired during the period until the vehicle 100 automatically parks in the parking lot 62 with the information related to the feature point F registered as the parking lot information Ipark, the positional relationship between the vehicle 100 and the parking lot 62 can be grasped more reliably. As a result, the vehicle 100 can park at a specified position in the parking lot 62 more reliably.

[0164] In addition, according to the vehicle parking assistance device 10, the feature point F is acquired when it is predicted that the vehicle 100 will go straight without turning hereafter, and the information related to these feature points F is registered as the parking lot information Ipark. Regardless of the positional relationship of the vehicle 100 with respect to the parking lot 62, fixed parking lot information Ipark can be stably acquired from the camera image CMR when the vehicle 100 goes straight. Therefore, by comparing the image information IMG acquired when the vehicle 100 goes straight with the parking lot information Ipark related to the feature point F acquired when the vehicle 100 goes straight, the positional relationship between the vehicle 100 and the parking lot 62 can be grasped more reliably.

[0165] <Specific operation of the vehicle parking assistance device>

[0166] Next, the specific operation of the vehicle parking assistance device 10 will be described. The CPU of the ECU 90 of the vehicle parking assistance device is configured to execute the Figure 20 routine shown.

[0167] Thus, if at a specified timing, the CPU starts processing from step 2000, advances the processing to step 2005, and determines whether the value of the parking assistance switch operation flag X1_auto is "1". When it is determined that the vehicle 100 has stopped near the entrance 62ent of the parking lot 62, it is determined that the parking lot 62 is not a registered parking lot, and the parking assistance switch image 51M is touched and operated, the value of the parking assistance switch operation flag X1_auto is set to "1", and when the vehicle 100 completes parking in the parking lot 62, the value of the parking assistance switch operation flag X1_auto is set to "0".

[0168] When the determination in step 2005 is "Yes", the CPU advances the process to step 2010 and determines whether the value of the first parking driving process flag X1_exe is "0". At the start of the first parking driving process, the value of the first parking driving process flag X1_exe is set to "1", and at the end of the first parking driving process, the value of the first parking driving process flag X1_exe is set to "0".

[0169] When the determination in step 2010 is "Yes", the CPU advances the process to step 2015 and displays the camera image 51C, the top view image 51P, and the registration main switch image 52M on the display 50 (refer to Figure 9 (B)). Next, the CPU advances the process to step 2020 and determines whether the value of the registration main switch operation flag Xmain is "1". When the registration main switch image 52M is touched and operated, the value of the registration main switch operation flag Xmain is set to "1", and when the parking of the vehicle 100 at the parking lot 62 is completed, the value of the registration main switch operation flag Xmain is set to "0".

[0170] When the determination in step 2020 is "Yes", the CPU advances the process to step 2025, clears the camera image 51C, the top view image 51P, and the registration main switch image 52M from the display 50, and displays the parking area selection image 52S on the display 50 (refer to Figure 9 (C)). Next, the CPU advances the process to step 2030 and determines whether the value of the selection completion flag Xselect is "1". When any parking area selection image 52S is touched and operated, the value of the selection completion flag Xselect is set to "1", and when the parking of the vehicle 100 at the parking lot 62 is completed, the value of the selection completion flag Xselect is set to "0".

[0171] When the determination in step 2030 is "Yes", the CPU advances the process to step 2035 and executes the Figure 21 routine shown. Thus, after the CPU advances the process to step 2035, it starts processing from step 2100 of Figure 21 , advances the process to step 2105, clears the parking area selection image 52S from the display 50, and displays the top view image 51P, the parking area line image 52, the setting button image 53, the movement button image 57, and the angle adjustment button image 58 on the display 50 (refer to Figure 9 (D)).

[0172] Next, the CPU advances the process to step 2110 to determine whether the value of the setting completion flag Xset is "1". When the setting button image 53 is touched, the value of the setting completion flag Xset is set to "1", and at the start of the first parking travel process, the value of the setting completion flag Xset is set to "0".

[0173] If the determination in step 2110 is "Yes", the CPU advances the process to step 2115, clears the setting button image 53, the movement button image 57, and the angle adjustment button image 58 from the display 50, and displays the camera image 51C, the top view image 51P, and the registration start button image 54 on the display 50 (see Figure 10 (A)). Next, the CPU advances the process to step 2120, sets the parking zone 61 corresponding to the parking zone line image 52 as the registration target parking zone 61set. Next, the CPU advances the process to step 2125, sets the travel route of the vehicle 100 up to the registration target parking zone 61set as the target travel route Rtgt. Next, the CPU advances the process to step 2130, acquires the temporary entrance information Ient_pre as described above and stores it in the RAM.

[0174] Next, the CPU advances the process to step 2135 to determine whether the value of the registration start flag Xreg_start is "1". When the registration start button image 54 is touched, the value of the registration start flag Xreg_start is set to "1", and at the start of the first parking travel process, the value of the registration start flag Xreg_start is set to "0".

[0175] If the determination in step 2135 is "Yes", the CPU advances the process to step 2140, clears the registration start button image 54 from the display 50, and displays the camera image 51C and the top view image 51P on the display 50 (see Figure 10 (B)). Next, the CPU advances the process to step 2145, starts the first parking travel process of driving the vehicle 100 along the target travel route Rtgt to the registration target parking zone 61set. Thereafter, the CPU advances the process via step 2195 to Figure 20 step 2095 of, and temporarily ends this routine.

[0176] On the other hand, if the determination in step 2135 is "No", the CPU advances the process via step 2195 to Figure 20 step 2095 of, and temporarily ends this routine.

[0177] In addition, if the determination in step 2120 is "No", the CPU also advances the process via step 2195 toFigure 20 In step 2095, this routine is temporarily terminated.

[0178] In Figure 20 If the determination in step 2030 is "No", the CPU directly advances the process to step 2095 to temporarily terminate this routine.

[0179] In addition, if the determination in step 2020 is "No", the CPU directly advances the process to step 2095 to temporarily terminate this routine.

[0180] In addition, if the determination in step 2010 is "No", the CPU advances the process to step 2060 to determine whether the value of the intermediate information acquisition flag Xmid is "1". When it is predicted that the vehicle 100 will go straight without turning after the vehicle 100 reverses, the value of the intermediate information acquisition flag Xmid is set to "1", and when the process of step 2065 is performed, the value of the intermediate information acquisition flag Xmid is set to "0".

[0181] If the determination in step 2060 is "Yes", the CPU advances the process to step 2065 to obtain the temporary intermediate information Imid_pre as described above and stores it in the RAM. Next, the CPU advances the process to step 2070.

[0182] On the other hand, if the determination in step 2060 is "No", the CPU advances the process to step 2070.

[0183] After the CPU advances the process to step 2070, it continues to execute the first parking driving process. Next, the CPU advances the process to step 2075 to determine whether the value of the parking completion flag Xpark_fin is "1". When the vehicle 100 enters the registered object parking area 61set, the value of the parking completion flag Xpark_fin is set to "1", and when the first parking driving process ends, the value of the parking completion flag Xpark_fin is set to "0".

[0184] If the determination in step 2075 is "Yes", the CPU advances the process to step 2080 to end the first parking driving process. Next, the CPU advances the process to step 2095 to temporarily terminate this routine.

[0185] On the other hand, if the determination in step 2075 is "No", the CPU directly advances the process to step 2095 to temporarily terminate this routine.

[0186] In addition, when the determination in step 2005 is "No", the CPU advances the process to step 2090, and ends the display of images such as the bird's-eye view image 51P on the display 50. Next, the CPU advances the process to step 2095, and temporarily ends this routine.

[0187] In addition, the CPU is configured to execute Figure 22 the routine shown whenever a predetermined time has elapsed. Thus, if it is at a predetermined timing, the CPU starts processing from Figure 22 step 2200, advances the process to step 2205, and determines whether the value of the information registration request flag Xreg_req is "1". When the parking of the vehicle 100 at the parking lot 62 is completed by the first parking driving process, the value of the information registration request flag Xreg_req is set to "1", and when the parking lot information Ipark is registered in the RAM, the value of the information registration request flag Xreg_req is set to "0".

[0188] When the determination in step 2205 is "Yes", the CPU advances the process to step 2210, clears the camera image 51C from the display 50, and displays the bird's-eye view image 51P, the movement button image 57, the angle adjustment button image 58, and the registration button image 55 on the display 50 (see Figure 10 (C)). Next, the CPU advances the process to step 2215, and determines whether the value of the registration decision flag Xreg_det is "1". When the registration button image 55 is touched, the value of the registration decision flag Xreg_det is set to "1", and when the process of step 2220 is performed, the value of the registration decision flag Xreg_det is set to "0".

[0189] When the determination in step 2215 is "Yes", the CPU advances the process to step 2220, and registers the registration entry information Ient_reg, the registration in-parking lot information Iin_reg, and the registration area information Iarea_reg as the parking lot information Ipark in the RAM as described above. Next, the CPU advances the process to step 2295, and temporarily ends this routine.

[0190] On the other hand, when the determination in step 2215 is "No", the CPU advances the process directly to step 2195, and temporarily ends this routine.

[0191] In addition, when the determination in step 2205 is "No", the CPU advances the process directly to step 2195, and temporarily ends this routine.

[0192] In addition, the CPU is configured to execute Figure 23The routine shown. Thus, when a specified timing is reached, the CPU starts processing from Figure 23 step 2300, advances the processing to step 2305, and determines whether the value of the parking assist switch operation flag X2_auto is "1". When it is determined that the vehicle 100 has stopped near the parking lot 62, and it is determined that the parking lot 62 is a registered parking lot, and the parking assist switch image 51M is touched and operated, the value of the parking assist switch operation flag X2_auto is set to "1". When the parking of the vehicle 100 at the parking lot 62 is completed, the value of the parking assist switch operation flag X2_auto is set to "0".

[0193] If it is determined to be "yes" in step 2305, the CPU advances the processing to step 2310 and determines whether the value of the second parking driving processing flag X2_exe is "0". At the start of the second parking driving process, the value of the second parking driving processing flag X2_exe is set to "1". At the end of the second parking driving process, the value of the second parking driving processing flag X2_exe is set to "0".

[0194] If it is determined to be "yes" in step 2310, the CPU advances the processing to step 2315, clears the parking assist switch image 51M from the display 50, and displays the camera image 51C, the top view image 51P, the registration main switch image 52M, and the parking start button image 56 on the display 50 (refer to Figure 19 (B) of

[0195] Next, the CPU advances the processing to step 2317 and determines whether the value of the registration main switch operation flag Xmain is "0". When the registration main switch image 52M is touched and operated, the value of the registration main switch operation flag Xmain is set to "1". When the parking of the vehicle 100 at the parking lot 62 is completed, the value of the registration main switch operation flag Xmain is set to "0".

[0196] If it is determined to be "yes" in step 2317, the CPU advances the processing to step 2320 and determines whether the value of the parking start flag Xpark_start is "1". When the parking start button image 56 is touched and operated, the value of the parking start flag Xpark_start is set to "1". At the start of the second parking driving process, the value of the parking start flag Xpark_start is set to "0".

[0197] If it is determined to be "yes" in step 2320, the CPU advances the processing to step 2325 and clears the registration main switch image 52M and the parking start button image 56 from the display 50. At this time, the CPU continues to display the camera image 51C and the top view image 51P on the display 50 (refer toFigure 19 In (C) below. Next, the CPU advances the process to step 2330, sets the parking area 61 registered as the parking lot information Ipark as the target parking area 61tgt. Next, the CPU advances the process to step 2335, sets the driving route for the vehicle 100 to travel to the target parking area 61tgt as the target driving route Rtgt. Next, the CPU advances the process to step 2340, and starts the second parking driving process. After that, the CPU advances the process to step 2395, and temporarily ends this routine.

[0198] On the other hand, when the determination in step 2320 is "No", the CPU directly advances the process to step 2395, and temporarily ends this routine.

[0199] In addition, when the determination in step 2317 is "No", the CPU advances the process to step 2342, and sets the value of the registered main switch operation flag Xmain to "1". Next, the CPU advances the process to step 2395, and temporarily ends this routine. Thus, in Figure 20 the determination in step 2005 is "Yes".

[0200] In addition, when the determination in step 2310 is "No", the CPU advances the process to step 2345, and continues to execute the second parking driving process. After that, the CPU advances the process to step 2350, and determines whether the value of the parking completion flag Xpark_fin is "1". When the vehicle 100 enters the target parking area 61tgt, the value of the parking completion flag Xpark_fin is set to "1", and when the second parking driving process ends, the value of the parking completion flag Xpark_fin is set to "0".

[0201] When the determination in step 2350 is "Yes", the CPU advances the process to step 2355, and ends the second parking driving process. Next, the CPU advances the process to step 2395, and temporarily ends this routine.

[0202] On the other hand, when the determination in step 2350 is "No", the CPU directly advances the process to step 2395, and temporarily ends this routine.

[0203] In addition, when the determination in step 2305 is "No", the CPU advances the process to step 2360, and ends the display of images such as the bird's-eye view image 51P on the display 50. Next, the CPU advances the process to step 2395, and temporarily ends this routine.

[0204] The above is a specific operation of the vehicle parking assistance device 10. According to this, the feature point F (refer to Figure 20 Step 2065, Figure 21 Step 2130 and Figure 22 In step 2220), the information related to these feature points F is registered as parking lot information Ipark (refer to Figure 22 Therefore, by comparing the image information IMG acquired until the vehicle 100 is automatically parked in the parking lot 62 with the information related to the feature point F registered as the parking lot information Ipark, the positional relationship between the vehicle 100 and the parking lot 62 can be more reliably determined. As a result, the vehicle 100 can be more reliably parked in a predetermined position in the parking lot 62.

[0205] Furthermore, according to the vehicle parking assistance device 10, when it is predicted that the vehicle 100 will not turn but go straight thereafter, the feature point F is acquired (see Figure 20 In step 2065), the information related to these feature points F is registered as parking lot information Ipark (refer to Figure 22 Therefore, by comparing the image information IMG acquired when the vehicle 100 is traveling straight with the parking lot information Ipark associated with the feature point F acquired when the vehicle 100 is traveling straight, the positional relationship between the vehicle 100 and the parking lot 62 can be more reliably grasped.

[0206] In addition, the present invention is not limited to the above-described embodiment, and various modifications can be adopted within the scope of the present invention.

[0207] Description of the label

[0208] 10: Vehicle parking assistance device; 11: Vehicle driving force generating device; 12: Braking device; 13: Steering device; 30: Sonar sensor device; 40: Camera device; 51C: Camera image; 51P: Top view image; 52: Parking partition line image; 61: Parking partition; 62: Parking lot; 63: Ground; 73: Left range; 74: Right range; 90: ECU; 100: Vehicle; F: Feature point.

Claims

1. A vehicle parking assistance device that performs parking assistance control for automatically parking the vehicle in a parking space using information about the parking space acquired from a camera image of a camera mounted on the vehicle. The vehicle parking assistance device comprises: a first registration unit that, when parking of the vehicle in the parking lot is completed, acquires a feature of a parking completion position in the parking lot from the camera image and registers the feature as a first feature; a second registration unit for acquiring a feature of the parking lot from the camera image at least once after the vehicle has parked adjacent to the parking lot and during a period from when the vehicle starts traveling to park the vehicle in the parking lot until parking of the vehicle in the parking lot is completed, if it is predicted that the vehicle will not turn but will travel straight thereafter, and registering the feature as a second feature; a first determination unit configured to determine a positional relationship between the vehicle and the parking lot by comparing a feature of the camera image currently acquired during execution of the parking assist control with the registered first feature; as well as The parking unit performs the parking assist control while using the positional relationship determined by the first determination unit and the result of comparing the feature of the camera image currently acquired during the execution of the parking assist control with the registered second feature as the information related to the parking space.

2. The vehicle parking assistance device according to claim 1, wherein: The second registration unit is configured to, after the vehicle starts traveling in order to park the vehicle in the parking lot, acquire a feature of the parking lot from the camera image at a time point when it is predicted that the vehicle will not turn but go straight thereafter, and register the feature as the second feature.

3. The vehicle parking assistance device according to claim 1 or 2, wherein: The second registration unit is configured to, after the vehicle starts traveling in order to park the vehicle in the parking lot, obtain a feature of the parking lot from the camera image at a time point when the vehicle has traveled a predetermined distance since it is predicted that the vehicle will not turn but go straight thereafter, and register the feature as the second feature.

4. The vehicle parking assistance device according to claim 1 or 2, wherein: The second registration unit is configured to, after the vehicle starts traveling in order to park the vehicle in the parking lot, obtain a feature of the parking lot from the camera image each time the vehicle travels a predetermined distance, starting from when it is predicted that the vehicle will go straight without turning, and register the feature as the second feature.

5. The vehicle parking assistance device according to claim 4, wherein: The first registration unit and the second registration unit are configured to register information related to feature points in the camera image as the first feature and the second feature, respectively.

6. The vehicle parking assistance device according to claim 5, wherein: The first registration unit and the second registration unit are configured to register the positions of the feature points with reference to a predetermined position as the first feature and the second feature, respectively.

7. The vehicle parking assistance device according to claim 1 or 2, wherein: The vehicle parking assistance device includes a correction unit that corrects a target driving route for parking the vehicle in the parking space based on the positional relationship between the vehicle and the parking space determined by the first determination unit during execution of the parking assistance control.

Images