Vehicle parking assistance device

Abstract

The present application provides a vehicle parking assistance device that enables automatic parking of a vehicle in a parking lot even when the conditions around the vehicle and the parking lot differ at the time of feature point registration and automatic parking. The vehicle parking assistance device acquires a feature point of the ground at the entrance of a parking lot as an entrance feature point from a captured image when the vehicle is stopped next to the parking lot, performs parking assistance control using parking lot information acquired from the captured image, and registers the parking lot information, which includes entrance feature point information. Thereafter, in a case where the vehicle is stopped next to an unjudged parking lot, a feature point of the ground at the entrance of the unjudged parking lot is acquired as an unverified feature point, and parking assistance control is performed using the parking lot information obtained this time and the registered parking lot information in a case where the unjudged parking lot is judged to be a registered parking lot by comparing the unverified feature point with the entrance feature point information.

Description

[0001] Divisional application statement

[0002] This application is a divisional application of Chinese invention patent application No. 202011035115.6, filed on September 27, 2020, entitled "Vehicle Parking Assist Device". Technical Field

[0003] This invention relates to a vehicle parking assistance device. Background Technology

[0004] A vehicle parking assistance device is known to automatically park a vehicle in a parking lot without designated parking areas (e.g., a parking lot at a private residence). This device is configured to register information related to the parking lot (hereinafter referred to as "parking lot information") when the vehicle parks there, and to compare the previously acquired parking lot information with the registered parking lot information the next time the vehicle needs to be automatically parked in the same parking lot, thus controlling the vehicle's position relative to the parking lot and automatically parking the vehicle there. As another known vehicle parking assistance device, one is known to register feature points of three-dimensional objects captured by a camera in an image (hereinafter referred to as "captured image") as parking lot information (e.g., see Japanese Patent Application Laid-Open No. 2017-138664). Summary of the Invention

[0005] When registering feature points of a 3D object (hereinafter referred to as "feature point registration"), only the driver is in the vehicle. However, when the vehicle is automatically parked in a parking lot (hereinafter referred to as "automatic parking"), other passengers besides the driver are in the vehicle. The vehicle's tilt may differ during feature point registration and automatic parking. Furthermore, the ground surface the vehicle travels on during feature point registration may differ from the ground surface the vehicle travels on during automatic parking. As a result, the vehicle's tilt may also differ depending on the slope of the ground traveled on. In this case, even if the same 3D object is captured in the image, the shape of the 3D object captured in the image may differ between feature point registration and automatic parking. In this case, even if feature points of a 3D object (hereinafter referred to as "registered 3D object") that have been registered are obtained during automatic parking, it may not be possible to determine that these obtained feature points are feature points of the registered 3D object. As a result, the vehicle may not be able to automatically park in the parking lot.

[0006] Furthermore, if feature point registration is performed in the morning but automatic parking occurs in the afternoon, the way sunlight illuminates the three-dimensional object and the way sunlight reflected from the ground illuminates the object differs between the two scenarios. Additionally, if feature point registration is performed during the day but automatic parking occurs at night, the way sunlight illuminates the three-dimensional object and the way sunlight reflected from the ground illuminates the object also differs between the two scenarios. In such cases, even if the same three-dimensional object is captured in the image, the feature points of the object presented in the image may differ between the two scenarios. In this situation, even if feature points of the registered three-dimensional object are obtained during automatic parking, it cannot be determined that these obtained feature points are those of the registered object, potentially resulting in the vehicle failing to automatically park in the parking lot.

[0007] Furthermore, if the feature points of other movable three-dimensional objects such as vehicles, bicycles, and flower pots are registered as parking lot information, and such movable three-dimensional objects have moved away from their registered locations during automatic parking, the feature points of the three-dimensional objects that were registered cannot be obtained during automatic parking. As a result, the vehicle cannot be automatically parked in the parking lot.

[0008] When feature points of a three-dimensional object are registered as parking lot information in this way, if the conditions surrounding the vehicle and parking lot are different when the feature points are registered and when automatic parking is performed, the vehicle may not be able to park automatically in the parking lot.

[0009] This invention was made to address the aforementioned problems. Specifically, one of the objectives of this invention is to provide a vehicle parking assistance device that enables a vehicle to automatically park in a parking lot even when the conditions surrounding the vehicle and the parking lot differ during feature point registration and automatic parking.

[0010] The vehicle parking assistance device of the present invention includes: a camera mounted on the vehicle in a manner capable of capturing images of the vehicle's surroundings; and a control unit capable of performing parking assistance control, the parking assistance control being to automatically park the vehicle in the parking lot using information related to the parking lot, the information related to the parking lot being obtained as parking lot information from images captured by the camera.

[0011] The control unit is configured to acquire an image of the parking lot taken by the camera when the vehicle stops next to the parking lot as a captured image, acquire feature points of the ground at the entrance of the parking lot from the captured image as entrance feature points, perform the parking assistance control using the parking information acquired from the captured image, and register the parking information, wherein the parking information includes information related to the acquired entrance feature points as entrance feature point information.

[0012] Furthermore, the control unit is configured to, after registering the parking information, if the vehicle stops next to an unidentified parking lot, acquire the captured image, obtain feature points of the ground at the entrance of the unidentified parking lot from the captured image as uncompared feature points, and determine whether the unidentified parking lot is the parking lot for which the parking information has been registered (i.e., a registered parking lot) by comparing the information related to the uncompared feature points with the entrance feature point information. If the unidentified parking lot is determined to be the registered parking lot, the control unit performs the parking assistance control using the parking information obtained and the registered parking information.

[0013] Therefore, instead of registering feature points of three-dimensional objects within or around the parking lot, information related to feature points of the ground within and around the parking lot is registered as parking lot information. Even if the conditions surrounding the vehicle and parking lot differ during feature point registration and automatic parking, the changes in the images of the ground within and around the parking lot captured by the camera are minimal. Therefore, even if the conditions surrounding the vehicle and parking lot differ during feature point registration and automatic parking, if a feature point corresponding to an entrance feature point registered as parking lot information (hereinafter referred to as a "registered entrance feature point") is obtained during automatic parking, it can be determined that this feature point is a registered entrance feature point. As a result, the previously unrecognized parking lot can be identified as a registered parking lot. Therefore, vehicles can be automatically parked in the parking lot.

[0014] In the vehicle parking assistance device of the present invention, the control unit may also be configured to divide the captured image into multiple predetermined segmentation ranges, and obtain at least one of the feature points from each of the predetermined segmentation ranges. In this case, the entry feature point is a feature point obtained from each of the predetermined segmentation ranges.

[0015] Therefore, it is possible to obtain entrance feature points from the ground without biasing towards a portion of the parking lot entrance area. This allows for a more reliable determination that an unidentified parking lot is a registered parking lot.

[0016] Furthermore, in the vehicle parking assistance device of the present invention, the specified segmentation range is, for example, the same size.

[0017] Furthermore, in the vehicle parking assistance device of the present invention, the defined division range is, for example, set to not overlap with each other.

[0018] In addition, in the vehicle parking assistance device of the present invention, the control unit may also be configured to obtain a larger number of the feature points from the predetermined segmentation range closer to the center of the entrance of the parking lot, compared with the predetermined segmentation range farther from the center of the entrance of the parking lot, as the entrance feature points.

[0019] This allows for the acquisition of numerous ground feature points near the center of the parking lot entrance. Consequently, it enables a more reliable determination that an unidentified parking lot is a registered parking lot.

[0020] In addition, in the vehicle parking assistance device of the present invention, the control unit may also be configured to obtain a predetermined number of the entrance feature points from each of the predetermined segmentation ranges, and in the case that there is a predetermined segmentation range from which the predetermined number of the entrance feature points cannot be obtained, obtain a greater number of the entrance feature points than the predetermined number from the predetermined segmentation range outside the predetermined segmentation range from which the predetermined number of the entrance feature points cannot be obtained.

[0021] Therefore, even if the required number of entrance feature points are not obtained in a portion of the area, the total number of entrance feature points obtained is the same as or close to the total number obtained when the required number of entrance feature points are obtained in all areas. Thus, it is possible to more reliably determine whether an unidentified parking lot is a registered parking lot.

[0022] Furthermore, in the vehicle parking assistance device of the present invention, the entrance feature point information includes, for example, information related to the density pattern of the image of the entrance feature point.

[0023] Therefore, the varying shades of patterns in the image of the entrance feature points are registered as parking information.

[0024] Furthermore, in the vehicle parking assistance device of the present invention, the entrance feature point information includes, for example, information related to the position of the entrance feature point with a predetermined position within the parking lot as a reference.

[0025] Therefore, the location of the entrance feature point, based on the designated location within the parking lot, is registered as parking lot information.

[0026] In addition, in the vehicle parking assistance device of the present invention, the control unit may also be configured to perform the parking assistance control using the parking information obtained this time when it is determined that the undetermined parking lot is not the registered parking lot.

[0027] Furthermore, in the vehicle parking assistance device of the present invention, the camera includes, for example, a front camera that captures images of the front of the vehicle, a rear camera that captures images of the rear of the vehicle, a left camera that captures images of the left side of the vehicle, and a right camera that captures images of the right side of the vehicle.

[0028] The embodiments will be described with reference to the following accompanying drawings. The structural elements of the present invention are not limited to the described embodiments. Other objects, features, and incidental advantages of the present invention can be readily understood from the description of embodiments of the present invention as described below with reference to the accompanying drawings. Attached Figure Description

[0029] Figure 1 This is a diagram illustrating a vehicle parking assistance device according to an embodiment of the present invention, and a vehicle using the vehicle parking assistance device.

[0030] Figure 2 This is a diagram showing the configuration and detection range of a sonar sensor device.

[0031] Figure 3 This is a diagram showing the configuration of the camera sensor device and the camera's field of view.

[0032] Figure 4 This is a diagram illustrating an example of a parking lot.

[0033] Figure 5 It is a diagram that represents the area in front and the area behind.

[0034] Figure 6 It is a diagram representing the left and right sides of the map.

[0035] Figure 7 It is a diagram showing the feature points.

[0036] Figure 8 It is a map showing the parking area divisions.

[0037] Figure 9 (A) through (D) are diagrams showing the display.

[0038] Figure 10 (A) through (C) are diagrams showing the display.

[0039] Figure 11 It is used for explanation Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0040] Figure 12 It is used for explanation Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0041] Figure 13 It is used for explanation Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0042] Figure 14 It is a diagram representing the feature points at the entrance.

[0043] Figure 15 It is used for explanation Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0044] Figure 16 It is used for explanation Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0045] Figure 17 It is used for explanation Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0046] Figure 18 It is used for explanation Figure 1 The diagram shows the operation of the vehicle parking assistance device.

[0047] Figure 19 (A) through (C) are diagrams showing the display.

[0048] Figure 20 It is shown Figure 1 The flowchart shown is a routine executed by the CPU of the ECU.

[0049] Figure 21 It is shown Figure 1 The flowchart shown is a routine executed by the CPU of the ECU.

[0050] Figure 22 It is shown Figure 1 The flowchart shown is a routine executed by the CPU of the ECU.

[0051] Figure 23 It is shown Figure 1 The flowchart shown is a routine executed by the CPU of the ECU. Detailed Implementation

[0052] Hereinafter, with reference to the accompanying drawings, the vehicle parking assistance device according to embodiments of the present invention will be described. Figure 1 The illustration shows a vehicle parking assistance device 10 according to an embodiment of the present invention, and a vehicle 100 in which the vehicle parking assistance device 10 is applied.

[0053] like Figure 1 As shown, the vehicle parking assistance device 10 has an ECU 90. ECU is short for Electronic Control Unit. The ECU 90 has a microcomputer as its main component. The microcomputer includes a CPU, ROM, RAM, non-volatile memory, and interfaces, etc. The CPU performs various functions by executing instructions, programs, or routines stored in the ROM.

[0054] The vehicle 100 is equipped with a vehicle drive force generating device 11, a braking device 12, and a steering device 13. The vehicle drive force generating device 11 is a device for generating a driving force to propel the vehicle 100 and applying this driving force to the drive wheels of the vehicle 100. The vehicle drive force generating device 11 is, for example, an internal combustion engine, an electric motor, etc. The braking device 12 is a device for applying a braking force to the wheels of the vehicle 100 to brake the vehicle 100. The steering device 13 is a device for applying a steering torque to the steering wheels of the vehicle 100 to steer the vehicle 100.

[0055] The vehicle drive force generating device 11, braking device 12, and steering device 13 are electrically connected to the ECU 90. The ECU 90 controls the driving force applied to the drive wheels of the vehicle 100 by controlling the operation of the vehicle drive force generating device 11. Furthermore, the ECU 90 controls the braking force applied to the wheels of the vehicle 100 by controlling the operation of the braking device 12. Additionally, the ECU 90 controls the steering torque applied to the steering wheels of the vehicle 100 by controlling the operation of the steering device 13.

[0056] <Sensors, etc.>

[0057] The vehicle parking assistance device 10 includes an accelerator pedal operation sensor 21, a brake pedal operation sensor 22, a steering angle sensor 23, a steering torque sensor 24, a vehicle speed sensor 25, a yaw rate sensor 26, a front-to-rear acceleration sensor 27, a lateral acceleration sensor 28, a sonar sensor device 30, a camera sensor device 40, and a display 50.

[0058] Accelerator pedal operation amount sensor 21 is electrically connected to ECU 90. ECU 90 detects and obtains the operation amount AP of accelerator pedal 14 via accelerator pedal operation amount sensor 21, which is taken as accelerator pedal operation amount AP. ECU 90 controls the operation of vehicle drive force generating device 11 so that the drive force corresponding to the obtained accelerator pedal operation amount AP is applied from vehicle drive force generating device 11 to the drive wheels of vehicle 100.

[0059] Brake pedal operation amount sensor 22 is electrically connected to ECU 90. ECU 90 detects and obtains the driver's operation amount BP on brake pedal 15 via brake pedal operation amount sensor 22, as the brake pedal operation amount BP. ECU 90 controls the operation of braking device 12 based on the obtained brake pedal operation amount BP, so that braking force is applied from braking device 12 to the wheels of vehicle 100.

[0060] Steering angle sensor 23 is electrically connected to ECU 90. ECU 90 detects and obtains the rotation angle θst of steering wheel 16 relative to the neutral position via steering angle sensor 23, which is taken as steering angle θst.

[0061] Steering torque sensor 24 is electrically connected to ECU 90. ECU 90 detects and obtains the torque TQst input from the driver to steering shaft 17 via steering torque sensor 24, which is taken as steering torque TQst.

[0062] The ECU 90 controls the operation of the steering device 13 so that the steering torque corresponding to the obtained steering angle θst and steering torque TQst is applied to the steering wheels of the vehicle 100.

[0063] Vehicle speed sensor 25 is electrically connected to ECU 90. ECU 90 detects and obtains the rotational speed Vrot of each wheel of vehicle 100 via vehicle speed sensor 25, and uses this as the rotational speed Vrot of each wheel. Based on the obtained rotational speed Vrot of each wheel, ECU 90 obtains the driving speed SPD of vehicle 100 as vehicle speed SPD.

[0064] Yaw rate sensor 26 is electrically connected to ECU 90. ECU 90 detects and obtains the yaw rate YR of vehicle 100 via yaw rate sensor 26, which is used as the vehicle yaw rate YR.

[0065] The front and rear acceleration sensor 27 is electrically connected to the ECU 90. The ECU 90 detects and obtains the front and rear acceleration Gx of the vehicle 100 via the front and rear acceleration sensor 27, and uses it as the vehicle's front and rear acceleration Gx.

[0066] The lateral acceleration sensor 28 is electrically connected to the ECU 90. The ECU 90 detects and obtains the lateral acceleration Gy of the vehicle 100 via the lateral acceleration sensor 28, which is taken as the vehicle's lateral acceleration Gy.

[0067] The sonar sensor device 30 has a first gap sonar 301 to a twelfth gap sonar 312.

[0068] like Figure 2As shown, the first gap sonar 301 is mounted on the vehicle 100 to emit sound waves from the front left end of the vehicle 100 towards the left front. The second gap sonar 302 is mounted on the vehicle 100 to emit sound waves from the left front end of the vehicle 100 towards the front. The third gap sonar 303 is mounted on the vehicle 100 to emit sound waves from the front right end of the vehicle 100 towards the right front. The fourth gap sonar 304 is mounted on the vehicle 100 to emit sound waves from the right front end of the vehicle 100 towards the front.

[0069] Additionally, the fifth gap sonar 305 is mounted on the vehicle 100 to emit sound waves from the rear left end of the vehicle 100 to the left rear. The sixth gap sonar 306 is mounted on the vehicle 100 to emit sound waves from the left rear end of the vehicle 100 to the rear. The seventh gap sonar 307 is mounted on the vehicle 100 to emit sound waves from the rear right end of the vehicle 100 to the right rear. The eighth gap sonar 308 is mounted on the vehicle 100 to emit sound waves from the right rear end of the vehicle 100 to the rear.

[0070] Additionally, the ninth gap sonar 309 is mounted on the vehicle 100 to emit sound waves from the front left side to the left. The tenth gap sonar 310 is mounted on the vehicle 100 to emit sound waves from the rear left side to the left. The eleventh gap sonar 311 is mounted on the vehicle 100 to emit sound waves from the front right side to the right. The twelfth gap sonar 312 is mounted on the vehicle 100 to emit sound waves from the rear right side to the right.

[0071] First gap sonar 301 to twelfth gap sonar 312 receive sound waves reflected from objects.

[0072] The sonar sensor device 30 is electrically connected to the ECU 90. The sonar sensor device 30 sends information related to "sound waves emitted by the first gap sonar 301 to the twelfth gap sonar 312" and "sound waves received by the first gap sonar 301 to the twelfth gap sonar 312" to the ECU 90. Based on the information received from the sonar sensor device 30 (hereinafter referred to as "Sonar Information SON"), the ECU 90 obtains information related to objects existing around the vehicle 100 as Object Information OBJ.

[0073] exist Figure 2 In this context, the direction represented by the symbol Dx is the front-rear direction of the vehicle 100, and this direction will be referred to as "vehicle front-rear direction Dx" below. The direction represented by the symbol Dy is the width direction of the vehicle 100, and this direction will be referred to as "vehicle width direction Dy" below.

[0074] The camera sensor device 40 includes a front camera 41, a rear camera 42, a left camera 43, and a right camera 44. Hereinafter, as needed, the front camera 41, the rear camera 42, the left camera 43, and the right camera 44 will be collectively referred to as "camera 45".

[0075] like Figure 3 As shown, the front camera 41 is mounted at the center of the front end of the vehicle 100 to capture the scene in front of the vehicle 100, with a field of view 41A of approximately 180°. The rear camera 42 is mounted at the center of the rear end of the vehicle 100 to capture the scene behind the vehicle 100, with a field of view 42A also of approximately 180°. The left-side camera 43 is mounted on the left side of the vehicle 100 to capture the scene on the left side of the vehicle 100, with a field of view 43A of approximately 180°. The right-side camera 44 is mounted on the right side of the vehicle 100 to capture the scene on the right side of the vehicle 100, with a field of view 44A of approximately 180°.

[0076] The camera sensor device 40 is electrically connected to the ECU 90. The ECU 90 can obtain information related to the images of the scene captured by each camera 45 via the camera sensor device 40.

[0077] Hereinafter, as needed, information related to the image of the scene captured by the front camera 41 will be referred to as "Front Image Information IMG1", information related to the image of the scene captured by the rear camera 42 will be referred to as "Rear Image Information IMG2", information related to the image of the scene captured by the left camera 43 will be referred to as "Left Image Information IMG3", and information related to the image of the scene captured by the right camera 44 will be referred to as "Right Image Information IMG4". Furthermore, in the following text, as needed, Front Image Information IMG1, Rear Image Information IMG2, Left Image Information IMG3, and Right Image Information IMG4 will be collectively referred to as "Image Information IMG".

[0078] When specified conditions are met, the vehicle parking assistance device 10 obtains feature points F based on image information IMG. Feature points F are images within a specified range of brightness variations captured by each camera 45.

[0079] For example, shooting with camera 45 Figure 4 In the case of the parking lot 62 shown, the corners of the concrete bricks 63B, the corners of the ground 63 composed of lawn 63L, and the boundary between the ground 63 composed of bricks 63B and the ground 63 composed of lawn 63L are taken as feature points F.

[0080] about Figure 4The ground 63 of the parking lot 62 shown is composed of a concrete surface 63C and a lawn surface 63L. Additionally, at the entrance 62ent of the parking lot 62, multiple concrete bricks 63B are arranged side-by-side to cover the side ditch. Therefore, the ground 63 of the entrance 62ent of the parking lot 62 is composed of the surface of the bricks 63B.

[0081] The vehicle parking assist device 10 obtains a specified range 71 in front of the vehicle 100 based on the forward image information IMG1 (see reference). Figure 5 The feature point F on the ground 63 (hereinafter referred to as "front feature point F1"). Additionally, the vehicle parking assist device 10 obtains a predetermined range 72 (see reference 10) behind the vehicle 100 based on the rear image information IMG2. Figure 5 The feature point F on the ground 63 (hereinafter referred to as "rear feature point F2"). Additionally, the vehicle parking assist device 10 obtains a defined range 73 on the left side of the vehicle 100 based on the left-side image information IMG3 (see reference). Figure 6 The feature point F on the ground 63 (hereinafter referred to as "left feature point F3"). The vehicle parking assist device 10 obtains a specified range 74 on the right side of the vehicle 100 based on the right image information IMG4 (refer to). Figure 6 Feature point F on the ground 63 (hereinafter referred to as "right feature point F4").

[0082] like Figure 5 As shown, the defined range 71 is the range defined by lines L711, L712, L713, and L714. Line L711 extends from the front camera 41 forward of the vehicle 100 at a defined distance Dset and in the vehicle width direction Dy. Line L712 passes through the front camera 41 and extends in the vehicle width direction Dy. Line L713 extends from the front camera 41 to the left of the vehicle 100 at a defined distance Dset and in the vehicle front-rear direction Dx. Line L714 extends from the front camera 41 to the right of the vehicle 100 at a defined distance Dset and in the vehicle front-rear direction Dx. Hereinafter, the defined range 71 will be referred to as the "front range 71".

[0083] The front area 71 is divided into eight areas 71D, which are divided into four equal parts in the vehicle width direction Dy and into two equal parts in the vehicle front-rear direction Dx. That is, the front area 71 is divided into multiple areas 71D of equal area. Hereinafter, these areas 71D are referred to as "front segmentation areas 71D". The two front segmentation areas 71D set at the left end in the vehicle width direction Dy are called "left segmentation areas 71D3", the two front segmentation areas 71D set at the right end in the vehicle width direction Dy are called "right segmentation areas 71D4", and the four front segmentation areas 71D set in the middle in the vehicle width direction Dy are called "middle segmentation areas 71D5".

[0084] In addition, such as Figure 5 As shown, the defined range 72 is the area defined by lines L721, L722, L723, and L724. Line L721 is a line that passes through the rear camera 42 and extends in the vehicle width direction Dy. Line L722 is a line that extends from the rear camera 42 to the rear of the vehicle 100 at a defined distance Dset and in the vehicle width direction Dy. Line L723 is a line that extends from the rear camera 42 to the left of the vehicle 100 at a defined distance Dset and in the vehicle front-rear direction Dx. Line L724 is a line that extends from the rear camera 42 to the right of the vehicle 100 at a defined distance Dset and in the vehicle front-rear direction Dx. Hereinafter, the defined range 72 will be referred to as the "rear range 72".

[0085] The rear area 72 is divided into eight areas 72D, which are divided into four equal parts in the vehicle width direction Dy and into two equal parts in the vehicle front-rear direction Dx. That is, the rear area 72 is divided into multiple areas 72D of equal area. Hereinafter, these areas 72D are referred to as "rear segmentation areas 72D". In addition, the two rear segmentation areas 72D that are set at the left end in the vehicle width direction Dy are called "left segmentation areas 72D3", the two rear segmentation areas 72D that are set at the right end in the vehicle width direction Dy are called "right segmentation areas 72D4", and the four rear segmentation areas 72D that are set in the middle in the vehicle width direction Dy are called "middle segmentation areas 72D5".

[0086] In addition, such as Figure 6As shown, the defined range 73 is the range defined by lines L731, L732, L733, and L734. Line L731 extends from the left camera 43 forward of the vehicle 100 at a defined distance Dset and in the vehicle width direction Dy. Line L732 extends from the left camera 43 backward of the vehicle 100 at a defined distance Dset and in the vehicle width direction Dy. Line L733 extends from the left camera 43 to the left of the vehicle 100 at a defined distance Dset and in the vehicle front-rear direction Dx. Line L734 is a line that passes through the left camera 43 and extends in the vehicle front-rear direction Dx. Hereinafter, the defined range 73 will be referred to as the "left range 73".

[0087] The left side range 73 is divided into eight ranges 73D, which are four-eighths in the vehicle's longitudinal direction Dx and two-half in the vehicle's width direction Dy. That is, the left side range 73 is divided into multiple ranges 73D of equal area. Hereinafter, these ranges 73D will be referred to as the "left side segmented ranges 73D". Furthermore, the two left side segmented ranges 73D located at the front end in the vehicle's longitudinal direction Dx will be referred to as the "front end segmented ranges 73D1", the two left side segmented ranges 73D located at the rear end in the vehicle's longitudinal direction Dx will be referred to as the "rear end segmented ranges 73D2", and the four left side segmented ranges 73D located in the middle in the vehicle's longitudinal direction Dx will be referred to as the "middle segmented ranges 73D5".

[0088] In addition, such as Figure 6 As shown, the defined range 74 is the area defined by lines L741, L742, L743, and L744. Line L741 extends from the right-side camera 44 forward of the vehicle 100 at a defined distance Dset and in the vehicle width direction Dy. Line L742 extends from the right-side camera 44 backward of the vehicle 100 at a defined distance Dset and in the vehicle width direction Dy. Line L743 passes through the right-side camera 44 and extends in the vehicle front-rear direction Dx. Line L744 extends from the right-side camera 44 to the right of the vehicle 100 at a defined distance Dset and in the vehicle front-rear direction Dx. Hereinafter, the defined range 74 will be referred to as the "right-side range 74".

[0089] The right-side region 74 is divided into eight regions 74D, which are quadrupled in the vehicle's longitudinal direction Dx and bisected in the vehicle's width direction Dy. That is, the right-side region 74 is divided into multiple regions 74D of equal area. Hereinafter, these regions 74D will be referred to as "right-side segmented regions 74D". The two right-side segmented regions 74D located at the front end in the vehicle's longitudinal direction Dx are referred to as "front-end segmented regions 74D1", the two right-side segmented regions 74D located at the rear end in the vehicle's longitudinal direction Dx are referred to as "rear-end segmented regions 74D2", and the four right-side segmented regions 74D located in the middle in the vehicle's longitudinal direction Dx are referred to as "middle segmented regions 74D5".

[0090] When the image of the area corresponding to feature point F captured by camera 45 is converted into a top-view image, the converted image of the area corresponding to feature point F is as follows: Figure 7 The image shown is a square range 75 of a predetermined length Lset. Under predetermined conditions, the vehicle parking assist device 10 divides the feature point F into 25 identical square ranges 75D and obtains the luminance LUM of each range 75D. Then, the vehicle parking assist device 10 calculates the value ΔLUM (=LUM-LUMave) obtained by subtracting the average value LUMave of each obtained luminance LUM from the ΔLUM, and obtains the intensity trend of the luminance LUM at the feature point F as density information CT based on these values ​​ΔLUM. That is, under predetermined conditions, the vehicle parking assist device 10 obtains the intensity pattern in the image of the feature point F captured by the camera 45 as density information CT.

[0091] The display 50 is positioned on a part of the vehicle 100 that the driver can visually inspect. In this example, the display 50 is the display of a so-called navigation device.

[0092] Display 50 is electrically connected to ECU 90. ECU 90 can display various images on display 50. In this example, ECU 90 can display the captured image 51C, top view image 51P, parking assist switch image 51M, registration main switch image 52M, parking zone selection image 52S, parking zone line image 52, setting button image 53, registration start button image 54, registration button image 55, parking start button image 56, movement button image 57, and angle adjustment button image 58 on display 50.

[0093] Image 51C is an image captured by camera 45.

[0094] The top-view image 51P is an image that includes a top-view image of the vehicle and images of the area surrounding the vehicle. The top-view image shows the vehicle 100 as viewed from above in a vertical direction. The images of the area surrounding the vehicle show the area around the vehicle 100 as viewed from above in a vertical direction, and at least include an image showing the parking lot 62. These top-view images and images of the area surrounding the vehicle are created by the ECU 90 based on image information IMG.

[0095] The parking assist switch image 51M is an image corresponding to a parking assist switch that is touched by the driver to enable the parking assist control described later in the vehicle parking assist device 10. In this example, the parking assist switch is displayed on the display 50 in the form of the parking assist switch image 51M, but it may not be displayed on the display 50, but may be provided as a physically operable switch in a part of the vehicle 100 that can be operated by the driver.

[0096] The registration master switch image 52M is an image corresponding to the registration master switch operated by the driver for registering (i.e., storing) parking information Ipark into the vehicle parking assist device 10 via parking assist control described later.

[0097] Parking zone selection image 52S Figure 9 (C) shows the left horizontal parking zone selection image 52SLa, the left vertical parking zone selection image 52SLb, the right horizontal parking zone selection image 52SRa, and the right vertical parking zone selection image 52SRb.

[0098] The left-hand horizontal parking zone selection image 52SLa is an image that the driver touches to select the parking zone 61 in the parking lot 62 to the left of vehicle 100, changing the orientation of vehicle 100 by 90 degrees. Parking zone 61 is the zone (or range or area) where vehicle 100 is parked via parking assistance controls described later. Figure 8 As shown, parking area 61 is set within parking lot 62. The left-hand column parking area selection image 52SLb is a touch operation image for selecting parking area 61 in parking lot 62 to the left of vehicle 100 without changing the vehicle 100's orientation, as performed by the driver. The right-hand column parking area selection image 52SRa is a touch operation image for selecting parking area 61 in parking lot 62 to the right of vehicle 100 by changing the vehicle 100's orientation by 90 degrees. The right-hand column parking area selection image 52SRb is a touch operation image for selecting parking area 61 in parking lot 62 to the right of vehicle 100 without changing the vehicle 100's orientation, as performed by the driver.

[0099] Furthermore, if there is no parking area 61 in the parking lot 62 to the left of vehicle 100 that allows parking by changing the orientation of vehicle 100 by 90 degrees, the left horizontal parking area selection image 52SLa is not displayed on the display 50 or is displayed on the display 50 with reduced brightness (so-called reduced color tone). Similarly, if there is no parking area 61 in the parking lot 62 to the left of vehicle 100 that allows parking without changing the orientation of vehicle 100, the left vertical parking area selection image 52SLb is not displayed on the display 50 or is displayed on the display 50 with reduced brightness (so-called reduced color tone). Similarly, if there is no parking area 61 in the parking lot 62 to the right of vehicle 100 that allows parking by changing the orientation of vehicle 100 by 90 degrees, the right horizontal parking area selection image 52SRa is not displayed on the display 50 or is displayed on the display 50 with reduced brightness (so-called reduced color tone). Similarly, if there is no parking area 61 in the parking lot 62 to the right of vehicle 100 that allows parking without changing the orientation of vehicle 100, the right column parking area selection image 52SRb is not displayed on display 50 or is displayed on display 50 with reduced brightness (so-called reduced hue).

[0100] Furthermore, if the driver touches and operates the parking area selection image 52S displayed on the display 50 in a reduced color tone, the ECU 90 does not determine that the parking area 61 corresponding to the touched parking area selection image 52S has been selected.

[0101] Parking zone marking image 52 is an image showing parking zone marking 61.

[0102] The setting button image 53 is an image corresponding to the setting button that is touched by the driver to set (or confirm or determine) the parking zone 61 for parking the vehicle 100 in the parking assist control described later.

[0103] The registration start button image 54 is an image corresponding to the registration start button that is touched by the driver to initiate the first parking driving process of the parking assistance control, which will be described later.

[0104] Image 55 is an image corresponding to a registration button operated by the driver to register the parking information Ipark obtained through the parking assistance control described later into the RAM of the vehicle parking assistance device 10 (specifically, the ECU 90). The parking information Ipark is information related to the parking lot 62 used by the vehicle parking assistance device 10 to automatically park the vehicle 100 in the parking lot 62.

[0105] The parking start button image 56 is an image corresponding to the parking start button operated by the driver to initiate parking assistance control (described later) in order to park the vehicle 100 into the parking zone 61 registered in the vehicle parking assistance device 10.

[0106] The movement button images 57 include an upper movement button image 57U, a lower movement button image 57D, a left movement button image 57L, and a right movement button image 57R. The upper movement button image 57U is operated by the driver to move the parking area marking image 52 upwards on the display 50. The lower movement button image 57D is operated by the driver to move the parking area marking image 52 downwards on the display 50. The left movement button image 57L is operated by the driver to move the parking area marking image 52 to the left on the display 50. The right movement button image 57R is operated by the driver to move the parking area marking image 52 to the right on the display 50.

[0107] The angle adjustment button image 58 includes a left-turn angle adjustment button image 58A and a right-turn angle adjustment button image 58B. The left-turn angle adjustment button image 58A is operated by the driver to rotate the parking area marking image 52 to the left on the display 50. The right-turn angle adjustment button image 58B is operated by the driver to rotate the parking area marking image 52 to the right on the display 50.

[0108] <Summary of Vehicle Parking Assist Device Operation>

[0109] The operation of the vehicle parking assist device 10 will now be described in summary. The vehicle parking assist device 10 is configured to perform parking assist control. Parking assist control is the control that parks the vehicle 100 in the parking area 61 without requiring the driver to operate the accelerator pedal 14, the brake pedal 15, or the steering wheel 16.

[0110] There are parking lots that delineate parking areas using white lines and other markings (hereinafter referred to as "zoning lines"). In such parking lots, when vehicles are automatically parked, the zoning lines, captured by cameras, can be used as markers to automatically drive the vehicle to park in the designated parking area.

[0111] On the other hand, there are also parking lots such as those in private residences that lack designated parking zones. In such parking lots, there are no marked zones for automatic parking. The parking assistance control implemented by the vehicle parking assistance device 10 includes controls that automatically park the vehicle in the parking lot and register parking information about that parking lot, as well as controls that automatically park the vehicle in the parking lot where the parking information has been registered.

[0112] When the vehicle speed SPD is below a predetermined speed SPDth, the vehicle parking assistance device 10 performs a search process. This search process involves searching for image portions in the captured image CMR that have a density pattern that matches the density patterns of multiple registered entry feature points Fent_reg. The vehicle parking assistance device 10 performs this search process using left image information IMG3 and right image information IMG4. The registered entry feature point Fent_reg is the entry feature point Fent that has been registered (i.e., stored) in the vehicle parking assistance device 10 through the registered entry density information CTent_reg, which will be described in detail later. The entry feature point Fent is the feature point F of the entry 62ent of the parking lot 62 obtained through the parking assistance control. In addition, the registered entry density information CTent_reg is the density information CT of the registered entry feature point Fent_reg. Furthermore, the captured image CMR is an image obtained by the camera 45.

[0113] When the vehicle parking assistance device 10 extracts image portions from the captured image CMR that have density patterns that are consistent with the density patterns of multiple registered entry feature points Fent_reg, it compares (or matches) the positional relationships between these extracted multiple image portions and the positional relationships between the multiple registered entry feature points Fent_reg that have density patterns consistent with the density patterns of these multiple image portions.

[0114] The vehicle parking assistance device 10 determines that a registered parking lot 62 exists near the vehicle 100 if the positional relationships between the extracted multiple image portions and the positional relationships between multiple registered entry feature points Fent_reg with a density pattern consistent with the density pattern of these multiple image portions are consistent. The registered parking lot 62 is the parking lot whose parking information Ipark has been registered (i.e., stored) in the vehicle parking assistance device 10 through parking assistance control.

[0115] More specifically, if the positional relationship between multiple image parts extracted based on the left image information IMG3 is consistent with the positional relationship between multiple registered entry feature points Fent_reg with a density pattern consistent with the density pattern of these multiple image parts, the vehicle parking assistance device 10 determines that there is a registered parking lot 62 on the left side of the vehicle 100.

[0116] On the other hand, if the positional relationship between the multiple image parts extracted based on the right image information IMG4 is consistent with the positional relationship between the multiple registered entry feature points Fent_reg with the same density pattern as the multiple image parts, the vehicle parking assistance device 10 determines that there is a registered parking lot 62 on the right side of the vehicle 100.

[0117] Furthermore, if the vehicle parking assist device 10 does not extract an image portion from the captured image CMR that has a density pattern consistent with the density patterns of multiple registered entrance feature points Fent_reg, it determines that there is no registered parking lot 62 near the vehicle 100. Alternatively, if the positional relationship between the extracted multiple image portions is inconsistent with the positional relationship between the multiple registered entrance feature points Fent_reg that have density patterns consistent with the density patterns of these multiple image portions, the vehicle parking assist device 10 also determines that there is no registered parking lot 62 near the vehicle 100.

[0118] Parking lot registration

[0119] When it is determined that vehicle 100 is stopped near entrance 62ent of parking lot 62 and the parking assist switch image 51M has been touched, and the registration start button image 54 has been touched, thus initiating the registration of parking information Ipark, the vehicle parking assist device 10 obtains temporary entrance information Ient_pre and temporary intermediate information Imid_pre in the following manner. Furthermore, the vehicle parking assist device 10 registers (i.e., stores) the registration entrance information Ient_reg, the registration parking lot information Iin_reg, and the registration area information Iarea_reg as parking information Ipark, as described later.

[0120] If it is determined that vehicle 100 is parked near entrance 62ent of parking lot 62, and it is determined that parking lot 62 is not a registered parking lot, and if Figure 9 As shown in (A), when the driver touches the parking assist switch image 51M displayed on the display 50, the vehicle parking assist device 10 clears the parking assist switch image 51M from the display 50, as... Figure 9As shown in (B), the captured image 51C, the top view image 51P, and the registered main switch image 52M are displayed on the display 50.

[0121] When the driver touches and operates the registration master switch image 52M, the vehicle parking assist device 10 clears the captured image 51C, the top view image 51P, and the registration master switch image 52M from the display 50. Figure 9 As shown in (C), the parking zone selection image 52S is displayed on the display 50.

[0122] When a driver touches one of the parking area selection images 52S displayed on display 50 or displayed on display 50 without reducing the color tone, the vehicle parking assist device 10 clears the parking area selection image 52S from display 50. Figure 9 As shown in (D), the display 50 shows a top view image 51P, a parking zone line image 52, a setting button image 53, a movement button image 57, and an angle adjustment button image 58. At this time, if an unregistered parking lot 62 exists to the left of the vehicle 100, the vehicle parking assistance device 10 displays the top view image 51P on the display 50 such that the parking lot image is displayed to the left of the vehicle image; if an unregistered parking lot 62 exists to the right of the vehicle 100, the top view image 51P is displayed on the display 50 such that the parking lot image is displayed to the right of the vehicle image.

[0123] Furthermore, the vehicle parking assistance device 10, based on image information (IMG) and sonar information (SON), sets the area within the parking lot 62 where the vehicle 100 can park as a parking zone 61, and displays the parking zone line image 52 as an image representing the set parking zone 61 on the display 50. The vehicle parking assistance device 10 uses sonar information (SON) for example, to obtain the width of the entrance 62ent of the parking lot 62.

[0124] The driver can move the parking zone marking image 52 on the display 50 by touching the operation button image 57 until the operation setting button image 53 is touched. The driver can change the position of the parking zone marking 61 that parks the vehicle 100 by moving the parking zone marking image 52 on the display 50. Furthermore, the driver can rotate the parking zone marking image 52 on the display 50 by touching the angle adjustment button image 58 until the operation setting button image 53 is touched. The driver can change the position of the parking zone marking 61 that parks the vehicle 100 by rotating the parking zone marking image 52 on the display 50.

[0125] When the driver touches and operates the setting button image 53, the vehicle parking assist device 10 clears the setting button image 53, the movement button image 57, and the angle adjustment button image 58 from the display 50, and as shown... Figure 10 The image 51C, the top view image 51P, and the registration start button image 54 are displayed as shown in (A).

[0126] Furthermore, when the driver touches and operates the setting button image 53, the vehicle parking assist device 10 sets the parking area 61 at the position corresponding to the parking area line image 52 displayed on the display 50 as the registered target parking area 61set.

[0127] Furthermore, when the driver touches and operates the setting button image 53, the vehicle parking assist device 10 is set to a target driving route Rtgt for parking the vehicle 100 in the registered target parking area 61set. For example, as Figure 11 As shown, when vehicle 100 is stopped on the right side of an unregistered parking lot 62, the vehicle parking assist device 10, as... Figure 12 Set the target driving route Rtgt as shown.

[0128] Furthermore, when the driver touches and operates the setting button image 53 while the vehicle 100 is stopped on the right side of the parking lot 62, the vehicle parking assist device 10 acquires one or more predetermined number of new left-side feature points F3new as entry feature points Fent for each of the middle segmentation range 73D5, the front segmentation range 73D1, and the rear segmentation range 73D2 of the left-side range 73. On the other hand, when the driver touches and operates the setting button image 53 while the vehicle 100 is stopped on the left side of the parking lot 62, the vehicle parking assist device 10 acquires one or more predetermined number of new right-side feature points F4new as entry feature points Fent for each of the middle segmentation range 74D5, the front segmentation range 74D1, and the rear segmentation range 74D2 of the right-side range 74.

[0129] In this example, when the driver touches and operates the setting button image 53 while the vehicle 100 is stopped on the right side of the parking lot 62, the vehicle parking assist device 10 acquires a greater number of entrance feature points Fent for each intermediate segmentation range 73D5 compared to the front segmentation range 73D1 and the rear segmentation range 73D2. That is, the vehicle parking assist device 10 acquires a greater number of entrance feature points Fent for the range 73D5, 73D1, and 73D2 that is closer to the center of the entrance 62ent of the parking lot 62 than for the ranges 73D1 and 73D2 that are farther from the center of the entrance 62ent of the parking lot 62.

[0130] On the other hand, when the driver touches and operates the setting button image 53 while the vehicle 100 is stopped on the left side of the parking lot 62, the vehicle parking assist device 10 acquires a greater number of entrance feature points Fent for each intermediate segmentation range 74D5 compared to the front segmentation range 74D1 and the rear segmentation range 74D2. That is, the vehicle parking assist device 10 acquires a greater number of entrance feature points Fent for the range 74D5, 74D1, and 74D2 that is closer to the center of the entrance 62ent of the parking lot 62 than for the ranges 74D1 and 74D2 that are farther from the center of the entrance 62ent of the parking lot 62.

[0131] For example, in vehicle 100 Figure 11 When the vehicle is parked on the right side of parking lot 62 as shown, the vehicle parking assist device 10... Figure 13 and Figure 14 As shown, for each of the four intermediate segmented ranges 73D5 of the left range 73, two new left-side feature points F3new are obtained as entry feature points Fent; for each of the two front segmented ranges 73D1, one new left-side feature point F3new is obtained as an entry feature point Fent; and for each of the two rear segmented ranges 73D2, one new left-side feature point F3new is obtained as an entry feature point Fent. Furthermore, when the vehicle 100 is stopped on the left side of the parking lot 62, the vehicle parking assistance device 10 obtains two new right-side feature points F4new for each of the four intermediate segmented ranges 74D5 of the right range 74 as entry feature points Fent; for each of the two front segmented ranges 74D1, one new right-side feature point F4new is obtained as an entry feature point Fent; and for each of the two rear segmented ranges 74D2, one new right-side feature point F4new is obtained as an entry feature point Fent.

[0132] Furthermore, the vehicle parking assist device 10 can also be configured such that, when the driver stops the vehicle 100 on the right side of the entrance 62ent of the parking lot 62, if the vehicle 100 stops more often at a position slightly less than directly to the side of the entrance 62ent of the parking lot 62, then for each front segment 73D1 and the two adjacent intermediate segment 73D5, more entrance feature points Fent are obtained compared to each rear segment 73D2 and the two adjacent intermediate segment 73D5. Similarly, the vehicle parking assist device 10 can also be configured such that, when the driver stops the vehicle 100 on the left side of the entrance 62ent of the parking lot 62, if the vehicle 100 stops more often at a position slightly less than directly to the side of the entrance 62ent of the parking lot 62, then for each front segment 74D1 and the two adjacent intermediate segment 74D5, more entrance feature points Fent are obtained compared to the rear segment 74D2 and the two adjacent intermediate segment 74D5.

[0133] Furthermore, if the vehicle parking assist device 10 cannot acquire a predetermined number of new left-side feature points F3new within a portion of the middle segmentation range 73D5, the front segmentation range 73D1, and the rear segmentation range 73D2 of the left-side range 73, it acquires an equal number of new left-side feature points F3new within the remaining ranges 73D5, 73D1, and 73D2, and uses these as entry feature points Fent. Similarly, if the vehicle parking assist device 10 cannot acquire a predetermined number of new right-side feature points F4new within a portion of the middle segmentation range 74D5, the front segmentation range 74D1, and the rear segmentation range 74D2 of the right-side range 74, it acquires an equal number of new right-side feature points F4new within the remaining ranges 74D5, 74D1, and 74D2, and uses these as entry feature points Fent.

[0134] After acquiring the entrance feature point Fent, the vehicle parking assistance device 10 acquires and stores the coordinates XY of Fent in the temporary coordinate system Cpre as temporary entrance coordinates XYent_pre, and also acquires and stores the density information CT of Fent as temporary entrance density information CTent_pre. The temporary coordinate system Cpre is a coordinate system with its origin at a specified position Ppre within the registered parking area 61set. Therefore, the temporary entrance coordinates XYent_pre are the position of Fent relative to the specified position Ppre. Furthermore, the temporary entrance information CTent_pre includes the temporary entrance coordinates XYent_pre and the temporary entrance density information CTent_pre.

[0135] When the driver touches and operates the registration start button image 54, the vehicle parking assist device 10 clears the registration start button image 54 from the display 50, as shown. Figure 10 As shown in (B), the captured image 51C and the top-view image 51P continue to be displayed on the display 50. At this time, if an unregistered parking lot 62 exists to the left of the vehicle 100, the vehicle parking assist device 10 displays the captured image 51C, which includes the unregistered parking lot 62, captured by the left-side camera 43, on the display 50, and displays the top-view image 51P on the display 50 in a manner that displays the parking lot image to the left of the vehicle image. On the other hand, if an unregistered parking lot 62 exists to the right of the vehicle 100, the vehicle parking assist device 10 displays the captured image 51C, which includes the unregistered parking lot 62, captured by the right-side camera 44, on the display 50, and displays the top-view image 51P on the display 50 in a manner that displays the parking lot image to the right of the vehicle image.

[0136] Furthermore, when the driver touches and operates the registration start button image 54, the vehicle parking assist device 10 performs a first parking driving process to drive the vehicle 100 along the target driving route Rtgt to the registered target parking area 61set. The first parking driving process controls the operation of the vehicle driving force generating device 11, the braking device 12, and the steering device 13 based on "image information IMG, object information OBJ, steering angle θst, steering torque TQst, vehicle speed SPD, vehicle yaw rate YR, vehicle longitudinal acceleration Gx, and vehicle lateral acceleration Gy" to make the vehicle 100 drive along the target driving route Rtgt.

[0137] For example, in vehicle 100 Figure 11 When the vehicle is stopped on the right side of an unregistered parking lot 62 as shown, the vehicle parking assist device 10 initiates the first parking and driving process, first as follows: Figure 15 As shown, make the vehicle turn 100 degrees right while simultaneously moving forward and then stopping. Next, as... Figure 16 As shown, the vehicle parking assist device 10 enables the vehicle 100 to reverse while turning left.

[0138] The vehicle parking assist device 10 acquires a feature point F at least once during the period until the vehicle 100 completes parking in the parking lot 62, and acquires the coordinates (XY) of the feature point F in the temporary coordinate system Cpre and the density information (CT) of the feature point F. Specifically, after the vehicle 100 starts driving in order to park in the parking lot 62, when it is predicted that the vehicle 100 will drive straight without turning thereafter, the vehicle parking assist device 10 acquires the feature point F at least once, and acquires the coordinates (XY) of the feature point F in the temporary coordinate system Cpre and the density information (CT) of the feature point F.

[0139] In this example, during the period when vehicle 100 is reversed, the point in time at which vehicle 100 is predicted to continue traveling straight without turning (refer to...) Figure 17 The vehicle parking assist device 10 acquires a rear feature point F2 as a new rear feature point F2new. Alternatively, the vehicle parking assist device 10 may acquire the rear feature point F2 at the point in time after the predicted straight-line driving without turning, during the period when the vehicle 100 is reversing. Or, the vehicle parking assist device 10 may acquire the rear feature point F2 at the point in time when the predicted straight-line driving without turning, and at the point in time after the predicted straight-line driving without turning, during the period when the vehicle 100 is reversing. Furthermore, the vehicle parking assist device 10 may acquire the rear feature point F2 at the point in time when the predicted straight-line driving without turning, and at each subsequent reversal of the vehicle 100 by the predicted straight-line driving without turning, during the period when the vehicle 100 is reversing. Additionally, the vehicle parking assist device 10 can also acquire a rear feature point F2 during the period when the vehicle 100 is reversing, after predicting that the vehicle 100 will not turn and will travel straight afterward, every time the vehicle 100 reverses a predetermined distance Dtravel_th. Furthermore, in addition to acquiring the rear feature point F2, the vehicle parking assist device 10 can also acquire at least one of the following: a front feature point F1, a left feature point F3, and a right feature point F4.

[0140] In this example, the specified distance Dtravel_th is set to the distance at which the rear range 72 when the rear feature point F2 was most recently acquired does not overlap with the rear range 72 when the rear feature point F2 was acquired this time.

[0141] Furthermore, the vehicle parking assist device 10 acquires at least one new rear feature point F2new existing in each rear segmentation range 72D as an intermediate feature point Fmid. The vehicle parking assist device 10 acquires and stores the coordinates XY of the acquired intermediate feature point Fmid in the temporary coordinate system Cpre as temporary intermediate coordinates XYmid_pre, and acquires and stores the density information CT of the acquired intermediate feature point Fmid as temporary intermediate density information CTmid_pre. The temporary intermediate coordinates XYmid_pre are the positions of the intermediate feature point Fmid with a predetermined position Ppre as a reference. The temporary intermediate information Imid_pre includes the temporary intermediate coordinates XYmid_pre and the temporary intermediate density information CTmid_pre.

[0142] During the execution of the first parking and driving process, the vehicle parking assistance device 10 performs a safety judgment process. This safety judgment process determines whether, when driving the vehicle 100 along the target driving route Rtgt, it can safely drive the vehicle 100 to the registered target parking area 61set without contacting any three-dimensional objects present in the parking lot 62. If the vehicle parking assistance device 10 determines that it cannot safely drive the vehicle 100 to the registered target parking area 61set, it corrects the target driving route Rtgt to ensure that the vehicle 100 can safely drive to the registered target parking area 61set. The vehicle parking assistance device 10 uses the image information IMG and object information OBJ obtained during the execution of the first parking and driving process to perform this safety judgment process.

[0143] In addition, during the execution of the first parking and driving process, the vehicle parking assistance device 10 performs route determination processing. This route determination processing involves determining whether driving the vehicle 100 along the target driving route Rtgt will allow the vehicle 100 to park within the registered target parking area 61set. If the vehicle parking assistance device 10 determines that it cannot park the vehicle 100 within the registered target parking area 61set, it corrects the target driving route Rtgt to ensure that the vehicle 100 can park within the registered target parking area 61set. The vehicle parking assistance device 10 uses image information IMG (specifically feature points F) acquired during the execution of the first parking and driving process to perform this route determination processing.

[0144] When the vehicle parking assist device 10 enters the registered parking area 61set (refer to...) Figure 18The vehicle 100 is brought to a stop, and the first parking process is completed. Thus, the parking of vehicle 100 in parking lot 62 is finished. At this time, the vehicle parking assist device 10 acquires the forward feature point F1, the left feature point F3, and the right feature point F4 as new forward feature point F1new, new left feature point F3new, and new right feature point F4new, respectively. At this time, the vehicle parking assist device 10 can also acquire the rear feature point F2 as a new rear feature point F2new.

[0145] Then, the vehicle parking assist device 10 acquires at least one new forward feature point F1new existing in each forward segmentation range 71D as the final feature point Ffin, acquires at least one new left-side feature point F3new existing in each left-side segmentation range 73D as the final feature point Ffin, and acquires at least one new right-side feature point F4new existing in each right-side segmentation range 74D as the final feature point Ffin. At this time, if the vehicle parking assist device 10 acquires a new rear feature point F2new, it can also acquire at least one new rear feature point F2new existing in each rear segmentation range 72D as the final feature point Ffin.

[0146] <Parking lot information registration>

[0147] Additionally, when the vehicle 100 finishes parking in the parking lot 62, the vehicle parking assist device 10 clears the captured image 51C from the display 50, such as... Figure 10 As shown in (C), the top view image 51P, the registration button image 55, the move button image 57, and the angle adjustment button image 58 are displayed on the display 50.

[0148] The driver can move the parking zone marking image 52 on the display 50 by touching the movement button image 57 until the registration button image 55 is touched. The driver can change the position of the parking zone marking 61 that parks the vehicle 100 by moving the parking zone marking image 52 on the display 50. Furthermore, the driver can rotate the parking zone marking image 52 on the display 50 by touching the angle adjustment button image 58 until the registration button image 55 is touched. The driver can change the position of the parking zone marking 61 that parks the vehicle 100 by rotating the parking zone marking image 52 on the display 50.

[0149] When the driver touches and operates the registration button image 55, the vehicle parking assistance device 10 acquires and registers (i.e., stores) the coordinates XY of the final feature point Ffin in the registration coordinate system Creg as the registration field coordinates XYin_reg, and acquires and registers (i.e., stores) the density information CT of the final feature point Ffin as the registration field density information CTin_reg. The registration coordinate system Creg is a coordinate system with the center of the vehicle width direction Dy of the axle connecting the left and right rear wheels of the vehicle 100 at the end of parking in the registration target parking area 61set as the origin (see reference). Figure 18 Therefore, the registered field coordinates XYin_reg are the positions of the final feature point Ffin, which are based on the specified position PREg.

[0150] Furthermore, the vehicle parking assistance device 10 converts the temporary intermediate coordinates XYmid_pre into coordinates XY in the registered coordinate system Creg, obtains and registers (i.e., stores) them as the registered field coordinates XYin_reg, and registers (i.e., stores) the temporary intermediate density information CTmid_pre as the registered field density information CTin_reg. Therefore, the registered field coordinates XYin_reg are the positions of the intermediate feature point Fmid based on the specified position PREc.

[0151] The registered field information Iin_reg includes the registered field coordinates XYin_reg and the registered field density information CTin_reg.

[0152] Furthermore, the vehicle parking assistance device 10 registers (i.e., stores) the coordinates XY of the registered object parking area 61set in the registration coordinate system Creg as the registration area coordinates XYarea_reg. The registration area coordinates XYarea_reg are the positions of the parking area 61 based on the specified position Preg. The registration area information Iarea_reg includes the registration area coordinates XYarea_reg.

[0153] Furthermore, the vehicle parking assistance device 10 converts the temporary entrance coordinates XYent_pre into coordinates XY in the registered coordinate system Creg, obtains and registers (i.e., stores) them as the registered entrance coordinates XYent_reg, and registers (i.e., stores) the temporary entrance density information CTent_pre as the registered entrance density information CTent_reg. Therefore, the registered entrance coordinates XYent_reg are the positions of the entrance feature point Fent based on the specified position Preg. The registered entrance information Ient_reg includes the registered entrance coordinates XYent_reg and the registered entrance density information CTent_reg.

[0154] As mentioned above, the parking lot information Ipark includes the registration entrance information Ient_reg, the registration inside the parking lot information Iin_reg, and the registration area information Iarea_reg.

[0155] Parking vehicles in registered parking lots

[0156] When the vehicle parking assist device 10 determines that the vehicle 100 has stopped near the entrance 62ent of the parking lot 62 and determines that the parking lot 62 is a registered parking lot, if Figure 10 As shown in (A), if the driver touches the parking assist switch image 51M displayed on the display 50, the vehicle parking assist device 10 will clear the parking assist switch image 51M from the display 50. Figure 19 As shown in (B), the captured image 51C, the top-view image 51P, the registration main switch image 52M, and the parking start button image 56 are displayed on the display 50. At this time, if a registered parking lot 62 exists to the left of the vehicle 100, the vehicle parking assist device 10 displays the captured image 51C, which includes the registered parking lot 62, as an image captured by the left-side camera 43, on the display 50, and displays the top-view image 51P on the display 50 in a manner that displays the parking lot image to the left of the vehicle image. On the other hand, if a registered parking lot 62 exists to the right of the vehicle 100, the vehicle parking assist device 10 displays the captured image 51C, which includes the registered parking lot 62, as an image captured by the right-side camera 44, on the display 50, and displays the top-view image 51P on the display 50 in a manner that displays the parking lot image to the right of the vehicle image.

[0157] In addition, the vehicle parking assistance device 10 determines the location of the parking area 61 based on the registered area coordinates XYarea_reg contained in the parking information Ipark related to the registered parking lot 62 where the vehicle 100 is parked.

[0158] When the driver touches and operates the parking start button image 56, the vehicle parking assist device 10 clears the registration master switch image 52M and the parking start button image 56 from the display 50, as shown. Figure 19 As shown in (C), the captured image 51C and the top view image 51P continue to be displayed on the display 50.

[0159] Furthermore, when the driver touches and operates the parking start button image 56, the vehicle parking assist device 10 will set the parking zone 61 registered as parking information Ipark as the target parking zone 61tgt.

[0160] Furthermore, when the driver touches and operates the parking start button image 56, the vehicle parking assist device 10 sets the target driving route Rtgt for the vehicle 100 to drive in order to park the vehicle 100 in the target parking area 61tgt.

[0161] Then, the vehicle parking assist device 10 performs a second parking driving process to drive the vehicle 100 along the target driving route Rtgt to the target parking area 61tgt. The second parking driving process controls the operation of the vehicle driving force generating device 11, the braking device 12, and the steering device 13 based on "image information IMG, object information OBJ, steering angle θst, steering torque TQst, vehicle speed SPD, vehicle yaw rate YR, vehicle longitudinal acceleration Gx, and vehicle lateral acceleration Gy" to make the vehicle 100 drive along the target driving route Rtgt.

[0162] In addition, the vehicle parking assistance device 10 performs a safety judgment process during the execution of the second parking and driving process. This safety judgment process determines whether, when driving the vehicle 100 along the target driving route Rtgt, it can safely drive the vehicle 100 to the target parking area 61tgt without contacting any three-dimensional objects present in the parking lot 62. If it is determined that the vehicle 100 cannot safely drive to the target parking area 61tgt, the vehicle parking assistance device 10 corrects the target driving route Rtgt to ensure the vehicle 100 can safely reach the target parking area 61tgt. The vehicle parking assistance device 10 uses the image information IMG and object information OBJ obtained during the execution of the second parking and driving process to perform this safety judgment process.

[0163] During the execution of the second parking and driving process, the vehicle parking assistance device 10 performs a search process. This search process searches for image portions in the captured image CMR that have a density pattern that matches the density patterns of multiple registered feature points Freg. The vehicle parking assistance device 10 performs this search process using rear image information IMG2, left image information IMG3, and right image information IMG4. The registered feature point Freg is a feature point F registered with the density information CTin_reg within the parking area through parking assistance control.

[0164] When an image portion with a density pattern that is consistent with the density pattern of multiple registered feature points Freg is extracted from the captured image CMR, the vehicle parking assist device 10 compares (or matches) the positional relationship between these extracted multiple image portions and the positional relationship between the multiple registered feature points Freg with density patterns consistent with the density patterns of these multiple image portions.

[0165] The vehicle parking assistance device 10 performs parking position determination processing when the positional relationships between the extracted multiple image portions and the positional relationships between multiple registered feature points Freg with a density pattern consistent with the density pattern of these multiple image portions are consistent. The parking position determination processing involves determining whether the position of the target parking area 61tgt within the parking lot 62 is consistent with the position shown by the registered area coordinates XY based on the positional relationship between the coordinates XY of the registered feature points Freg with a density pattern consistent with the extracted image portions and the registered area coordinates XYarea_reg. If it is determined that the position of the target parking area 61tgt in the parking lot 62 is inconsistent with the position shown by the registered area coordinates XYarea_reg, the vehicle parking assistance device 10 corrects the position of the target parking area 61tgt to make it consistent with the position shown by the registered area coordinates XYarea_reg, and corrects the target driving route Rtgt so that the vehicle 100 can park in the corrected target parking area 61tgt.

[0166] When the vehicle 100 has fully entered the target parking area 61tgt, the vehicle parking assist device 10 stops the vehicle 100 and ends the second parking process. Thus, the parking of the vehicle 100 in the parking lot 62 is completed.

[0167] The above is a summary of the operation of the vehicle parking assistance device 10. Therefore, instead of targeting feature points of three-dimensional objects within the parking lot 62 or feature points of three-dimensional objects surrounding the parking lot 62, information related to feature points F of the ground 63 within the parking lot 62 and the ground 63 surrounding the parking lot 62 is registered as entry information Ient_reg. Even if the conditions surrounding the vehicle 100 and the parking lot 62 differ when the entry information Ient_reg is registered and subsequently when the vehicle 100 arrives at the registered entry 62ent of the parking lot 62, the changes in the images of the ground 63 within the parking lot 62 and the ground 63 surrounding the parking lot 62 captured by the camera 45 are minimal. Therefore, even if the conditions surrounding vehicle 100 and parking lot 62 differ when vehicle 100 arrives at the entrance 62ent of the registered parking lot 62 after registering the entrance information Int_reg, if an entrance feature point Fent corresponding to the entrance feature point Fent registered with the entrance information Int_reg is obtained upon arrival at the entrance 62ent of the registered parking lot 62, it can be determined that the entrance feature point Fent is the entrance feature point Fent registered with the entrance information Int_reg. As a result, it can be determined that parking lot 62 is a registered parking lot 62. Therefore, vehicle 100 can automatically park in the registered parking lot 62.

[0168] <Specific Operations of Vehicle Parking Assist Devices>

[0169] Next, the specific operation of the vehicle parking assist device 10 will be explained. The CPU of the ECU 90 of the vehicle parking assist device 10 is configured to execute [operations] every predetermined time interval. Figure 20 The example shown.

[0170] Therefore, when the designated time is reached, the CPU starts processing from step 2000, causing the process to proceed to step 2005, where it determines whether the value of the parking assist switch operation flag X1_auto is "1". When it is determined that vehicle 100 has stopped near the entrance 62ent of parking lot 62, and that parking lot 62 is not a registered parking lot, and the parking assist switch image 51M has been touched, the value of the parking assist switch operation flag X1_auto is set to "1". It is set to "0" when vehicle 100 finishes parking in parking lot 62.

[0171] If the CPU determines "yes" in step 2005, the process proceeds to step 2010, where it checks whether the value of the first parking and driving processing identifier X1_exe is "0". The value of the first parking and driving processing identifier X1_exe is set to "1" when the first parking and driving process begins and set to "0" when the first parking and driving process ends.

[0172] If the CPU determines "yes" in step 2010, the process proceeds to step 2015, whereby the captured image 51C, the top-view image 51P, and the registered main switch image 52M are displayed on the monitor 50 (see reference). Figure 9 (B) Next, the CPU causes the processing to proceed to step 2020, which determines whether the value of the registration master switch operation flag Xmain is "1". The value of the registration master switch operation flag Xmain is set to "1" when the registration master switch image 52M is touched, and is set to "0" when the vehicle 100 finishes parking in the parking lot 62.

[0173] If the CPU determines "yes" in step 2020, the process proceeds to step 2025, whereby the captured image 51C, the top-view image 51P, and the registered main switch image 52M are cleared from the display 50, and the parking zone selection image 52S is displayed on the display 50 (see reference). Figure 9 (C)). Then, the CPU causes the processing to proceed to step 2030, which determines whether the value of the selection completion flag Xselect is "1". The value of the selection completion flag Xselect is set to "1" when any of the parking zone selection images 52S is touched, and is set to "0" when the vehicle 100 completes parking in the parking zone 62.

[0174] If the CPU determines "yes" in step 2030, the process proceeds to step 2035 and is executed. Figure 21 The example shown. Therefore, after the CPU causes processing to proceed to step 2035, that is, from... Figure 21 Step 2100 begins processing, leading to step 2105, where the parking zone selection image 52S is cleared, and the top view image 51P, parking zone line image 52, setting button image 53, movement button image 57, and angle adjustment button image 58 are displayed on the monitor 50 (see reference). Figure 9 (D)).

[0175] Next, the CPU initiates the process to step 2110, determining whether the value of the setting completion flag Xset is "1". The value of the setting completion flag Xset is set to "1" when the setting button image 53 is touched, and set to "0" when the first parking and driving process begins.

[0176] If the CPU determines "yes" in step 2110, the process proceeds to step 2115, clearing the setting button image 53, the movement button image 57, and the angle adjustment button image 58 from the display 50, and displaying the captured image 51C, the top view image 51P, and the registration start button image 54 on the display 50 (see...). Figure 10 A). Next, the CPU proceeds to step 2120, setting the parking zone 61 corresponding to the parking zone line image 52 as the registered target parking zone 61set. Next, the CPU proceeds to step 2125, setting the driving route of the vehicle 100 up to the registered target parking zone 61set as the target driving route Rtgt. Next, the CPU proceeds to step 2130, and as described above, obtains the temporary entry information Int_pre and stores it in RAM.

[0177] Next, the CPU causes the process to proceed to step 2135, which determines whether the value of the registration start flag Xreg_start is "1". The value of the registration start flag Xreg_start is set to "1" when the registration start button image 54 is touched, and is set to "0" when the first parking and driving process begins.

[0178] When the CPU determines "yes" in step 2135, the process proceeds to step 2140, clearing the registration start button image 54 from the display 50, and displaying the captured image 51C and the top-view image 51P (see reference) on the display 50. Figure 10 (B)). Next, the CPU causes the process to proceed to step 2145, initiating the first parking and driving process where vehicle 100 travels along the target driving route Rtgt to the registered object parking area 61set. Afterwards, the CPU causes the process to proceed via step 2195. Figure 20 Step 2095, temporarily end this routine.

[0179] On the other hand, if the CPU determines "no" in step 2135, the processing proceeds via step 2195. Figure 20 Step 2095, temporarily end this routine.

[0180] Additionally, if the CPU determines "no" in step 2120, the process will proceed via step 2195. Figure 20 Step 2095, temporarily end this routine.

[0181] CPU Figure 20 If the result in step 2030 is "no", the process will proceed directly to step 2095, temporarily ending this routine.

[0182] In addition, if the CPU determines "no" in step 2020, the process will directly proceed to step 2095, temporarily ending this routine.

[0183] Furthermore, if the CPU determines "no" in step 2010, the process proceeds to step 2060 to determine whether the value of the intermediate information acquisition identifier Xmid is "1". The value of the intermediate information acquisition identifier Xmid is set to "1" when it is predicted that the vehicle 100 will not turn and will travel straight afterward during the reversing period, and is set to "0" when the processing in step 2065 is performed.

[0184] If the CPU determines "yes" in step 2060, the process proceeds to step 2065, whereby the temporary intermediate information Imid_pre is obtained and stored in RAM as described above. Then, the CPU proceeds to step 2070.

[0185] On the other hand, if the CPU determines "no" in step 2060, the processing will proceed directly to step 2070.

[0186] After the CPU initiates the process to step 2070, it continues to execute the first parking and driving process. Next, the CPU initiates the process to step 2075, and determines whether the value of the parking completion flag Xpark_fin is "1". The value of the parking completion flag Xpark_fin is set to "1" when the vehicle 100 has entered the registered object parking area 61set, and is set to "0" when the first parking and driving process ends.

[0187] If the CPU determines "yes" in step 2075, the process proceeds to step 2080, ending the first stop-and-go process. Then, the CPU proceeds to step 2095, temporarily ending the current routine.

[0188] On the other hand, if the CPU determines "no" in step 2075, the process will proceed directly to step 2095, temporarily ending the current routine.

[0189] Additionally, if the CPU determines "no" in step 2005, it proceeds to step 2090, ending the display of the top-view image 51P and other images to the display 50. Then, the CPU proceeds to step 2095, temporarily terminating this routine.

[0190] Therefore, the CPU executes at specified intervals. Figure 22 The example shown. Therefore, at the designated time, the CPU from Figure 22 Step 2200 begins processing, leading to step 2205, where it is determined whether the value of the information registration request identifier Xreg_req is "1". The value of the information registration request identifier Xreg_req is set to "1" when the vehicle 100 completes parking in the parking lot 62 through the first parking and driving process, and is set to "0" when the parking lot information Ipark is registered to RAM.

[0191] If the CPU determines "yes" in step 2205, the process proceeds to step 2210, clearing the captured image 51C from the display 50 and displaying the top view image 51P, the movement button image 57, the angle adjustment button image 58, and the registration button image 55 (see reference) on the display 50. Figure 10 (C) Next, the CPU causes the processing to proceed to step 2215, which determines whether the value of the registration confirmation flag Xreg_det is "1". The value of the registration confirmation flag Xreg_det is set to "1" when the registration button image 55 is touched, and is set to "0" when the processing in step 2120 is performed.

[0192] If the CPU determines "yes" in step 2215, it proceeds to step 2220, whereby, as described above, the registration entry information Ient_reg, the registration site information Iin_reg, and the registration zoning information Iarea_reg are registered in RAM as parking lot information Ipark. Then, the CPU proceeds to step 2295, temporarily terminating this routine.

[0193] On the other hand, if the CPU determines "no" in step 2215, the process will proceed directly to step 2295, temporarily ending the current routine.

[0194] Furthermore, if the CPU determines "no" in step 2205, the processing will proceed directly to step 2295, temporarily terminating the routine.

[0195] Therefore, the CPU executes at specified intervals. Figure 23 The example shown. Therefore, at the designated time, the CPU from Figure 23 Step 2300 begins processing, leading to step 2305, where it is determined whether the value of the parking assist switch operation flag X2_auto is "1". When it is determined that vehicle 100 has stopped near parking lot 62, and that parking lot 62 is a registered parking lot, and the parking assist switch image 51M has been touched, the value of the parking assist switch operation flag X2_auto is set to "1". It is set to "0" when vehicle 100 finishes parking in parking lot 62.

[0196] If the CPU determines "yes" in step 2305, the process proceeds to step 2310, where it checks whether the value of the second parking and driving processing identifier X2_exe is "0". The value of the second parking and driving processing identifier X2_exe is set to "1" when the second parking and driving process begins and set to "0" when the second parking and driving process ends.

[0197] If the CPU determines "yes" in step 2310, the process proceeds to step 2315, clearing the parking assist switch image 51M from the display 50 and displaying the captured image 51C, the top view image 51P, the registration main switch image 52M, and the parking start button image 56 on the display 50 (see reference). Figure 19 (B)

[0198] Next, the CPU initiates step 2317, determining whether the value of the registration master switch operation flag Xmain is "0". The value of the registration master switch operation flag Xmain is set to "1" when the registration master switch image 52M is touched, and set to "0" when the vehicle 100 finishes parking in the parking lot 62.

[0199] If the CPU determines "yes" in step 2317, the process proceeds to step 2320, where it checks whether the value of the parking start flag Xpark_start is "1". The value of the parking start flag Xpark_start is set to "1" when the parking start button image 56 is touched, and set to "0" when the second parking and driving process begins.

[0200] If the CPU determines "yes" in step 2320, the process proceeds to step 2325, clearing the registration master switch image 52M and the parking start button image 56 from the display 50. At this time, the CPU continues to display the captured image 51C and the top-view image 51P (see reference) on the display 50. Figure 19C) Then, the CPU proceeds to step 2330, setting the parking zone 61 registered as parking information Ipark as the target parking zone 61tgt. Next, the CPU proceeds to step 2335, setting the route for driving vehicle 100 to the target parking zone 61tgt as the target driving route Rtgt. Next, the CPU proceeds to step 2340, starting the second parking and driving process. Afterward, the CPU proceeds to step 2395, temporarily ending this routine.

[0201] On the other hand, if the CPU determines "no" in step 2320, the process will proceed directly to step 2395, temporarily ending the current routine.

[0202] Additionally, if the CPU determines "no" in step 2317, it proceeds to step 2342, setting the value of the parking assist switch operation flag X1_auto to "1". Then, the CPU proceeds to step 2395, temporarily terminating the current routine. Thus, in Figure 20 In step 2005, the judgment was "yes".

[0203] Additionally, if the CPU determines "no" in step 2310, it proceeds to step 2345 to continue executing the second parking and driving process. Then, the CPU proceeds to step 2350 to determine if the parking completion flag Xpark_fin is "1". The parking completion flag Xpark_fin is set to "1" when the vehicle 100 has fully entered the target parking area 61tgt, and is set to "0" at the end of the second parking and driving process.

[0204] If the CPU determines "yes" in step 2350, it proceeds to step 2355, ending the second parking and driving process. Then, the CPU proceeds to step 2395, temporarily terminating this routine.

[0205] On the other hand, if the CPU determines "no" in step 2350, the processing will proceed directly to step 2395, temporarily ending the current routine.

[0206] Furthermore, if the CPU determines "no" in step 2305, it proceeds to step 2360, ending the display of images such as the top-view image 51P on the display 50. Next, the CPU proceeds to step 2395, temporarily terminating this routine.

[0207] The above describes the specific actions of the vehicle parking assistance device 10. Therefore, instead of targeting the feature points of three-dimensional objects within the parking lot 62 or the three-dimensional objects surrounding the parking lot 62, it registers information related to the feature points F of the ground 63 within the parking lot 62 and the ground 63 surrounding the parking lot 62 as the registration entry information Int_reg (refer to...). Figure 22 (Step 2220). Therefore, even if the situation surrounding the vehicle 100 and the parking lot 62 is different when the registration entry information Int_reg is registered and when the vehicle 100 arrives at the entrance 62ent of the registered parking lot 62, if an entry feature point Fent corresponding to the entry feature point Fent registered with the registration entry information Int_reg is obtained when arriving at the entrance 62ent of the registered parking lot 62, it can be determined that the entry feature point Fent is the entry feature point Fent registered with the registration entry information Int_reg. As a result, it can be determined that the parking lot 62 is a registered parking lot 62. Therefore, the vehicle 100 can be automatically parked in the registered parking lot 62.

[0208] Furthermore, the present invention is not limited to the described embodiments, and various modifications can be adopted within the scope of the present invention.

[0209] Explanation of reference numerals in the attached figures

[0210] 10…Vehicle parking assistance devices;

[0211] 11… Vehicle drive force generating device;

[0212] 12…braking device;

[0213] 13…Steering mechanism;

[0214] 30…Sonar sensor device;

[0215] 40…camera equipment;

[0216] 51C…captures images;

[0217] 51P…Top view image;

[0218] 52… Parking area marking image;

[0219] 61…Parking zone designation;

[0220] Parking lot 62…

[0221] 63…ground;

[0222] 73…left side range;

[0223] 74…Right side range;

[0224] 90…ECU;

[0225] 100… vehicles;

[0226] F…feature point.

Claims

1. A vehicle parking assistance device capable of performing parking assistance control to automatically park a vehicle in a parking lot, The vehicle parking assistance device has the following features: A camera, mounted on the vehicle in a manner capable of capturing images of the area surrounding the vehicle; and The control unit obtains features of the ground inside and around the parking lot from images captured by a camera, using these features as entrance feature points. The control unit includes: The storage unit stores the features of the ground within the parking lot and the ground surrounding the parking lot as parking lot information; and The judgment unit determines whether the features of the ground inside and around the parking lot captured by the camera have been registered as parking lot information by comparing them with the stored parking lot information. If the control unit determines that the features of the ground inside and around the parking lot have been registered as parking lot information, the control unit starts the parking assistance control.

2. The vehicle parking assistance device according to claim 1, wherein, The control unit is configured to, The image captured by the camera is divided into multiple defined segments. At least one of the features is obtained from the specified segmentation range. The features of the ground around the starting position of the parking assistance control are features obtained from the defined segmented range.

3. The vehicle parking assistance device according to claim 2, wherein, The specified segmentation ranges are all the same size.

4. The vehicle parking assistance device according to claim 2 or 3, wherein, The specified division ranges are set to be non-overlapping.

5. The vehicle parking assistance device according to claim 2 or 3, wherein, The control unit is configured to obtain a greater number of the features from the predetermined segmentation range closer to the center of the entrance to the parking lot, compared to the predetermined segmentation range farther from the center of the entrance to the parking lot.

6. The vehicle parking assistance device according to claim 2 or 3, wherein, The control unit is configured to, A specified number of features are obtained from each of the specified segmentation ranges. In the case where there is a specified segmentation range in which the specified number of the features cannot be obtained, a greater number of the features than the specified number are obtained from a specified segmentation range outside the specified segmentation range in which the specified number of the features cannot be obtained.

7. The vehicle parking assistance device according to any one of claims 1 to 3, wherein, The parking information includes information related to the intensity pattern of the image of the feature.

8. The vehicle parking assistance device according to any one of claims 1 to 3, wherein, The parking information includes information related to the location of the feature, with a specified location within the parking lot as a reference.

9. The vehicle parking assistance device according to any one of claims 1 to 3, wherein, The control unit is configured to execute the parking assistance control using the parking lot information obtained this time, in a case where it is determined that a feature of a ground around the vehicle is not the registered parking lot information.

10. The vehicle parking assistance apparatus according to any one of claims 1 to 3, wherein The cameras include a front camera that captures a front of the vehicle, a rear camera that captures a rear of the vehicle, a left camera that captures a left side of the vehicle, and a right camera that captures a right side of the vehicle.

Images