Vehicle parking assist device

The camera captures the ground feature point information of the parking lot for registration and comparison, and solves the problem of automatic parking accuracy caused by different conditions of vehicles and parking lots, and realizes accurate parking of vehicles under different conditions.

CN119078802AActive Publication Date: 2024-12-06TOYOTA JIDOSHA KK
View PDF 12 Cites 0 Cited by

Patent Information

Application Number
CN202411270409.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2019-10-11
Filing Date
2020-09-27
Publication Date
2024-12-06
Estimated Expiration
2040-09-27

AI Technical Summary

Technical Problem

When existing vehicle parking assist devices register and automatically park at feature points, the different conditions of the vehicle and parking lots lead to the inability to accurately judge the parking lot, resulting in the vehicle being unable to automatically park to the correct position.

Method used

The camera is used to capture the ground and surrounding feature points of the parking lot as parking information for registration, and compare the entry feature points information to ensure that the vehicle can park accurately under different conditions.

Benefits of technology

Even if the conditions of the vehicle and parking lot change, the ground feature point information captured by the camera can accurately determine the parking lot location to ensure that the vehicle can automatically park to the correct parking lot.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119078802A_ABST
    Figure CN119078802A_ABST
Patent Text Reader

Abstract

The present invention provides a vehicle parking assistance device capable of automatically parking a vehicle to a parking lot even when the conditions of the parking lot are different when a feature point is registered and when the vehicle is automatically parked. When a vehicle is stopped beside a parking lot, a feature point of a ground surface at an entrance of the parking lot is acquired from a captured image as an entrance feature point, parking assistance control is performed using parking lot information acquired from the captured image, and parking lot information including entrance feature point information is registered. Afterwards, when the vehicle is stopped beside the undetermined parking lot, the feature point of the ground at the entrance of the undetermined parking lot is acquired as an uncontrasted feature point, and when it is determined that the undetermined parking lot is a registered parking lot by contrasting the feature point with the entrance feature point information, the feature point of the ground at the entrance of the undetermined parking lot is determined to be a registered parking lot. Parking assist control is performed using the parking lot information obtained this time and the registered parking lot information.
Need to check novelty before this filing date? Find Prior Art

Description

Divisional application statement This application is a divisional application of the Chinese invention patent application filed on September 27, 2020, with the invention name “Vehicle Parking Assist Device” and application number 202011035115.6. Technical Field

[0001] The invention relates to a vehicle parking assistance device. Background Art

[0002] A vehicle parking assistance device is known that enables a vehicle to automatically park in a parking lot (e.g., a parking lot of a private residence) that is not provided with demarcation lines such as white lines that demarcate parking areas. The vehicle parking assistance device is configured to register information related to the parking lot (hereinafter referred to as "parking lot information") when the vehicle parks in the parking lot, and when the vehicle is to be automatically parked in the parking lot next time, the parking lot information obtained at that time is compared with the registered parking lot information to grasp the positional relationship between the vehicle and the parking lot, and the vehicle is automatically parked in the parking lot at the same time. As such a vehicle parking assistance device, a vehicle parking assistance device is known that registers feature points of a three-dimensional object captured in an image (hereinafter referred to as a "captured image") obtained by capturing a three-dimensional object in a parking lot or a three-dimensional object around a parking lot by a camera as parking lot information (e.g., refer to Japanese Patent Application Laid-Open No. 2017-138664). Summary of the invention

[0003] When only the driver is on the vehicle when registering the feature points of a three-dimensional object (hereinafter referred to as "feature point registration"), but when the vehicle is automatically parked in a parking lot (hereinafter referred to as "automatic parking"), there are other passengers in addition to the driver on the vehicle, and the tilt of the vehicle may be different when the feature points are registered and when the vehicle is automatically parked. In addition, the ground on which the vehicle travels when the feature points are registered is different from the ground on which the vehicle travels when the vehicle is automatically parked. As a result, when the slope of the ground on which the vehicle travels is different, the tilt of the vehicle may be different when the feature points are registered and when the vehicle is automatically parked. In this case, even if the same three-dimensional object is captured in the captured image, the shape of the three-dimensional object captured in the captured image may be different when the feature points are registered and when the vehicle is automatically parked. In this case, even if the feature points of the three-dimensional object (hereinafter referred to as "registered three-dimensional object") with registered feature points are obtained during automatic parking, it may not be possible to determine that these acquired feature points are the feature points of the registered three-dimensional object, and as a result, it may be impossible to automatically park the vehicle in the parking lot.

[0004] In addition, in a case where feature point registration is performed in the morning but automatic parking is performed in the afternoon, the way in which sunlight irradiates the three-dimensional object and the way in which sunlight reflected from the ground irradiates the three-dimensional object are different when the feature point registration is performed and when the automatic parking is performed. In addition, in a case where feature point registration is performed during the day but automatic parking is performed at night, the way in which light irradiates the three-dimensional object and the way in which light reflected from the ground irradiates the three-dimensional object are different when the feature point registration is performed and when the automatic parking is performed. In such a case, even if the same three-dimensional object is captured in the captured image, the feature points of the three-dimensional object presented in the captured image may be different when the feature point registration is performed and when the automatic parking is performed. In this case, even if the feature points of the registered three-dimensional object are obtained during automatic parking, it cannot be determined that these obtained feature points are the feature points of the registered three-dimensional object, and as a result, the vehicle may not be able to be automatically parked in the parking lot.

[0005] Furthermore, when feature points of movable three-dimensional objects such as other vehicles, bicycles, and flower pots are registered as parking lot information, and such movable three-dimensional objects have moved from the locations at which the feature points were registered during automatic parking, the feature points of the three-dimensional objects for which the feature points were registered are not obtained during automatic parking, resulting in the inability to automatically park the vehicle in the parking lot.

[0006] When the feature points of a three-dimensional object are registered as parking lot information in this manner, if the conditions of the surrounding vehicle and the parking lot are different between when the feature points are registered and when the vehicle is automatically parked, the vehicle may not be automatically parked in the parking lot.

[0007] The present invention is made to address the above-mentioned problems. That is, one of the objects of the present invention is to provide a vehicle parking assistance device that can automatically park a vehicle in a parking lot even when the conditions of the surrounding vehicle and the parking lot are different during feature point registration and automatic parking.

[0008] The vehicle parking assistance device of the present invention comprises: a camera, which is installed on the vehicle in a manner capable of photographing the surroundings of the vehicle; and a control unit, which is capable of executing parking assistance control, wherein the parking assistance control is to automatically park the vehicle in the parking lot using information related to the parking lot, wherein the information related to the parking lot is obtained as parking lot information from an image photographed by the camera.

[0009] The control unit is configured to obtain an image of the parking lot taken by the camera when the vehicle stops next to the parking lot as a captured image, obtain feature points of the ground at the entrance of the parking lot from the captured image as entrance feature points, perform the parking assist control using the parking lot information obtained from the captured image obtained this time, and register the parking lot information, wherein the parking lot information includes information related to the obtained entrance feature points as entrance feature point information.

[0010] Furthermore, the control unit is configured to obtain the captured image when the vehicle stops next to an undetermined parking lot after the parking lot information is registered, obtain feature points of the ground at the entrance of the undetermined parking lot from the captured image obtained this time as unchecked feature points, and determine whether the undetermined parking lot is the parking lot where the parking lot information is registered, i.e., the registered parking lot, by comparing information related to the unchecked feature points with the entrance feature point information; and when it is determined that the undetermined parking lot is the registered parking lot, perform the parking assist control using the parking lot information obtained this time and the registered parking lot information.

[0011] Thus, information related to feature points of the ground in the parking lot and the ground around the parking lot is registered as parking lot information, not feature points of three-dimensional objects in the parking lot and three-dimensional objects around the parking lot. Even if the conditions of the surrounding vehicle and the parking lot are different when the feature points are registered and when the parking lot is automatically parked, the change in the image of the ground in the parking lot and the ground around the parking lot captured by the camera is small. Therefore, even if the conditions of the surrounding vehicle and the parking lot are different when the feature points are registered and when the parking lot is automatically parked, when a feature point corresponding to an entrance feature point registered as parking lot information (hereinafter referred to as a "registered entrance feature point") is obtained during automatic parking, it can be determined that the feature point is a registered entrance feature point, and as a result, it can be determined that the parking lot that has not been determined is a registered parking lot. Therefore, the vehicle can be automatically parked in the parking lot.

[0012] In the vehicle parking assistance device of the present invention, the control unit may also be configured to divide the captured image into a plurality of predetermined division ranges and obtain at least one of the feature points from each of the predetermined division ranges. In this case, the entry feature point is a feature point obtained from each of the predetermined division ranges.

[0013] This makes it possible to acquire the entrance feature point without biasing the ground surface in a part of the range of the entrance of the parking lot. Therefore, it is possible to more reliably determine that the undetermined parking lot is a registered parking lot.

[0014] In the vehicle parking assistance device of the present invention, the predetermined divided ranges are, for example, of the same size.

[0015] Furthermore, in the vehicle parking assist device of the present invention, the predetermined division ranges are set, for example, so as not to overlap with each other.

[0016] In addition, in the vehicle parking assistance device of the present invention, the control unit can also be configured to obtain a relatively large number of the feature points as the entrance feature points from the prescribed segmentation range closer to the center of the entrance of the parking lot than from the prescribed segmentation range farther from the center of the entrance of the parking lot.

[0017] This makes it possible to acquire a large number of entrance feature points on the ground surface close to the center of the entrance to the parking lot, so that it is possible to more reliably determine that an undetermined parking lot is a registered parking lot.

[0018] In addition, in the vehicle parking assistance device of the present invention, the control unit can also be configured to obtain a specified number of the entry feature points from each of the specified division ranges, and when there is a specified division range in which the specified number of the entry feature points cannot be obtained, obtain a number of the entry feature points greater than the specified number from the specified division range outside the specified division range in which the specified number of the entry feature points cannot be obtained.

[0019] Thus, even when the specified number of entrance feature points is not obtained in a part of the range, the same or close number of entrance feature points as the total number of entrance feature points when the specified number of entrance feature points is obtained in all the ranges can be obtained. Therefore, it is possible to more reliably determine that the parking lot that has not been determined is a registered parking lot.

[0020] In the vehicle parking assistance device of the present invention, the entry feature point information includes, for example, information on a light and dark pattern of an image of the entry feature point.

[0021] Thereby, the light and dark pattern in the image of the entrance feature point is registered as parking lot information.

[0022] In the vehicle parking assistance device of the present invention, the entrance feature point information includes, for example, information related to the position of the entrance feature point with respect to a predetermined position in the parking lot as a reference.

[0023] Thus, the position of the entrance feature point based on the predetermined position in the parking lot is registered as the parking lot information.

[0024] In addition, in the vehicle parking assistance device of the present invention, the control unit may be configured to, when it is determined that the undetermined parking lot is not the registered parking lot, execute the parking assistance control using the parking lot information obtained this time.

[0025] In addition, in the vehicle parking assistance device of the present invention, the camera includes, for example, a front camera for photographing the front of the vehicle, a rear camera for photographing the rear of the vehicle, a left camera for photographing the left side of the vehicle, and a right camera for photographing the right side of the vehicle.

[0026] The embodiments are described with reference to the following drawings, and the various structural elements of the present invention are not limited to the described embodiments. Other purposes, other features and additional advantages of the present invention can be easily understood from the description of the embodiments of the present invention described with reference to the following drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a diagram showing a vehicle parking assistance device according to an embodiment of the present invention and a vehicle to which the vehicle parking assistance device is applied. Figure 2 It is a diagram showing the arrangement and detection range of the sonar sensor device. Figure 3 It is a diagram showing the arrangement and imaging range of a camera sensor device. Figure 4 This is a diagram showing an example of a parking lot. Figure 5 It is a diagram showing the front range and the rear range. Figure 6 It is a diagram showing the left range and the right range. Figure 7 is a diagram showing feature points. Figure 8 This is a diagram showing parking areas. Fig. 9 (A) to (D) are diagrams showing displays. Fig.10 (A) to (C) are diagrams showing displays. Fig.11 It is used to illustrate Figure 1 A diagram showing the operation of the vehicle parking assistance device. Fig.12 It is used to illustrate Figure 1 A diagram showing the operation of the vehicle parking assistance device. Fig.13 It is used to illustrate Figure 1 A diagram showing the operation of the vehicle parking assistance device. Fig.14 It is a graph showing the entry feature points. Fig.15 It is used to illustrate Figure 1 A diagram showing the operation of the vehicle parking assistance device. Fig.16 It is used to illustrate Figure 1 A diagram showing the operation of the vehicle parking assistance device. Fig.17 It is used to illustrate Figure 1 A diagram showing the operation of the vehicle parking assistance device. Fig.18 It is used to illustrate Figure 1 A diagram showing the operation of the vehicle parking assistance device. Fig.19 (A) to (C) are diagrams showing displays. Fig. 20 It is shown Figure 1 A flowchart of a routine executed by the CPU of the ECU is shown. Fig.21 It is shown Figure 1 A flowchart of a routine executed by the CPU of the ECU shown. Fig. 22 It is shown Figure 1 A flowchart of a routine executed by the CPU of the ECU shown. Fig.23 It is shown Figure 1 A flowchart of a routine executed by the CPU of the ECU is shown. DETAILED DESCRIPTION

[0028] Hereinafter, a vehicle parking assistance device according to an embodiment of the present invention will be described with reference to the drawings. Figure 1 A vehicle parking assistance device 10 according to an embodiment of the present invention and a vehicle 100 to which the vehicle parking assistance device 10 is applied are shown.

[0029] like Figure 1 As shown, the vehicle parking assist device 10 has an ECU 90. ECU is an abbreviation for an electronic control unit. The ECU 90 has a microcomputer as a main part. The microcomputer includes a CPU, a ROM, a RAM, a nonvolatile memory, an interface, etc. The CPU implements various functions by executing instructions, programs, or routines stored in the ROM.

[0030] The vehicle 100 is equipped with a vehicle driving force generating device 11, a braking device 12, and a steering device 13. The vehicle driving force generating device 11 is a device for generating a driving force for driving the vehicle 100 and applying the driving force to the driving wheels of the vehicle 100. The vehicle driving force generating device 11 is, for example, an internal combustion engine, an electric motor, etc. The braking device 12 is a device for applying a braking force for braking the vehicle 100 to the wheels of the vehicle 100. The steering device 13 is a device for applying a steering torque for steering the vehicle 100 to the steering wheels of the vehicle 100.

[0031] The vehicle driving force generating device 11, the braking device 12, and the steering device 13 are electrically connected to the ECU 90. The ECU 90 controls the driving force applied to the driving wheels of the vehicle 100 by controlling the operation of the vehicle driving force generating device 11. In addition, the ECU 90 controls the braking force applied to the wheels of the vehicle 100 by controlling the operation of the braking device 12. In addition, the ECU 90 controls the steering torque applied to the steering wheels of the vehicle 100 by controlling the operation of the steering device 13.

[0032] <Sensors, etc.> The vehicle parking assistance device 10 includes an accelerator pedal operation amount sensor 21, a brake pedal operation amount sensor 22, a steering angle sensor 23, a steering torque sensor 24, a vehicle speed sensor 25, a yaw rate sensor 26, a front and rear acceleration sensor 27, a lateral acceleration sensor 28, a sonar sensor device 30, a camera sensor device 40 and a display 50.

[0033] The accelerator pedal operation amount sensor 21 is electrically connected to the ECU 90. The ECU 90 detects and acquires the operation amount AP of the accelerator pedal 14 as the accelerator pedal operation amount AP via the accelerator pedal operation amount sensor 21. The ECU 90 controls the operation of the vehicle driving force generating device 11 so that the driving force corresponding to the acquired accelerator pedal operation amount AP is applied from the vehicle driving force generating device 11 to the driving wheels of the vehicle 100.

[0034] The brake pedal operation amount sensor 22 is electrically connected to the ECU 90. The ECU 90 detects and obtains the driver's operation amount BP of the brake pedal 15 as the brake pedal operation amount BP via the brake pedal operation amount sensor 22. The ECU 90 controls the operation of the brake device 12 according to the obtained brake pedal operation amount BP so that the braking force is applied from the brake device 12 to the wheels of the vehicle 100.

[0035] The steering angle sensor 23 is electrically connected to the ECU 90. The ECU 90 detects and acquires the rotation angle θst of the steering wheel 16 with respect to the neutral position via the steering angle sensor 23 as the steering angle θst.

[0036] The steering torque sensor 24 is electrically connected to the ECU 90. The ECU 90 detects and acquires the torque TQst input from the driver to the steering shaft 17 via the steering torque sensor 24 as the steering torque TQst.

[0037] The ECU 90 controls the operation of the steering device 13 so that a steering torque corresponding to the acquired steering angle θst and steering torque TQst is applied to the steered wheels of the vehicle 100 .

[0038] The vehicle speed sensor 25 is electrically connected to the ECU 90. The ECU 90 detects and acquires the rotation speed Vrot of each wheel of the vehicle 100 as the rotation speed Vrot of each wheel via the vehicle speed sensor 25. The ECU 90 acquires the running speed SPD of the vehicle 100 as the vehicle speed SPD based on the acquired rotation speed Vrot of each wheel.

[0039] The yaw rate sensor 26 is electrically connected to the ECU 90. The ECU 90 detects and acquires the yaw rate YR of the vehicle 100 via the yaw rate sensor 26 as the vehicle yaw rate YR.

[0040] The longitudinal acceleration sensor 27 is electrically connected to the ECU 90. The ECU 90 detects and acquires the longitudinal acceleration Gx of the vehicle 100 via the longitudinal acceleration sensor 27 as the vehicle longitudinal acceleration Gx.

[0041] The lateral acceleration sensor 28 is electrically connected to the ECU 90. The ECU 90 detects and acquires the lateral acceleration Gy of the vehicle 100 via the lateral acceleration sensor 28 as the vehicle lateral acceleration Gy.

[0042] The sonar sensor device 30 includes first to twelfth clearance sonars 301 to 312 .

[0043] like Figure 2 As shown, the first clearance sonar 301 is installed on the vehicle 100 in such a manner as to emit sound waves from the front left end portion of the vehicle 100 to the left front. The second clearance sonar 302 is installed on the vehicle 100 in such a manner as to emit sound waves from the left front end portion of the vehicle 100 to the front. The third clearance sonar 303 is installed on the vehicle 100 in such a manner as to emit sound waves from the front right end portion of the vehicle 100 to the right front. The fourth clearance sonar 304 is installed on the vehicle 100 in such a manner as to emit sound waves from the right front end portion of the vehicle 100 to the front.

[0044] In addition, the fifth clearance sonar 305 is installed on the vehicle 100 in such a manner as to emit sound waves from the rear left end portion of the vehicle 100 to the left rear. The sixth clearance sonar 306 is installed on the vehicle 100 in such a manner as to emit sound waves from the left rear end portion of the vehicle 100 to the rear. The seventh clearance sonar 307 is installed on the vehicle 100 in such a manner as to emit sound waves from the rear right end portion of the vehicle 100 to the right rear. The eighth clearance sonar 308 is installed on the vehicle 100 in such a manner as to emit sound waves from the right rear end portion of the vehicle 100 to the rear.

[0045] In addition, the ninth clearance sonar 309 is installed on the vehicle 100 in such a manner as to emit sound waves from the front left side portion of the vehicle 100 to the left. The tenth clearance sonar 310 is installed on the vehicle 100 in such a manner as to emit sound waves from the rear left side portion of the vehicle 100 to the left. The eleventh clearance sonar 311 is installed on the vehicle 100 in such a manner as to emit sound waves from the front right side portion of the vehicle 100 to the right. The twelfth clearance sonar 312 is installed on the vehicle 100 in such a manner as to emit sound waves from the rear right side portion of the vehicle 100 to the right.

[0046] The first to twelfth interstitial sonars 301 to 312 receive sound waves reflected by the object.

[0047] The sonar sensor device 30 is electrically connected to the ECU 90. The sonar sensor device 30 transmits information related to "sound waves emitted by the first clearance sonar 301 to the twelfth clearance sonar 312" and "sound waves received by the first clearance sonar 301 to the twelfth clearance sonar 312" to the ECU 90. The ECU 90 acquires information related to objects existing around the vehicle 100 as object information OBJ based on the information received from the sonar sensor device 30 (hereinafter referred to as "sonar information SON").

[0048] exist Figure 2 In the figure, the direction indicated by the symbol Dx is the front-rear direction of the vehicle 100, hereinafter referred to as the "vehicle front-rear direction Dx", and the direction indicated by the symbol Dy is the width direction of the vehicle 100, hereinafter referred to as the "vehicle width direction Dy".

[0049] The camera sensor device 40 includes a front camera 41, a rear camera 42, a left camera 43, and a right camera 44. Hereinafter, the front camera 41, the rear camera 42, the left camera 43, and the right camera 44 are collectively referred to as "camera 45" as needed.

[0050] like Figure 3As shown, the front camera 41 is installed at the center of the front end of the vehicle 100 to shoot the scene in front of the vehicle 100, and its field of view 41A is approximately 180°. The rear camera 42 is installed at the center of the rear end of the vehicle 100 to shoot the scene behind the vehicle 100, and its field of view 42A is also approximately 180°. The left camera 43 is installed at the left side of the vehicle 100 to shoot the scene on the left side of the vehicle 100, and its field of view 43A is also approximately 180°. The right camera 44 is installed at the right side of the vehicle 100 to shoot the scene on the right side of the vehicle 100, and its field of view 44A is also approximately 180°.

[0051] The camera sensor device 40 is electrically connected to the ECU 90. The ECU 90 can acquire information on the image of the scene captured by each camera 45 via the camera sensor device 40.

[0052] Hereinafter, as required, information related to the image of the scene captured by the front camera 41 is referred to as "front image information IMG1", information related to the image of the scene captured by the rear camera 42 is referred to as "rear image information IMG2", information related to the image of the scene captured by the left camera 43 is referred to as "left image information IMG3", and information related to the image of the scene captured by the right camera 44 is referred to as "right image information IMG4". In addition, hereinafter, as required, the front image information IMG1, the rear image information IMG2, the left image information IMG3, and the right image information IMG4 are collectively referred to as "image information IMG".

[0053] The vehicle parking assistance device 10 acquires the feature point F based on the image information IMG when a predetermined condition is satisfied. The feature point F is an image within a predetermined range where the brightness change is large in the image captured by each camera 45 .

[0054] For example, when camera 45 captures Figure 4 In the case of the parking lot 62 shown, the corners of the concrete bricks 63B, the corners of the ground 63 consisting of the lawn 63L, and the boundary between the ground 63 consisting of the bricks 63B and the ground 63 consisting of the lawn 63L are obtained as feature points F.

[0055] about Figure 4 The floor 63 of the parking lot 62 shown is composed of a floor 63 made of concrete 63C and a floor 63 made of lawn 63L. In addition, a plurality of concrete bricks 63B covering the side ditch are arranged side by side at the entrance 62ent of the parking lot 62. Therefore, the floor 63 of the entrance 62ent of the parking lot 62 is composed of the surface of the bricks 63B.

[0056] The vehicle parking assist device 10 obtains a predetermined range 71 (see FIG. 1 ) in front of the vehicle 100 based on the front image information IMG1. Figure 5 ) on the ground 63 (hereinafter referred to as "front feature point F1"). In addition, the vehicle parking assistance device 10 obtains a predetermined range 72 (refer to Figure 5 ) on the ground 63 (hereinafter referred to as "rear feature point F2"). In addition, the vehicle parking assistance device 10 obtains a predetermined range 73 on the left side of the vehicle 100 based on the left side image information IMG3 (refer to Figure 6 ) on the ground 63 (hereinafter referred to as "left feature point F3"). The vehicle parking assistance device 10 obtains a predetermined range 74 on the right side of the vehicle 100 based on the right side image information IMG4 (refer to Figure 6 ) on the ground 63 (hereinafter referred to as “right feature point F4”).

[0057] like Figure 5 As shown, the prescribed range 71 is a range defined by lines L711, L712, L713, and L714. Line L711 is a line L711 extending in the vehicle width direction Dy from the front camera 41 at a prescribed distance Dset to the front of the vehicle 100. Line L712 is a line L712 passing through the front camera 41 and extending in the vehicle width direction Dy. Line L713 is a line extending in the vehicle front-rear direction Dx from the front camera 41 at a prescribed distance Dset to the left of the vehicle 100. Line L714 is a line extending in the vehicle front-rear direction Dx from the front camera 41 at a prescribed distance Dset to the right of the vehicle 100. Hereinafter, the prescribed range 71 will be referred to as the "front range 71".

[0058] The front range 71 is divided into eight ranges 71D that are divided into four equal parts in the vehicle width direction Dy and divided into two equal parts in the vehicle front-rear direction Dx. That is, the front range 71 is divided into a plurality of ranges 71D of equal area. Hereinafter, these ranges 71D are referred to as "front divided ranges 71D". The two front divided ranges 71D set at the left end in the vehicle width direction Dy among the front divided ranges 71D are referred to as "left end divided ranges 71D3", the two front divided ranges 71D set at the right end in the vehicle width direction Dy are referred to as "right end divided ranges 71D4", and the four front divided ranges 71D set in the middle in the vehicle width direction Dy are referred to as "middle divided ranges 71D5".

[0059] In addition, if Figure 5As shown, the prescribed range 72 is a range defined by lines L721, L722, L723, and L724. Line L721 is a line that passes through the rear camera 42 and extends in the vehicle width direction Dy. Line L722 is a line that is a prescribed distance Dset away from the rear camera 42 to the rear of the vehicle 100 and extends in the vehicle width direction Dy. Line L723 is a line that is a prescribed distance Dset away from the rear camera 42 to the left of the vehicle 100 and extends in the vehicle front-rear direction Dx. Line L724 is a line that is a prescribed distance Dset away from the rear camera 42 to the right of the vehicle 100 and extends in the vehicle front-rear direction Dx. Hereinafter, the prescribed range 72 will be referred to as the "rear range 72".

[0060] The rear range 72 is divided into eight ranges 72D that are divided into four equal parts in the vehicle width direction Dy and divided into two equal parts in the vehicle front-rear direction Dx. That is, the rear range 72 is divided into a plurality of ranges 72D of equal area. Hereinafter, these ranges 72D are referred to as "rear divided ranges 72D". In addition, the two rear divided ranges 72D set at the left end in the vehicle width direction Dy among the rear divided ranges 72D are referred to as "left end divided ranges 72D3", the two rear divided ranges 72D set at the right end in the vehicle width direction Dy are referred to as "right end divided ranges 72D4", and the four rear divided ranges 72D set in the middle in the vehicle width direction Dy are referred to as "middle divided ranges 72D5".

[0061] In addition, if Figure 6 As shown, the prescribed range 73 is a range defined by lines L731, L732, L733, and L734. Line L731 is a line extending in the vehicle width direction Dy from the left camera 43 at a prescribed distance Dset to the front of the vehicle 100. Line L732 is a line extending in the vehicle width direction Dy from the left camera 43 at a prescribed distance Dset to the rear of the vehicle 100. Line L733 is a line extending in the vehicle front-rear direction Dx from the left camera 43 at a prescribed distance Dset to the left of the vehicle 100. Line L734 is a line extending in the vehicle front-rear direction Dx through the left camera 43. Hereinafter, the prescribed range 73 will be referred to as the "left side range 73".

[0062] The left side range 73 is divided into eight ranges 73D that are divided into four equal parts in the vehicle front-rear direction Dx and divided into two equal parts in the vehicle width direction Dy. That is, the left side range 73 is divided into a plurality of ranges 73D of equal area. Hereinafter, these ranges 73D are referred to as "left side divided ranges 73D". In addition, the two left side divided ranges 73D set at the front end in the vehicle front-rear direction Dx among the left side divided ranges 73D are referred to as "front end divided ranges 73D1", the two left side divided ranges 73D set at the rear end in the vehicle front-rear direction Dx are referred to as "rear end divided ranges 73D2", and the four left side divided ranges 73D set in the middle in the vehicle front-rear direction Dx are referred to as "middle divided ranges 73D5".

[0063] In addition, if Figure 6 As shown, the prescribed range 74 is a range defined by lines L741, L742, L743, and L744. Line L741 is a line extending in the vehicle width direction Dy from the right camera 44 at a prescribed distance Dset to the front of the vehicle 100. Line L742 is a line extending in the vehicle width direction Dy from the right camera 44 at a prescribed distance Dset to the rear of the vehicle 100. Line L743 is a line extending in the vehicle front-rear direction Dx through the right camera 44. Line L744 is a line extending in the vehicle front-rear direction Dx from the right camera 44 at a prescribed distance Dset to the right of the vehicle 100. Hereinafter, the prescribed range 74 will be referred to as the "right side range 74".

[0064] The right side range 74 is divided into eight ranges 74D that are divided into four equal parts in the vehicle front-rear direction Dx and divided into two equal parts in the vehicle width direction Dy. That is, the right side range 74 is divided into a plurality of ranges 74D of equal area. Hereinafter, these ranges 74D are referred to as "right side divided ranges 74D". The two right side divided ranges 74D set at the front end in the vehicle front-rear direction Dx among the right divided ranges 74D are referred to as "front end divided ranges 74D1", the two right side divided ranges 74D set at the rear end in the vehicle front-rear direction Dx are referred to as "rear end divided ranges 74D2", and the four right side divided ranges 74D set in the middle in the vehicle front-rear direction Dx are referred to as "middle divided ranges 74D5".

[0065] When the image of the range corresponding to the feature point F captured by the camera 45 is converted into a bird's-eye view image, the converted image of the range corresponding to the feature point F is as follows: Figure 7The image of the range 75 of a square with a predetermined length Lset on one side is shown. When the predetermined conditions are met, the vehicle parking assistance device 10 divides the feature point F into 25 identical square ranges 75D, and obtains the brightness LUM of each range 75D. Then, the vehicle parking assistance device 10 respectively obtains the value ΔLUM (=LUM-LUMave) obtained by subtracting the average value LUMave of these brightnesses LUM from each brightness LUM obtained, and obtains the tendency of the brightness LUM at the feature point F as the shading information CT based on these values ​​ΔLUM. That is, when the predetermined conditions are met, the vehicle parking assistance device 10 obtains the shading pattern in the image of the feature point F taken by the camera 45 as the shading information CT.

[0066] The display 50 is disposed at a position of the vehicle 100 that can be visually recognized by the driver. In this example, the display 50 is a so-called display of a navigation device.

[0067] The display 50 is electrically connected to the ECU 90. The ECU 90 can display various images on the display 50. In this example, the ECU 90 can display a captured image 51C, a top view image 51P, a parking assist switch image 51M, a registered main switch image 52M, a parking area selection image 52S, a parking area dividing line image 52, a setting button image 53, a registration start button image 54, a registration button image 55, a parking start button image 56, a movement button image 57, and an angle adjustment button image 58 on the display 50.

[0068] The captured image 51C is an image captured by the camera 45 .

[0069] The overhead image 51P includes a vehicle overhead image and a vehicle surrounding image. The vehicle overhead image is an image showing the vehicle 100 when viewed from above in the vertical direction. The vehicle surrounding image is an image showing the surroundings of the vehicle 100 when viewed from above in the vertical direction, and includes at least an image showing the parking lot 62. These vehicle overhead image and vehicle surrounding image are produced by the ECU 90 based on the image information IMG.

[0070] The parking assist switch image 51M is an image corresponding to a parking assist switch that is touched and operated by the driver in order to start parking assist control described later by the vehicle parking assist device 10. In addition, in this example, the parking assist switch is displayed on the display 50 in the form of the parking assist switch image 51M, but it may not be displayed on the display 50, but may be provided as a physically operable switch at a location in the vehicle 100 that can be operated by the driver.

[0071] The registration main switch image 52M is an image corresponding to a registration main switch that is touched and operated by the driver in order to register (ie, store) parking lot information Ipark in the vehicle parking assistance device 10 by parking assistance control described later.

[0072] The parking area selection image 52S is as follows Fig. 9 As shown in (C), the left-side horizontal parking area selection image 52SLa, the left-side vertical parking area selection image 52SLb, the right-side horizontal parking area selection image 52SRa, and the right-side vertical parking area selection image 52SRb are included.

[0073] The left side horizontal parking area selection image 52SLa is an image that is touched and operated by the driver in order to select a parking area 61 of a parking lot 62 for parking the vehicle 100 on the left side of the vehicle 100 by changing the orientation of the vehicle 100 by 90 degrees. The parking area 61 is an area (or range or region) where the vehicle 100 is to be parked by the parking assist control described later. Figure 8 As shown, the parking area 61 is set in the parking lot 62. The left longitudinal parking area selection image 52SLb is an image that the driver touches to select the parking area 61 of the parking lot 62 on the left side of the vehicle 100 without changing the orientation of the vehicle 100. The right transverse parking area selection image 52SRa is an image that the driver touches to select the parking area 61 of the parking lot 62 on the right side of the vehicle 100 by changing the orientation of the vehicle 100 by 90 degrees. The right longitudinal parking area selection image 52SRb is an image that the driver touches to select the parking area 61 of the parking lot 62 on the right side of the vehicle 100 without changing the orientation of the vehicle 100.

[0074] Furthermore, when the parking lot 62 on the left side of the vehicle 100 does not have a parking zone 61 where the vehicle 100 can be parked by changing the direction of the vehicle 100 by 90 degrees, the left side horizontal parking zone selection image 52SLa is not displayed on the display 50 or is displayed with reduced brightness (so-called lowered tone) on the display 50. Similarly, when the parking lot 62 on the left side of the vehicle 100 does not have a parking zone 61 where the vehicle 100 can be parked without changing the direction of the vehicle 100, the left side vertical parking zone selection image 52SLb is not displayed on the display 50 or is displayed with reduced brightness (so-called lowered tone) on the display 50. Similarly, when the parking lot 62 on the right side of the vehicle 100 does not have a parking zone 61 where the vehicle 100 can be parked by changing the direction of the vehicle 100 by 90 degrees, the right side horizontal parking zone selection image 52SRa is not displayed on the display 50 or is displayed with reduced brightness (so-called lowered tone) on the display 50. Similarly, when parking lot 62 on the right side of vehicle 100 does not have parking area 61 where vehicle 100 can be parked without changing the orientation, right longitudinal parking area selection image 52SRb is not displayed on display 50 or is displayed on display 50 with reduced brightness (so-called lowered tone).

[0075] Furthermore, when the parking zone selection image 52S displayed in a lowered tone on the display 50 is touched by the driver, the ECU 90 does not determine that the parking zone 61 corresponding to the touched parking zone selection image 52S is selected.

[0076] The parking area dividing line image 52 is an image showing the parking area dividing line 61 .

[0077] The setting button image 53 is an image corresponding to a setting button that is touched and operated by the driver in order to set (or confirm or determine) the parking zone 61 in which the vehicle 100 is to be parked in parking assist control described later.

[0078] The registration start button image 54 is an image corresponding to a registration start button that is touched and operated by the driver to start a first parking travel process of parking assist control described later.

[0079] The registration button image 55 is an image corresponding to a registration button that is touched and operated by the driver in order to register parking lot information Ipark acquired by parking assistance control described later in the vehicle parking assistance device 10 (particularly, the RAM of the ECU 90). The parking lot information Ipark is information related to the parking lot 62 that is used by the vehicle parking assistance device 10 to automatically park the vehicle 100 in the parking lot 62.

[0080] The parking start button image 56 is an image corresponding to a parking start button that is touched and operated by the driver to start parking assist control described later in order to park the vehicle 100 in the parking area 61 registered in the vehicle parking assist device 10 .

[0081] The move button image 57 includes an upward move button image 57U, a downward move button image 57D, a leftward move button image 57L, and a rightward move button image 57R. The upward move button image 57U is an image operated by the driver to move the parking area dividing line image 52 upward on the display 50. The downward move button image 57D is an image operated by the driver to move the parking area dividing line image 52 downward on the display 50. The leftward move button image 57L is an image operated by the driver to move the parking area dividing line image 52 to the left on the display 50. The rightward move button image 57R is an image operated by the driver to move the parking area dividing line image 52 to the right on the display 50.

[0082] The angle adjustment button image 58 includes a left-turn angle adjustment button image 58A and a right-turn angle adjustment button image 58B. The left-turn angle adjustment button image 58A is an image operated by the driver to rotate the parking area dividing line image 52 in the left direction on the display 50. The right-turn angle adjustment button image 58B is an image operated by the driver to rotate the parking area dividing line image 52 in the right direction on the display 50.

[0083] <Overview of Operation of Vehicle Parking Assist Device> Next, the operation of the vehicle parking assist device 10 will be described in general. The vehicle parking assist device 10 is configured to be able to execute parking assist control. The parking assist control is a control for parking the vehicle 100 in the parking space 61 without the driver having to operate the accelerator pedal 14, the brake pedal 15, and the steering wheel 16.

[0084] There are parking lots that are divided into parking areas by white lines or the like (hereinafter referred to as "dividing lines"). When a vehicle is automatically parked in such a parking lot, the vehicle can be automatically driven and parked in the parking area using the dividing lines captured by the camera as markers.

[0085] On the other hand, there are parking lots such as parking lots of private houses that do not have demarcation lines for demarcating parking areas. When a vehicle is automatically parked in such a parking lot, there are no demarcation lines as a mark. The parking assistance control performed by the vehicle parking assistance device 10 is a control including control for automatically parking the vehicle in the parking lot and registering parking lot information about the parking lot, and control for automatically parking the vehicle in the parking lot where the parking lot information is registered.

[0086] The vehicle parking assistance device 10 performs a search process when the vehicle speed SPD is below the prescribed vehicle speed SPDth, and the search process is to search for an image portion having a density pattern that is respectively consistent with the density patterns of a plurality of registered entry feature points Fent_reg in the captured image CMR. The vehicle parking assistance device 10 uses the left image information IMG3 and the right image information IMG4 to perform the search process. The registered entry feature point Fent_reg is an entry feature point Fent whose registered entry density information CTent_reg has been registered (i.e., stored) in the vehicle parking assistance device 10 through parking assistance control, which will be described in detail later. The entry feature point Fent is a feature point F of the entrance 62ent of the parking lot 62 obtained through parking assistance control. In addition, the registered entry density information CTent_reg is the density information CT of the registered entry feature point Fent_reg. In addition, the captured image CMR is an image obtained by being captured by the camera 45.

[0087] When the vehicle parking assistance device 10 extracts image parts having light and dark patterns that are respectively consistent with the light and dark patterns of multiple registered entry feature points Fent_reg from the captured image CMR, it compares (or matches) the positional relationship between the multiple extracted image parts and the positional relationship between the multiple registered entry feature points Fent_reg having light and dark patterns that are consistent with the light and dark patterns of the multiple image parts.

[0088] When the positional relationship between the extracted multiple image parts and the positional relationship between the multiple registered entry feature points Fent_reg having the same light and dark patterns as the multiple image parts are consistent, the vehicle parking assistance device 10 determines that there is a registered parking lot 62 near the vehicle 100. The registered parking lot 62 is a parking lot whose parking lot information Ipark has been registered (i.e., stored) in the vehicle parking assistance device 10 through parking assistance control.

[0089] More specifically, when the positional relationship between multiple image parts extracted based on the left image information IMG3 and the positional relationship between multiple registered entry feature points Fent_reg having a light and dark pattern consistent with the light and dark pattern of these multiple image parts are consistent, the vehicle parking assistance device 10 determines that there is a registered parking lot 62 on the left side of the vehicle 100.

[0090] On the other hand, when the positional relationship between multiple image parts extracted based on the right image information IMG4 and the positional relationship between multiple registered entry feature points Fent_reg having a light and dark pattern consistent with the light and dark patterns of these multiple image parts are consistent, the vehicle parking assistance device 10 determines that there is a registered parking lot 62 on the right side of the vehicle 100.

[0091] Furthermore, when the vehicle parking assistance device 10 does not extract an image portion having a density pattern that matches the density patterns of the plurality of registered entry feature points Fent_reg from the captured image CMR, it is determined that there is no registered parking lot 62 near the vehicle 100. Alternatively, when the positional relationship between the plurality of extracted image portions and the positional relationship between the plurality of registered entry feature points Fent_reg having a density pattern that matches the density patterns of the plurality of image portions are inconsistent, the vehicle parking assistance device 10 also determines that there is no registered parking lot 62 near the vehicle 100.

[0092] <Parking lot registration> After it is determined that the vehicle 100 stops near the entrance 62ent of the parking lot 62 and the parking assist switch image 51M is touched, the registration start button image 54 is touched to start the registration of the parking lot information Ipark, and the vehicle parking assist device 10 obtains the temporary entrance information Ient_pre and the temporary intermediate information Imid_pre as follows. In addition, the vehicle parking assist device 10 registers (i.e., stores) the registered entrance information Ient_reg, the registered in-field information Iin_reg, and the registered area information Iarea_reg as the parking lot information Ipark as described later.

[0093] If it is determined that the vehicle 100 is stopped near the entrance 62ent of the parking lot 62 and it is determined that the parking lot 62 is not a registered parking lot, and if Fig. 9 When the driver touches and operates the parking assist switch image 51M displayed on the display 50 as shown in (A) of FIG. 5 , the vehicle parking assist device 10 clears the parking assist switch image 51M from the display 50. Fig. 9 As shown in FIG. 5B , the captured image 51C, the overhead image 51P, and the registered main switch image 52M are displayed on the display 50 .

[0094] When the driver touches and operates the registered main switch image 52M, the vehicle parking assist device 10 clears the captured image 51C, the overhead image 51P, and the registered main switch image 52M from the display 50. Fig. 9 As shown in (C) , the parking area selection image 52S is displayed on the display 50 .

[0095] When the driver touches and operates any of the parking area selection images 52S displayed on the display 50 or displayed on the display 50 without lowering the tone, the vehicle parking assistance device 10 clears the parking area selection image 52S from the display 50. Fig. 9 As shown in (D), the overhead image 51P, the parking area line image 52, the setting button image 53, the moving button image 57, and the angle adjustment button image 58 are displayed on the display 50. At this time, when there is an unregistered parking lot 62 on the left side of the vehicle 100, the vehicle parking assistance device 10 displays the overhead image 51P on the display 50 in a manner that displays the parking lot image on the left side of the vehicle image, and when there is an unregistered parking lot 62 on the right side of the vehicle 100, the overhead image 51P is displayed on the display 50 in a manner that displays the parking lot image on the right side of the vehicle image.

[0096] In addition, the vehicle parking assistance device 10 sets the range in which the vehicle 100 can be parked in the parking lot 62 as the parking area 61 based on the image information IMG and the sonar information SON, and displays the parking area dividing line image 52 as an image representing the set parking area 61 on the display 50. The vehicle parking assistance device 10 uses the sonar information SON, for example, to obtain the width of the entrance 62ent of the parking lot 62.

[0097] The driver can move the parking area dividing line image 52 on the display 50 by touching the move button image 57 until the setting button image 53 is touched. The driver can change the position of the parking area dividing line image 61 where the vehicle 100 is to be parked by moving the parking area dividing line image 52 on the display 50. Furthermore, the driver can rotate the parking area dividing line image 52 on the display 50 by touching the angle adjustment button image 58 until the setting button image 53 is touched. The driver can change the position of the parking area dividing line image 61 where the vehicle 100 is to be parked by rotating the parking area dividing line image 52 on the display 50.

[0098] When the driver touches the setting button image 53, the vehicle parking assist device 10 clears the setting button image 53, the moving button image 57, and the angle adjustment button image 58 from the display 50, and displays the setting button image 53, the moving button image 57, and the angle adjustment button image 58 as shown in FIG. Fig.10 As shown in FIG. 5(A) , a captured image 51C, a bird's-eye view image 51P, and a registration start button image 54 are displayed.

[0099] When the driver touches and operates the setting button image 53 , the vehicle parking assistance device 10 sets the parking area 61 at a position corresponding to the parking area dividing line image 52 displayed on the display 50 as the registration target parking area 61 set.

[0100] When the driver touches and operates the setting button image 53, the vehicle parking assistance device 10 sets the target travel route Rtgt along which the vehicle 100 should travel in order to park the vehicle 100 in the registered parking area 61set. Fig.11 As shown in FIG. 1 , when the vehicle 100 stops on the right side of the unregistered parking lot 62, the vehicle parking assist device 10 Fig.12 The target travel route Rtgt is set as shown.

[0101] Furthermore, when the driver touches and operates the setting button image 53 when the vehicle 100 stops at the right side of the parking lot 62, the vehicle parking assistance device 10 acquires one or more predetermined number of newly added left feature points F3new as entry feature points Fent for each of the middle division range 73D5, the front division range 73D1, and the rear division range 73D2 of the left side range 73. On the other hand, when the driver touches and operates the setting button image 53 when the vehicle 100 stops at the left side of the parking lot 62, the vehicle parking assistance device 10 acquires one or more predetermined number of newly added right feature points F4new as entry feature points Fent for each of the middle division range 74D5, the front division range 74D1, and the rear division range 74D2 of the right side range 74.

[0102] In this example, when the driver touches and operates the setting button image 53 when the vehicle 100 stops on the right side of the parking lot 62, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent for each intermediate segmentation range 73D5 than for the front segmentation range 73D1 and the rear segmentation range 73D2. That is, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent for the range 73D5 that is close to the center of the entrance 62ent of the parking lot 62 among the plurality of ranges 73D5, 73D1, and 73D2 than for the ranges 73D1 and 73D2 that are far from the center of the entrance 62ent of the parking lot 62 among the plurality of ranges.

[0103] On the other hand, when the driver touches and operates the setting button image 53 when the vehicle 100 stops on the left side of the parking lot 62, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent for each intermediate segmentation range 74D5 than for the front segmentation range 74D1 and the rear segmentation range 74D2. That is, the vehicle parking assistance device 10 obtains a larger number of entrance feature points Fent for the range 74D5 that is close to the center of the entrance 62ent of the parking lot 62 among the plurality of ranges 74D5, 74D1, and 74D2 than for the ranges 74D1 and 74D2 that are far from the center of the entrance 62ent of the parking lot 62 among the plurality of ranges.

[0104] For example, in the vehicle 100 Fig.11 When the vehicle stops at the right side of the parking lot 62 as shown in FIG. Fig.13 and Fig.14 As shown, two newly added left feature points F3new are obtained for each of the four middle segmented ranges 73D5 of the left range 73 as the entry feature point Fent, one newly added left feature point F3new is obtained for each of the two front segmented ranges 73D1 as the entry feature point Fent, and one newly added left feature point F3new is obtained for each of the two rear segmented ranges 73D2 as the entry feature point Fent. In addition, when the vehicle 100 stops at the left side of the parking lot 62, the vehicle parking assistance device 10 obtains two newly added right feature points F4new as the entry feature point Fent for each of the four middle segmented ranges 74D5 of the right range 74, one newly added right feature point F4new is obtained for each of the two front segmented ranges 74D1 as the entry feature point Fent, and one newly added right feature point F4new is obtained for each of the two rear segmented ranges 74D2 as the entry feature point Fent.

[0105] Furthermore, the vehicle parking assisting device 10 may be configured such that, when the driver stops the vehicle 100 on the right side of the entrance 62ent of the parking lot 62, if the vehicle 100 is often stopped at a position slightly short of the right side of the entrance 62ent of the parking lot 62, a larger number of entrance feature points Fent are obtained for each front segmentation range 73D1 and the two adjacent middle segmentation ranges 73D5 than for each rear segmentation range 73D2 and the two adjacent middle segmentation ranges 73D5. Similarly, the vehicle parking assisting device 10 may be configured such that, when the driver stops the vehicle 100 on the left side of the entrance 62ent of the parking lot 62, if the vehicle 100 is often stopped at a position slightly short of the right side of the entrance 62ent of the parking lot 62, a larger number of entrance feature points Fent are obtained for the front segmentation range 74D1 and the two adjacent middle segmentation ranges 74D5 than for the rear segmentation range 74D2 and the two adjacent middle segmentation ranges 74D5.

[0106] In addition, when the vehicle parking assistance device 10 cannot obtain the prescribed number of newly added left feature points F3new in the range of the middle division range 73D5, the front division range 73D1, and a part of the rear division range 73D2 of the left range 73, the vehicle parking assistance device 10 obtains the same number of newly added left feature points F3new as the number of newly added left feature points F3new that cannot be obtained in the remaining ranges 73D5, 73D1, and 73D2 as the entry feature points Fent. Similarly, when the vehicle parking assistance device 10 cannot obtain the prescribed number of newly added right feature points F4new in the range of the middle division range 74D5, the front division range 74D1, and a part of the rear division range 74D2 of the right range 74, the vehicle parking assistance device 10 obtains the same number of newly added right feature points F4new as the number of newly added right feature points F4new that cannot be obtained in the remaining ranges 74D5, 74D1, and 74D2 as the entry feature points Fent.

[0107] After obtaining the entrance feature point Fent, the vehicle parking assistance device 10 obtains and stores the coordinates XY of the obtained entrance feature point Fent in the temporary coordinate system Cpre as temporary entrance coordinates XYent_pre, and obtains and stores the density information CT of the obtained entrance feature point Fent as temporary entrance density information CTent_pre. The temporary coordinate system Cpre is a coordinate system with a specified position Ppre in the registered object parking area 61set as the origin. Therefore, the temporary entrance coordinate XYent_pre is the position of the entrance feature point Fent based on the specified position Ppre. In addition, the temporary entrance information Ient_pre includes the temporary entrance coordinates XYent_pre and the temporary entrance density information CTent_pre.

[0108] When the driver touches and operates the registration start button image 54, the vehicle parking assist device 10 clears the registration start button image 54 from the display 50. Fig.10 As shown in (B), the captured image 51C and the overhead image 51P continue to be displayed on the display 50. At this time, when there is an unregistered parking lot 62 on the left side of the vehicle 100, the vehicle parking assistance device 10 displays the image including the unregistered parking lot 62 as the captured image 51C on the display 50, and displays the overhead image 51P on the display 50 in a manner that the parking lot image is displayed on the left side of the vehicle image. On the other hand, when there is an unregistered parking lot 62 on the right side of the vehicle 100, the vehicle parking assistance device 10 displays the image including the unregistered parking lot 62 as the captured image 51C on the display 50, and displays the overhead image 51P on the display 50 in a manner that the parking lot image is displayed on the right side of the vehicle image.

[0109] When the driver touches and operates the registration start button image 54, the vehicle parking assist device 10 implements the first parking driving process for driving the vehicle 100 along the target driving route Rtgt to the registration target parking area 61set. The first parking driving process is a process for controlling the operation of the vehicle driving force generating device 11, the operation of the braking device 12, and the operation of the steering device 13 based on the "image information IMG, object information OBJ, steering angle θst, steering torque TQst, vehicle speed SPD, vehicle yaw rate YR, vehicle longitudinal acceleration Gx, and vehicle lateral acceleration Gy" so that the vehicle 100 drives along the target driving route Rtgt.

[0110] For example, in the vehicle 100 Fig.11 When the vehicle stops on the right side of the unregistered parking lot 62 as shown in the figure, the vehicle parking assistance device 10 starts the first parking driving process, firstly as shown in the figure. Fig.15 As shown in FIG. 1 , the vehicle 100 is made to move forward while turning right and then stop. Fig.16 As shown, the vehicle parking assist device 10 causes the vehicle 100 to back up while turning left.

[0111] The vehicle parking assistance device 10 acquires the feature point F at least once during the period until the parking of the vehicle 100 in the parking lot 62 is completed, and acquires the coordinates XY of the feature point F in the temporary coordinate system Cpre and the density information CT of the feature point F. In particular, after the vehicle 100 starts traveling in order to park the vehicle 100 in the parking lot 62, when it is predicted that the vehicle 100 will travel straight without turning thereafter, the vehicle parking assistance device 10 acquires the feature point F at least once, and acquires the coordinates XY of the feature point F in the temporary coordinate system Cpre and the density information CT of the feature point F.

[0112] In this example, while the vehicle 100 is being moved backward, at a time point when it is predicted that the vehicle 100 will not turn but will travel straight thereafter (see Fig.17), the vehicle parking assistance device 10 acquires the rear feature point F2 as the newly added rear feature point F2new. In addition, the vehicle parking assistance device 10 may acquire the rear feature point F2 at a time point when the vehicle 100 is predicted to travel straight without turning thereafter and the vehicle 100 is retreated a predetermined distance Dtravel_th during the period of retreating the vehicle 100. Alternatively, the vehicle parking assistance device 10 may acquire the rear feature point F2 at a time point when the vehicle 100 is predicted to travel straight without turning thereafter and the vehicle 100 is retreated a predetermined distance Dtravel_th thereafter during the period of retreating the vehicle 100. In addition, the vehicle parking assistance device 10 may acquire the rear feature point F2 at a time point when the vehicle 100 is predicted to travel straight without turning thereafter and each time the vehicle 100 is retreated a predetermined distance Dtravel_th thereafter during the period of retreating the vehicle 100. Furthermore, the vehicle parking assistance device 10 may acquire the rear feature point F2 each time the vehicle 100 moves backward a predetermined distance Dtravel_th after it is predicted that the vehicle 100 will move straight without turning thereafter, while the vehicle 100 is moving backward. Furthermore, the vehicle parking assistance device 10 may acquire at least one of the front feature point F1, the left feature point F3, and the right feature point F4 in addition to the rear feature point F2.

[0113] In this example, the predetermined distance Dtravel_th is set to a distance at which the rear range 72 when the rear feature point F2 is acquired most recently does not overlap with the rear range 72 when the rear feature point F2 is acquired this time.

[0114] Furthermore, the vehicle parking assistance device 10 obtains at least one newly added rear feature point F2new existing in each rear divided range 72D as the intermediate feature point Fmid. The vehicle parking assistance device 10 obtains and stores the coordinates XY of the intermediate feature point Fmid in the temporary coordinate system Cpre as temporary intermediate coordinates XYmid_pre, and obtains and stores the density information CT of the intermediate feature point Fmid as temporary intermediate density information CTmid_pre. The temporary intermediate coordinates XYmid_pre are the positions of the intermediate feature points Fmid based on the predetermined position Ppre. The temporary intermediate information Imid_pre includes the temporary intermediate coordinates XYmid_pre and the temporary intermediate density information CTmid_pre.

[0115] The vehicle parking assistance device 10 performs a safety determination process during the execution of the first parking driving process, wherein the safety determination process determines whether the vehicle 100 can be safely driven to the registered target parking area 61set without contacting a solid object existing in the parking lot 62 when the vehicle 100 is driven along the target driving route Rtgt. If the vehicle 100 is determined to be unable to be safely driven to the registered target parking area 61set, the vehicle parking assistance device 10 corrects the target driving route Rtgt so that the vehicle 100 can be safely driven to the registered target parking area 61set. The vehicle parking assistance device 10 performs the safety determination process using the image information IMG and the object information OBJ acquired during the execution of the first parking driving process.

[0116] In addition, the vehicle parking assistance device 10 performs a route determination process during the execution of the first parking driving process, and the route determination process is to determine whether the vehicle 100 can be parked in the registered parking area 61set when the vehicle 100 is driven along the target driving route Rtgt. If the vehicle 100 is determined to be unable to be parked in the registered parking area 61set, the vehicle parking assistance device 10 corrects the target driving route Rtgt so that the vehicle 100 can be parked in the registered parking area 61set. The vehicle parking assistance device 10 performs the route determination process using the image information IMG (particularly the feature point F) acquired during the execution of the first parking driving process.

[0117] When the vehicle 100 as a whole enters the registered parking area 61set (see Fig.18 ), the vehicle 100 stops and the first parking driving process is terminated. Thus, the parking of the vehicle 100 in the parking lot 62 is completed. At this time, the vehicle parking assistance device 10 obtains the front feature point F1, the left feature point F3, and the right feature point F4 as the newly added front feature point F1new, the newly added left feature point F3new, and the newly added right feature point F4new. At this time, the vehicle parking assistance device 10 may also obtain the rear feature point F2 as the newly added rear feature point F2new.

[0118] Then, the vehicle parking assistance device 10 obtains at least one newly added front feature point F1new existing in each front segmentation range 71D as the final feature point Ffin, obtains at least one newly added left feature point F3new existing in each left segmentation range 73D as the final feature point Ffin, and obtains at least one newly added right feature point F4new existing in each right segmentation range 74D as the final feature point Ffin. At this time, when the vehicle parking assistance device 10 obtains the newly added rear feature point F2new, it may also obtain at least one newly added rear feature point F2new existing in each rear segmentation range 72D as the final feature point Ffin.

[0119] <Registration of parking lot information> When the parking of the vehicle 100 in the parking lot 62 is completed, the vehicle parking assisting device 10 clears the captured image 51C from the display 50. Fig.10 As shown in FIG. 5C , a bird's-eye view image 51P, a registration button image 55 , a move button image 57 , and an angle adjustment button image 58 are displayed on the display 50 .

[0120] The driver can move the parking area line image 52 on the display 50 by touching the move button image 57 until the registration button image 55 is touched. The driver can change the position of the parking area line image 61 where the vehicle 100 is to be parked by moving the parking area line image 52 on the display 50. Furthermore, the driver can rotate the parking area line image 52 on the display 50 by touching the angle adjustment button image 58 until the registration button image 55 is touched. The driver can change the position of the parking area line image 61 where the vehicle 100 is to be parked by rotating the parking area line image 52 on the display 50.

[0121] When the driver touches and operates the registration button image 55, the vehicle parking assistance device 10 obtains and registers (i.e. stores) the coordinates XY of the final feature point Ffin in the registration coordinate system Creg as the registration field coordinates XYin_reg, and obtains and registers (i.e. stores) the density information CT of the final feature point Ffin as the registration field density information CTin_reg. The registration coordinate system Creg is a coordinate system having as its origin a predetermined position Preg in the vehicle width direction Dy of the axles connecting the left rear wheel and the right rear wheel of the vehicle 100 when the vehicle 100 is parked in the registration target parking area 61set (see Fig.18 ). Therefore, the registration field coordinate XYin_reg is the position of the final feature point Ffin based on the prescribed position Preg.

[0122] In addition, the vehicle parking assistance device 10 converts the temporary intermediate coordinates XYmid_pre into the coordinates XY in the registration coordinate system Creg, obtains and registers (i.e., stores) them as the registration field coordinates XYin_reg, and registers (i.e., stores) the temporary intermediate shading information CTmid_pre as the registration field shading information CTin_reg. Therefore, the registration field coordinates XYin_reg are the positions of the intermediate feature points Fmid based on the predetermined position Preg.

[0123] The registration field information Iin_reg includes the registration field coordinates XYin_reg and the registration field shading information CTin_reg.

[0124] The vehicle parking assist device 10 registers (ie, stores) the coordinates XY of the registration target parking area 61set in the registration coordinate system Creg as the registration area coordinates XYarea_reg. The registration area coordinates XYarea_reg are the positions of the parking area 61 with reference to the predetermined position Preg. The registration area information Iarea_reg includes the registration area coordinates XYarea_reg.

[0125] In addition, the vehicle parking assistance device 10 converts the temporary entrance coordinates XYent_pre into the coordinates XY in the registration coordinate system Creg, obtains and registers (i.e., stores) as the registered entrance coordinates XYent_reg, and registers (i.e., stores) the temporary entrance density information CTent_pre as the registered entrance density information CTent_reg. Therefore, the registered entrance coordinates XYent_reg are the positions of the entrance feature points Fent based on the specified position Preg. The registered entrance information Ient_reg includes the registered entrance coordinates XYent_reg and the registered entrance density information CTent_reg.

[0126] As described above, the parking lot information Ipark includes the registered entrance information Ient_reg, the registered in-park information Iin_reg, and the registered area information Iarea_reg.

[0127] <Parking at a registered parking lot> When the vehicle parking assist device 10 determines that the vehicle 100 is stopped near the entrance 62ent of the parking lot 62 and determines that the parking lot 62 is a registered parking lot, Fig.10 As shown in (A) of FIG. 1 , the driver touches and operates the parking assist switch image 51M displayed on the display 50, and the vehicle parking assist device 10 clears the parking assist switch image 51M from the display 50. Fig.19As shown in (B) of FIG. 1 , the vehicle parking assisting device 10 displays the captured image 51C, the overhead image 51P, the registered main switch image 52M, and the parking start button image 56 on the display 50. At this time, when there is a registered parking lot 62 on the left side of the vehicle 100, the vehicle parking assisting device 10 displays the image including the registered parking lot 62 as the captured image 51C on the display 50, and displays the overhead image 51P on the display 50 in such a manner that the parking lot image is displayed on the left side of the vehicle image. On the other hand, when there is a registered parking lot 62 on the right side of the vehicle 100, the vehicle parking assisting device 10 displays the image including the registered parking lot 62 as the captured image 51C on the display 50, and displays the overhead image 51P on the display 50 in such a manner that the parking lot image is displayed on the right side of the vehicle image.

[0128] Furthermore, the vehicle parking assisting device 10 specifies the position of the parking block 61 based on the registered block coordinates XYarea_reg included in the parking lot information Ipark related to the registered parking lot 62 where the vehicle 100 is parked this time.

[0129] When the driver touches and operates the parking start button image 56, the vehicle parking assist device 10 clears the registered main switch image 52M and the parking start button image 56 from the display 50. Fig.19 As shown in (C) of FIG. 5 , the captured image 51C and the overhead image 51P continue to be displayed on the display 50 .

[0130] Furthermore, when the driver touches and operates the parking start button image 56 , the vehicle parking assistance device 10 sets the parking area 61 registered in the parking lot information Ipark as the target parking area 61tgt.

[0131] Then, when the driver touches and operates the parking start button image 56 , the vehicle parking assistance device 10 sets a target travel route Rtgt along which the vehicle 100 is to travel in order to park the vehicle 100 in the target parking area 61tgt.

[0132] Then, the vehicle parking assist device 10 implements a second parking driving process to drive the vehicle 100 along the target driving route Rtgt to the target parking area 61tgt. The second parking driving process is a process for controlling the operation of the vehicle driving force generating device 11, the operation of the braking device 12, and the operation of the steering device 13 based on the "image information IMG, object information OBJ, steering angle θst, steering torque TQst, vehicle speed SPD, vehicle yaw rate YR, vehicle longitudinal acceleration Gx, and vehicle lateral acceleration Gy" so that the vehicle 100 drives along the target driving route Rtgt.

[0133] In addition, the vehicle parking assistance device 10 performs a safety determination process during the execution of the second parking driving process, wherein the safety determination process determines whether the vehicle 100 can be safely driven to the target parking area 61tgt without contacting a solid object existing in the parking lot 62 when the vehicle 100 is driven along the target driving route Rtgt. If it is determined that the vehicle 100 cannot be safely driven to the target parking area 61tgt, the vehicle parking assistance device 10 corrects the target driving route Rtgt so that the vehicle 100 can be safely driven to the target parking area 61tgt. The vehicle parking assistance device 10 performs the safety determination process using the image information IMG and the object information OBJ acquired during the execution of the second parking driving process.

[0134] The vehicle parking assistance device 10 performs a search process during the execution of the second parking driving process, wherein the search process searches for an image portion having a shading pattern that matches the shading patterns of the plurality of registered feature points Freg in the captured image CMR. The vehicle parking assistance device 10 performs the search process using the rear image information IMG2, the left image information IMG3, and the right image information IMG4. The registered feature point Freg is a feature point F for which the registration field shading information CTin_reg is registered by the parking assistance control.

[0135] When image portions having light and dark patterns respectively consistent with light and dark patterns of multiple registered feature points Freg are extracted from the captured image CMR, the vehicle parking assistance device 10 compares (or matches) the positional relationship between the multiple extracted image portions and the positional relationship between the multiple registered feature points Freg having light and dark patterns consistent with the light and dark patterns of the multiple image portions.

[0136] The vehicle parking assisting device 10 performs a parking position determination process for determining whether the position of the target parking area 61tgt in the parking lot 62 is consistent with the position indicated by the registration area coordinates XYarea_reg based on the positional relationship between the coordinates XY of the registration feature point Freg having the shading pattern consistent with the shading pattern of the extracted image portion and the registration area coordinates XYarea_reg when the positional relationship between the extracted plurality of image portions and the positional relationship between the plurality of registered feature points Freg having the shading pattern consistent with the shading pattern of the plurality of image portions is consistent. When it is determined that the position of the target parking area 61tgt in the parking lot 62 is inconsistent with the position indicated by the registration area coordinates XYarea_reg, the vehicle parking assisting device 10 corrects the position of the target parking area 61tgt in the parking lot 62 so that the position of the target parking area 61tgt in the parking lot 62 is consistent with the position indicated by the registration area coordinates XYarea_reg, and corrects the target driving route Rtgt so that the vehicle 100 can park in the target parking area 61tgt after the position correction.

[0137] When the entire vehicle 100 enters the target parking area 61tgt, the vehicle parking assistance device 10 stops the vehicle 100 and ends the second parking travel process. Thus, the parking of the vehicle 100 in the parking lot 62 is completed.

[0138] The above is an outline of the operation of the vehicle parking assistance device 10. Thus, information related to the feature points F of the ground 63 in the parking lot 62 and the ground 63 around the parking lot 62 is registered as the registered entrance information ient_reg, rather than the feature points of the three-dimensional objects in the parking lot 62 and the three-dimensional objects around the parking lot 62. Even if the conditions around the vehicle 100 and the parking lot 62 are different when the registered entrance information ient_reg is registered and when the vehicle 100 arrives at the entrance 62ent of the registered parking lot 62 thereafter, the change in the image of the ground 63 in the parking lot 62 and the ground 63 around the parking lot 62 captured by the camera 45 is small. Therefore, even if the conditions around the vehicle 100 and the parking lot 62 are different when the registered entrance information Ient_reg is registered and when the vehicle 100 arrives at the entrance 62ent of the registered parking lot 62 thereafter, when an entrance feature point Fent corresponding to the entrance feature point Fent registered with the registered entrance information Ient_reg is obtained when arriving at the entrance 62ent of the registered parking lot 62, it can be determined that the entrance feature point Fent is the entrance feature point Fent registered with the registered entrance information Ient_reg, and as a result, it can be determined that the parking lot 62 is the registered parking lot 62. Therefore, the vehicle 100 can be automatically parked at the registered parking lot 62.

[0139] <Specific Operation of Vehicle Parking Assist Device> Next, the specific operation of the vehicle parking assistance device 10 will be described. The CPU of the ECU 90 of the vehicle parking assistance device 10 is configured to execute the following operation every time a predetermined time has passed. Fig. 20 The routine shown.

[0140] Therefore, when the predetermined time is reached, the CPU starts the process from step 2000, and the process proceeds to step 2005, where it is determined whether the value of the parking assist switch operation flag X1_auto is "1". When it is determined that the vehicle 100 is stopped near the entrance 62ent of the parking lot 62, and it is determined that the parking lot 62 is not a registered parking lot, and the parking assist switch image 51M is touched, the value of the parking assist switch operation flag X1_auto is set to "1", and is set to "0" when the parking of the vehicle 100 in the parking lot 62 is completed.

[0141] If the CPU determines "yes" in step 2005, the process proceeds to step 2010 to determine whether the value of the first parking driving process flag X1_exe is "0". The value of the first parking driving process flag X1_exe is set to "1" when the first parking driving process starts, and is set to "0" when the first parking driving process ends.

[0142] If the CPU determines "yes" in step 2010, the process proceeds to step 2015, and the captured image 51C, the overhead image 51P, and the registered main switch image 52M are displayed on the display 50 (see FIG. 2 ). Fig. 9 (B)). Next, the CPU advances the processing to step 2020 to determine whether the value of the registered main switch operation flag Xmain is "1". The value of the registered main switch operation flag Xmain is set to "1" when the registered main switch image 52M is touched, and is set to "0" when the parking of the vehicle 100 in the parking lot 62 is completed.

[0143] If the CPU determines "yes" in step 2020, the process proceeds to step 2025, clears the captured image 51C, the bird's-eye view image 51P, and the registered main switch image 52M from the display 50, and displays the parking area selection image 52S on the display 50 (see FIG. 20). Fig. 9 Then, the CPU advances the processing to step 2030 to determine whether the value of the selection completion flag Xselect is "1". The value of the selection completion flag Xselect is set to "1" when any of the parking area selection images 52S is touched, and is set to "0" when the parking of the vehicle 100 in the parking area 62 is completed.

[0144] If the CPU determines "yes" in step 2030, the process proceeds to step 2035 and executes Fig.21 Therefore, after the CPU enters the processing into step 2035, that is, from Fig.21 The process starts at step 2100, and the process proceeds to step 2105, where the parking area selection image 52S is cleared, and the overhead image 51P, the parking area line image 52, the setting button image 53, the moving button image 57, and the angle adjustment button image 58 are displayed on the display 50 (see Fig. 9 (D)).

[0145] Next, the CPU advances the process to step 2110 to determine whether the value of the setting completion flag Xset is “1.” The value of the setting completion flag Xset is set to “1” when the setting button image 53 is touched, and is set to “0” when the first parking travel process is started.

[0146] If the CPU determines "yes" in step 2110, the process proceeds to step 2115, the setting button image 53, the moving button image 57, and the angle adjustment button image 58 are cleared from the display 50, and the shooting image 51C, the top view image 51P, and the registration start button image 54 are displayed on the display 50 (see Fig.10 A). Then, the CPU advances the processing to step 2120, and sets the parking area 61 corresponding to the parking area dividing line image 52 as the registration target parking area 61set. Then, the CPU advances the processing to step 2125, and sets the driving route of the vehicle 100 to the registration target parking area 61set as the target driving route Rtgt. Then, the CPU advances the processing to step 2130, and acquires the temporary entrance information Ient_pre as described above, and stores it in the RAM.

[0147] Next, the CPU advances the process to step 2135 to determine whether the value of the registration start flag Xreg_start is “1.” The value of the registration start flag Xreg_start is set to “1” when the registration start button image 54 is touched, and is set to “0” when the first parking travel process is started.

[0148] If the CPU determines "yes" in step 2135, the process proceeds to step 2140, the registration start button image 54 is cleared from the display 50, and the captured image 51C and the overhead image 51P are displayed on the display 50 (see Fig.10 (B)). Then, the CPU causes the processing to proceed to step 2145, and starts the first parking driving processing of causing the vehicle 100 to drive along the target driving route Rtgt to the registered parking area 61set. Thereafter, the CPU causes the processing to proceed to step 2195. Fig. 20 Step 2095 temporarily ends this routine.

[0149] On the other hand, when the CPU determines "No" in step 2135, the processing proceeds to step 2195. Fig. 20 Step 2095 temporarily ends this routine.

[0150] In addition, when the CPU determines "No" in step 2120, the processing also enters step 2195. Fig. 20 Step 2095 temporarily ends this routine.

[0151] CPU Fig. 20 If the judgment is "No" in step 2030, the processing directly enters step 2095 and temporarily ends this routine.

[0152] Also, when the CPU makes a "No" determination in step 2020, the CPU directly proceeds to step 2095 and temporarily terminates this routine.

[0153] If the CPU determines "No" in step 2010, the CPU proceeds to step 2060 to determine whether the value of the intermediate information acquisition flag Xmid is "1". The value of the intermediate information acquisition flag Xmid is set to "1" when it is predicted that the vehicle 100 will not turn but will travel straight after that while the vehicle 100 is moving backward, and is set to "0" when the process of step 2065 is performed.

[0154] When the CPU makes a “yes” determination at step 2060 , the CPU advances the process to step 2065 , acquires temporary intermediate information Imid_pre as described above, and stores it in the RAM. Next, the CPU advances the process to step 2070 .

[0155] On the other hand, when the CPU makes a “No” determination at step 2060 , the CPU directly proceeds to step 2070 .

[0156] After the CPU advances the processing to step 2070, it continues to execute the first parking driving processing. Next, the CPU advances the processing to step 2075 to determine whether the value of the parking completion flag Xpark_fin is "1". The value of the parking completion flag Xpark_fin is set to "1" when the vehicle 100 as a whole enters the registered parking area 61set, and is set to "0" when the first parking driving processing is completed.

[0157] When the CPU makes a "yes" determination in step 2075, the CPU proceeds to step 2080 to terminate the first parking travel process. Next, the CPU proceeds to step 2095 to temporarily terminate this routine.

[0158] On the other hand, when the CPU makes a "No" determination at step 2075, the CPU directly proceeds to step 2095 and temporarily terminates this routine.

[0159] If the CPU makes a "No" determination in step 2005, the CPU advances the process to step 2090 to terminate the display of the overhead image 51P and other images on the display 50. Next, the CPU advances the process to step 2095 to temporarily terminate the present routine.

[0160] Furthermore, the CPU executes every time a specified period of time has passed. Fig. 22 Therefore, when the specified time is reached, the CPU starts Fig. 22 The process starts from step 2200, and the process proceeds to step 2205, where it is determined whether the value of the information registration request flag Xreg_req is "1". The value of the information registration request flag Xreg_req is set to "1" when the parking of the vehicle 100 in the parking lot 62 is completed by the first parking driving process, and is set to "0" when the parking lot information Ipark is registered in the RAM.

[0161] If the CPU determines "yes" in step 2205, the process proceeds to step 2210, clears the captured image 51C from the display 50, and displays the top view image 51P, the move button image 57, the angle adjustment button image 58, and the registration button image 55 on the display 50 (see FIG. Fig.10 (C)). Next, the CPU advances the process to step 2215 to determine whether the value of the registration confirmation flag Xreg_det is "1". The value of the registration confirmation flag Xreg_det is set to "1" when the registration button image 55 is touched, and is set to "0" when the process of step 2120 is performed.

[0162] When the CPU determines "yes" in step 2215, the CPU advances the processing to step 2220, and registers the registered entrance information Ient_reg, the registered in-field information Iin_reg, and the registered area information Iarea_reg as the parking lot information Ipark in the RAM as described above. Then, the CPU advances the processing to step 2295, and temporarily ends this routine.

[0163] On the other hand, when the CPU makes a "No" determination at step 2215, the CPU directly proceeds to step 2295 and temporarily terminates this routine.

[0164] If the CPU makes a "No" determination at step 2205, the CPU directly proceeds to step 2295 and temporarily terminates the routine.

[0165] Furthermore, the CPU executes every time a specified period of time has passed. Fig.23 Therefore, when the specified time is reached, the CPU starts Fig.23 The process starts at step 2300, and the process proceeds to step 2305, where it is determined whether the value of the parking assist switch operation flag X2_auto is "1". When it is determined that the vehicle 100 is stopped near the parking lot 62, and the parking lot 62 is a registered parking lot, and the parking assist switch image 51M is touched and operated, the value of the parking assist switch operation flag X2_auto is set to "1", and is set to "0" when the vehicle 100 is parked in the parking lot 62.

[0166] If the CPU determines "yes" in step 2305, the process proceeds to step 2310 to determine whether the value of the second parking driving process flag X2_exe is "0". The value of the second parking driving process flag X2_exe is set to "1" when the second parking driving process starts, and is set to "0" when the second parking driving process ends.

[0167] If the CPU determines "yes" in step 2310, the process proceeds to step 2315, the parking assist switch image 51M is cleared from the display 50, and the captured image 51C, the top view image 51P, the registered main switch image 52M, and the parking start button image 56 are displayed on the display 50 (see FIG. 23). Fig.19 (B)).

[0168] Next, the CPU advances the processing to step 2317 to determine whether the value of the registered main switch operation flag Xmain is “0.” The value of the registered main switch operation flag Xmain is set to “1” when the registered main switch image 52M is touched, and is set to “0” when the parking of the vehicle 100 in the parking lot 62 is completed.

[0169] If the CPU determines "yes" in step 2317, the process proceeds to step 2320 to determine whether the value of the parking start flag Xpark_start is "1". The value of the parking start flag Xpark_start is set to "1" when the parking start button image 56 is touched, and is set to "0" when the second parking travel process is started.

[0170] If the CPU determines "yes" in step 2320, the process proceeds to step 2325, and clears the registered main switch image 52M and the parking start button image 56 from the display 50. At this time, the CPU continues to display the captured image 51C and the overhead image 51P on the display 50 (see Fig.19C) Then, the CPU advances the processing to step 2330, and sets the parking area 61 registered as the parking lot information Ipark as the target parking area 61tgt. Next, the CPU advances the processing to step 2335, and sets the driving route for the vehicle 100 to drive to the target parking area 61tgt as the target driving route Rtgt. Next, the CPU advances the processing to step 2340, and starts the second parking driving processing. Thereafter, the CPU advances the processing to step 2395, and temporarily ends this routine.

[0171] On the other hand, when the CPU makes a "No" determination at step 2320, the CPU directly proceeds to step 2395 and temporarily terminates this routine.

[0172] If the CPU determines "No" in step 2317, the CPU proceeds to step 2342 to set the value of the parking assist switch operation flag X1_auto to "1". Then, the CPU proceeds to step 2395 to temporarily terminate this routine. Fig. 20 In step 2005, it is judged as "yes".

[0173] In addition, if the CPU determines "No" in step 2310, the process proceeds to step 2345 to continue the second parking travel process. Thereafter, the CPU proceeds to step 2350 to determine whether the value of the parking completion flag Xpark_fin is "1". The value of the parking completion flag Xpark_fin is set to "1" when the vehicle 100 as a whole enters the target parking area 61tgt, and is set to "0" when the second parking travel process is completed.

[0174] When the CPU makes a "yes" determination in step 2350, the CPU proceeds to step 2355 to terminate the second parking travel process. Next, the CPU proceeds to step 2395 to temporarily terminate this routine.

[0175] On the other hand, when the CPU makes a "No" determination in step 2350, the CPU directly proceeds to step 2395 and temporarily terminates this routine.

[0176] If the CPU makes a "No" determination at step 2305, the CPU advances the process to step 2360 to terminate the display of the overhead image 51P and other images on the display 50. The CPU then advances the process to step 2395 to temporarily terminate the present routine.

[0177] The above is a specific operation of the vehicle parking assistance device 10. Thus, instead of the feature points of the three-dimensional objects in the parking lot 62 and the three-dimensional objects around the parking lot 62, information related to the feature points F of the ground 63 in the parking lot 62 and the ground 63 around the parking lot 62 is registered as the registration entry information lent_reg (refer to Fig. 22 Therefore, even if the conditions around the vehicle 100 and the parking lot 62 are different when the registered entrance information Ient_reg is registered and when the vehicle 100 arrives at the entrance 62ent of the registered parking lot 62, when the entrance feature point Fent corresponding to the entrance feature point Fent registered with the registered entrance information Ient_reg is obtained when arriving at the entrance 62ent of the registered parking lot 62, it can be determined that the entrance feature point Fent is the entrance feature point Fent registered with the registered entrance information Ient_reg, and as a result, it can be determined that the parking lot 62 is the registered parking lot 62. Therefore, the vehicle 100 can be automatically parked at the registered parking lot 62.

[0178] In addition, the present invention is not limited to the above-described embodiment, and various modifications can be adopted within the scope of the present invention. Description of Reference Numerals

[0179] 10… Vehicle parking assistance device; 11…Vehicle driving force generating device; 12… brake device; 13…steering device; 30…Sonar sensor device; 40…Camera device; 51C…capture images; 51P…Top view image; 52…parking area marking image; 61…Parking zoning; 62…Parking lot; 63…ground; 73…left range; 74…Right Range; 90…ECU; 100…Vehicles; F…Feature point.

Claims

1. A vehicle parking assistance device, comprising: a camera mounted on the vehicle in such a manner as to be able to capture the surroundings of the vehicle; a control unit capable of executing parking assist control for automatically parking the vehicle in the parking lot using information related to the parking lot, the information related to the parking lot being acquired as parking lot information from an image captured by the camera, The control unit is configured to: acquiring, as a captured image, an image of the parking lot captured by the camera when the vehicle is parked next to the parking lot, acquiring feature points of the ground surface of the entrance of the parking lot from the captured image as entrance feature points, executing the parking assist control using the parking lot information acquired from the captured image acquired this time, registering the parking lot information, wherein the parking lot information includes information related to the obtained entrance feature point as the entrance feature point information, The control unit is configured to: After the parking lot information is registered, when the vehicle stops next to an undetermined parking lot, acquiring the captured image, The feature points of the ground surface of the entrance of the parking lot that has not been determined are obtained from the captured image obtained this time as unchecked feature points. By comparing the information related to the unchecked feature point with the entrance feature point information, it is determined whether the unchecked parking lot is the parking lot for which the parking lot information is registered, that is, the registered parking lot. When it is determined that the undetermined parking lot is the registered parking lot, the parking assist control is executed using the parking lot information obtained this time and the registered parking lot information.

2. The vehicle parking assistance device according to claim 1, wherein: The control unit is configured to: The captured image is divided into a plurality of predetermined division ranges, At least one of the feature points is obtained from each of the predetermined segmentation ranges. The entry feature points are feature points acquired from the predetermined segmentation ranges.

3. The vehicle parking assistance device according to claim 2, wherein: The specified segmentation ranges are of the same size.

4. The vehicle parking assistance device according to claim 2 or 3, wherein: The prescribed division ranges are set so as not to overlap with each other.

5. The vehicle parking assistance device according to any one of claims 2 to 4, wherein: The control unit is configured to obtain a larger number of the feature points as the entrance feature points from the prescribed division range closer to the center of the entrance of the parking lot than from the prescribed division range farther from the center of the entrance of the parking lot.

6. The vehicle parking assistance device according to any one of claims 2 to 5, wherein: The control unit is configured to obtain a predetermined number of the entry feature points from each of the predetermined segmentation ranges. When there is the prescribed segmentation range in which the prescribed number of the entry feature points cannot be obtained, the entry feature points of a number greater than the prescribed number are obtained from the prescribed segmentation range other than the prescribed segmentation range in which the prescribed number of the entry feature points cannot be obtained.

7. The vehicle parking assistance device according to any one of claims 1 to 6, wherein: The entry feature point information includes information related to a pattern of light and dark of an image of the entry feature point.

8. The vehicle parking assistance device according to any one of claims 1 to 7, wherein: The entrance feature point information includes information related to the position of the entrance feature point with reference to a predetermined position in the parking lot.

9. The vehicle parking assistance device according to any one of claims 1 to 8, wherein: The control unit is configured to, when it is determined that the undetermined parking lot is not the registered parking lot, execute the parking assist control using the parking lot information obtained this time.

10. The vehicle parking assistance device according to any one of claims 1 to 9, wherein: The cameras include a front camera for photographing the front of the vehicle, a rear camera for photographing the rear of the vehicle, a left camera for photographing the left side of the vehicle, and a right camera for photographing the right side of the vehicle.

Citation Information

Patent Citations

  • Automatic drive control device, vehicle and automatic drive control method

    JP2017138664A

  • Parking assistance device, component for parking assistance device, parking assistance method, parking assistance program, method and program for calculating vehicle travel parameter, device for calcu

    CN101573257A

  • Parking assistance device, vehicle-side device for parking assistance device, parking assistance method, and parking assistance program

    CN101765527A

  • Automatic walking suggestion system and method by analyzing ground textures and combining with data records

    CN106996781A

  • Automatic driving control device, vehicle, and automatic driving control method

    CN108617167A