Vehicle parking assistance device

Abstract

Vehicle parking assistance device with a camera (40, 41-44) configured to capture an image representing an area surrounding a vehicle, and a control device (90, 11, 12, 13) which is configured to register information about a parking space as parking information, which contains a parking area in which the vehicle will be parked, based on a registration image which is an image representing the parking space which the camera captures when the control device receives a registration request to register the parking area from a driver of the vehicle, and autonomous parking of the vehicle in the parking area based on the parking information, when it is determined that the vehicle has reached the parking space specified by the parking information after the parking information has been registered, where the control unit is set up for Obtaining a feature image, which is an image of a predetermined area size and has a specific feature, from the registration image (step 2450) to register the feature image as the parking information (step 2545, step 2565, step 2620), and Determine that the vehicle has reached the parking area specified by the parking information (step 2720) when an image segment containing the same feature as the feature image is included in a post-registration image (“Yes” in step 2175) in order to autonomously park the vehicle in the parking area in the parking lot based on the parking information, wherein the post-registration image is an image of an area captured by the camera after the control unit has registered the parking information. and whereby the control unit is further set up to Specifying an object image, which is an image representing an object, from the registration image (step 2440), if the object is present in the area corresponding to the registration image (step 2420), and Procurement of the feature image based on the registration image, from which the specified object image is excluded (step 2540, step 2565, step 2620).

Description

BACKGROUND Technical area

[0001] The present disclosure relates to a vehicle parking assistance device. Related technology

[0002] A vehicle parking assistance device for autonomously parking a vehicle in a parking space that is not defined / partitioned by parking area lines, such as white lines, is known. The parking space not defined / partitioned by parking area lines is, for example, a parking space belonging to a house or apartment building. This type of known vehicle parking assistance device registers / stores parking space information about a parking space not defined / partitioned by parking area lines once the vehicle has finished parking in that space. Since the known vehicle parking assistance device autonomously moves the vehicle to park it in the same parking space where it was previously parked autonomously, the device periodically obtains information about the parking space.The vehicle parking assistance device compares the acquired information with the registered / stored parking information to recognize a positional relationship between the vehicle and the parking space in order to park the vehicle in the parking space.

[0003] Such a vehicle parking assistance device is disclosed, for example, in Japanese patent application JP 2017-138664 A. The disclosed vehicle parking assistance device obtains an image containing an object image of an object located in or around the parking space using a camera, in order to register / store at least one of the object's feature points in the image as the parking information. The image containing the object image is hereinafter referred to as the "camera image".

[0004] US 2019 / 0039605A1 discloses a control device for automated driving that includes a recording unit configured to automatically generate automated driving information used for driving a vehicle based on a first image obtained by capturing the vehicle's surroundings in a driver-drive mode, and a control unit configured to automatically drive the vehicle based on the automated driving information and a second image obtained by capturing the vehicle's surroundings in an automated driving mode.The registration unit comprises an extraction unit configured to extract candidate feature points present in the vehicle's environment based on the first image, and a generation unit configured to select the candidate feature points, determined as structures fixed around a target position of the vehicle, as feature points based on the first images taken during the vehicle's movement.

[0005] DE 10 2011 083745 A1 discloses a method for the monocular motion-stereo-based automatic measurement of available parking spaces from a vehicle, in which a multiple images of the same scene are captured by an imaging system while the vehicle is in motion. The images captured from different vehicle positions are processed taking the vehicle's movement into account in order to determine the shape and dimensions of the parking space. At least two images are read out within an exposure window of an image-capturing element. From these at least two images, one or more areas with a first, measured motion blur of a respective edge are determined. For the determined area(s), a second, expected motion blur of the respective edge, dependent on the vehicle's movement, is calculated.Furthermore, for each area, a measure of the deviation between the first and second motion blur of corresponding edges is determined. Depending on the degree of deviation, inferences are drawn about moving objects, which are then processed in a predefined manner when determining depth information for the parking lot. Summary

[0006] A situation may arise where only a driver is in the vehicle when the parking information is registered / saved, but one or more passengers are also in the vehicle when the vehicle is autonomously parked in the parking space whose parking information was registered / saved. In this case, the vehicle's tilt when the parking information is registered / saved may differ from the vehicle's tilt when the vehicle is autonomously parked in the same parking space using the registered / saved parking information.There may be another case where "the parking space whose parking information is registered / stored" and / or "the surroundings of this parking space" have a slope and / or undulation, and a route when the parking information is registered / stored differs from a route when the vehicle is autonomously parked in the same parking space using the registered / stored parking information. In this case, too, the vehicle's slope when the parking information is registered / stored may differ from the vehicle's slope when the vehicle is autonomously parked in the parking space using the registered / stored parking information.

[0007] In the aforementioned cases, the shape of an object in the camera image, when the parking information (i.e., the feature points) is recorded / stored, may differ from the shape of the same object in the camera image when the vehicle is autonomously parked in the parking space using the recorded / stored parking information. This can make it difficult to determine that the feature points of an object acquired when the vehicle is autonomously parked are the recorded / stored feature points of the same object. In this case, it may not be possible to park the vehicle autonomously in the parking space.

[0008] Meanwhile, there may be another scenario where the parking information is registered in the morning, and the vehicle is parked autonomously in the afternoon. In this case, the way sunlight shines directly onto the object and how it is reflected back onto the object through the ground may differ between the time the parking information is registered / stored and the time the vehicle is parked autonomously. There may also be another scenario where the parking information is registered / stored during the day, and the vehicle is parked autonomously at night. In this case, the way sunlight shines directly onto the object and how it is reflected back onto the object through the ground varies between the time the parking information is registered and the time the vehicle is parked autonomously.In these cases, the object's feature points in the camera image may differ between the time the parking information is registered / saved and the time the vehicle parks autonomously. In this case, it may not be possible to determine that the feature points acquired at the time the vehicle parks autonomously are the registered feature points of the object, even if the acquired feature points are indeed the registered feature points of the object. Therefore, it may not be possible to park the vehicle autonomously in the parking space.

[0009] There may be another scenario where the feature points of a movable object, such as another vehicle, a bicycle, or a planter, are registered / stored as parking information, and the movable object has been moved when the vehicle is parked autonomously. In this case, the registered / stored feature points of the movable object may not be retrieved when the vehicle is parked autonomously. Consequently, the vehicle may not be able to park autonomously. There may also be another scenario where the movable object is not present in / around the parking space when the parking information is registered / stored, and the movable object is present in / around the parking space when the vehicle is parked autonomously. In this case, the feature points of this movable object were not registered / stored. Consequently, the vehicle may not be able to park autonomously.

[0010] There may be another scenario where the camera's position varies between the time the parking information is registered / saved and the time the vehicle autonomously parks in the parking space. In this case, even if the object photographed when the parking information is registered / saved is the same as the object photographed when the vehicle autonomously parks, the object's shape may still vary between these two times. Therefore, it may not be possible to determine that the feature points acquired at the time the vehicle autonomously parks are the registered / saved feature points of the object, even if the acquired feature points are indeed the registered / saved feature points of the object.As a result, it may not be possible to park the vehicle autonomously in the parking space.

[0011] If the vehicle parking assistance device is configured to register / store the object's feature points as the parking information as described above, the vehicle may not be able to park autonomously in the parking space if the conditions surrounding the vehicle and the parking space differ between the time of registration / storage of the feature points and the time of autonomous parking of the vehicle using the stored feature points (parking information).

[0012] The present disclosure serves to overcome the problems described above. The present disclosure has a function for providing a vehicle parking assistance device that can autonomously park the vehicle in the parking space even if the conditions surrounding the vehicle and the parking space change between the time of registration / storage of the feature points and the time of autonomous parking of the vehicle using the stored feature points.

[0013] A vehicle parking assistance device according to the present disclosure comprises: a vehicle parking assistance device with a camera (40, 41-44) designed to take an image depicting the surrounding area of ​​a vehicle, and a control device (90, 11, 12, 13) which is configured to Registering information about a parking space as parking information, which includes a parking area in which the vehicle will be parked, based on a registration image that is an image representing the parking space captured by the camera when the control unit receives a registration request to register the parking area from a driver of the vehicle, and autonomous parking of the vehicle in the parking area based on the parking information, when it is determined that the vehicle has reached the parking space specified by the parking information after the parking information has been registered.

[0014] The control unit is set up for Obtaining a feature image from the registration image, which is an image of a predetermined area size and has a specific feature (step 2450), to register the feature image as the parking information (step 2545, step 2565, step 2620), and Determine that the vehicle has reached the parking area specified by the parking information (step 2720) when an image section containing the same feature as the feature image is included in a post-registration image (“Yes” in step 2715) in order to autonomously park the vehicle in the parking area in the parking space based on the parking information, wherein the post-registration image is an image of an area captured by the camera after the control unit has registered the parking information. and the control unit is further configured to Specifying an object image, which is an image representing an object, from the registration image (step 2440), if the object exists in the area corresponding to the registration image (step 2420), and Creating the feature image based on the registration image from which the specific object image is excluded (step 2540, step 2565, step 2620).

[0015] According to the aforementioned vehicle parking assistance device, the feature image is obtained from a different image than the object image, which is the image representing the object, in order to be registered as parking information. The feature image representing the ground at / around the parking space is registered as the parking information. The object image, which is significantly affected by changes in the situation around the vehicle and the parking space, is masked / excepted. Even if the situation around the vehicle and the parking space changes between the time the parking information is registered and the time after the parking information is registered, the vehicle parking assistance device can reliably determine whether or not the vehicle reaches the parking space specified by the parking information.Even if the situation surrounding the vehicle and the parking space changes between the time the parking information is registered and the time after the parking information is registered, the vehicle can therefore be parked autonomously in the parking area in the parking lot.

[0016] According to one embodiment of the present disclosure The vehicle parking assistance device further comprises a detection sensor (30, 301-312) which is configured to emit a radio carrier for the detection of the object by receiving the radio carrier reflected by the object.

[0017] The control unit is set up for Determine that the object is present in the area corresponding to the registration image (“yes” in step 2420) in a case where the detection sensor detects the object when the control unit receives the registration request, and Specifying the object image from the registration image based on the object detected by the detection sensor (steps 2425-2445).

[0018] According to this design, the vehicle parking assistance device can correctly determine that the object is present.

[0019] According to one interpretation of the present disclosure Is the control unit set up for Obtaining a top-down image, wherein the top-down image is an image when an image captured by the camera is viewed from an observation point positioned above the camera (step 2415), Procuring a virtual line between a furthest point of capture results in the top-down image representing the object captured by the capture sensor and the camera, wherein the furthest point is a point that has the greatest distance to the camera among the capture results (step 2430), Obtaining a first virtual line extending from the furthest point in the same direction as the virtual line, where the direction is a direction away from the camera (step 2430), Procuring a second virtual line extending from a nearest point of the acquisition results in a direction parallel to the central axis of the radiation area of ​​the acquisition sensor, the nearest point being a point that has the shortest distance to the camera (step 2435), and Specifying an image of an area as the object image defined by the acquisition results, the first virtual line, and the second virtual line (step 2440).

[0020] According to this configuration, the image of the area defined by the first and second virtual lines in the top-down image, representing the object detected by the sensor, is specified as the object image. The object height at its furthest point is transformed to extend in the same direction as the virtual line and away from the camera (i.e., along the first virtual line) in the top-down image when the image captured by the camera is converted into the top-down image. The first virtual line is used to define the object image so that the vehicle parking assist system can correctly specify the object image.

[0021] The detection sensor cannot detect another object located behind a reflective surface of the object using distance sonar. The reflective surface is a surface that reflects the radio carrier emitted by the detection sensor. The second virtual line extends from the nearest point in a direction parallel to the central axis of the detection sensor's radiation field. This second virtual line is used to define the object image, allowing the vehicle parking assist system to include an area behind the reflective surface (i.e., an area where something else may be present) in the object image. The image representing the object, which can be easily affected by a change in the situation, is therefore reliably masked / excluded.

[0022] According to one interpretation of the present disclosure The detection sensor and the camera are configured for mounting on the vehicle such that a direction of the central axis of the radiation area of ​​the detection sensor coincides with a direction of a central axis of a recording area of ​​the camera.

[0023] According to this design, the vehicle parking assistance device can reduce the possibility that a line image representing the object at the nearest point protrudes from the second virtual line. The vehicle parking assistance device can therefore reduce the possibility that an area which is part of the image representing the object and which protrudes from the second virtual line is not included in the object image.

[0024] According to one interpretation of the present disclosure Is the control unit set up for Specifying the object image, which is an image representing the object, from the post-registration image (step 2440), if the object is present in an area corresponding to the post-registration image ("Yes" in step 2420), and Determine whether an image different from the specified object image in the post-registration image contains the feature image (step 2710, step 2715).

[0025] According to this design, the object image is masked / excepted in a post-registration image taken by the camera after the parking space information has been registered. The vehicle parking assistance system can therefore determine whether the vehicle reaches the parking space specified by the parking information.

[0026] Components of the present disclosure are not limited to components of embodiments and modified examples of the present disclosure, which are described together with the drawings. Further problems, features and accompanying advantages of the present disclosure can be easily understood with reference to the embodiments and modified examples of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 shows a representation of a vehicle parking assistance device according to an embodiment of the present disclosure and a vehicle in which the vehicle parking assistance device according to the embodiment of the present disclosure is applied. Fig. Figure 2 shows a representation of a sonar sensor device and detection areas of the sonar sensor device. Fig. Figure 3 shows a representation of a camera sensor device and recording areas of the camera sensor device. Fig. Figure 4 shows an example of a parking lot. Fig. Figure 5 shows a representation of a front area and a rear area. Fig. Figure 6 shows a representation of a left area and a right area. Fig. Figure 7 shows a representation of feature points. Fig. Figure 8 shows a representation of a parking area. The Fig. 9A to 9D show illustrations of advertisements. Fig. Figure 10 shows a representation describing the operating methods of the vehicle parking assistance device according to the embodiment of the present disclosure. Fig. Figure 11 shows a representation describing the operating methods of the vehicle parking assistance device according to the embodiment of the present disclosure. Fig. Figure 12 shows a representation describing the operating methods of the vehicle parking assistance device according to the embodiment of the present disclosure. Fig. Figure 13 shows a representation of entry feature points. Fig. Figure 14 shows a representation describing the operating methods of the vehicle parking assistance device according to the embodiment of the present disclosure. Fig. Figure 15 shows a representation describing the operating methods of the vehicle parking assistance device according to the embodiment of the present disclosure. Fig. Figure 16 shows a representation describing the operating methods of the vehicle parking assistance device according to the embodiment of the present disclosure. Fig. Figure 17 shows a representation describing the operating methods of the vehicle parking assistance device according to the embodiment of the present disclosure. The Fig. 18A and Fig. 18B shows illustrations of the advertisements. Fig. Figure 19 shows an illustration of an example of a parking lot where a wall is present. Fig. Figure 20 shows a representation of a parking lot image (left camera image) taken by a left camera. Fig. Figure 21 shows a representation of a top-down image into which the left camera image has been converted. Fig. Figure 22 shows a representation of the recording results from the sonar sensor device. Fig. Figure 23 shows a representation describing a process for specifying an object. Fig. Figure 24 shows a flowchart of a routine executed by a CPU of a system Fig. The ECU shown in section 1 is executed. Fig. Figure 25 shows a flowchart of a routine executed by the CPU. Fig. Figure 26 shows a flowchart of a routine executed by the CPU. Fig. Figure 27 shows a flowchart of a routine executed by the CPU. Fig. Figure 28 shows a flowchart of a routine executed by the CPU. DETAILED DESCRIPTION

[0027] A vehicle parking assistance device according to an embodiment of the present disclosure is described with reference to the drawings. Fig. Figure 1 shows the vehicle parking assistance device 10 according to the embodiment of the present disclosure and a vehicle 100 in which the vehicle parking assistance device 10 is used.

[0028] As in Fig. As shown in Figure 1, the vehicle parking assistance device 10 contains an ECU 90. ECU stands for electronic control unit. The ECU 90 contains a microcomputer as its main component. The microcomputer includes a CPU, ROM, RAM, non-volatile memory, and an interface. The CPU is configured or programmed to perform various functions by executing instructions, programs, or routines stored in the ROM.

[0029] The vehicle 100 has a vehicle propulsion power generation device 11, a braking device 12, and a steering device 13 installed. The vehicle propulsion power generation device 11 generates a driving force for driving the vehicle 100 and applies this driving force to the vehicle's drive wheels. The vehicle propulsion power generation device 11 is, for example, an internal combustion engine and / or an electric motor. The braking device 12 applies braking force to the wheels of the vehicle 100 for braking the vehicle 100. The steering device 13 applies steering torque to the steering wheels for steering the vehicle 100.

[0030] The vehicle propulsion power generation device 11, the brake device 12, and the steering device 13 are electrically connected to the ECU 90. The ECU 90 controls the operation of the vehicle propulsion power generation device 11 to control the propulsion force applied to the drive wheels of the vehicle 100. The ECU 90 controls the operation of the brake device 12 to control the braking force applied to the wheels of the vehicle 100. The ECU 90 controls the operation of the steering device 13 to control the steering torque applied to the steering wheels for steering the vehicle 100. <sensoren>

[0031] The vehicle parking assist device 10 includes an accelerator pedal operation extent sensor 21, a brake pedal operation extent sensor 22, a steering angle sensor 23, a steering torque sensor 24, a vehicle movement speed sensor 25, a yaw rate sensor 26, a longitudinal acceleration sensor 27, a lateral acceleration sensor 28, a sonar sensor device 30, a camera sensor device 40, a parking assist switch 48 and a display 50.

[0032] The accelerator pedal input sensor 21 is electrically connected to the ECU 90. The ECU 90 uses the accelerator pedal input sensor 21 to detect the input AP of an accelerator pedal 14 and obtains this input AP as the accelerator pedal input AP. The ECU 90 controls the operation of the vehicle drive force generation device 11 to apply the drive force to the vehicle's drive wheels 10 according to the obtained accelerator pedal input AP.

[0033] The brake pedal actuation level sensor 22 is electrically connected to the ECU 90. The ECU 90 uses the brake pedal actuation level sensor 22 to detect the degree BP of an actuation applied to a brake pedal 15 and obtains this degree BP as the brake pedal actuation level BP. The ECU 90 controls the operation of the brake device 12 to apply the braking force to the wheels according to the obtained brake pedal actuation level BP.

[0034] The steering angle sensor 23 is electrically connected to the ECU 90. The ECU 90 uses the steering angle sensor 23 to detect an angle θst of a rotation of a steering wheel 16 relative to a center position and obtains the angle θst as the steering angle θst.

[0035] The steering torque sensor 24 is electrically connected to the ECU 90. The ECU 90 detects a torque TQst via the steering torque sensor 24, with which a driver of the vehicle 10 applies a steering shaft 17, and obtains the torque TQst as steering torque TQst.

[0036] The ECU 90 controls the operation of the steering device 13 to apply the steering torque to the steering wheels of the vehicle 10 according to the obtained steering angle θst and the obtained steering torque TQst.

[0037] The vehicle motion speed sensor 25 is electrically connected to the ECU 90. The ECU 90 uses the vehicle motion speed sensor 25 to detect the rotational speeds Vrot of the wheels of the vehicle 100 and obtains the rotational speeds Vrot of the wheels of the vehicle 100. The ECU 90 obtains a motion speed SPD of the vehicle 100 based on the obtained rotational speeds Vrot of the wheels of the vehicle 100.

[0038] The yaw rate sensor 26 is electrically connected to the ECU 90. The ECU 90 detects the yaw rate YR of the vehicle 100 via the yaw rate sensor 26 and obtains the yaw rate YR of the vehicle 100 as the vehicle yaw rate YR.

[0039] The longitudinal acceleration sensor 27 is electrically connected to the ECU 90. The ECU 90 detects a longitudinal acceleration Gx of the vehicle 100 via the longitudinal acceleration sensor 27 and obtains the longitudinal acceleration Gx as the vehicle longitudinal acceleration Gx.

[0040] The lateral acceleration sensor 28 is electrically connected to the ECU 90. The ECU 90 detects a lateral acceleration Gy of the vehicle 100 via the lateral acceleration sensor 28 and obtains the lateral acceleration Gy as the vehicle lateral acceleration Gy.

[0041] The sonar sensor device 30 contains a first distance sonar 301 up to a twelfth distance sonar 312.

[0042] One in Fig. The direction Dx shown in section 2 is a longitudinal direction of vehicle 100. Hereinafter, direction Dx will be referred to as the "vehicle longitudinal direction Dx". A Fig. The direction Dy shown in point 2 is a width direction of the vehicle 100. The direction Dy is referred to below as the "vehicle width direction Dy".

[0043] As in Fig. As shown in Figure 2, the first distance sonar 301 is mounted at the left end of a front section of the vehicle 100 for emitting sonar waves forward and to the left. The first distance sonar 301 is mounted such that a central axis SA1 of the radiation area of ​​the sound waves emitted by the first distance sonar 301 is inclined 45 degrees to the left with respect to the longitudinal direction Dx of the vehicle. The second distance sonar 302 is mounted on the front section of the vehicle 100 between the left end of the front section of the vehicle 100 and a center point of the front section of the vehicle 100 for emitting sound waves straight forward. The second distance sonar 302 is mounted such that a central axis SA2 of the radiation area of ​​the sound waves emitted by the second distance sonar 302 coincides with the longitudinal direction Dx of the vehicle.The third distance sonar 303 is mounted at the right end of the front section of the vehicle 100 for emitting sound waves forward to the right. The third distance sonar 303 is mounted such that a central axis SA3 of the radiation area of ​​the sound waves emitted by the third distance sonar is inclined 45 degrees to the right with respect to the longitudinal direction Dx of the vehicle. The fourth distance sonar 304 is mounted on the front section of the vehicle 100 between the right end of the front section of the vehicle 100 and the center of the front section of the vehicle 100 for emitting sound waves straight forward. The fourth distance sonar 304 is mounted such that a central axis SA4 of the radiation area of ​​the sound waves emitted by the fourth distance sonar 304 coincides with the longitudinal direction Dx of the vehicle.

[0044] The fifth distance sonar 305 is mounted at the left end of a rear section of the vehicle 100 for emitting sound waves to the left rear. The fifth distance sonar 305 is specifically mounted such that a central axis SA5 of a radiation pattern of the sound waves emitted by the fifth distance sonar 305 is inclined 45 degrees to the left with respect to the longitudinal direction Dx of the vehicle. The sixth distance sonar 306 is mounted at the rear section of the vehicle 100 between the left end of the rear section and the center of the rear section for emitting sound waves straight rearward. The sixth distance sonar 306 is specifically mounted such that a central axis SA6 of the radiation pattern of the sound waves emitted by the sixth distance sonar 306 coincides with the longitudinal direction Dx of the vehicle.The seventh distance sonar 307 is mounted at the right end of the rear section of the vehicle 100 for emitting sound waves to the right rear. The seventh distance sonar 307 is mounted such that a central axis SA7 of the radiation pattern of the sound waves emitted by the seventh distance sonar 307 is inclined 45 degrees to the right with respect to the longitudinal direction Dx of the vehicle. The eighth distance sonar 308 is mounted at the rear section of the vehicle 100 between the right end of the rear section and the center of the rear section for emitting sound waves straight rearward. The eighth distance sonar 308 is mounted such that a central axis SA8 of the radiation pattern of the sound waves emitted by the eighth distance sonar 308 coincides with the longitudinal direction Dx of the vehicle.

[0045] The ninth distance sonar 309 is mounted on the front side of a left-hand section of the vehicle 100 for emitting sound waves straight to the left. The ninth distance sonar 309 is specifically mounted such that a central axis SA9 of the radiation area of ​​the sound waves emitted by the ninth distance sonar 309 coincides with the vehicle's width direction Dy. The tenth distance sonar 310 is mounted on the rear side of the left-hand section of the vehicle 100 for emitting sound waves straight to the left. The tenth distance sonar 310 is specifically mounted such that a central axis SA10 of the radiation area of ​​the sound waves emitted by the tenth distance sonar 310 coincides with the vehicle's width direction Dy. The eleventh distance sonar 311 is mounted on the front side of a right-hand section of the vehicle 100 for emitting sound waves straight to the right.The eleventh distance sonar 311 is mounted such that a central axis SA11 of the radiation area of ​​the sound waves emitted by the eleventh distance sonar 311 coincides with the vehicle width direction Dy. The twelfth distance sonar 312 is mounted on a rear side of the right-hand section of the vehicle 100 for emitting sound waves directly to the right. The twelfth distance sonar 312 is mounted such that a central axis SA12 of the radiation area of ​​the sound waves emitted by the twelfth distance sonar 312 coincides with the vehicle width direction Dy.

[0046] The first distance sonar 301 to the twelfth distance sonar 312 each receive the sound waves reflected by a three-dimensional object. The three-dimensional object is referred to below as the "object".

[0047] The sonar sensor device 30 is electrically connected to the ECU 90. The sonar sensor device 30 transmits information about (i) the sound waves emitted by the first distance sonar 301 to the twelfth distance sonar 312 and (ii) the sound waves received by the first distance sonar 301 to the twelfth distance sonar 312. The ECU 90 obtains information about the objects around the vehicle 100 as object information OBJ based on the information transmitted by the sonar sensor device 30. Hereinafter, the information about the objects obtained based on the sonar sensor device 30 (i.e., object information OBJ) is referred to as "sonar information SON".

[0048] The camera sensor device 40 includes a front camera 41, a rear camera 42, a left camera 43, and a right camera 44. The cameras 45 below comprise the front camera 41, the rear camera 42, the left camera 43, and the right camera 44.

[0049] As in Fig. As shown in Figure 3, the front camera 41 is mounted in the center of the front section of the vehicle 100 for capturing images of a view in front of the vehicle 100. The viewing angle 41A of the front camera 41 is approximately 180 degrees. The rear camera 42 is mounted in the center of the rear end section of the vehicle 100 for capturing images of a view behind the vehicle 100. The viewing angle 42A of the rear camera 42 is approximately 180 degrees. It is noted that the recording area of ​​the front camera 41 and the rear camera 42 each extends in the longitudinal direction Dx of the vehicle. The left camera 43 is mounted on the left-side section of the vehicle 100 for capturing images of a view to the left of the vehicle 100. The field of view 43a of the left camera 43 is approximately 180 degrees. The right camera 44 is attached to the right-hand section of vehicle 100 for taking pictures of a view to the right of vehicle 100.The field angle 44A of the right camera 44 is approximately 180 degrees. It is noted that the recording area of ​​both the left camera 43 and the right camera 44 extends in the vehicle width direction Dy.

[0050] The camera sensor device 40 is electrically connected to the ECU 90. The ECU 90 obtains information about the images of the views captured by the cameras 45 of the camera sensor device 40.

[0051] The information about the images of the view captured by the front camera 41 is referred to below as "Front Image Information IMG1". Similarly, the information about the images of the view captured by the rear camera 42 is referred to as "Rear Image Information IMG2". Furthermore, the information about the images of the view captured by the left camera 43 is referred to as "Left Image Information IMG3". The information about the images of the view captured by the right camera 44 is referred to as "Right Image Information IMG4". Furthermore, the Front Image Information IMG1, the Rear Image Information IMG2, the Left Image Information IMG3, and the Right Image Information IMG4 are collectively referred to as "Image Information IMG".

[0052] The vehicle parking assistance device 10 obtains information about one or more feature points F based on the image information IMG when a predetermined low-speed condition is met. The predetermined low-speed condition is a condition that is met when the vehicle's speed SPD is equal to or less than a speed threshold SPDth. The feature point F is a part of the image captured by the cameras 45 in which a luminance in the image changes significantly. The feature point F can be referred to as a "feature image".

[0053] The device 10 generates, in particular, a plane view image or top view image based on the image information IMG and extracts / cuts out the part in which the luminance varies significantly from the top view image as the feature point F (i.e., as the feature image), which is described in more detail below. The feature image thus extracted, which is described below with reference to Fig. 7, which is described in more detail, has a rectangular shape with one side whose length corresponds to a predetermined actual distance Lset.

[0054] If the cameras, for example, 45, capture the images of a person in Fig. By recording parking space 62 shown in section 4, various feature points described below can be obtained. The one in Fig. Parking space 62 shown in Figure 4 has a floor 63 comprising a concrete section 63C, a grass-covered section (grass section) 63L, and concrete slabs (blocks) 63B that close / cover a road gutter at the entrance (62ent) of parking space 62. The floor 63 of the entrance 62ent of parking space 62 is therefore formed by the surfaces of the concrete slabs 63B.

[0055] Based on the images of the in Fig. The following feature points F are procured for parking space 62 shown in section 4. • Images corresponding to the four corners of each of the concrete slabs 63B, • Images corresponding to the projected corners of the lawn 63L, and • Images corresponding to a section where the concrete slab 63B and the lawn 63L meet.

[0056] The vehicle parking assistance device 10 procures the feature points (feature images) F in a predetermined area (a front area) 71 of the floor 63, which is located at the front of the vehicle 100, based on the front image information IMG1 (compare Fig. 5) Hereinafter, the feature points F in the predetermined area 71 are referred to as “front feature points F1”. The vehicle parking assistance device 10 also procures the feature points (feature images F) in a predetermined area (a rear area) 72 of the floor 63, which is located on the rear side of the vehicle 100, based on the rear image information IMG2 (compare Fig. 5) The feature points F in the predetermined area 72 are hereinafter referred to as “reverse feature points 2”. The vehicle parking assistance device 10 obtains the feature points (feature images F) in a predetermined area (a left-side area) 73 of the floor 63, which is located on the left side of the vehicle 100, based on the left-side image information IMG3 (compare Fig. 6) The feature points F in the predetermined area 73 are hereinafter referred to as “left feature points F3”. The vehicle parking assistance device 10 also procures the feature points (feature points F) in a predetermined area (a right-side area) 74 of the floor 63, which is located on the right side of the vehicle 100, based on the right image information IMG4 (compare Fig. 6) The feature points F in the predetermined area 74 are hereinafter referred to as “legal feature points F4”.

[0057] As described above, to obtain the feature points, the vehicle parking assistance device 10 converts the image captured by each of the cameras 45 into a plane view image or top view image as described below, in order to specify an object image as described below within the top view image, and obtains the feature points F from an image within the top view image that is different from the object image Pobj. The top view image is an image viewed from a virtual observation point (bird's-eye view) positioned directly above each of the respective cameras 45. The vehicle parking assistance device 10 specifies "a portion of the image that is likely to represent an object" as "object image Pobj".

[0058] As in Fig. As shown in Figure 5, the predetermined area 71 is a region defined (surrounded) by a line L711, a line L712, a line L713, and a line L714. Line L711 extends parallel to the vehicle width direction Dy, passing through a point located a predetermined distance Dset forward from the front camera 41. Line L712 extends parallel to the vehicle width direction Dy, passing through the front camera 41. Line L713 extends parallel to the vehicle length direction Dx, passing through a point located a predetermined distance Dset to the left of the front camera 41. Line L714 extends parallel to the vehicle length direction Dx, passing through a point located a predetermined distance Dset to the right of the front camera 41. The predetermined area 71 is referred to below as the "front area 71".

[0059] The front section 71 is divided into four sections in the vehicle width direction Dy and into two sections in the vehicle length direction Dx. The front section 71 therefore contains eight sections 71D. That is, the front section 71 is divided into eight sections 71D of the same size (and shape). Each of the sections 71D is referred to below as a "front section section 71D". Each of the two front section sections 71D located at the left end of the front section 71 in the vehicle width direction Dy is referred to below as "left section section 71D3". Furthermore, each of the two front section sections 71D located at the right end of the front section 71 in the vehicle width direction Dy is referred to as "right section section 71D4". Each of the four front section sections 71D located in the middle of the front section 71 in the vehicle width direction Dy is referred to below as "middle section section 71D5".

[0060] As in Fig. As shown in Figure 5, the predetermined area 72 is a region defined (surrounded) by a line L721, a line L722, a line L723, and a line L724. Line L721 extends parallel to the vehicle width direction Dy and passes through the rear camera 42. Line L722 extends parallel to the vehicle width direction Dy and passes through a point located a predetermined distance Dset rearward from the rear camera 42. Line L723 extends parallel to the vehicle length direction Dx and passes through a point a predetermined distance Dset to the left of the rear camera 42. Line L724 extends parallel to the vehicle length direction Dx and passes through a point a predetermined distance Dset to the right of the rear camera 42. The predetermined area 72 is referred to below as "Rear Area 72".

[0061] The rear section 72 is subdivided into four sections in the vehicle width direction Dy and into two sections in the vehicle length direction Dx. The rear section 72 therefore contains eight sections 72D. That is, the rear section 72 is subdivided into eight sections 72D of the same size (and shape). Hereinafter, each of the sections 72D will be referred to as a "rear section 72D". Furthermore, each of the two rear section sections 72D located at the left end of the rear section 72 in the vehicle width direction Dy will be referred to as "left section 72D3". Furthermore, each of the two rear section sections 72D located at the right end of the rear section 72 in the vehicle width direction Dy will be referred to as "right section 72D4". Furthermore, each of the four rear section sections 72D located in the middle of the rear section 72 in the vehicle width direction Dy will be referred to as "middle section 72D5".

[0062] As in Fig. As shown in Figure 6, the predetermined area 73 is a region defined (surrounded) by a line L731, a line L732, a line L733, and a line L734. Line L731 extends parallel to the vehicle width direction Dy, passing through a point located forward of the left camera 43 by the predetermined distance Dset. Line L732 extends parallel to the vehicle width direction Dy, passing through a point located rearward of the left camera 43 by the predetermined distance Dset. Line L733 extends parallel to the vehicle length direction Dx, passing through a point located to the left of the left camera 43 by the predetermined distance Dset. Line L734 extends parallel to the vehicle length direction Dx, passing through the left camera 43. The predetermined area 73 is referred to below as "left area 73".

[0063] The left section 73 is subdivided into four sections in the longitudinal direction Dx and into two sections in the transverse direction Dy. The left section 73 is therefore subdivided into eight sections 73D. That is, the left section 73 is subdivided into eight sections 73D of the same size (and shape). Hereinafter, each of the sections 73D will be referred to as a "left section 73D". Furthermore, each of the two left sections 73D located at the front end of the left section 73 in the longitudinal direction Dx will be referred to as "front section 73D1". Furthermore, each of the two left sections 73D located at the rear end of the left section 73 in the longitudinal direction Dx will be referred to as "rear section 73D2". Furthermore, each of the four left sub-areas 73D that is located in the middle of the left area 73 in the vehicle longitudinal direction Dx is referred to as the "middle sub-area 73D5".

[0064] As in Fig. As shown in Figure 6, the predetermined area 74 is a region defined (surrounded) by a line L741, a line L742, a line L743, and a line L744. Line L741 extends parallel to a vehicle width direction Dy, passing through a point located forward from the right camera 44 by the predetermined distance Dset. Line L742 extends parallel to the vehicle width direction Dy, passing through a point located rearward from the right camera 44 by the predetermined distance Dset. Line L743 extends parallel to the vehicle length direction Dx, passing through the right camera 44. Line L744 extends parallel to the vehicle length direction Dx, passing through a point located to the right of the right camera 44 by the predetermined distance Dset. The predetermined area 74 is referred to below as "right area 74".

[0065] The right section 74 is subdivided into four sections in the longitudinal direction Dx and into two sections in the transverse direction Dy. The right section 74 therefore contains eight sections 74D. That is, the right section 74 is subdivided into eight sections 74D of the same size (and shape). Each of the sections 74D is referred to below as a "right section 74D". Furthermore, each of the two right sections 74D located at the front end of the right section 74 in the longitudinal direction Dx is referred to as "front section 74D1". Furthermore, each of the two right sections 74D located at the rear end of the right section 74 in the longitudinal direction Dx is referred to as "rear section 74D2". Furthermore, each of the four right-hand sub-areas 74D that is located in the middle of the right-hand area 74 in the longitudinal direction of the vehicle Dx is referred to as the "middle sub-area 74D5".

[0066] As described above, each of the feature points F (that is, each of the feature images) corresponds to the one in Fig. Figure 7 shows a rectangular area 75. The length of each side is the predetermined length Lset. When a predetermined condition is met, the vehicle parking assist device 10 divides each of the feature points F into twenty-five rectangular areas 75D, which are equal to each other, and obtains luminance values ​​LUM of the areas 75D. Then, the vehicle parking assist device 10 obtains values ​​DLUM by subtracting a mean LUMave of the obtained luminance values ​​LUM from each of the luminance values ​​LUM (DLUM = LUM - LUMave). Then, based on the values ​​DLUM, the vehicle parking assist device 10 obtains differences between the luminance values ​​LUM of the feature point F. The vehicle parking assist device 10 then obtains a pattern of the obtained differences as luminance pattern information CT.If the predetermined condition is met, the vehicle parking assistance device 10 generally obtains the luminance pattern of each of the feature points (feature images) F based on the image information IMG obtained by the cameras 45 as the luminance pattern information CT.

[0067] The parking assist switch 48 is located in a section near the steering wheel 16. The parking assist switch 48 is electrically connected to the ECU 90. The driver operates the parking assist switch 48 to start a parking assist control described below.

[0068] The display 50 is provided on a section of the vehicle 100 where the driver can see it. In this embodiment, the display 50 is a display contained in a so-called navigation device.

[0069] Display 50 is electrically connected to ECU 90. ECU 90 can cause display 50 to show various images. In this embodiment, ECU 90 can cause display 50 to show a camera image 51C, a plane view or top view image 51P, a parking area line image 52, an adjustment key image 53, a registration start key image 54, a registration key image 55, a parking start key image 56, and a displacement key image 57.

[0070] Camera image 51C is an image taken by any one of the cameras 45.

[0071] The top-down image 51P is an image containing a vehicle plan view image and a vehicle environment image. The vehicle plan view image is an image depicting vehicle 100 viewed vertically from above. The vehicle environment image is an image depicting the area surrounding vehicle 100 viewed vertically from above. The vehicle environment image contains at least one image depicting parking space 62. The vehicle plan view image and the vehicle environment image (the bird's-eye view of vehicle 100) are generated by the ECU 90 based on the image information IMG.

[0072] The parking area line image 52 is an image depicting parking area 61. Parking area 61 is an area, space, or region where the vehicle 100 is to be parked by the parking assistance control. As shown in Fig. As shown in Figure 8, parking area 61 is located in parking lot 62.

[0073] The setting button image 53 is an image that represents a setting button which the driver can press with a touch interaction to set or fix or determine the parking area 61 into which the driver wants to park the vehicle 100 by means of the parking assistance control.

[0074] The registration start button image 54 is an image that represents a registration (memory) start button which the driver can press with touch interaction to cause the vehicle parking assist device 10 to start an execution of a first parking movement process of the parking assist control described below.

[0075] The registration button image 55 is an image representing a registration (save) button that the driver can activate via touch interaction to instruct the vehicle parking assistance device 10 to register / save the parking space information Ipark (specifically the RAM of the ECU 90) obtained by the parking assistance control unit. The parking space information Ipark is information about parking space 62, which is used by the vehicle parking assistance device 10 to autonomously park the vehicle 100 in parking space 62.

[0076] The parking start button image 56 is an image that represents a parking start button which the driver can press with touch interaction to cause the vehicle parking assistance device 10 to start the execution of the parking assistance control to park the vehicle 100 in the parking area 61 which is registered / stored in the vehicle parking assistance device 10.

[0077] The shift key image 57 contains an upward shift key image 57U, a downward shift key image 57D, a left shift key image 57L, and a right shift key image 57R. The upward shift key image 57U is an image that the driver can apply using touch interaction to shift the parking area line image 52 upwards on display 50. The downward shift key image 57D is an image that the driver can apply using touch interaction to shift the parking area line image 53 downwards on display 50. The left shift key image 57L is an image that the driver can apply using touch interaction to shift the parking area line image 52 to the left on display 50. The right-shift button image 57R is an image that the driver can apply to the right using touch interaction to shift the parking area line image 52 on the display 50. <Overview of the parking assistance control>

[0078] Next, an overview (brief summary) of a parking assistance control system is described. The vehicle parking assistance device 10 is configured to execute the parking assistance control system. The parking assistance control system is a control system for autonomously / automatically parking the vehicle 100 in the parking area 61 without requiring any operation by the driver of the accelerator pedal 14, the brake pedal 15, and the steering wheel 16.

[0079] There are parking lots where the parking areas are partitioned (defined) by lines, such as white lines. Hereinafter, each line that partitions (defines) the parking areas is referred to as a "parking area line." In the parking lot where the parking areas are partitioned by parking area lines, the vehicle parking assistance device 10 can use the parking area lines detected by the cameras 45 to autonomously park the vehicle in the parking area.

[0080] On the other hand, there are parking spaces, such as the parking space of a private house / apartment, where the parking areas are not partitioned by parking space lines. In a parking space where the parking areas are not partitioned by parking space lines, the vehicle parking assistance device 10 cannot use the parking space lines to autonomously park the vehicle 100 in the parking area. The parking assistance control performed by the vehicle parking assistance device 10 includes (i) control for autonomously parking the vehicle in the parking space and registering / saving the parking space information while the vehicle is parked in the parking space, and (ii) control for autonomously parking the vehicle in the parking space whose parking space information has already been registered / saved.

[0081] In a case where parking information is already registered / stored, the vehicle parking assist device 10 searches a left camera image Pleft and a right camera image Pright to find image sections with substantially the same luminance patterns as the luminance patterns of registered input luminance pattern information CTent_reg when the vehicle 100 has stopped. The left camera image is an image captured by the left camera 43. The right camera image is an image captured by the right camera 44. The registered input luminance pattern information CTent_reg is the luminance pattern information CT of input feature points Fent, which were registered / stored in the vehicle parking assist device 10 via the parking assist controller. The input feature points Fent are the feature points of input 62ent of parking space 62, which are obtained via the parking assist controller.The left camera image Pleft and the right camera image Pright, which are respectively taken by the left camera 43 and the right camera 44 when the vehicle 100 has stopped (before parking), are referred to as "post-registration images" or "post-registration images" if the parking information has already been registered / saved.

[0082] If the vehicle parking assist device 10 has successfully located the image sections with substantially the same luminance patterns as the luminance patterns of registered input luminance pattern information CTent_reg in the left camera image Pleft, the vehicle parking assist device 10 determines that the registered parking space 62 is present on the left side of the stopped vehicle 10. The registered parking space 62 is the parking space whose parking information Ipark has already been registered / stored in the vehicle parking assist device 10 via the parking assist control.

[0083] If the vehicle parking assist device 10 has succeeded in finding the image sections with substantially the same luminance patterns as the luminance patterns of registered input luminance pattern information CTent_reg in the right camera image Pright, the vehicle parking assist device 10 determines that the registered parking space 62 is present on the right side of the stopped vehicle 100. <Registrierung des Parkplatzes>

[0084] When a registration (storage) start condition is met, the vehicle parking assist control device 10 obtains pre-input information Ient_pre and pre-intermediate information Imid_pre, as described below. The registration start condition is met when (i) the vehicle parking assist device 10 determines that vehicle 100 has stopped, (ii) the parking assist switch 48 is operated, and (iii) the vehicle parking assist device 10 determines that the parking space 62 located near vehicle 100 is not the registered parking space 62. Additionally, the vehicle parking assist device 10 registers / stores (i) registration input information Ient_reg, (ii) registration internal information Iin_reg, and (iii) registration area information Iarea_reg as the parking space information Ipark, as described below. When the registration start condition is met, the vehicle parking assist device 10 displays various in Fig. Images 9A shown, which include the top view image 51P, the parking area line image 52 (in Fig. (9A not shown), the setting key image 53 and the displacement key image 57 with key images 57U, 57L, 57R and 57D are displayed on the screen 50. If the parking space 62 in which the vehicle 100 can be parked is on the left side of the vehicle 100, the vehicle parking assist device 10 displays the top view image 51P on the screen 50 such that a parking space image is shown on the left side of a vehicle image. If, on the other hand, the parking space 62 in which the vehicle 100 can be parked is on the right side of the vehicle 100, the vehicle parking assist device 10 displays the top view image 51P on the screen 50 such that the parking space image is shown on the right side of the vehicle image.

[0085] The vehicle parking assistance device 10 also defines an area within parking space 62, designated as parking area 61, in which the vehicle 100 can be parked, based on the image information IMG and the sonar information SON. The vehicle parking assistance device 10 then displays the parking area line image 52 on the display 50, representing the defined parking area 61. For example, the vehicle parking assistance device 10 uses the sonar information SON to determine the size of the entrance 62 of parking space 62.

[0086] The driver can reposition the parking area line image 52 on the display 50 by applying a touch interaction to the reposition button image 57 before applying a touch interaction to the setting button image 53. The driver can change the position of the parking area 61 to a position where the driver wishes to park the vehicle 100 by repositioning the parking area line image 52 on the display 50.

[0087] When the driver applies touch interaction to the setting button image 53, the vehicle parking assist device 10 stops displaying the setting button image 53 and the displacement button image 57 on the display 50. Instead, the device 10 begins displaying the registration start button image 54 in an area where the setting button image 53 was previously displayed on the display 50, as shown in Fig. 9B is shown.

[0088] When the driver interacts with the setting button image 53 via touch, that is, when the vehicle parking assistance device 10 receives a registration request to register the parking area 61 in which the driver wishes to park the vehicle 100, the vehicle parking assistance device 10 also obtains the position of the parking area 61 that corresponds to the position of the parking area line image 52 displayed on the display 50. The vehicle parking assistance device 10 then sets (determines) the parking area 61 that corresponds to the parking area line image 52 displayed on the display 50 as the registered target parking area 61set.

[0089] When the driver applies touch interaction to the setting button image 53, the vehicle parking assistance device 10 also sets / determines a target movement route Rtgt along which the vehicle 100 is to be moved to park the vehicle 100 in the registered target parking area 61set. If the vehicle 100, for example, is on the right side of the unregistered parking space 62 as shown in Fig. 10 shown stops, sets / determines the vehicle parking assistance device 10 the target movement route Rtgt as in Fig. 11 shown.

[0090] When the driver applies touch interaction to the setting button image 53 while the vehicle stops on the right side of the parking space 62, the vehicle parking assistance device 10 also acquires a predetermined number of new left feature points F3new in each of the four middle sub-areas 73D5, the two front sub-areas 73D1, and the two rear sub-areas 73D2 of the left area 73. At this time, the vehicle parking assistance device 10 acquires the new left feature points F3new as input feature points Fent. When the driver applies touch interaction to the setting button image 53 while the vehicle stops on the left side of the parking space 62, the vehicle parking assistance device 10, on the other hand, procures a predetermined number of new right feature points F4new in each of the four middle sub-areas 74D5, the two front sub-areas 74D1 and the two rear sub-areas 74D2 of the right area 74.At this point, the vehicle parking assistance device 10 procures the new right feature points F4new as input feature points Fent.

[0091] In this embodiment, when the driver applies touch interaction to the setting button image 53 while the vehicle stops on the right side of the parking space 62, the vehicle parking assistance device 10 procures the input feature points Fent such that the number of input feature points Fent procured in each of the middle sub-areas 73D5 is greater than the number of input feature points Fent procured in each of the front part areas 73D1 and rear part areas 73D2. Basically, the vehicle parking assistance device 10 procures the entry feature points Fent such that the number of entry feature points Fent procured in each of the areas 73D5 near a center of the entrance 62ent of parking lot 62 is greater than the number of entry feature points Fent procured in each of the areas 73D1 and 73D2 away from the center of the entrance 62ent of parking lot 62.

[0092] When the driver applies touch interaction to the setting button image 53 when the vehicle has stopped on the left side of the parking space 62, the vehicle parking assistance device 10, on the other hand, procures the input feature points Fent such that the number of input feature points Fent procured in each of the middle sub-areas 74D5 is greater than the number of input feature points Fent procured in each of the front part areas 74D1 and the rear part areas 74D2. Basically, the vehicle parking assistance device 10 procures the entry feature points Fent such that the number of entry feature points Fent procured in each of the areas 74D5 near the center of the entrance 62ent of parking lot 62 is greater than the number of entry feature points Fent procured in each of the areas 74D1 and 74D2 away from the center of the entrance 62ent of parking lot 62.

[0093] If vehicle 100, for example, stops on the right side of parking space 62, as shown in Fig. As shown in Figure 10, the vehicle parking assistance device 10 (i) procures the two new left feature points F3new as the input feature points Fent from each of the four middle sub-areas 73D5 of the left area 73, (ii) the one new left feature point F3new as the input feature point Fent from each of the two front sub-areas 73D1 of the left area 73, and (iii) the one new left feature point F3new as the input feature point Fent from each of the two rear sub-areas 73D2 of the left area 73 (compare Figure 10). Fig. 12 and Fig. 13) If, on the other hand, the vehicle 100 stops on the left side of the parking space 62, the vehicle parking assist device 10 (i) obtains the two new right feature points F4new as input feature points Fent from each of the four middle sub-areas 74D5 of the right area 74, (ii) the one new right feature point F4new as input feature point Fent from each of the two front sub-areas 74D1 of the right area 74, and (iii) the one new right feature point F4new as input feature point Fent from each of the two rear sub-areas 74D2 of the right area 74.

[0094] It is noted that the vehicle parking assistance device 10 is configured to obtain the entry feature points Fent such that the number of entry feature points Fent obtained from each of the two front part areas 73D1 and the two middle part areas 73D5 adjacent to the front part areas 73D1 is greater than the number of entry feature points Fent obtained from each of the two rear part areas 73D2 and the two middle part areas 73D5 adjacent to the rear part areas 73D2 when the driver tends to stop the vehicle 100 on the right side of the entrance 62ent of the parking space 62 and just before a position immediately perpendicular to the entrance 62ent of the parking space 62.Similarly, the vehicle parking assistance device 10 can be configured to obtain the entry feature points Fent such that the number of entry feature points Fent obtained from each of the two front part areas 74D1 and the two middle part areas 74D5 adjacent to the front part areas 74D1 is greater than the number of entry feature points Fent obtained from each of the two rear part areas 74D2 and the two middle part areas 74D5 adjacent to the rear part areas 74D2, if the driver tends to stop the vehicle 100 on the left side of the entrance 62ent of the parking space 62 and just before the position immediately perpendicular to the entrance 62ent of the parking space 62.

[0095] If the vehicle parking assist device 10 cannot procure the predetermined number of new left feature points F3new from at least one of the middle sub-areas 73D5, the front sub-areas 73D1 and the rear sub-areas 73D2 of the left area 73, the vehicle parking assist device 10 procures the input feature points Fent from the remaining middle sub-areas 73D5, the front sub-areas 73D1 and the rear sub-areas 73D2 to compensate for a deficit in the number of procured input feature points Fent.If the vehicle parking assistance device 10 cannot procure the predetermined number of new right feature points F4new from at least one of the middle sub-areas 74D5, the front sub-areas 74D1 and the rear sub-areas 74D2 of the right area 74, the vehicle parking assistance device 10 procures the input feature points Fent from the remaining middle sub-areas 74D5, the front sub-areas 74D1 and the rear sub-areas 74D2 to compensate for the deficit in the number of procured input feature points Fent.

[0096] After the vehicle parking assistance device 10 has acquired the input feature points Fent, it acquires the coordinates XY of each of the acquired input feature points Fent in a preliminary (non-binding) coordinate system CT and stores the acquired coordinates XY as preliminary (non-binding) input coordinates XYent_pre. Furthermore, the vehicle parking assistance device 10 acquires the luminance pattern information CT for each of the acquired input feature points Fent and stores the acquired luminance pattern information CT as preliminary (non-binding) input luminance pattern information CTent_pre. The preliminary coordinate system CT is a coordinate system that has a predetermined point Ppre in the registration target parking area 61set as its origin. The preliminary input coordinates XYent_pre therefore specify a position of the input feature point Fent relative to the predetermined position Ppre.The pre-entry information Ient_pre contains the pre-entry coordinates XYent_pre and the pre-entry luminance pattern information CTent_pre.

[0097] When the driver applies touch interaction to the registration start button image 54, the vehicle parking assist device 10 stops displaying the registration start button image 54 on the display 50, but continues to display the camera image 51C and the top-down image 51P on the display, as shown in Fig. Figure 9C illustrates this. If parking space 62, in which vehicle 100 can be parked, is located on the left side of vehicle 100 at that time, the vehicle parking assistance device 10 obtains the image depicting this parking space 62 from the left camera 43 and displays the obtained image on the display 50 as camera image 51C. The vehicle parking assistance device 10 also displays the top-down image 51P on the display 50 such that the parking space image is shown on the left side of the vehicle image. Conversely, if parking space 62, in which vehicle 100 can be parked, is located on the right side of vehicle 100 at that time, the vehicle parking assistance device 10 obtains the image depicting this parking space 62 from the right camera 44 and displays the obtained image on the display 50 as camera image 51C.Furthermore, the vehicle parking assistance device 10 displays the top view image 51P on the display 50 in such a way that the parking image is shown on the right side of the vehicle image.

[0098] When the driver applies touch interaction to the registration start button image 54, the vehicle parking assistance device 10 also initiates the execution of the first parking movement process to move the vehicle 100 to the registration target parking area 61set along the target movement route Rtgt. The first parking movement process is a process for controlling the operation of the vehicle propulsion force generation device 11, the braking device 12, and the steering device 13 such that the vehicle 100 moves along the target movement route Rtgt, based on (i) the image information IMG, (ii) the object information OBJ, (iii) the steering angle θst, (iv) the steering torque TQst, (v) the vehicle movement speed SPD, (vi) the vehicle yaw rate Yr, (vii) the vehicle longitudinal acceleration Gx, and (viii) the vehicle lateral acceleration Gy.

[0099] If, for example, vehicle 100 stops on the right side of parking space 62, where vehicle 100 can be parked, as shown in Fig. As shown in Figure 10, the vehicle parking assistance device 10 starts the execution of the first parking movement process to move / rotate the vehicle 100 forward to the right, and then stops the vehicle 100 as shown in Figure 10. Fig. 14 is shown. Next, the vehicle parking assist device 10 moves / rotates the vehicle 100 backwards to the left, as shown in Fig. 15 is shown.

[0100] In this embodiment, the vehicle parking assistance device 10 acquires the feedback points F2 as new feedback points F2new when the vehicle parking assistance device 10 completes the parking of the vehicle 100 in the parking space 62 after the direction of movement of the vehicle 100 has straightened out, while the vehicle parking assistance device 10 allowed the vehicle to move backwards via the parking assistance control (compare Fig. 16) It is noted that the vehicle parking assist device 10 can acquire the feedback points F2 after the direction of travel of the vehicle 100 has become straight, while the vehicle parking assist device 10 is driving the vehicle backward via the parking assist control, before the vehicle parking assist device 10 finishes parking the vehicle 100 in the parking space 62. Alternatively, the vehicle parking assist device 10 can (i) acquire the feedback points F2 at a time when the direction of travel of the vehicle 100 becomes straight, and (ii) acquire the feedback points F2 when the vehicle 100 has traveled the predetermined distance Dtravel_th backward after the direction of travel of the vehicle 100 has become straight, while the vehicle parking assist device 10 is driving the vehicle backward.Furthermore, the vehicle parking assistance device 10 can procure not only the rear feature points F2, but also the front feature points F2 and / or the left feature points F3 and / or the right feature points F4 if the device 10 procures the rear feature points F2.

[0101] The vehicle parking assistance device 10 then obtains one or more of the new feature points F2new from each of the rear section areas 72D and sets the obtained new feature points F2new as intermediate feature points Fmid. Next, the vehicle parking assistance device 10 obtains the coordinates XY of the obtained intermediate feature points Fmid in the preliminary (non-binding) coordinate system Cpre and stores the obtained coordinates XY as preliminary intermediate coordinates XYmid_pre. The vehicle parking assistance device 10 also obtains the luminance pattern information CT for the obtained intermediate feature points Fmid and stores the obtained luminance pattern information CT as preliminary intermediate luminance pattern information CTmid_pre. The preliminary intermediate coordinates XYmid_pre specify the positions of the intermediate feature points Fmid relative to the predetermined position Ppre.The pre-intermediate information Imid_pre contains the pre-intermediate coordinates XYmid_pre and the pre-intermediate luminance pattern information CTmid_pre.

[0102] While the vehicle parking assistance device 10 executes the initial parking movement process to move the vehicle 100 along the target movement route Rtgt, the vehicle parking assistance device 10 also executes a safety determination process to determine whether it can safely move the vehicle 100 to the registration target parking area 61set without the vehicle 100 coming into contact with / collising on the object present in the parking space 62. If the vehicle parking assistance device 10 determines that it cannot safely move the vehicle 100 to the registration target parking area 61set, it corrects the target movement route Rtgt such that it can safely move the vehicle 100 to the registration target parking area 61set without the vehicle 100 coming into contact with / collising on the object present in the parking space 62.The vehicle parking assistance device 10 performs the safety determination process based on the image information IMG and the object information OBJ, which the vehicle parking assistance device 10 obtains during the execution of the first parking movement process.

[0103] While the vehicle parking assist device 10 is executing the first parking movement process, it also performs a route determination / verification process. The route determination process is a process to determine whether the vehicle parking assist device 10 can reliably park the vehicle 100 in the registered target parking area 61set when the vehicle 10 is moved along the current target movement route Rtgt. If the vehicle parking assist device 10 determines that it cannot park the vehicle 100 in the registered target parking area 61set using the current target movement route Rtgt, it corrects the current target movement route Rtgt so that it can reliably park the vehicle 100 in the registered target parking area 61set.The vehicle parking assistance device 10 performs the route determination process based on the image information IMG (especially the feature points F) that the vehicle parking assistance device 10 obtains during the execution of the first parking movement process.

[0104] When the entire vehicle has moved 100 into the registration target parking area 61set (compare Fig. 17) The vehicle parking assistance device 10 stops the vehicle 100 and completes the execution of the first parking movement process. This completes the parking of the vehicle 100 in parking space 62 by the parking assistance control. At this point, the vehicle parking assistance device 10 acquires the front feature points F1, the left feature points F3, and the right feature points F4 as new front feature points F1new, new left feature points F3new, and new right feature points F4new, respectively. At this point, the vehicle parking assistance device 10 can acquire the rear feature points F2 as new rear feature points F2new.

[0105] Then, the vehicle parking assist device 10 acquires one or more of the acquired new front feature points F1new from each of the front part areas 71D as final feature points Ffin. The vehicle parking assist device 10 also acquires one or more of the acquired new left feature points F3new from each of the left part areas 73D as the final feature points Ffin. The vehicle parking assist device 10 also acquires one or more of the acquired new right feature points F4new from each of the right part areas 74D as the final feature points Ffin. Once the vehicle parking assist device 10 has acquired the new rear feature points F2new, it acquires one or more of the acquired new rear feature points F2new from each of the rear part areas 72D as the final feature points Ffin. <Registrierung der Parkplatzinformationen>

[0106] When the vehicle parking assist device 10 completes the parking of the vehicle 100 in the parking space 62 via the parking assist control, the vehicle parking assist device 10 displays the registered key image 55 on the display 50 as shown in Fig. 9D illustrates this.

[0107] When the driver applies touch interaction to the registration key image 55, the vehicle parking assistance device 10 obtains the XY coordinates of the acquired final feature points Ffin in a registration coordinate system Creg and registers or stores the acquired XY coordinates as registration inner coordinates XYin_reg. The vehicle parking assistance device 10 also obtains the luminance pattern information CT of the acquired final feature points Ffin and registers or stores the acquired luminance pattern information CT as registration inner luminance pattern information CTin_reg. The registration coordinate system Creg is a coordinate system that has a predetermined point Preg as its origin (compare Fig. 17) Vehicle 100 contains a shaft connecting a left rear wheel and a right rear wheel. The predetermined point Preg is the midpoint of the shaft in the vehicle's transverse direction Dy when the parking of vehicle 100 in the registration target parking area 61set is completed by the parking assistance control. The registration inner coordinates XYin_reg therefore specify the positions of the final feature points Ffin relative to the predetermined position Preg.

[0108] The vehicle parking assistance device 10 also converts the pre-intermediate coordinates XYmid_pre into the coordinates XY in the registration coordinate system Creg and registers or stores the converted coordinates XY as the registration inner coordinates XYin_reg. The vehicle parking assistance device 10 also registers or stores the pre-intermediate luminance pattern information CTmid_pre as the registration inner luminance pattern information CTin_reg. The registration inner coordinates XYin_reg therefore specify the positions of the intermediate feature points Fmid relative to the predetermined position Preg.

[0109] The registration information Iin_reg contains the registration coordinates XYin_reg and the registration luminance pattern information CTin_reg.

[0110] The vehicle parking assistance device 10 also registers or stores the XY coordinate of the target parking area 61set in the CTreg coordinate system as registration area coordinates XYarea_reg. The XYarea_reg coordinates indicate the position of parking area 61 relative to the predetermined position Preg. The registration area information Iarea_reg contains the XYarea_reg coordinates.

[0111] The vehicle parking assistance device 10 also converts the pre-entry coordinates XYent_pre into the coordinates XY in the registration coordinate system CTreg and registers or stores the converted coordinates XY as registration entry coordinates XYent_reg.

[0112] The vehicle parking assistance device 10 also registers or stores the pre-input luminance pattern information CTent_pre as the registered input luminance pattern information CTent_reg. The registration input coordinates XYent_reg therefore specify the positions of the input feature points Fent relative to the pre-input Preg. The registration input information Ient_reg contains the registration input coordinates XYent_reg and the registered input luminance pattern information CTent_reg.

[0113] As described above, the parking information Ipark contains the registration entry information Ient_reg, the registration interior information Iin_reg and the registration area information Iarea_reg. <Parken des Fahrzeugs im registrierten Parkplatz>

[0114] When an autonomous parking start condition is met, the vehicle parking assist device 10 displays the camera image 51C, the top-down image 51P, the parking area line image 52 (not shown), and the parking start button image 56 on the display 50 as shown in Fig. 18A illustrates this.

[0115] The autonomous parking start condition is met when (i) the vehicle parking assistance device 10 determines that the vehicle 100 has stopped at the entrance 62 of parking space 62, (ii) the vehicle parking assistance device 10 determines that the parking space 62 in question is the registered parking space, and (iii) the driver activates the parking assistance switch 48 with the touch interaction. If the registered parking space 62 is present on the left side of the vehicle 100, the vehicle parking assistance device 10 obtains the image representing the registered parking space 62 from the left camera 43 and displays the obtained image on the display 50 as the camera image 51C. The vehicle parking assistance device 10 also displays the top-down image 51P on the display 50 such that the parking space image is shown on the left side of the vehicle image.If, on the other hand, the registered parking space 62 is located on the right side of the vehicle 100, the vehicle parking assistance device 10 obtains the image depicting the registered parking space 62 from the right camera 44 and displays the obtained image on the display 50 as the camera image 51C. The vehicle parking assistance device 10 also displays the top-down image 51P on the display 50 in such a way that the parking space image on the right side of the vehicle is shown.

[0116] The vehicle parking assistance device 10 also fixes the position of the parking area 61 based on the registration area coordinates XYarea_reg, which are contained in the parking space information Ipark, referring to the registered parking space 62 in which the vehicle parking assistance device 10 will park the vehicle 100 at that time. The vehicle parking assistance device 10 displays an image as the parking area line image 52 on the display 50, which represents this parking area 61, whose position is fixed.

[0117] When the driver applies touch interaction to the parking start button image 56, the vehicle parking assistance device 10 ends the display of the parking start button image 56 on the display 50 and continues with the display of the camera image 51C and the top-down image 51P on the display 50, as shown in Fig. 18B is shown.

[0118] When the driver applies touch interaction to the parking start button image 56, the vehicle parking assistance device 10 also sets a target parking area 61tgt to the parking area 61, which corresponds to the position of the parking area line image 52 shown on the display 50.

[0119] When the driver applies touch interaction to the parking start button image 56, the vehicle parking assist device 10 also sets the target movement route Rtgt, along which the vehicle parking assist device 10 moves the vehicle 100 to park the vehicle 100 in the target parking area 61tgt.

[0120] The vehicle parking assistance device 10 then executes a second parking movement process to move the vehicle 100 to the target parking area 61tgt along the target movement route Rtgt. The second parking movement process is a process for controlling the operation of the vehicle propulsion force generation device 11, the braking device 12, and the steering device 13 to move the vehicle 100 along the target movement route Rtgt based on (i) the image information IMG, (ii) the object information OBJ, (iii) the steering angle θst, (iv) the steering torque TQst, (v) the vehicle movement velocity SPD, (vi) the vehicle yaw rate YR, (vii) the vehicle longitudinal acceleration Gx, and (viii) the vehicle lateral acceleration Gy.

[0121] While the vehicle parking assistance device 10 executes the second parking movement process to move the vehicle 100 along the target movement route Rtgt, the vehicle parking assistance device 10 performs a safety determination process to determine whether it can safely move the vehicle 100 to the target parking area 61tgt, preventing the vehicle 10 from contacting the object in the parking space 62. If the vehicle parking assistance device 10 determines that it cannot safely move the vehicle 100 to the target parking area 61tgt, it corrects the target movement route Rtgt such that it can safely move the vehicle 100 to the target parking area 61tgt, preventing the vehicle 100 from contacting the object in the parking space 62.The vehicle parking assistance device 10 performs the safety determination process based on the image information IMG and the object information OBJ, which the vehicle parking assistance device 10 obtains during the execution of the second parking movement process.

[0122] While the vehicle parking assistance device 10 executes the second parking movement process, it also performs a parking position determination process. This parking position determination process comprises the following steps.

[0123] One step of scanning the camera image to find image sections with the same luminance patterns as the luminance patterns of the registered input luminance pattern information CTent_reg or the registered internal luminance pattern information CTin_reg.

[0124] One step of determining whether the position of the target parking area 61tgt in the parking lot 62 matches the position specified by the registration area coordinates XYarea_reg, based on the positional relationship between the coordinates XY of the found image sections and the registration area coordinates XYarea_reg.

[0125] If the vehicle parking assistance device 10 does not determine that the position of the target parking area 61tgt in the parking space 62 corresponds to the position specified by the registration area coordinates XYarea_reg, the vehicle parking assistance device 10 corrects the position of the target parking area 61tgt so that the position of the target parking area 61tgt corresponds to the position specified by the registration area coordinates XYarea_reg, and corrects the target movement route Rtgt so that the vehicle parking assistance device 10 can move the vehicle 100 to the corrected target parking area 61tgt.

[0126] Once the entire vehicle 100 has been moved into the target parking area 61tgt, the vehicle parking assistance device 10 stops the vehicle 100 and completes the execution of the second parking movement process. Thus, the parking of the vehicle 100 in parking space 62 is completed by the parking assistance control. <Zusammenfassung des Betriebs der Fahrzeugparkassistenzvorrichtung>

[0127] The object's image is easily affected by changes in the situation / condition around the vehicle and the parking space between the time the parking information (Ipark) is registered / stored and the time thereafter. For example, the object's position may change between the time the parking information (Ipark) is registered / stored and the time thereafter. Furthermore, the object's image may change if the camera's recording position changes between the time the parking information (Ipark) is registered / stored and the time thereafter. Therefore, the vehicle parking assistance device 10 specifies an object image (Pobj) within the camera image and extracts (masks) the specified object image (Pobj) from the camera image to obtain the extracted camera image.When the parking information Ipark is registered, the vehicle parking assistance device 10 obtains the feature points F from the extracted camera image. After registering the parking information Ipark, the vehicle parking assistance device 10 searches the captured camera image to identify image sections with the same luminance patterns as the luminance patterns of the registered input luminance pattern information CTent_reg.

[0128] A in Fig. Parking space 62' shown in 19 differs from the one in Fig. The parking space 62 shown in section 4 is characterized by the fact that a wall 64, which is the object in question, is located on the right side of parking space 62. A wall 64 is located on the right side of parking space 62. Fig. The left camera image shown in Figure 20, Pleft, is an image taken by the left camera 43 when the vehicle stopped near entrance 62' of parking lot 62'. The vehicle assistance device 10 obtains a Fig. 21 Top view image Pheimen shown by converting the left camera image Pleft into a plan view or top view image. That is, the left top view image Pheimen is generated such that the left camera image Pleft is viewed from an upper observation point VP of the left camera, which is positioned vertically above the left camera 43.

[0129] If the left top-down image Pheimen is generated such that the left camera image Pleft is viewed from an observation point located vertically above wall 64, then only an image of the upper surface of wall 64 is contained in this left top-down image Pheimen as an image representing wall 64. However, the left top-down image Pheimen is generated as described above such that the left camera image Pleft is viewed from the upper observation point VP of the left camera. As in Fig. Figure 21 shows that an image 640 of wall 64 (an object image) in the left top view image Pheimen contains an image corresponding to a front surface Sfront of wall 64, and an image corresponding to a left side surface Sleft of wall 64.

[0130] The vehicle parking assistance device 10 specifies a position of the object around the vehicle 100 based on the sonar information SON. In a Fig. In the example shown in Figure 19, the tenth distance sonar 310 detects a distance L between the front surface Sfront and the tenth distance sonar 310. Based on the sonar information SON received from the tenth distance sonar 310, the vehicle parking assistance device 10 detects / determines that the object (the front surface Sfront of the wall 64) is located at a point that is a distance L from the left side of the vehicle 100. Fig. The 22 shown areas L22 to L25 are removed. It is noted that the vehicle parking assist device 10 divides the left-side area 73 and the right-side area 74 each into twenty-five areas. The twenty-five areas of the left-side area 73 are each referred to as "areas L1 to L25". The twenty-five areas of the right-side area 74 are each referred to as "areas R1 to R25". The vehicle parking assist device 10 divides the front area 71 and the rear area 72 each into twenty-five areas. Each of the sub-areas of the front area 71 and the rear area 72 has a rectangular shape with one long side in the longitudinal direction.

[0131] The vehicle parking assistance device 10 detects only a reflective surface of the object that reflects the sound waves emitted by the distance sonar. In the Fig. In the example shown in Figure 21, the vehicle parking assistance device 10 only detects the front surface of the wall 64. This means that the vehicle parking assistance device 10 cannot detect the shape of the object behind the reflective surface with respect to the distance sonar. As shown in Fig. As shown in Figure 23, the vehicle parking assistance device 10 plots the positions (sonar detection results 2310) of the object (the front surface Sfront of the wall 64), which are detected based on the sonar information SON, in the left top view image Pheimen at positions that are away from the tenth distance sonar 310 by the distance detected by the tenth distance sonar 310.

[0132] The vehicle parking assistance device 10 specifies the object image Pobj in the left top view image Pheimen based on the sonar detection results 2310, which are plotted in the left top view image Pheimen.

[0133] Specifically, the vehicle parking assistance device 10 first obtains a furthest point 2320 and a nearest point 2330 using the sonar detection results 2310. The furthest point 2320 is a point that is furthest from the left camera 43 according to the sonar detection results 2310. The nearest point 2320 is a point that is closest to the left camera 43 according to the sonar detection results 2310. Next, the vehicle parking assistance device 10 obtains a gradient of a virtual line segment 2335 from the furthest point 2320 to the left camera 43. Then, the vehicle parking assistance device 10 obtains a first virtual line 2340 that extends from the furthest point 2320 in a direction with gradient GR and away from the left camera 43.

[0134] The vehicle parking assistance device 10 procures a second virtual line 2350, which extends from the nearest point 2330 in a direction parallel to the central axis SA10 of the sound wave radiation area of ​​the tenth distance sonar 310 and away from the left camera 43. It is noted that the second virtual line 2350 extends in a direction orthogonal to the left side surface of the vehicle 100 (that is, in the vehicle width direction Dy), since the direction of the central axis SA10 is orthogonal to the left side surface of the vehicle 100 (that is, in the vehicle width direction Dy).

[0135] Subsequently, the vehicle parking assistance device 10 specifies an image of an area as the object image Pobj, which is defined by the sonar detection results 2310, the first virtual line 2340 and the second virtual line 2350.

[0136] The reason why the vehicle parking assistance device 10 uses the first virtual line 2340 to define the object image Pobj is that a line image representing the object height at the furthest point 2320 in the top-down image Pheimen tilts (lean) away from the left camera 43 by the gradient GR. The object image Pobj therefore obscures an object located behind the front surface Sfront, and it is possible to exclude the image representing the object from the excluded camera image.

[0137] If another object is located behind the reflective surface that reflects the sound waves from the distance sonar, the distance sonar cannot detect this other object because the sound waves from the distance sonar do not reach the other object, and the other object therefore does not reflect the sound waves. Thus, another object may be present in the area behind the reflective surface. In light of this, the vehicle parking assistance device 10 uses the second virtual line 2350 to define the object image Pobj, so that the object image Pobj can contain the area where the other object may or may not be present. The vehicle parking assistance device 10 can therefore reliably prevent the vehicle 10 from acquiring feature points F based on the image, which may contain a part representing an object.The central axis CA3 of the recording area of ​​the left camera 43 coincides with the central axis SA9 of the ninth distance sonar 309 and the central axis SA10 of the tenth distance sonar 310. Therefore, a line image representing the object height at the nearest point 2330 does not protrude from the second virtual line 2350 to be capable of being included in the object image Pobj. The object image Pobj thus covers an object from the image that is present behind the front surface Sfront, and the image representing the object can be excluded from the excluded camera image.

[0138] Finally, the vehicle parking assistance device 10 obtains the feature points F from a masked / excluded image that does not contain the object image Pobj in the left top-down image Pheimen. The masked image is an image that the vehicle parking assistance device 10 generates by excluding the object Pobj from the left top-down image Pheimen (that is, the left top-down image Pheimen in which the object Pobj is masked).

[0139] As described above, the vehicle parking assistance device 10 masks the object image Pobj from the top-down image Pheimen, into which a registration image has been converted, in order to obtain the masked image. The registration image is captured by the left camera 43 and the right camera 44 respectively when the parking information Ipark is registered. The vehicle parking assistance device 10 obtains the feature points F from the masked image. The masked image is an image that contains only the surface of the ground. The registered input luminance pattern information CTent_reg, which is obtained based on the feature points F thus acquired, is registered / stored as the registration input information Ient_reg.Even if the situation / condition around vehicle 100 and parking space 62 changes after the parking space information Ipark has been registered, the vehicle parking assistance device 10 can therefore correctly determine whether vehicle 100 has stopped on the right or left side of the registered parking space 62, whose parking space information Ipark has already been registered, because the luminance pattern information CT of the feature points F, which are obtained from the masked image, are registered as the registration input information Ient_reg.

[0140] The vehicle parking assistance device 10 masks the object image Pobj from the top-down image Pheimen, into which the post-registration image is converted. The post-registration image is captured by cameras 45 after the parking information Ipark has been registered. The vehicle parking assistance device 10 searches the masked image to identify the image sections with substantially the same luminance patterns as the luminance patterns of the registered input luminance pattern information CTent_reg. Therefore, even if the situation around vehicle 100 and parking space 62 has changed after the parking information Ipark was registered, the vehicle parking assistance device 10 can correctly determine whether vehicle 100 stopped on the right or left side of the registered parking space 62, whose parking information Ipark has already been registered.

[0141] It is noted that the vehicle parking assistance device 10 specifies the object image Pobj in the top-down image Pheimen, into which the camera image captured by the right camera 44 is converted when the object is located on the right side of the vehicle 100. The process for specifying the object image Pobj has been described above.

[0142] The vehicle parking assistance device 10 specifies the object image Pobj in the top-down image Pheimen, into which the camera image captured by the front camera 41 is converted when the object is located at the front of the vehicle 100. In this case, the second virtual line 2350 extends from the nearest point 2330 in the same direction as the directions of the center axis SA2 of the second distance sonar 302 and the center axis SA4 of the fourth distance sonar 304. The direction is away from the front camera 41.

[0143] When the first distance sonar 301 or the third distance sonar 303 detects the object, the vehicle parking assist device 10 cannot mask the line pattern representing the object's height at the nearest point 2330 if the second virtual line 2350 extends in the same direction as the center axis SA1 of the first distance sonar 301 or the center axis SA3 of the third distance sonar 303 (that is, if the second virtual line 2350 extends in a direction inclined 45 degrees to the left or right with respect to the vehicle's longitudinal direction Dx). Therefore, even if the first distance sonar 301 or the third distance sonar 303 detects the object, the vehicle parking assist device 10 extends the second virtual line 2350 from the nearest point 2330 in the same direction as the directions of the center axis SA2 of the second distance sonar 302 and the center axis SA4 of the fourth distance sonar 304 (that is, the vehicle's longitudinal direction Dx).The direction is a direction away from the front camera 41. The direction of the central axis CA1 of the front camera 41's recording area is the vehicle's longitudinal direction Dx. Therefore, the vehicle parking assist device 10 extends the second virtual line 2350 into the vehicle's longitudinal direction Dx in order to incorporate the line image representing the object's height at the nearest point 2330 into the object image Pobj. The vehicle parking assist device 10 can also incorporate the image representing the area where the distance sonars cannot detect the other object due to the object's presence into the object image Pobj.

[0144] It is noted that the vehicle parking assist device 10 specifies the object image Pobj in the top-down image Pheimen, into which the camera image captured by the rear camera 42 is converted when the object is present at the rear of the vehicle 100. The process for specifying the object image Pobj is the same as the process that is performed when the object is present at the front of the vehicle 100. <Spezifischer Betrieb der Fahrzeugparkassistenzvorrichtung >

[0145] Next, the specific operation of the vehicle parking assistance device 10 is described. The CPU of the ECU 90 of the vehicle parking assistance device 10 is used to execute a command in Fig. The routine shown in section 24 is configured or programmed each time a predetermined time expires.

[0146] When a suitable time arrives, the CPU starts processing step 2400 and proceeds to step 2405 to obtain the image information from camera sensor device 40 and the sonar information SON from sonar sensor device 30. The CPU then proceeds to step 2410.

[0147] In step 2410, the CPU determines whether the low-speed condition, that the vehicle speed SPD is equal to or less than the threshold speed SPDth, is met. If the low-speed condition is met, the CPU performs a "Yes" determination in step 2410 and proceeds to step 2415. In step 2415, the CPU converts the camera image captured by each of the 45 cameras into the top-down image and proceeds to step 2420.

[0148] In step 2420, the CPU determines, based on the sonar information (SON), whether the object is present or not. If at least one distance sonar from the first distance sonar 301 to the fourth distance sonar 304 detects the object, the CPU determines that the object is present in the recording area of ​​the front camera 41. If at least one distance sonar from the fifth distance sonar 305 to the eighth distance sonar 308 detects the object, the CPU determines that the object is present in the recording area of ​​the rear camera 41. If at least one distance sonar from the ninth distance sonar 309 and the tenth distance sonar 310 detects the object, the CPU determines that the object is present in the recording area of ​​the left camera 41. If at least one distance sonar from the eleventh distance sonar 311 and the twelfth distance sonar 312 detects the object, the CPU determines that the object is present in the recording area of ​​the right camera 41.

[0149] If the object exists, the CPU performs a "Yes" determination in step 2420 and executes steps 2425 to 2450. Afterward, the CPU proceeds to step 2495 to temporarily terminate the current routine.

[0150] Step 2425: The CPU plots the sonar acquisition results on the top-down images. A length in a longitudinal direction and a length in a transverse direction of an image element in the top-down image each correspond to a predetermined actual length (for example, approximately 15 cm). The CPU specifies image elements of the object in the top-down image based on the lengths of the image elements, the position of the reconnaissance sonar detecting the object, and the distance between the reconnaissance sonar and the object. The CPU plots the sonar acquisition results on the specified image elements.

[0151] Step 2420: The CPU procures the first virtual line extending from the furthest point contained in the sonar acquisition results relative to the camera.

[0152] Step 2435: The CPU procures the second virtual line extending from the nearest point contained in the sonar acquisition results with respect to the camera.

[0153] Step 2440: The CPU specifies an image of an area as the object image defined / surrounded by the sonar detection results, the first virtual line, and the second virtual line.

[0154] Step 2445: The CPU obtains the masked image by masking the object image in the top view image.

[0155] Step 2450: The CPU obtains the feature points F from the masked image.

[0156] If the vehicle speed SPD is greater than the threshold speed SPDth when the CPU moves to step 2410, the CPU performs a "No" determination in step 2410 and moves to step 2495 to temporarily terminate the current routine.

[0157] If the object is not present when the CPU proceeds to step 2420, the CPU performs a "No" determination in step 2420 and proceeds to step 2450 to obtain the feature points F (from the top view image).

[0158] The CPU is used to execute a command in Fig. The routine shown in Figure 25 is configured or programmed each time a predetermined time elapses. When a suitable time arrives, the CPU starts processing step 2500 and proceeds to step 2505 to determine whether the value of a registry flag Xreg is "1" or not. The value of the registry flag Xreg is set to "1" if the registry start condition is met (in a Fig. (Step 2727 shown in step 2727). The value of the registration flag Xreg is set to “0” when the parking of vehicle 100 in parking space 62 is completed.

[0159] If the CPU determines "yes" in step 2505, it proceeds to step 2510 to determine whether the value of the first parking process flag X1_exe is "0" or not. The value of the first parking process flag X1_exe is set to "1" if the execution of the first parking process has started. The value of the first parking process flag X1_exe is set to "0" if the execution of the first parking process has finished.

[0160] If the CPU performs a determination of "yes" in step 2510, the CPU goes to step 2515 to display the top view image 51P, the parking area line image 52, the setting key image 53 and the displacement key image 57 on the display 50.

[0161] Next, the CPU proceeds to step 2520 to determine whether the value of a setting-complete flag Xset is "1" or not. The value of the setting-complete flag Xset is set to "1" when the setting key image 53 is subjected to touch interaction. The value of the setting-complete flag Xset is set to "0" when the execution of the first parking motion process has started.

[0162] If the CPU determines "Yes" in step 2520, it proceeds to step 2525 to stop displaying the setting key image 53 and the displacement key image 357 on display 50 and displays the registration start key image 54 on display 50. The CPU then proceeds to step 2530 to set the parking area 61 corresponding to the parking area line image 52 as the registration target parking area 61set. Next, the CPU proceeds to step 2535 to set the target movement route Rtgt to a movement route for vehicle 100 to the registration target parking area 61set. Finally, the CPU proceeds to step 2540 to obtain the pre-entry information Ient_pre, as described above, and registers / stores the obtained pre-entry information Ient_pre in RAM.It is noted that the pre-entry coordinates XYent_pre and the pre-entry luminance pattern information CTent_pre, which are contained in the pre-entry information Ient_pre, are obtained based on the feature points F, which are specified in step 2450 of the document. Fig. The routine shown in section 24 will be procured, which is executed immediately before the present time.

[0163] Next, the CPU proceeds to step 2545 to determine whether the value of the registration start flag Xreg_start is "1" or not. The value of the registration start flag Xreg_start is set to "1" when the registration start button image 54 is subjected to touch interaction. Conversely, the value of the registration start flag Xreg_start is set to "0" when the execution of the first parking motion process has started.

[0164] If the CPU executes a "Yes" determination in step 2545, the CPU proceeds to step 2550 to stop displaying the registration start key image 54 on display 50 and displays the camera image 51C and the top-down image 51P on display 50. Next, the CPU proceeds to step 2555 to start executing the first parking movement process to move vehicle 100 to the registration target parking area 61set along the target movement route Rtgt. Next, the CPU proceeds to step 2595 to temporarily terminate the current routine.

[0165] If, on the other hand, the CPU performs a determination of “No” in step 2545, the CPU goes to step 2595 to temporarily terminate the current routine.

[0166] Even if the CPU performs a "No" determination in step 2525, the CPU proceeds to step 2595 to temporarily terminate the current routine.

[0167] If the CPU makes a determination of "No" in step 2510, it proceeds to step 2560 to determine whether the value of an intermediate information gathering flag Xmid is "1" or not. The value of the intermediate information gathering flag Xmid is set to "1" if the CPU predicts that vehicle 100 will continue reversing straight ahead without turning until the CPU completes the parking of vehicle 100 in parking space 62 via the parking assistance control. That is, the value of the intermediate information gathering flag Xmid is set to "0" when the execution of a process from step 2565 is complete.

[0168] If the CPU determines "Yes" in step 2560, it proceeds to step 2565 to obtain the pre-intermediate information Imid_pre as described above and registers / stores the obtained pre-intermediate information Imid_pre in RAM. Next, the CPU proceeds to step 2570. It should be noted that the pre-intermediate coordinates XYmid_pre and the pre-intermediate luminance pattern information CTmid_pre, contained within the pre-intermediate information Imid_pre, are obtained based on the feature points F obtained in step 2450 of the [reference to relevant section]. Fig. The routine shown in section 24 will be procured, which is executed immediately before the present time.

[0169] If, on the other hand, the CPU performs a determination of "No" in step 2560, the CPU goes to step 2570.

[0170] When the CPU proceeds to step 2570, it continues executing the first parking movement process. Next, the CPU proceeds to step 2575 to determine whether the value of the parking-completed flag Xpark_fin is "1" or not. The value of the parking-completed flag Xpark_fin is set to "1" if the entire vehicle 100 has moved into the registration target parking area 61set. Conversely, the value of the parking-completed flag Xpark_fin is set to "0" if the execution of the first parking movement process is complete.

[0171] If the CPU determines "Yes" in step 2575, it proceeds to step 2580 to complete the execution of the first parking motion process. Then, the CPU proceeds to step 2595 to temporarily terminate the current routine.

[0172] If, on the other hand, the CPU makes a determination of "No" in step 2575, the CPU proceeds to step 2595 to temporarily terminate the current routine. If the CPU makes a determination of "No" in step 2505, the CPU proceeds to step 2590 to stop displaying the top view image 51P, and so on on display 50. Next, the CPU proceeds to step 2595 to temporarily terminate the current routine.

[0173] Furthermore, the CPU is used to execute a task in Fig. The routine shown in section 26 is configured or programmed each time the predetermined time expires. When a suitable time arrives, the CPU starts processing a command in Fig. Step 2600, shown in section 26, proceeds to step 2605 to determine whether the value of an information registration request flag Xreg_reg is "1" or not. The value of the information registration request flag Xreg_reg is set to "1" if the parking of vehicle 100 in parking space 62 is completed by the first parking movement process. Conversely, the value of the information registration request flag Xreg_reg is set to "0" if the parking information Ipark is registered in RAM.

[0174] If the CPU determines "Yes" in step 2605, it proceeds to step 2615 to display the registered key image 65 on display 50. Next, the CPU proceeds to step 2615 to determine whether the value of a registration-fixed flag Xreg_det is "1" or not. The value of the registration-fixed flag Xreg_det is set to "1" if the registered key image 55 is subjected to touch interaction. Conversely, the value of the registration-fixed flag Xreg_det is set to "0" if a process from step 2620 is executed.

[0175] If the CPU performs a "Yes" determination in step 2615, the CPU proceeds to step 2620 to register / store the registration input information Ient_reg, the registration interior information Iin_reg, and the registration area information Iarea_reg in RAM as described above as the parking space information Ipark. It is noted that the registration interior coordinates XYin_reg and the registration interior luminance pattern information CTin_reg, which are contained in the parking space information Ipark, are obtained based on the feature points F determined in step 2450 of the process described above. Fig. The routine shown in step 24, which is executed immediately before the present time, is procured. Next, the CPU goes to step 2695 to temporarily terminate the current routine.

[0176] If, on the other hand, the CPU performs a determination of “No” in step 2615, the CPU goes to step 2695 to temporarily terminate the current routine.

[0177] If the CPU performs a determination of "No" in step 2605, the CPU proceeds to step 2695 to temporarily terminate the current routine.

[0178] The CPU is also used to execute a command in Fig. The routine shown in section 27 is configured or programmed each time the predetermined time expires. When a suitable time arrives, the CPU starts processing a command in Fig. The CPU performs step 2700 as shown in step 2705 and proceeds to step 2705 to determine whether a start condition is met. The start condition is met if (i) the vehicle speed SPD is "0" and (ii) the driver operates the parking assist switch 48. If the start condition is met, the CPU performs a "Yes" determination in step 2705 and proceeds to step 2710.

[0179] In step 2710, the CPU obtains the masked image (the left masked image) of the left top-down image and the masked image (the right masked image) of the right top-down image, which were obtained in step 2445 of the process. Fig. The routine shown in step 24, which is executed immediately before the present time, is obtained, and the process proceeds to step 2715. In step 2715, the CPU determines whether the image sections with the same luminance patterns as the luminance patterns of the registered input luminance pattern information CTent_reg are contained in the left masked image and / or the right masked image.

[0180] If the image sections with the same luminance patterns as the luminance patterns of the registered input luminance pattern information CTent_reg are contained in the left masked image and / or the right masked image, the CPU determines that (i) vehicle 100 stops at input 62ent of registered parking space 62, and the automatic parking start condition is met. Then, in step 2715, the CPU performs a "yes" determination and proceeds to step 2720 to set the value of an assistance flag Xassist to "1". Afterward, the CPU proceeds to step 2795 to temporarily terminate the current routine.

[0181] If no image segments with the same luminance patterns as the luminance patterns of the registered input luminance pattern information CTent_reg are found in the left masked image and the right masked image, the CPU determines that (i) there is no registered parking space 62 near vehicle 100, and (ii) the registration start condition is met. Then, in step 2715, the CPU performs a "No" determination and proceeds to step 2725 to set the registration flag Xreg to "1". Afterward, the CPU proceeds to step 2795 to temporarily terminate the current routine.

[0182] If, on the other hand, the start condition is not met when the CPU moves to step 2705, the CPU executes a "No" determination in step 2705 and goes to step 2795 to temporarily terminate the current routine.

[0183] In step 2715, the CPU can execute a "Yes" determination if (i) the image sections with the same luminance patterns as the luminance patterns of the registered input luminance pattern information CTent_reg are contained in the left masked image and / or the right masked image, and (ii) a positional relationship between the input feature points Fent matches a positional relationship between the image sections with the same luminance patterns as the luminance patterns of the registered input luminance pattern information CTent_reg.

[0184] Furthermore, the CPU is used to execute a task in Fig. The routine shown in section 28 is configured or programmed each time the predetermined time expires. When a suitable time arrives, the CPU starts processing a command in Fig. Step 2800, as shown in section 28, is then performed, and the process continues to step 2805 to determine whether the value of an assistance flag Xassist is "1" or not. The value of the assistance flag Xassist is set to "1" when vehicle 100 stops at entrance 63 of registered parking space 62. Conversely, the value of the assistance flag Xassist is set to "0" when the vehicle moves away from registered parking space 63, or when parking of vehicle 100 in parking space 62 is complete.

[0185] If the CPU determines "Yes" in step 2805, it proceeds to step 2810 to determine whether the value of the second parking process flag X2_exe is "0" or not. The value of the second parking process flag X2_exe is set to "1" if the execution of the second parking process has started. Conversely, the value of the second parking process flag X2_exe is set to "0" if the execution of the second parking process has finished.

[0186] If the CPU performs a determination of “Yes” in step 2810, the CPU goes to step 2815 to display the camera image 51C, the top view image 51P, the parking area line image 52 and the parking start button image 56 on the display 50.

[0187] Next, the CPU proceeds to step 2820 to determine whether the value of the park start flag Xpark_start is "1" or not. The value of the park start flag Xpark_start is set to "1" when the park start button image 56 is subjected to touch interaction. Conversely, the value of the park start flag Xpark_start is set to "0" when the execution of the second park movement process has started.

[0188] If the CPU determines "Yes" in step 2820, it proceeds to step 2825 to stop displaying the parking start button image 56 on display 50. Next, the CPU proceeds to step 2830 to set the target parking area 61tgt to parking area 61, which corresponds to the parking area line image 57. Then, the CPU proceeds to step 2825 to set the target movement route Rtgt to a movement route to move vehicle 100 to the target parking area 61tgt. Next, the CPU proceeds to step 2840 to start executing the second parking movement process. Finally, the CPU proceeds to step 2895 to temporarily terminate the current routine.

[0189] If, on the other hand, the CPU makes a determination of “No” in step 2820, the process proceeds to step 2895 to temporarily terminate the current routine.

[0190] If the CPU determines "No" in step 2810, it proceeds to step 2845 to execute the second parking movement process. Next, the CPU goes to step 2850 to determine whether the value of the parking-completed flag Xpark_fin is "1" or not. The value of the parking-completed flag Xpark_fin is set to "1" if the entire vehicle 100 has moved into the target parking area 61tgt. Conversely, the value of the parking-completed flag Xpark_fin is set to "0" if the execution of the second parking movement process is complete.

[0191] If the CPU executes a "Yes" determination in step 2850, the CPU proceeds to step 2855 to terminate the execution of the second parking motion process. Then, the CPU proceeds to step 2895 to temporarily terminate the current routine.

[0192] If, on the other hand, the CPU executes a determination of “No” in step 2850, the CPU goes to step 2895 to temporarily terminate the current routine.

[0193] If the CPU executes a determination of "No" in step 2805, the CPU goes to step 2860 to stop displaying the top view image 51P and so on on display 50. Then the CPU goes to step 2895 to temporarily terminate the current routine.

[0194] The specific operation of the vehicle parking assistance device 10 has been described. According to the vehicle parking assistance device 10, the information is not registered / stored via the feature points of the objects in and / or around the parking space 62, but rather via the feature points F of the ground 63 in and / or around the parking space 62 as the registration input information Ient_reg (compare step 2020 in Fig. 20) This makes it possible to reliably determine that the entry feature points Fent acquired at that time are the registration entry feature points Fent_reg when vehicle 100 arrives at entrance 62ent of registered parking space 62, if the situation around vehicle 100 and parking space 62 changes between the time the registration entry information Ient_reg was recorded and the time vehicle 100 arrives at entrance 62ent of registered parking space 62 this time. Consequently, it can be reliably determined that parking space 62, where vehicle 100 arrives this time, is registered parking space 62. Thus, vehicle 100 can be parked autonomously in registered parking space 62.

[0195] The present disclosure is not limited to the foregoing embodiment, and various modifications may be applied within the scope of protection of the present disclosure.

[0196] In the preceding embodiment, the distance sonares 301 to 312 detect the object. The vehicle parking assistance device 10 can detect the object in another way. For example, the CPU obtains an image with a predetermined feature point and a predetermined feature size from the camera image, which is captured by each of the cameras 45 every time the predetermined time expires, as a defining feature point. The CPU uses the defining feature point to detect the object from the camera image.If the first feature point acquired at a first time point is the same as the second feature point acquired at a second time point, the CPU estimates a motion route from the first time point to the second time point based on the vehicle speed Vs and the yaw rate Yr to estimate a position (a second position) of vehicle 100 at the second time point relative to a position (a first position) of vehicle 100 at the first time point. The CPU specifies a position and height of the feature point relative to the camera based on the second time point, the position of the first feature point in the camera image acquired at the first time point, and the position of the second feature point in the camera image acquired at the second time point.If the height is equal to or greater than a predetermined threshold height, the CPU determines that the object exists.

[0197] The distance sonars can be replaced by any other sensor type, as long as each is configured to emit a radio wave (for example, an infrared beam or a radar wave) and to receive a reflected radio wave to detect an object. That is, the distance sonars can be replaced by infrared radar sensors and / or millimeter-wave band radar sensors. Alternatively, the infrared beam radar sensors and / or millimeter-wave band radar sensors can be used in addition to the distance sonars.

[0198] The number of camera sensors (21) and the number of distance sonars are not based on those in the Fig. 2 and Fig. The number described is limited to 3.

[0199] A vehicle parking assistance device obtains a feature image from a registration image representing a parking space to register the feature image as parking information. The registration image is an image captured by a camera when the vehicle parking assistance device receives a registration request to register a parking area in which the vehicle is parked. The vehicle parking assistance device determines that the vehicle has reached the parking area specified by the parking information when a portion of an image containing the same feature as the feature image is included in a subsequent registration image. The vehicle parking assistance device specifies an object image, which is an image representing an object, from the registration image if the object is present in the area corresponding to the registration image, and obtains the feature image based on the registration image from which the specified object image is excluded.< / sensoren>

Claims

[1] Vehicle parking assist device with a camera (40, 41-44) configured to capture an image representing an area surrounding a vehicle, and a control device (90, 11, 12, 13) which is configured to register information about a parking space as parking information, which contains a parking area in which the vehicle will be parked, based on a registration image which is an image representing the parking space which the camera captures when the control device receives a registration request to register the parking area from a driver of the vehicle, and autonomous parking of the vehicle in the parking area based on the parking information, when it is determined that the vehicle has reached the parking space specified by the parking information after the parking information has been registered, where the control unit is set up for Obtaining a feature image, which is an image of a predetermined area size and has a specific feature, from the registration image (step 2450) to register the feature image as the parking information (step 2545, step 2565, step 2620), and Determine that the vehicle has reached the parking area specified by the parking information (step 2720) when an image segment containing the same feature as the feature image is included in a post-registration image (“Yes” in step 2175) in order to autonomously park the vehicle in the parking area in the parking lot based on the parking information, wherein the post-registration image is an image of an area captured by the camera after the control unit has registered the parking information. and whereby the control unit is further set up to Specifying an object image, which is an image representing an object, from the registration image (step 2440), if the object is present in the area corresponding to the registration image (step 2420), and Procurement of the feature image based on the registration image, from which the specified object image is excluded (step 2540, step 2565, step 2620). [2] Vehicle parking assistance device according to claim 1, further comprising a detection sensor (30, 301-312) which is configured to emit a radio carrier for detecting the object by receiving the radio carrier reflected by the object, the control unit is set up to Determine that the object is present in the area corresponding to the registration image (“Yes” in step 2420), in a case where the detection sensor detects the object when the control unit receives the registration request, and Specifying the object image from the registration image based on the object detected by the detection sensor (steps 2425-2445). [3] Vehicle parking assistance device according to claim 2, wherein the detection sensor is configured to emit the radio carrier to a predetermined radiation area in which a central axis is present, the control unit is set up to Obtaining a top-down image, wherein the top-down image is an image when an image captured by the camera is viewed from an observation point positioned above the camera (step 2415), Procuring a virtual line between a furthest point of capture results representing the object captured by the capture sensor in the top view image and the camera, wherein the furthest point is a point that has the greatest distance to the camera among the capture results (step 2430), Obtaining a first virtual line extending from the furthest point in the same direction as the virtual line, where the direction is a direction away from the camera (step 2430), Procuring a second virtual line extending from a nearest point of the acquisition results in a direction parallel to the central axis of the radiation area of ​​the acquisition sensor, the nearest point being a point that has the shortest distance to the camera (step 2435), and Determining an image of an area as the object image defined by the acquisition results, the first virtual line, and the second virtual line (step 2440). [4] Vehicle parking assistance device according to claim 3, wherein the detection sensor and the camera are configured for mounting on the vehicle such that a direction of the central axis of the radiation area of ​​the detection sensor coincides with a direction of a central axis of a recording area of ​​the camera. [5] Vehicle parking assistance device according to claim 1, the control unit is set up to Specifying the object image, which is an image representing the object, from the post-registration image (step 2440), if the object exists in an area corresponding to the post-registration image ("Yes" in step 2420), and Determine whether an image different from the specified object image in the post-registration image contains the feature image or not (step 2710, step 2715).

Images