Parking Assistance System
The parking assistance system addresses the issue of undetectable registered parking lots by displaying current and registered images side by side, providing clear guidance for vehicle alignment, thus improving convenience and enabling effective parking assistance.
Patent Information
- Application Number
- JP2021150871
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-09-16
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2041-09-16
AI Technical Summary
Conventional parking assistance systems fail to detect registered parking lots when the vehicle's current location significantly differs from the location at the time of registration, leading to inadequate guidance for the driver to align the vehicle position, thus hindering effective parking assistance control.
The system includes a control device that collates the current surroundings captured by an imaging device with registered parking lot images, displaying them side by side on a display if detection fails, and uses position information to determine if the vehicle is within a predetermined distance range, providing appropriate guidance to the driver.
This approach enhances convenience by clearly indicating to the driver when the vehicle is far from the registered position, allowing for easier alignment and enabling reliable parking assistance control.
Smart Images

Figure 0007578084000001 
Figure 0007578084000002 
Figure 0007578084000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a parking assistance system for a vehicle. [Background technology]
[0002] Conventionally, there is known a parking assistance system in which an image of a parking lot taken by a camera mounted on the vehicle is registered in advance, and the next time the vehicle is parked in that parking lot, the system automatically parks the vehicle in the registered parking lot by determining the relative position of the vehicle and the parking lot based on feature points of the image of the surroundings of the vehicle taken by the camera at that time and feature points of the registered parking lot image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2020-117128 A Summary of the Invention
[0004] When the above system receives a parking assistance request from the driver, it performs a matching process between the image of the surroundings of the vehicle captured by the camera at that time and the registered parking lot image, and starts parking assistance control if it can detect a parking space in the registered parking lot. Therefore, if the current position of the vehicle when the parking assistance request is received is significantly away from the parking position of the vehicle when the parking lot was registered in the system, specifically, the position when the vehicle was parked near the entrance of the parking lot and the registration operation was performed (hereinafter, the vehicle position at the time of registration), it is not possible to detect the parking space in the parking lot and it is not possible to start parking assistance control.
[0005] In such a situation, in order for the parking assistance system to start parking assistance control, the driver needs to align the current position of the vehicle with the registered vehicle position. However, the system is unable to detect parking spaces in registered parking lots, and is unable to properly inform the driver of where to move the vehicle to align it with the registered vehicle position, so there is room for improvement in convenience.
[0006] The present invention has been made to solve the above problems. That is, one of the objects of the present invention is to provide a parking assistance system that can improve convenience when a registered parking lot cannot be detected.
[0007] The parking assistance system (10) of the present invention comprises: an imaging device (45) mounted on the vehicle (100) for capturing an image (51P) of the surroundings of the vehicle (100) and a parking lot image (60) of the parking lot (62) for use in automatically parking the vehicle (100) in the parking lot; a control device (90) that registers the parking lot image (62) captured by the imaging device (45) as parking lot information for the parking lot, and performs parking assistance control to automatically park the vehicle (100) in the parking lot (62) based on the parking lot information when a parking assistance request is received from a driver; and and a display device (50) capable of displaying an image captured by the imaging device (45).
[0008] The control device (90) The surrounding image (51P) captured by the imaging device (45) is compared with the parking lot image (60) registered as the parking lot information, and the parking assistance control is started when the parking lot (62) is detected by the comparison process. If the parking lot (62) cannot be detected by the matching process, the parking lot image (60) registered as the parking lot information and the current surrounding image (51P) captured by the imaging device (45) are displayed on the display device (50).
[0009] According to the above configuration, when a registered parking lot (62) cannot be detected, a registered parking lot image (60) and an image (51P) of the surroundings of the vehicle (100) are displayed on the display device (50). This makes it possible to inform the driver that the current position of the vehicle (100) is far from the vehicle position (P) at the time of registration in a situation where parking assistance control cannot be started, thereby effectively improving convenience.
[0010] In another embodiment of the present invention, The vehicle further includes a position information acquisition device (30) for acquiring position information of the vehicle (100), The control device (90) The parking lot image (60) captured by the imaging device (45) is associated with location information acquired by the location information acquisition device (30) when the image is captured, and registered as the parking lot information of the parking lot (62); If the parking lot (62) cannot be detected by the matching process, a determination is made as to whether the current position of the vehicle (100) is within a predetermined distance range from the parking lot (62) based on the current position information of the vehicle (100) acquired by the position information acquisition device (30) and the position information of the parking lot (62) registered as the parking lot information; When it is determined that the current position of the vehicle (100) is within a predetermined distance range from the parking lot (62), the parking lot image (60) registered as the parking lot information and the current surrounding image (51P) captured by the imaging device (45) are displayed on the display device (50), whereas when it is determined that the current position of the vehicle (100) is not within a predetermined distance range from the parking lot (62), a message image indicating that the parking assistance control cannot be executed is displayed on the display device (50).
[0011] According to this embodiment, different images are displayed on the display device (50) when the vehicle (100) is within a predetermined distance range from the registered parking lot (62) and when the vehicle (100) is farther away than the predetermined distance range from the registered parking lot (62). This makes it possible to present the driver with appropriate information according to the distance between the current position of the vehicle (100) and the registered parking lot (62), thereby making it possible to more reliably improve convenience.
[0012] In another embodiment of the present invention, The control device (90) The parking lot images (60) of a plurality of parking lots (62) captured by the imaging device (45) are associated with location information acquired by the location information acquisition device (30) when the images are captured, and are registered as parking lot information of each parking lot, If none of the multiple parking lots (62) can be detected by the matching process, the parking lot (62) closest to the current position of the vehicle (100) is identified among the multiple parking lots (62) based on the current position information of the vehicle (100) acquired by the position information acquisition device (30) and the position information of the multiple parking lots (62) registered as the parking lot information, and if it is determined that the current position of the vehicle (100) is within a predetermined distance range from the identified parking lot (62), a parking lot image (60) of the identified parking lot (62) and the current surrounding image (51P) captured by the imaging device (45) are displayed on the display device (50).
[0013] According to this embodiment, the parking lot image (60) of the parking lot (62) that is located closest to the current position of the vehicle (100) among the multiple parking lots (62) is displayed on the display device (50). This makes it possible to effectively present to the driver the parking lot (62) that is most suitable for the current position of the vehicle (100) even when multiple parking lots (62) are registered.
[0014] In the above description, in order to facilitate understanding of the invention, the symbols used in the embodiments are added in parentheses to the constituent elements of the invention corresponding to the embodiments, but each constituent element of the invention is not limited to the embodiment defined by the said symbols. [Brief description of the drawings]
[0015] [Figure 1] 1 is a diagram showing a parking assistance system according to an embodiment of the present invention and a vehicle to which the parking assistance system is applied; [Diagram 2] FIG. 2 is a diagram showing the arrangement and detection range of a sonar sensor. [Diagram 3] FIG. 2 is a diagram showing the arrangement and imaging range of a camera sensor. [Figure 4] FIG. 1 is a diagram showing an example of a parking lot. [Diagram 5] FIG. 2 shows the front and rear ranges. [Figure 6] FIG. 2 is a diagram showing a left range and a right range. [Figure 7] FIG. 13 is a diagram showing feature points. [Figure 8] FIG. [Figure 9] FIG. [Figure 10] FIG. 2 is a diagram for explaining the operation of the parking assistance system shown in FIG. [Figure 11] FIG. 2 is a diagram for explaining the operation of the parking assistance system shown in FIG. [Figure 12] FIG. 2 is a diagram for explaining the operation of the parking assistance system shown in FIG. [Figure 13] FIG. 13 is a diagram showing entry feature points. [Figure 14] FIG. 2 is a diagram for explaining the operation of the parking assistance system shown in FIG. [Figure 15] FIG. 2 is a diagram for explaining the operation of the parking assistance system shown in FIG. [Figure 16] FIG. 2 is a diagram for explaining the operation of the parking assistance system shown in FIG. [Figure 17]FIG. 2 is a diagram for explaining the operation of the parking assistance system shown in FIG. [Figure 18] FIG. [Figure 19] 2 is a flowchart showing a display control routine executed by the ECU shown in FIG. 1. [Figure 20] FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0016] Hereinafter, the parking assistance system according to the present embodiment will be described with reference to the drawings. The same components are given the same reference numerals, and their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. In the following, the parking assistance system may also be referred to as the "embodiment device".
[0017] [Overall configuration] FIG. 1 shows a parking assistance system 10 according to this embodiment and a vehicle 100 to which the parking assistance system 10 is applied.
[0018] As shown in Fig. 1, the parking assistance system 10 includes an ECU 90 (control device). ECU is an abbreviation for Electronic Control Unit. The ECU 90 includes a microcomputer as a main part. The microcomputer includes a CPU, a ROM, a RAM, a non-volatile memory, an interface, and the like. The CPU is configured to realize various functions by executing instructions, programs, or routines stored in the ROM.
[0019] The vehicle 100 is equipped with a drive device 11, a steering device 12, and a braking device 13. The drive device 11 generates a driving force to be transmitted to the driving wheels of the vehicle 100. Examples of the drive device 11 include an electric motor and an engine. The vehicle 100 may be any of an engine vehicle, a hybrid vehicle (HEV), a plug-in hybrid vehicle (PHEV), a fuel cell vehicle (FCEV), and an electric vehicle (BEV). The steering device 12 is, for example, an electric power steering device, and applies a steering force to the wheels of the vehicle 100. The braking device 13 is, for example, a disc brake device, and applies a braking force to the wheels of the vehicle 100.
[0020] The drive device 11, the steering device 12, and the braking device 13 are electrically connected to the ECU 90. The ECU 90 controls the operation of the drive device 11 to control the drive force applied to the driving wheels of the vehicle 100. The ECU 90 also controls the operation of the steering device 12 to control the steering torque applied to the steered wheels of the vehicle 100. The ECU 90 also controls the operation of the braking device 13 to control the braking force applied to the wheels of the vehicle 100.
[0021] The ECU 90 is connected to a parking assist switch 15, a vehicle state acquisition device 20, a position information acquisition device 30, a surrounding recognition device 40, a display 50 (display device), and the like.
[0022] The parking assist switch 15 is a switch that is turned ON / OFF by the driver. When the driver turns the parking assist switch 15 ON, the ECU 90 accepts a parking assist request.
[0023] The vehicle state acquisition device 20 is a group of sensors that acquires the state of the vehicle 100. Specifically, the vehicle state acquisition device 20 includes an accelerator sensor 21, a brake sensor 22, a steering angle sensor 23, a steering torque sensor 24, a vehicle speed sensor 25, and the like.
[0024] The accelerator sensor 21 detects the amount of operation of an accelerator pedal (not shown) of the vehicle 100. The brake sensor 22 detects the amount of operation of a brake pedal (not shown) of the vehicle 100. The steering angle sensor 23 detects the steering angle θst of a steering wheel (or steering shaft) (not shown) of the vehicle 100. The steering torque sensor 24 detects the steering torque TQst acting on a steering shaft (not shown) of the vehicle 100 due to operation of the steering wheel. The vehicle speed sensor 25 detects the traveling speed (vehicle speed) SPD of the vehicle 100. Various state information of the vehicle 100 acquired by the vehicle state acquisition device 20 is transmitted to the ECU 90.
[0025] The position information acquisition device 30 acquires current position information of the vehicle 100. For example, a global positioning system (GPS) or a global navigation satellite system (GNSS) can be used as the position information acquisition device 30. The current position information of the vehicle 100 acquired by the position information acquisition device 30 is transmitted to the ECU 90.
[0026] The surrounding recognition device 40 is a type of sensor that recognizes targets around the vehicle 100. Specifically, the surrounding recognition device 40 includes a sonar sensor 41 and a camera sensor 44 (imaging device). Target information around the vehicle 100 acquired by the surrounding recognition device 40 is transmitted to the ECU 90.
[0027] Fig. 2 is a diagram showing the arrangement and detection range of the sonar sensor 41. The sonar sensor 41 is equipped with a first clearance sonar 411 to a twelfth clearance sonar 422. In Fig. 2, the direction indicated by the symbol Dx is the front-rear direction of the vehicle 100, and the direction indicated by the symbol Dw is the width direction of the vehicle 100.
[0028] 2, the first clearance sonar 411 is attached to the vehicle 100 so as to radiate sound waves from the front left end of the vehicle 100 to the left front. The second clearance sonar 412 is attached to the vehicle 100 so as to radiate sound waves forward from the left front end of the vehicle 100. The third clearance sonar 413 is attached to the vehicle 100 so as to radiate sound waves from the front right end of the vehicle 100 to the right front. The fourth clearance sonar 414 is attached to the vehicle 100 so as to radiate sound waves forward from the right front end of the vehicle 100.
[0029] The fifth clearance sonar 415 is attached to the vehicle 100 so as to radiate sound waves from the rear left end of the vehicle 100 to the left rear. The sixth clearance sonar 416 is attached to the vehicle 100 so as to radiate sound waves from the rear left end of the vehicle 100 to the rear. The seventh clearance sonar 417 is attached to the vehicle 100 so as to radiate sound waves from the rear right end of the vehicle 100 to the right rear. The eighth clearance sonar 418 is attached to the vehicle 100 so as to radiate sound waves from the rear right end of the vehicle 100 to the rear.
[0030] The ninth clearance sonar 419 is attached to the vehicle 100 so as to radiate sound waves to the left from the front left side of the vehicle 100. The tenth clearance sonar 420 is attached to the vehicle 100 so as to radiate sound waves to the left from the rear left side of the vehicle 100. The eleventh clearance sonar 421 is attached to the vehicle 100 so as to radiate sound waves to the right from the front right side of the vehicle 100. The twelfth clearance sonar 422 is attached to the vehicle 100 so as to radiate sound waves to the right from the rear right side of the vehicle 100.
[0031] The first clearance sonar 411 to the twelfth clearance sonar 422 receive sound waves reflected by objects. The sonar sensor 41 is electrically connected to the ECU 90. The sonar sensor 41 transmits information relating to the sound waves emitted by the first clearance sonar 411 to the twelfth clearance sonar 422 and the sound waves received by the first clearance sonar 411 to the twelfth clearance sonar 422 to the ECU 90. The ECU 90 acquires information relating to objects present around the vehicle 100 as object information OBJ based on the information received from the sonar sensor 41 (hereinafter referred to as "sonar information SON").
[0032] 3 is a diagram showing the arrangement and imaging range of the camera sensor 44. The camera sensor 44 captures images of the surroundings of the vehicle 100 and processes the captured image data to obtain an image of the surroundings of the vehicle 100. The camera sensor 44 is, for example, a stereo camera or a monocular camera, and a digital camera having an imaging element such as a CMOS or a CCD can be used.
[0033] The camera sensor 44 includes a front camera 44a, a rear camera 44b, a left side camera 44c, and a right side camera 44d. When there is no need to distinguish between the multiple cameras 44a to 44d, they are simply referred to as "camera 45."
[0034] The front camera 44a is provided at approximately the center in the vehicle width direction at the front of the vehicle 100. The optical axis of the front camera 44a faces forward of the vehicle 100. The rear camera 44b is provided at the rear of the vehicle 100. The optical axis of the rear camera 44b faces rearward of the vehicle 100. The left side camera 44c is provided on the left side of the vehicle 100. The optical axis of the left side camera 44c faces leftward of the vehicle 100. The right side camera 44d is provided on the right side of the vehicle 100. The optical axis of the right side camera 44d faces rightward of the vehicle 100.
[0035] The camera 45 has a wide angle of view. Therefore, the imaging range of the camera 45 includes "ranges to the right, left, below, and above" based on the optical axis. The entire periphery of the vehicle 100 is included in the imaging range of the four cameras 44a to 44d. The camera 45 captures an image of the area around the vehicle 100 corresponding to the imaging range every time a predetermined time elapses, thereby acquiring image information (image data). The camera 45 transmits the acquired image data to the ECU 90.
[0036] Specifically, the front camera 44a captures an image of the surrounding area in front of the vehicle 100 and transmits the generated image data (hereinafter referred to as "front image data IMG1") to the ECU 90. The back camera 44b captures an image of the surrounding area behind the vehicle 100 and transmits the generated image data (hereinafter referred to as "rear image data IMG2") to the ECU 90. The left side camera 44c captures an image of the surrounding area on the left side of the vehicle 100 and transmits the generated image data (hereinafter referred to as "left side image data IMG3") to the ECU 90. The right side camera 44d captures an image of the surrounding area on the right side of the vehicle 100 and transmits the generated image data (hereinafter referred to as "right side image data IMG4") to the ECU 90. In the following, the front image data IMG1, rear image data IMG2, left side image data IMG3, and right side image data IMG4 will be collectively referred to as "image data IMG" as necessary.
[0037] When a predetermined condition is met, the parking assistance system 10 acquires a characteristic point F based on the image data IMG. The characteristic point F is an image in a predetermined range in which the brightness of the image captured by the camera 45 changes significantly.
[0038] For example, when camera 45 captures an image of parking lot 62 shown in Figure 4, the corners of concrete cover member 63B, the corners of ground 63 made of grass 63L, and the boundary between ground 63 made of cover member 63B and ground 63 made of grass 63L are acquired as feature points F.
[0039] The ground 63 of the parking lot 62 shown in Fig. 4 is made up of a ground 63 made of concrete 63C and a ground 63 made of grass 63L. In addition, at the entrance 62ent of the parking lot 62, a plurality of concrete cover members 63B that cover the side gutter are installed side by side. Therefore, the ground 63 at the entrance 62ent of the parking lot 62 is made up of the surface of the cover members 63B.
[0040] The parking assistance system 10 acquires a feature point F (hereinafter, "front feature point F1") on the ground 63 in a predetermined range 71 (see FIG. 5) in front of the vehicle 100 based on the front image data IMG1. The parking assistance system 10 also acquires a feature point F (hereinafter, "rear feature point F2") on the ground 63 in a predetermined range 72 (see FIG. 5) behind the vehicle 100 based on the rear image data IMG2. The parking assistance system 10 also acquires a feature point F (hereinafter, "left side feature point F3") on the ground 63 in a predetermined range 73 (see FIG. 6) on the left side of the vehicle 100 based on the left side image data IMG3. The parking assistance system 10 acquires a feature point F (hereinafter, "right side feature point F4") on the ground 63 in a predetermined range 74 (see FIG. 6) on the right side of the vehicle 100 based on the right side image data IMG4.
[0041] As shown in FIG. 5, the predetermined range 71 is a range defined by lines L711, L712, L713, and L714. The line L711 is a line L711 extending in the vehicle width direction Dy from the front camera 44a forward of the vehicle 100 at a distance Dset. The line L712 is a line L712 extending in the vehicle width direction Dy through the front camera 44a. The line L713 is a line extending in the vehicle front-rear direction Dx from the front camera 44a to the left of the vehicle 100 at a distance Dset. The line L714 is a line extending in the vehicle front-rear direction Dx from the front camera 44a to the right of the vehicle 100 at a distance Dset. Hereinafter, the predetermined range 71 is referred to as the "forward range 71".
[0042] The front area 71 is divided into eight areas 71D, for example, divided into four equal areas in the vehicle width direction Dy and into two equal areas in the vehicle front-rear direction Dx. That is, the front area 71 is divided into a plurality of areas 71D having equal areas. Hereinafter, these areas 71D are referred to as "front divided areas 71D." Among the front divided areas 71D, the two front divided areas 71D set at the left end in the vehicle width direction Dy are referred to as "left end divided area 71D3," the two front divided areas 71D set at the right end in the vehicle width direction Dy are referred to as "right end divided area 71D4," and the four front divided areas 71D set in the middle in the vehicle width direction Dy are referred to as "middle divided area 71D5."
[0043] As shown in FIG. 5, the predetermined range 72 is a range defined by lines L721, L722, L723, and L724. The line L721 is a line that passes through the rear camera 44b and extends in the vehicle width direction Dy. The line L722 is a line that extends in the vehicle width direction Dy from the rear camera 44b to the rear of the vehicle 100 at a distance of a predetermined distance Dset. The line L723 is a line that extends in the vehicle front-rear direction Dx from the rear camera 44b to the left of the vehicle 100 at a distance of a predetermined distance Dset. The line L724 is a line that extends in the vehicle front-rear direction Dx from the rear camera 44b to the right of the vehicle 100 at a distance of a predetermined distance Dset. Hereinafter, the predetermined range 72 is referred to as the "rear range 72".
[0044] The rear area 72 is divided into eight areas 72D, which are divided into four equal areas in the vehicle width direction Dy and into two equal areas in the vehicle front-rear direction. That is, the rear area 72 is divided into a plurality of areas 72D having equal areas. Hereinafter, these areas 72D are referred to as "rear divided areas 72D." Among the rear divided areas 72D, the two rear divided areas 72D set at the left end in the vehicle width direction Dy are referred to as "left end divided area 72D3," the two rear divided areas 72D set at the right end in the vehicle width direction Dy are referred to as "right end divided area 72D4," and the four rear divided areas 72D set in the middle in the vehicle width direction Dy are referred to as "middle divided area 72D5."
[0045] As shown in FIG. 6, the predetermined range 73 is a range defined by lines L731, L732, L734, and L733. The line L731 is a line extending in the vehicle width direction Dy from the left side camera 44c forward of the vehicle 100 at a distance Dset. The line L732 is a line extending in the vehicle width direction Dy from the left side camera 44c rearward of the vehicle 100 at a distance Dset. The line L733 is a line extending in the vehicle front-rear direction Dx from the left side camera 44c to the left of the vehicle 100 at a distance Dset. The line L734 is a line extending in the vehicle front-rear direction Dx through the left side camera 44c. Hereinafter, the predetermined range 73 is referred to as the "left side range 73".
[0046] The left area 73 is divided into eight areas 73D, which are divided into four equal parts in the vehicle front-rear direction Dx and two equal parts in the vehicle width direction Dy. That is, the left area 73 is divided into a plurality of areas 73D having equal areas. Hereinafter, these areas 73D are referred to as "left divided area 73D." Among the left divided area 73D, the two left divided area 73D set at the front end in the vehicle front-rear direction Dx are referred to as "front end divided area 73D1," the two left divided area 73D set at the rear end in the vehicle front-rear direction Dx are referred to as "rear end divided area 73D2," and the four left divided area 73D set in the middle in the vehicle front-rear direction Dx are referred to as "middle divided area 73D5."
[0047] As shown in FIG. 6, the predetermined range 74 is a range defined by lines L741, L742, L743, and L744. The line L741 is a line extending in the vehicle width direction Dy from the right side camera 44d forward of the vehicle 100 at a distance Dset. The line L742 is a line extending in the vehicle width direction Dy from the right side camera 44d rearward of the vehicle 100 at a distance Dset. The line L743 is a line extending in the vehicle front-rear direction Dx through the right side camera 44d. The line L744 is a line extending in the vehicle front-rear direction Dx from the right side camera 44d to the right of the vehicle 100 at a distance Dset. Hereinafter, the predetermined range 74 is referred to as the "right side range 74".
[0048] The right side area 74 is divided into eight areas 74D, which are divided into four equal areas in the vehicle front-rear direction Dx and two equal areas in the vehicle width direction Dy. That is, the right side area 74 is divided into a plurality of areas 74D having equal areas. Hereinafter, these areas 74D are referred to as "right side divided areas 74D." Among the right side divided areas 74D, the two right side divided areas 74D set at the front end in the vehicle front-rear direction Dx are referred to as "front end divided areas 74D1," the two right side divided areas 74D set at the rear end in the vehicle front-rear direction Dx are referred to as "rear end divided areas 74D2," and the four right side divided areas 74D set in the middle in the vehicle front-rear direction Dx are referred to as "middle divided areas 74D5."
[0049] When an image of a range corresponding to the characteristic point F captured by the camera 45 is converted into an image in a plan view, the converted image of the range corresponding to the characteristic point F is an image of a square range 75 with a side of a predetermined length Lset, as shown in FIG. 7. When a predetermined condition is met, the parking assistance system 10 divides the characteristic point F into 25 equal square ranges 75D and acquires the luminance LUM of each range 75D. Then, the parking assistance system 10 calculates a value ΔLUM (=LUM-LUMave) by subtracting the average value LUMave of the acquired luminance LUM from each luminance LUM, and acquires the tendency of the high and low luminance LUM at the characteristic point F as shading information CT based on these values ΔLUM. That is, when a predetermined condition is met, the parking assistance system 10 acquires the shading pattern in the image of the characteristic point F captured by the camera 45 as shading information CT.
[0050] 1, the display 50 is electrically connected to the ECU 90. The display 50 is disposed in a location of the vehicle 100 that is visible to the driver. In this embodiment, the display 50 is, for example, a display of a navigation device.
[0051] The ECU 90 can display various images on the display 50. In this embodiment, the ECU 90 can display at least a plan view image 51P, a parking space line image 52, a setting button image 53, a registration start button image 54, a registration button image 55, a parking start button image 56, a movement button image 57, and a parking lot image 60 on the display 50.
[0052] The planar view image 51P is an image including a vehicle planar view image and a vehicle surroundings image. The vehicle planar view image is an image that displays the vehicle 100 when the vehicle 100 is viewed from above in the vertical direction. The vehicle surroundings image is an image that displays the surroundings of the vehicle 100 when the surroundings of the vehicle 100 are viewed from above in the vertical direction. The vehicle planar view image and the vehicle surroundings image are generated by the ECU 90 based on the image data IMG.
[0053] The parking section line image 52 is an image that displays a parking section. A parking section is a section (or a range or an area) where the vehicle 100 is parked by parking assistance control, which will be described later. As shown in FIG. 8, a parking section 61 is set in a parking lot 62.
[0054] The setting button image 53 is an image corresponding to a setting button that is touched by the driver to set (confirm) the parking space 61 in which the vehicle 100 is to be parked in the parking assist control described later.
[0055] The registration start button image 54 is an image corresponding to a registration start button that is touched by the driver to start the registration time parking and driving process of the parking assist control described later.
[0056] The parking start button image 56 is an image corresponding to the parking start button that is touched by the driver to start the automatic parking and driving process described later when parking the vehicle 100 in a parking space 61 registered in the parking assistance system 10.
[0057] The registered button images 55 include a first registered button image 551, a second registered button image 552, and a third registered button image 553.
[0058] The parking assistance system 10 can register (i.e., store) multiple parking lot information Ipark that differs depending on the parking assistance control described below. The parking lot information Ipark is information about a parking lot that is used by the parking assistance system 10 to automatically park the vehicle 100 in the parking lot. In this embodiment, the parking assistance system 10 can register up to three parking lot information Ipark. Hereinafter, these parking lot information Ipark are referred to as "first parking lot information Ipark_1," "second parking lot information Ipark_2," and "third parking lot information Ipark_3," respectively. The number of parking lot information Ipark that can be registered is not limited to three, and may be two, or four or more.
[0059] The first registration button image 551 to the third registration button image 553 are images that are touched by the driver to register the parking lot information Ipark, acquired by the parking assistance control described below, in the parking assistance system 10. When the first registration button image 551 is touched by the driver, the parking lot information Ipark is registered in the parking assistance system 10 as the first parking lot information Ipark_1. When the second registration button image 552 is touched by the driver, the parking lot information Ipark is registered in the parking assistance system 10 as the second parking lot information Ipark_2. When the third registration button image 553 is touched by the driver, the parking lot information Ipark is registered in the parking assistance system 10 as the third parking lot information Ipark_3.
[0060] The parking lot image 60 includes a first parking lot image, a second parking lot image, and a third parking lot image. The first parking lot image is an image of the parking lot captured by the camera 45 and registered, and corresponds to the first parking lot information Ipark_1. The second parking lot image is an image of the parking lot captured by the camera 45 and registered, and corresponds to the second parking lot information Ipark_2. The third parking lot image is an image of the parking lot captured by the camera 45 and registered, and corresponds to the third parking lot information Ipark_3.
[0061] [Parking Assist System Operation Overview] Next, an overview of the operation of the parking assistance system 10 will be described. The parking assistance system 10 is configured to be able to execute parking assistance control. The parking assistance control is a control for parking the vehicle 100 in a parking space without requiring the driver to operate the accelerator pedal, the brake pedal, or the steering wheel.
[0062] There are parking lots where parking spaces are divided by lines such as white lines (hereinafter referred to as "dividing lines"). When vehicle 100 is automatically parked in such a parking lot, the vehicle can be automatically driven to park in the parking space using the dividing lines captured by camera 45 as a landmark.
[0063] On the other hand, there are parking lots, such as private parking lots, that do not have demarcation lines that define parking spaces. When automatically parking the vehicle 100 in such parking lots, there are no demarcation lines to serve as landmarks. The parking assistance control performed by the parking assistance system 10 includes control to automatically park the vehicle 100 in the parking lot and to register parking lot information related to the parking lot, and control to automatically park the vehicle 100 in the parking lot whose parking lot information is registered.
[0064] [Parking a vehicle in an unregistered parking lot and registering it] The parking assistance system 10 automatically parks the vehicle 100 in an unregistered parking lot 62 and also performs a process of registering the unregistered parking lot. Note that the process is simply reversed between when the unregistered parking lot 62 is on the right side of the vehicle 100 and when it is on the left side of the vehicle 100. For this reason, the following describes the case where the unregistered parking lot 62 is on the left side of the vehicle 100, and a description of the case where the unregistered parking lot 62 is on the right side of the vehicle 100 will be omitted.
[0065] As shown in Fig. 10, when the vehicle 100 stops near the entrance 62ent of an unregistered parking lot 62 and the parking assist switch 15 (see Fig. 1) is turned ON, the parking assist system 10 displays a planar image 51P, a setting button image 53, and a movement button image 57 (571-574) on the display 50 as shown in Fig. 9(A). In addition, the parking assist system 10 sets an area in which the vehicle 100 can be parked in the parking lot 62 based on the image data IMG as a parking space 61 (see Fig. 8), and displays a parking space line image 52 representing the set parking space 61 superimposed on the planar image 51P.
[0066] Here, the movement button image 57 includes an upward movement button image 571, a downward movement button image 572, a left movement button image 573, and a right movement button image 574. The upward movement button image 571 is an image operated by the driver to move the parking space line image 52 upward on the display 50. The downward movement button image 572 is an image operated by the driver to move the parking space line image 52 downward on the display 50. The left movement button image 573 is an image operated by the driver to move the parking space line image 52 leftward on the display 50. The right movement button image 574 is an image operated by the driver to move the parking space line image 52 rightward on the display 50.
[0067] The driver can move the parking space line image 52 arbitrarily on the display 50 by touching the movement button image 57 until touching the setting button image 53. By moving the parking space line image 52 on the display 50, the driver can change the position of the parking space 61 where the vehicle 100 is to be parked.
[0068] When the setting button image 53 is touched by the driver, the parking assistance system 10 acquires and stores an image of the parking lot 62 (particularly, an image of the entrance 62ent of the parking lot 62) captured by the camera 45 when the vehicle 100 is stopped near the entrance 62ent of the parking lot 62 as a parking lot image 60. The parking assistance system 10 associates the parking lot image 60 with the position information of the vehicle 100 acquired by the position information acquisition device 30 and stores it.
[0069] Furthermore, when the driver touches the set button image 53, the parking assistance system 10 erases the set button image 53 and the move button image 57 from the display 50, and displays the registration start button image 54, as shown in Fig. 9(B). Furthermore, when the driver touches the set button image 53, the parking assistance system 10 sets the parking space 61 at the position corresponding to the parking space line image 52 displayed on the display 50 as the parking space 61set to be registered.
[0070] Furthermore, when the driver touches the setting button image 53, the parking assistance system 10 sets a target driving route Rtgt along which the vehicle 100 travels to park the vehicle 100 in the registration target parking space 61set. For example, as shown in Fig. 10, when the vehicle 100 stops so that an unregistered parking space 62 is located on the left side of the vehicle 100, the parking assistance system 10 sets a target driving route Rtgt shown in Fig. 11.
[0071] 10, when the driver touches the setting button image 53, the parking assistance system 10 acquires two left side feature points F3 for each of the four intermediate divided ranges 73D5 of the left side range 73, acquires one left side feature point F3 for each of the two front end divided ranges 73D1, and acquires one left side feature point F3 for each of the two rear end divided ranges 73D2, as shown in FIG. 12 and FIG. 13, and stores these feature points F3 as entrance feature points Fent of the unregistered parking lot 62. The entrance feature point Fent is the feature point F of the entrance 62ent of the parking lot 62.
[0072] After acquiring the entrance characteristic point Fent, the parking assistance system 10 stores the coordinates XY of the acquired entrance characteristic point Fent in the coordinate system Cpre as the entrance coordinates XYent, and stores the shading information CT of the acquired entrance characteristic point Fent as the entrance shading information CTent, as shown in Fig. 12. Here, the coordinate system Cpre refers to a coordinate system whose origin is a predetermined position Ppre in the registration target parking space 61set.
[0073] When the registration start button image 54 is touched by the driver, the parking assistance system 10 performs a parking driving process at the time of registration to drive the vehicle 100 along the target driving route Rtgt to the registration target parking space 61set. The parking driving process at the time of registration is a process for controlling the operation of the drive device 11, the steering device 12, and the braking device 13 so that the vehicle 100 drives along the target driving route Rtgt based on the image data IMG, the object information OBJ, the steering angle θst, the steering torque TQst, the vehicle speed SPD, and the like.
[0074] For example, as shown in Fig. 10, when the vehicle 100 stops so that an unregistered parking lot 62 is located to the left of the vehicle 100, the parking assistance system 10 starts the registered parking and running process, and first moves the vehicle 100 forward while turning right and stops it, as shown in Fig. 14. Next, the parking assistance system 10 moves the vehicle 100 backward while turning left, as shown in Fig. 15.
[0075] The parking assistance system 10 acquires the feature point F at least once before parking of the vehicle 100 in the parking lot 62 is completed, and acquires the coordinates XY of the feature point F in the coordinate system Cpre and the shading information CT of the feature point F. In particular, after the vehicle 100 starts traveling in order to park the vehicle 100 in the parking lot 62, the parking assistance system 10 acquires the feature point F at least once when the vehicle 100 is predicted to travel straight without turning thereafter, and acquires the coordinates XY of the feature point F in the coordinate system Cpre and the shading information CT of the feature point F.
[0076] In this embodiment, the parking assistance system 10 acquires a rear feature point F2 while the vehicle 100 is being reversed at a point in time when the vehicle 100 is predicted to continue to drive straight without turning thereafter (see FIG. 16). Specifically, the parking assistance system 10 acquires at least one rear feature point F2 present in each of the rear divided ranges 72D as an in-park intermediate feature point Fmid. The parking assistance system 10 also stores the coordinates XY of the acquired in-park intermediate feature point Fmid in the coordinate system Cpre as in-park intermediate coordinates XYmid, and stores the shading information of the acquired in-park intermediate feature point Fmid as in-park intermediate shading information CTmid.
[0077] 17, when the entire vehicle 100 is within the registration target parking section 61set, the parking assistance system 10 stops the vehicle 100 and ends the registration time parking and driving process. This completes parking of the vehicle 100 in the parking lot 62.
[0078] When parking of the vehicle 100 in the parking lot 61 is completed, the parking assistance system 10 acquires at least one or more front feature points F1 present in each of the front divided ranges 71D, acquires at least one or more left feature points F3 present in each of the left divided ranges 73D, acquires at least one or more right feature points F4 present in each of the right divided ranges 74D, and stores these feature points F1, F3, and F4 as the final feature points Ffin in the lot. In addition, the parking assistance system 10 stores the coordinates XY of the acquired final feature points Ffin in the coordinate system Creg as the final coordinates XYfin in the lot, and stores the shading information CT of the acquired final feature points Ffin as the final shading information CTfin in the lot. In addition, the parking assistance system 10 stores the coordinates XY of the parking space 61set to be registered in the coordinate system Creg as the registration space coordinates XYarea.
[0079] When parking of the vehicle 100 in the parking lot 62 is completed, the parking assistance system 10 displays the first to third registered button images 551 to 553 on the display 50, as shown in FIG. 9(C).
[0080] When the driver touches the first registration button image 551, the parking assistance system 10 registers the acquired entrance feature point Fent, the intermediate feature point Fmid, the final feature point Ffin, the registered section coordinate XYarea, and the parking lot image 60 as the first parking lot information Ipark_1. When the driver touches the second registration button image 552, the parking assistance system 10 registers the acquired entrance feature point Fent, the intermediate feature point Fmid, the final feature point Ffin, the registered section coordinate XYarea, and the parking lot image 60 as the second parking lot information Ipark_2. When the driver touches the third registration button image 553, the parking assistance system 10 registers the acquired entrance feature point Fent, the intermediate feature point Fmid, the final feature point Ffin, the registered section coordinate XYarea, and the parking lot image 60 as the third parking lot information Ipark_3.
[0081] [Parking a vehicle in a registered parking lot] When the vehicle speed SPD of the vehicle 100 becomes equal to or lower than a predetermined vehicle speed SPDth (for example, equal to or lower than 15 km / h), the parking assistance system 10 performs a search process to search the camera image CMR for image portions having shading patterns that match those of the registered entrance feature point Fent. The parking assistance system 10 performs this search process using the left side image data IMG3 and the right side image data IMG4. Here, the registered entrance feature point Fent is the entrance feature point Fent whose shading information is stored in the parking assistance system 10 by the above-mentioned registration process. The camera image CMR is an image of the current surroundings of the vehicle 100 captured by the camera 45.
[0082] When image portions having shading patterns that match the shading patterns of multiple entrance feature points Fent are extracted from the camera image CMR, the parking assistance system 10 performs a comparison process (matching process) between the positional relationship between the extracted multiple image portions and the positional relationship between the multiple entrance feature points Fent having shading patterns that match the shading patterns of the multiple image portions.
[0083] When the positional relationship between the extracted image portions coincides with the positional relationship between the entrance feature points Fent having a shading pattern that coincides with the shading pattern of the extracted image portions, the parking assistance system 10 determines that it has detected a parking space 61 in a registered parking lot 62. A registered parking lot 62 is a parking lot whose parking lot information Ipark has been stored in the parking assistance system 10 by the above-mentioned registration process.
[0084] More specifically, if the positional relationship between multiple image portions extracted based on the left side image data IMG3 matches the positional relationship between multiple entrance feature points Fent having shading patterns that match the shading patterns of the multiple image portions, the parking assistance system 10 determines that it has detected a parking space 61 of a registered parking lot 62 on the left side of the vehicle 100.
[0085] On the other hand, if the positional relationship between the multiple image portions extracted based on the right-side image data IMG4 coincides with the positional relationship between the multiple entrance feature points Fent having shading patterns that match the shading patterns of the multiple image portions, the parking assistance system 10 determines that it has detected a parking space 61 of a registered parking lot 62 on the right side of the vehicle 100.
[0086] Furthermore, if no image portion having a shading pattern matching the shading patterns of the entrance feature points Fent is extracted from the camera image CMR, the parking assistance system 10 determines that the parking space 61 in the registered parking lot 62 could not be detected. Alternatively, the parking assistance system 10 also determines that the parking space 61 in the registered parking lot 62 could not be detected if the positional relationship between the extracted image portions does not match the positional relationship between the entrance feature points Fent having shading patterns matching the shading patterns of the image portions.
[0087] When the parking assistance system 10 determines that the parking assistance switch 15 (see FIG. 1) is turned ON and that a registered parking lot 62 has been detected, it displays a planar image 51P and a parking start button image 56 on the display 50, as shown in FIG. 18(A). At this time, a parking space line image 52 corresponding to the registered parking space 61 is superimposed on the planar image 51P.
[0088] When the driver touches the parking start button image 56, the parking assistance system 10 erases the parking start button image 56 from the display 50, as shown in Fig. 18(B). Also, when the driver touches the parking start button image 56, the parking assistance system 10 sets the parking space 61 at a position corresponding to the parking space line 52 in the planar image 51P displayed on the display 50 as the target parking space 61tgt.
[0089] Furthermore, when the parking start button image 56 is touched by the driver, the parking assistance system 10 sets a target driving route Rtgt along which the vehicle 100 should travel in order to park the vehicle 100 in the target parking space 61tgt.
[0090] Then, the parking assistance system 10 performs an automatic parking and driving process for driving the vehicle 100 to the target parking space 61tgt along the target driving route Rtgt. The automatic parking and driving process is a process for controlling the operation of the drive device 11, the steering device 12, and the braking device 13 so that the vehicle 100 drives along the target driving route Rtgt based on the image data IMG, the object information OBJ, the steering angle θst, the steering torque TQst, the vehicle speed SPD, and the like.
[0091] During the execution of the automatic parking and driving process, the parking assistance system 10 performs a safety judgment process to determine whether or not the vehicle 100 can be safely driven to the target parking space 61tgt without coming into contact with a three-dimensional object present in the parking lot 62 when the vehicle 100 is driven along the target driving route Rtgt. When the parking assistance system 10 determines that the vehicle 100 cannot be safely driven to the target parking space 61tgt, it corrects the target driving route Rtgt so that the vehicle 100 can be safely driven to the target parking space 61tgt. The vehicle parking assistance device 10 performs the above safety judgment process using the image data IMG and object information OBJ acquired during the execution of the automatic parking and driving process.
[0092] During the automatic parking and driving process, the parking assistance system 10 performs a search process to search the camera image CMR for image portions having shading patterns that match the shading patterns of the multiple registered feature points Freg. The parking assistance system 10 performs this search process using the rear image data IMG2, the left side image data IMG3, and the right side image data IMG4. Here, the registered feature points Freg are feature points F for which the intermediate feature point Fmid and the final feature point Ffin are registered by the parking assistance control described above.
[0093] When image portions having shading patterns matching those of a plurality of registered feature points Freg are extracted from the camera image CMR, the parking assistance system 10 performs a matching process between the positional relationship between the extracted image portions and the positional relationship between the plurality of registered feature points Freg having shading patterns matching those of the image portions.
[0094] When the positional relationship between the extracted image portions coincides with the positional relationship between the registered feature points Freg having a shading pattern that coincides with the shading pattern of the extracted image portions, the parking assistance system 10 performs a parking position determination process to determine whether the position of the target parking section 61tgt in the parking lot 62 coincides with the position indicated by the registered section coordinate XYarea based on the positional relationship between the coordinate XY of the registered feature point Freg having a shading pattern that coincides with the shading pattern of the extracted image portions and the registered section coordinate XYarea. When the parking assistance system 10 determines that the position of the target parking section 61tgt in the parking lot 62 does not coincide with the position indicated by the registered section coordinate XYarea, it corrects the position of the target parking section 61tgt so that the position of the target parking section 61tgt in the parking lot 62 coincides with the position indicated by the registered section coordinate XYarea, and corrects the target driving route Rtgt so that the vehicle 100 can be parked in the target parking section 61tgt whose position has been corrected.
[0095] When the entire vehicle 100 is within the target parking section 61tgt, the parking assistance system 10 stops the vehicle 100 and ends the automatic parking and driving process. This completes parking of the vehicle 100 in the parking lot 62.
[0096] [Display control of Display 50] The parking assistance control for the above-mentioned registered parking lot 62 can be started when the parking assistance system 10 matches the feature points of the camera image CMR (left side image data IMG3, right side image data IMG4) captured by the camera 45 with the entrance feature point Fent of the registered parking lot 62 and detects the parking space 61 of the parking lot 62.
[0097] Therefore, if the current position of the vehicle 100 is significantly away from the position where the vehicle 100 was parked when the parking lot 62 was registered, specifically, from the position where the vehicle 100 was parked near the entrance 62ent of the parking lot 62 and the registration operation was performed (hereinafter referred to as the vehicle position at the time of registration), the parking assistance system 10 will fail the matching process and will not be able to start parking assistance control.
[0098] In such a situation, in order for the parking assistance system 10 to start parking assistance control, the driver needs to align the current position of the vehicle 100 with the vehicle position at the time of registration. However, the parking assistance system 10 is in a state where it is unable to detect the parking space 61 in the parking lot 62, and there is a problem in that it is unable to properly present to the driver where to move the vehicle 100 to align it with the vehicle position at the time of registration.
[0099] On the other hand, the driver may forget the vehicle position at the time of registration, or may not know the registered parking lot 62 because the registration operation was performed by someone other than the driver (for example, a family member who shares the vehicle 100). In such a case, even if the current position of the vehicle 100 is far from the vehicle position at the time of registration, the driver may not realize this, and may not understand the reason why the parking assistance system 10 cannot start parking assistance control.
[0100] In order to solve such problems, the parking assistance system 10 of the present embodiment performs display control to display a registered parking lot image 60 and an image (planar image) of the current surroundings of the vehicle 100 side by side on the display 50 when the current position of the vehicle 100 is within a predetermined distance range from a registered parking lot 62 and a parking space 61 in the parking lot 62 cannot be detected. By displaying these images side by side, it is possible to appropriately inform the driver that the current position of the vehicle 100 is far from the vehicle position at the time of registration. Details of the display control will be described below.
[0101] 19 is a flowchart showing a display control routine. The ECU 90 (CPU) of the parking assistance system 10 starts the routine shown in FIG.
[0102] In step S100, the ECU 90 determines whether or not the vehicle speed SPD of the vehicle 100 is equal to or less than a predetermined vehicle speed SPDth based on detection by the vehicle speed sensor 25. If the vehicle speed SPD is equal to or less than the predetermined vehicle speed SPDth (Yes), the ECU 90 advances the process to step S110. On the other hand, if the vehicle speed SPD is not equal to or less than the predetermined vehicle speed SPDth (No), the ECU 90 repeats the determination process of step S100.
[0103] In step S110, the ECU 90 acquires a camera image CMR of the surroundings of the vehicle 100 captured by the camera 45. Next, in step S120, the ECU 90 performs a matching process between the feature points of the camera image CMR (left side image data IMG3 and right side image data IMG4) acquired in step S110 and the registered entrance feature point Fent.
[0104] In step S130, the ECU 90 determines whether the parking assist switch 15 has been turned ON. If the parking assist switch 15 has been turned ON (Yes), the ECU 90 accepts a parking assist request from the driver and proceeds to step S140. On the other hand, if the parking assist switch 15 has not been turned ON (No), in other words, if the parking assist switch 15 is OFF, the ECU 90 returns the process to step S100.
[0105] In step S140, the ECU 90 determines whether or not the matching process in step S120 has detected a parking space 61 in the registered parking lot 62. If the parking space 61 in the parking lot 62 has been detected (Yes), the ECU 90 advances the process to step S150. In step S150, the ECU 90 displays the planar image 51P, the registered parking space line image 52, and the parking start button image 56 on the display 50 (see FIG. 18(A)).
[0106] Next, in step S152, the ECU 90 determines whether or not a predetermined end condition is satisfied. Here, the end conditions include (1) when the parking assist switch 15 is turned OFF, (2) when the vehicle speed SPD of the vehicle 100 becomes higher than a predetermined vehicle speed SPDth, and (3) when a predetermined upper limit time has elapsed since the image display in step S150. If any of these end conditions (1) to (3) is satisfied, the ECU 90 determines that the end condition is satisfied and proceeds to the process in step S190. In step S190, the ECU 90 ends the image display and then ends (returns) this routine.
[0107] On the other hand, if it is determined in step S152 that the termination condition is not satisfied, the ECU 90 advances the process to step S154. In step S154, the ECU 90 determines whether or not the parking start button image 56 has been touched by the driver. If the parking start button image 56 has not been touched (No), the ECU 90 returns the process to step S150. That is, the image display is continued. On the other hand, if the parking start button image 56 has been touched (Yes), the ECU 90 advances the process to step S156.
[0108] In step S156, the ECU 90 performs an automatic parking and driving process to automatically park the vehicle 100 in a registered parking lot 62, and then ends this routine.
[0109] On the other hand, when it is determined in step S140 that the parking space 61 of the registered parking lot 62 cannot be detected (No), the ECU 90 advances the process to step S145. In step S145, the ECU 90 determines whether the current position of the vehicle 100 is within a predetermined distance range from the registered parking lot 62. For the current position of the vehicle 100, the position information acquired by the position information acquisition device 30 is used. For the position information of the registered parking lot 62, the position information associated with the parking lot image 60 and stored at the time of registration is used. The ECU 90 compares these position information to determine whether a registered parking lot 62 exists within a predetermined distance range from the current position of the vehicle 100. If there are multiple registered parking lots 62, the ECU 90 identifies the parking lot 62 closest to the current position of the vehicle 100, and compares the position information of the identified parking lot 62 with the current position of the vehicle 100.
[0110] If it is determined in step S145 that the current position of vehicle 100 is within a predetermined distance range from registered parking lot 62 (Yes), ECU 90 advances the process to step S160. Here, the predetermined distance used in the determination in step S145 is not particularly limited, but it is preferable to set it based on a distance (e.g., several meters to several tens of meters) that allows the driver to easily move vehicle 100 to the vehicle position at the time of registration while referring to images 60, 51P displayed on display 50 in step S160.
[0111] In step S160, the ECU 90 displays the parking lot image 60 of the registered parking lot 62 and the image of the current surroundings of the vehicle 100 captured by the camera 45 (planar image 51P) side by side on the display 50. Furthermore, when there are multiple registered parking lots 62, the ECU 90 displays the parking lot image 60 of the parking lot 62 identified in the process of step S145 as being closest to the current position of the vehicle 100 side by side with the image of the current surroundings of the vehicle 100 (planar image 51P). In other words, even when multiple parking lots (62) are registered, the parking lot image 60 of the parking lot 62 that is most suitable for the current position of the vehicle (100) is displayed on the display 50.
[0112] 20 is an example of an image displayed on the display 50 in the process of step S160. The ECU 90 displays a parking lot image 60 of a registered parking lot 62 on the left side of the display 50, and displays an image of the current surroundings of the vehicle 100 captured by the camera 45 (planar image 51P) on the right side of the display 50. Note that the layout relationship of these images is not limited to the illustrated example, and the left-right positional relationship may be interchanged, or these images may be arranged one above the other.
[0113] The parking lot image 60 is an image of the parking lot 62 registered by the above-mentioned registration process viewed from above in the vertical direction. In the parking lot image 60, an image showing the vehicle 100 parked at the vehicle position P at the time of registration and a parking space line image 52 showing the parking space (61) are superimposed and displayed. The image of the current surroundings of the vehicle 100 is a planar view image 51P generated based on image data IMG captured by the camera 45, and includes a vehicle planar view image and a vehicle surroundings image. The vehicle planar view image is an image showing the vehicle 100 when the vehicle 100 is viewed from above in the vertical direction. The vehicle surroundings image is an image showing the surroundings of the vehicle 100 when the surroundings of the vehicle 100 are viewed from above in the vertical direction. In the example shown in FIG. 20, a part of the parking lot 62 is captured in front of the vehicle 100 in the planar view image 51P.
[0114] By referring to the parking lot image 60 and the image of the current surroundings of the vehicle 100 (planar image 51P) displayed side by side on the display 50, the driver can easily grasp how far the current position of the vehicle 100 is from the vehicle position P at the time of registration, and further, in which direction the vehicle 100 should be moved to align with the vehicle position P at the time of registration. In the illustrated example, the driver can grasp that the vehicle position P at the time of registration is located ahead of the vehicle 100, and that moving the vehicle 100 forward will align the vehicle with the vehicle position P at the time of registration. In other words, by appropriately presenting to the driver through an image in which direction and how far the vehicle 100 should be moved to align with the vehicle position P at the time of registration, convenience can be improved.
[0115] In step S162, the ECU 90 again determines whether or not the matching process has detected a parking space 61 in the registered parking lot 62. If the matching process has detected a parking space 61 in the parking lot 62 (Yes), the ECU 90 advances the process to step S150. That is, the ECU 90 can start parking assistance control. On the other hand, if the matching process has not detected a parking space 61 in the parking lot 62 (No), the ECU 90 advances the process to step S164.
[0116] In step S164, the ECU 90 determines whether or not a predetermined end condition is satisfied. The end conditions include (1) when the parking assist switch 15 is turned OFF, (2) when the vehicle speed SPD of the vehicle 100 becomes higher than a predetermined vehicle speed SPDth, and (3) when a predetermined upper limit time has elapsed since the image display in step S160. If any of the end conditions (1) to (3) is satisfied, the ECU 90 determines that the end condition is satisfied and advances the process to step S190. In step S190, the ECU 90 ends the image display and then ends this routine.
[0117] On the other hand, if it is determined in step S145 that the current position of the vehicle 100 is not within a predetermined distance range from the registered parking lot 62 (No), that is, if the vehicle 100 is far away from the registered parking lot 62, the ECU 90 advances the process to step S170 and displays a message image on the display 50 indicating that "parking assistance is not possible." In a situation in which the vehicle 100 is far away from the registered parking lot 62, even if the registered parking lot image 60 and the image 51P of the surroundings of the vehicle 100 are displayed on the display 50 as in step S160, it is difficult for the driver to align the vehicle 100 with the vehicle position P at the time of registration, and this is ineffective. In such a case, the ECU 90 notifies the driver that the system cannot execute parking assistance control.
[0118] Next, in step S175, the ECU 90 determines whether or not a predetermined end condition is satisfied. Here, the end conditions include (1) when the parking assist switch 15 is turned OFF, (2) when the vehicle speed SPD of the vehicle 100 becomes higher than a predetermined vehicle speed SPDth, and (3) when a predetermined upper limit time has elapsed since the image display in step S170. If any of the end conditions (1) to (3) is satisfied, the ECU 90 determines that the end condition is satisfied and proceeds to step S190. In step S190, the ECU 90 ends the image display and then ends this routine.
[0119] The above is a specific operation of the parking assistance system 10. According to the parking assistance system 10, when a parking assistance request is received from the driver, if the current position of the vehicle 100 is far from the vehicle position P at the time of registration and the parking space 61 of the registered parking lot 62 cannot be detected by the matching process, the registered parking lot image 60 and the image 51P of the current surroundings of the vehicle 100 are displayed side by side on the display 50. That is, it is possible to appropriately notify the driver that the current position of the vehicle 100 is far from the vehicle position P at the time of registration. This allows the driver to easily know in which direction to move the vehicle 100 to align it with the vehicle position P at the time of registration, and it is possible to reliably improve convenience.
[0120] The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the present invention. [Explanation of symbols]
[0121] 10... parking assist system, 11... drive device, 12... steering device, 13... braking device, 15... parking assist switch, 30... position information acquisition device, 40... surrounding recognition device, 41... sonar sensor, 44... camera sensor, 51P... planar image, 52... parking space line image, 60... parking space image, 62... parking space, 90... ECU, 100... vehicle
Claims
1. an imaging device mounted on the vehicle for capturing images of the surroundings of the vehicle and images of a parking lot of the parking lot for use in automatically parking the vehicle in the parking lot; a control device that registers the parking lot image captured by the imaging device as parking lot information for the parking lot, and performs parking assistance control to automatically park the vehicle in the parking lot based on the parking lot information when a parking assistance request is received from a driver; a display device capable of displaying an image captured by the imaging device; In parking assistance systems, The control device includes: The surrounding image captured by the imaging device is compared with the parking lot image registered as the parking lot information, and the parking assistance control is started when the parking lot is detected by the comparison process. If the parking lot cannot be detected by the comparison process, the parking lot image registered as the parking lot information and the current surrounding image captured by the imaging device are displayed on the display device. Parking assistance system.
2. 2. The parking assistance system according to claim 1, A position information acquisition device for acquiring position information of the vehicle is further provided. The control device includes: The parking lot image captured by the imaging device is associated with location information acquired by the location information acquisition device when the image is captured, and is registered as the parking lot information of the parking lot; If the parking lot cannot be detected by the matching process, it is determined whether the current position of the vehicle is within a predetermined distance range from the parking lot based on the current position information of the vehicle acquired by the position information acquisition device and the position information of the parking lot registered as the parking lot information, When it is determined that the current position of the vehicle is within a predetermined distance range from the parking lot, the parking lot image registered as the parking lot information and the current surrounding image captured by the imaging device are displayed on the display device, whereas when it is determined that the current position of the vehicle is not within a predetermined distance range from the parking lot, a message to the effect that the parking assistance control cannot be executed is displayed on the display device. Parking assistance system.
3. 3. The parking assistance system according to claim 2, The control device includes: The parking lot images of the plurality of parking lots captured by the imaging device can be registered as parking lot information of each parking lot in association with location information acquired by the location information acquisition device when the images are captured, If none of the plurality of parking lots can be detected by the comparison process, the parking lot closest to the current position of the vehicle among the plurality of parking lots is identified based on the current position information of the vehicle acquired by the position information acquisition device and the position information of the plurality of parking lots registered as the parking lot information, and if it is determined that the current position of the vehicle is within a predetermined distance range from the identified parking lot, a parking lot image of the identified parking lot and the current surrounding image captured by the imaging device are displayed on the display device. Parking assistance system.
Citation Information
Patent Citations
Parking support device
JP2014172411A
Parking support device
JP2020117128A
Vehicle parking support device
JP2021064104A
