Parking assistance device

Abstract

To provide a parking assistance device capable of executing accurately passage determination with respect to an obstacle while preventing changing of a parking path after cancellation of parking assistance or a start of assistance as much as possible.SOLUTION: In the process where a vehicle enters a parking space, until detection of an obstacle by ultrasonic sensors 9A-9L is enabled, the obstacle forming the vehicle width direction of the parking space is detected on the basis of images obtained by imaging vehicle surroundings by a front camera 6, a rear camera 7, and side cameras 8A and 8B installed in the vehicle. After the detection of the obstacle by the ultrasonic sensors 9A-9L has been enabled, the obstacle forming the vehicle width direction of the parking space is detected on the basis of range-finding values detected by the ultrasonic sensors 9A-9L installed in the vehicle.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a parking assistance device that assists a vehicle in parking. [Background technology]

[0002] Conventionally, parking assistance devices have been proposed that provide parking assistance by displaying camera images of the vehicle's surrounding environment when the vehicle is parking, issuing warnings about people or obstacles in the vicinity, or performing some or all of the user's driving operations on the vehicle side.

[0003] Here, when some or all of the driving operations are performed by the vehicle as the parking assistance, it is important to determine whether the vehicle can pass through the parking route from the vehicle's current position to the parking space where the vehicle is to be parked without risk of contact with obstacles, and if it is predicted that the vehicle cannot pass through, to cancel the parking assistance or change the parking route. For example, Japanese Patent Application Laid-Open No. 2021-126950 proposes technology for a parking assistance device that automatically parks into a parking space, in which the width of the parking space's entrance is detected from an image captured by a camera, and if the entrance is narrow, the parking route is set so that the vehicle passes through the entrance in a straight line. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-126950 A (paragraph 0049) Summary of the Invention [Problem to be solved by the invention]

[0005] In particular, when a parking space is located in a closed space surrounded by obstacles, such as a garage, the area through which a vehicle can pass is limited compared to when parking in an open space, and the above-mentioned pass-through determination requires accuracy. In the above-mentioned Patent Document 1, the width of the entrance is detected using a camera installed in the vehicle, but detection using a camera is easily affected by the surrounding environment, such as weather, and the detection accuracy is lower than that of detection using a distance sensor. Therefore, it is possible that the entrance may be determined to be inaccessible even when the entrance is sufficiently wide.

[0006] On the other hand, although distance measuring sensors have higher detection accuracy than cameras, their detection range is narrower and more directional than cameras, so in order to detect the width of an entrance using a distance measuring sensor, for example, it is necessary to move the vehicle to a position where the distance measuring sensor passes through the entrance. However, if parking assistance is discontinued or the parking route is significantly changed after the vehicle has reached such a position, parking operations must be performed again from scratch, which places a heavy burden on the occupants.

[0007] The present invention has been made to solve the above-mentioned problems in the conventional technology, and aims to provide a parking assistance device that, when detecting obstacles that form the vehicle width direction of a parking space, appropriately uses the detection results using an imaging device and the detection results using a distance measuring sensor while the vehicle is entering the parking space, thereby preventing as much as possible the suspension of parking assistance or the change of the parking route after assistance has started, while also making accurate judgments about whether the obstacle has been passed. [Means for solving the problem]

[0008] In order to achieve the above object, a parking assistance device according to the present invention comprises: a parking space identification means for identifying a parking space in which a vehicle is to be parked; a parking assistance means for providing parking assistance to park the vehicle in the parking space identified by the parking space identification means; a first obstacle detection means for detecting an obstacle that forms part of the parking space in the vehicle width direction based on an image of the area around the vehicle captured by an imaging device installed in the vehicle; and a second obstacle detection means for detecting an obstacle that forms part of the parking space in the vehicle width direction based on a distance measurement value detected by a distance measurement sensor installed in the vehicle. During the process of the vehicle entering the parking space, the parking assistance means detects the obstacle using the first obstacle detection means until it becomes possible for the distance measurement sensor to detect the obstacle, and continues parking assistance even if it is determined based on the detection result of the first obstacle detection means that it is difficult for the vehicle to enter the parking space, and after it becomes possible for the distance measurement sensor to detect the obstacle, it detects the obstacle using the second obstacle detection means, and discontinues parking assistance if it is determined based on the detection result of the second obstacle detection means that it is difficult for the vehicle to enter the parking space. [Effects of the Invention]

[0009] The parking assistance device according to the present invention having the above configuration includes both an imaging device and a distance measuring sensor, and when detecting obstacles that form the vehicle width direction of a parking space, the detection results using the imaging device and the distance measuring sensor are appropriately used as the vehicle enters the parking space. This makes it possible to accurately determine whether the vehicle has passed through an obstacle while minimizing the need to stop parking assistance or change the parking path after assistance has started. In particular, while the first obstacle detection means can grasp the general situation of the parking space and its surroundings in advance, the important decision on whether to continue parking assistance can be made at a stage when accurate information can be obtained from the second obstacle detection means. [Brief explanation of the drawings]

[0010] [Figure 1]1 is a schematic configuration diagram of a vehicle according to an embodiment of the present invention. [Figure 2] 1 is a diagram showing an example of the arrangement of ultrasonic sensors on the front of a vehicle; [Figure 3] 1 is a diagram showing an example of the arrangement of ultrasonic sensors on the side of a vehicle; [Figure 4] 1 is a block diagram showing the configuration of a parking assistance device according to an embodiment of the present invention; [Figure 5] 4 is a flowchart of a parking assistance processing program according to the present embodiment. [Figure 6] 1 is a diagram showing a detection range in which an ultrasonic sensor installed in a vehicle can detect an obstacle; [Figure 7] FIG. 10 is a diagram illustrating a method for generating a parking trajectory when no obstacles exist. [Figure 8] FIG. 10 is a diagram illustrating a method for setting a target parking position. [Figure 9] 10A and 10B are diagrams illustrating a method for generating a parking trajectory when an obstacle is present. [Figure 10] 10A and 10B are diagrams illustrating a method for setting a target parking position using an ultrasonic sensor. [Figure 11] FIG. 10 is a diagram illustrating a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0011] A specific embodiment of a parking assistance device according to the present invention will be described below in detail with reference to the drawings. First, a vehicle 2 equipped with a parking assistance device 1 according to this embodiment will be described below. Figure 1 is a schematic diagram of the vehicle 2 according to this embodiment.

[0012] Here, the vehicle 2 may be, for example, an automobile (internal combustion engine automobile) that uses an internal combustion engine (engine, etc.) as a drive source, an automobile (electric automobile, fuel cell automobile, etc.) that uses an electric motor (motor, etc.) as a drive source, or an automobile that uses both of these as a drive source (hybrid automobile). Furthermore, the vehicle type is not limited, and it may be a standard car, or a large commercial truck, bus, construction machinery, etc. Furthermore, although the following description will be of a four-wheeled automobile, it may also be a two-wheeled or three-wheeled vehicle.

[0013] However, vehicle 2 is a vehicle capable of manual driving, in which the vehicle drives based on the driving operation of the user, as well as assisted driving using automatic driving assistance, in which the vehicle drives automatically without the driving operation of the user.

[0014] Furthermore, autonomous driving assistance may be performed only under specific circumstances, such as when parking or leaving a parking lot, or may be performed for all road sections, or may be performed only while the vehicle is traveling on a specific road section (for example, a highway with a gate (manned or unmanned, toll or free) at the boundary). In the following description, the autonomous driving section in which autonomous driving assistance is performed includes all road sections, including general roads and highways, as well as parking lots, and is performed only when the user selects to perform autonomous driving assistance (for example, turns on the autonomous driving start button) and it is determined that autonomous driving assistance is possible. On the other hand, vehicle 2 may be a vehicle that is only capable of assisted driving with autonomous driving assistance. Alternatively, autonomous driving assistance may be performed only when the vehicle is traveling to a parking space when parking (i.e., parking assistance).

[0015] In the vehicle control in the automated driving assistance of this embodiment, for example, the current position of the vehicle, the lane the vehicle is traveling on, and the positions of surrounding obstacles are detected as needed, and vehicle control of the steering, drive source, brakes, etc. is automatically performed so that the vehicle travels along the generated travel trajectory at a speed according to the generated speed plan. In particular, when performing parking assistance, as described below, the system uses detection results from sensors and cameras to check the parking space where the vehicle is to park and its surrounding conditions, calculates a parking trajectory to the parking space, and automatically controls the vehicle to enter the parking space along the calculated parking trajectory and complete parking. However, it is also possible to automatically perform only the steering operation and manually control the drive source and brakes. Alternatively, it is also possible to provide only guidance on the parking trajectory to the parking space or guidance on vehicle operation, and have the user manually perform the parking operation into the parking space. Furthermore, when providing the above-mentioned autonomous driving assistance, the scenery (actual scene) around the vehicle captured by a camera installed in the vehicle is displayed on the vehicle's display, and if there is a warning object such as a pedestrian around the vehicle, a warning image indicating the presence of the warning object is superimposed on the position of the warning object within the scenery.

[0016] 1, the vehicle 2 has an operation unit 3 that accepts operations from the occupant, a liquid crystal display 4 that displays bird's-eye and overhead images of the vehicle's surroundings and other driving assistance-related information to the occupant, a speaker 5 that outputs audio guidance related to the driving assistance, a front camera 6, a rear camera 7, and side cameras 8A and 8B for capturing images of the vehicle's surroundings, ultrasonic sensors 9A to 9L that detect obstacles around the vehicle, and a driving assistance ECU (electronic control unit) 10 that performs various calculations based on input information. The driving assistance ECU 10 and other components are collectively referred to as the parking assistance device 1.

[0017] Each component of the vehicle 2 will be described below. First, the operation unit 3 is arranged, for example, on the front of the handle (also called the steering wheel), and includes operation buttons and the like that are operated when starting automatic driving assistance. By operating the operation unit 3, the user can switch between manual driving, in which the vehicle travels based on the user's driving operation, and automatic driving assistance, in which the vehicle travels automatically without the user's driving operation. The operation unit 3 may have a touch panel provided on the front of the liquid crystal display 4. It may also have a microphone and a voice recognition device.

[0018] The liquid crystal display 4 is mounted on the instrument panel of the vehicle 2, and displays bird's-eye and overhead images of the vehicle surroundings generated by performing viewpoint conversion and synthesis processing on images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B while autonomous driving assistance is being performed. In addition, if there is a warning object such as a pedestrian around the vehicle 2, a warning image indicating the presence of the warning object at the position of the warning object in the bird's-eye and overhead images is also displayed. The liquid crystal display 4 may also be used for a navigation device.

[0019] The speaker 5 is mounted on the instrument panel of the vehicle 2 and outputs voice guidance and warning sounds related to driving assistance. The speaker 5 may also be used for a navigation device.

[0020] The forward camera 6 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, with its optical axis facing forward in the direction of travel of the vehicle.

[0021] The rear camera 7 is an imaging device having a camera that also uses a solid-state imaging element such as a CCD, and is installed, for example, near the upper center of the license plate attached to the rear of the vehicle 2, with the optical axis facing toward the rear of the vehicle.

[0022] Furthermore, the side cameras 8A and 8B are imaging devices each having a camera using a solid-state imaging element such as a CCD, and are attached to the left and right side mirrors of the vehicle 2, for example, with their optical axes directed to the sides of the vehicle.

[0023] The driving assistance ECU 10 then performs viewpoint conversion and synthesis processing on the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B to generate bird's-eye and overhead images of the vehicle's surroundings. During autonomous driving assistance, the driving assistance ECU 10 also performs image recognition processing on the captured images to detect lane lines, parking frame lines, and obstacles (other vehicles, pedestrians, bicycles, walls, guardrails, and other structures) around the vehicle, and performs autonomous driving assistance based on the detection results. Particularly when performing parking assistance, the system also uses the obstacle detection results from the cameras to check the parking space and its surroundings.

[0024] Meanwhile, the ultrasonic sensors 9A-9L are arranged at predetermined intervals on the front, rear, and sides of the vehicle 2. They transmit ultrasonic waves as search waves around the vehicle 2 and receive reflected waves from objects around the vehicle, thereby detecting the objects that reflect the search waves. Specifically, they are a type of distance measurement sensor that can measure the distance (measured distance) to the object that reflected the search wave by measuring the time from transmission to reception. The ultrasonic sensors 9A-9L are also configured to generate output signals (including the distance to the detected object) corresponding to the reception results of the received waves and output them to the control unit. Examples of objects that can be detected by the ultrasonic sensors 9A-9L include obstacles that the vehicle 2 must avoid when traveling, such as people, bicycles, other vehicles, and walls, as well as obstacles that form parking spaces. Instead of ultrasonic sensors, millimeter-wave sensors or laser sensors may be used as distance measurement sensors.

[0025] The installation position and installation direction of each ultrasonic sensor 9A-9L can be set as appropriate. In this embodiment, to detect objects in all directions (forward, backward, left, and right) of the vehicle 2, for example, ultrasonic sensors 9A-9D are installed on the front of the vehicle 2 facing the vehicle's traveling direction so that the transmission direction of the search wave is forward. Ultrasonic sensors 9E and 9F are installed on the left side of the vehicle 2 facing left so that the transmission direction of the search wave is to the left of the vehicle's traveling direction. Ultrasonic sensors 9G and 9H are installed on the right side of the vehicle 2 facing right so that the transmission direction of the search wave is to the right of the vehicle's traveling direction. Ultrasonic sensors 9I-9L are installed on the rear of the vehicle 2 facing the opposite direction to the vehicle's traveling direction so that the transmission direction of the search wave is to the rear of the vehicle. The ultrasonic sensors 9A-9L are all approximately the same height from the ground surface.

[0026] Taking ultrasonic sensors 9A to 9D as an example in particular, it is desirable that ultrasonic sensors 9A to 9D be installed at different positions around the front bumper or the front grille above the front bumper of vehicle 2, as shown in Figure 2, with equal spacing between them without bias in the left and right directions, so that they can transmit detection waves over a wider range in front of the vehicle (i.e., the range in which objects can be detected is wider).

[0027] Specifically, as shown in FIG. 2 , ultrasonic sensor 9A is installed near the left front corner of vehicle 2 with the transmission direction of the probe wave tilted slightly leftward from the direction of travel of vehicle 2 so that it transmits the probe wave to the left front of vehicle 2. Ultrasonic sensor 9B is installed slightly left of the center line of vehicle 2 with the transmission direction of the probe wave tilted toward the vehicle's direction of travel so that it transmits the probe wave mainly from the front, particularly the left side, of vehicle 2. Ultrasonic sensor 9C is installed slightly right of the center line of vehicle 2 with the transmission direction of the probe wave tilted toward the vehicle's direction of travel so that it transmits the probe wave mainly from the front, particularly the right side, of vehicle 2. Ultrasonic sensor 9D is installed near the right front corner of vehicle 2 with the transmission direction of the probe wave tilted slightly rightward from the direction of travel of vehicle 2 so that it transmits the probe wave to the right front of vehicle 2. Ultrasonic sensors 9A and 9D, and ultrasonic sensors 9B and 9C are each arranged symmetrically across the vehicle's center line in a plan view. Although not shown, the ultrasonic sensors 9I to 9L on the rear surface of the vehicle 2 are similarly arranged in a vertically symmetrical manner.

[0028] On the other hand, as shown in Figure 3, the ultrasonic sensors 9E and 9F on the sides are each installed so as to emit search waves in a direction that intersects at 90 degrees with the direction of travel of the vehicle 2. Since there are fewer sensors installed relative to the range on the sides compared to the front and rear of the vehicle as described above, there are areas where the ultrasonic sensors 9E and 9F cannot directly detect objects. However, in these areas, it is possible to estimate the presence or position of an object from the detection history of the object by the ultrasonic sensors 9A to 9L. Although not shown, the ultrasonic sensors 9G and 9H on the right side of the vehicle 2 are similarly symmetrical.

[0029] In this embodiment, among the ultrasonic sensors 9A-9L, the ultrasonic sensors 9A-9D on the front of the vehicle 2 and the ultrasonic sensors 9I-9L on the rear of the vehicle 2 are particularly installed in positions where they can receive reflected waves from adjacent sensors as indirect waves, so that by receiving direct and indirect waves as received waves, it is possible to determine not only the distance to an object but also the specific position of the object (its relative position to the vehicle) using triangulation. The ultrasonic sensors 9E-9H on the sides are installed at a distance from each other and cannot receive indirect waves, but as the vehicle moves, it is also possible to determine the specific position of the object (its relative position to the vehicle) by triangulation using the measured distances at the previous and current positions and the distance traveled between them.

[0030] Meanwhile, the driving assistance ECU 10 is an electronic control unit that performs various processes related to autonomous driving assistance. For example, it constantly detects the vehicle's current position, the lane the vehicle is traveling in, and the positions of surrounding obstacles, and controls the vehicle, including steering, drive source, and braking, so that the vehicle travels along the generated travel path and at a speed according to the generated speed plan. In particular, when performing parking assistance, it uses the detection results of the front camera 6, rear camera 7, side cameras 8A and 8B, and ultrasonic sensors 9A-9L to check the parking space and its surrounding conditions, calculates a parking path to the parking space, and controls the vehicle to enter the parking space along the calculated parking path and complete parking. The LCD display 4 also displays the scenery (real scene) around the vehicle, and if there are warning objects such as pedestrians around the vehicle, it superimposes a warning image indicating the presence of the warning object on the scenery. The driving assistance ECU 10 is connected to the operation unit 3, LCD display 4, speaker 5, front camera 6, rear camera 7, side cameras 8A and 8B, and ultrasonic sensors 9A to 9L via an in-vehicle network such as a CAN. The driving assistance ECU 10 is also connected to various sensors mounted on the vehicle 2, such as a vehicle speed sensor, an acceleration sensor, a gyro sensor, a steering sensor, and a shift position sensor, as well as to a navigation device or other in-vehicle device. The detailed configuration of the driving assistance ECU 10 will be described later.

[0031] In addition, vehicle 2 has basic components as vehicle 2 in addition to the components shown in Figure 1, but we will only explain the configuration related to the control of automatic driving assistance and the control related to that configuration.

[0032] Next, a detailed description will be given of the driving assistance ECU 10 in particular of the parking assistance device 1 provided in the vehicle 2. Fig. 4 is a block diagram showing the configuration of the parking assistance device 1 according to this embodiment.

[0033] As shown in FIG. 4, the driving assistance ECU (electronic control unit) 10 is an electronic control unit that controls the entire parking assistance device 1. It includes a CPU 31, which functions as a calculation device and a control device; a RAM 32, which serves as a working memory for the CPU 31 to perform various calculation processes and stores driving trajectory data and other information used when the driving trajectory is calculated; a ROM 33, which stores control programs and a parking assistance processing program (see FIG. 5 ), which will be described later; and a flash memory 34, which stores programs read from the ROM 33. The driving assistance ECU 10 also includes various processing algorithms. For example, the parking space identification means identifies a parking space for the vehicle. The parking assistance means provides parking assistance for parking the vehicle in the parking space identified by the parking space identification means. The first obstacle detection means detects obstacles that form a width direction of the parking space based on images of the vehicle's surroundings captured by the front camera 6, rear camera 7, and side cameras 8A and 8B, which are image capture devices installed on the vehicle. The second obstacle detection means detects obstacles that form the parking space in the vehicle width direction based on distance measurements detected by the ultrasonic sensors 9A to 9L installed in the vehicle.

[0034] The driving assistance ECU 10 is also connected to various sensors 36 for detecting vehicle behavior, such as a vehicle speed sensor, an acceleration sensor, a gyro sensor, a steering sensor, and a shift position sensor, as well as to each of the vehicle's drive units 37, such as the steering, brake, accelerator, and transmission, and detects the current vehicle behavior based on the detection results of these sensors 36, while controlling each of the drive units 37 to provide automatic driving assistance for the vehicle 2. Specific details of the automatic driving assistance include, for example, constantly detecting the current vehicle position, the lane the vehicle is traveling on, and the positions of surrounding obstacles, and controlling the vehicle, such as the steering, drive source, and brakes, so that the vehicle travels along a generated travel trajectory at a speed in accordance with a speed plan that is also generated. However, it is also possible to automatically perform only the steering operation, while manually controlling the drive source and brakes.

[0035] The flash memory 34 also includes a vehicle information DB 35, which stores various information related to the vehicle 2. For example, the vehicle information DB 35 stores the installation positions (height from the ground and left-right positions) of the cameras and ultrasonic sensors 9A-9L installed on the vehicle 2, the detection axes (optical axes for cameras), overall length, vehicle width, wheelbase, minimum turning radius, etc. This information is input in advance by the occupants or a person from the vehicle manufacturer.

[0036] Next, a parking assistance processing program executed by the driving assistance ECU 10 in the parking assistance device 1 having the above configuration will be described with reference to Fig. 5. Fig. 5 is a flowchart of the parking assistance processing program according to this embodiment. Here, the parking assistance processing program is executed after the ACC power supply (accessory power supply) of the vehicle 2 is turned on, and is a program that provides parking assistance, particularly when parking the vehicle, as one type of automatic driving assistance. The program shown in the flowchart in Fig. 5 below is stored in the RAM 32 and ROM 33 provided in the parking assistance device 1, and is executed by the CPU 31.

[0037] First, in step (hereinafter abbreviated as S) 1, the CPU 31 determines whether or not to start parking assistance. In particular, the parking assistance in this embodiment is to automatically move the vehicle to a parking space where the vehicle is to be parked.

[0038] For example, parking assistance may be started when the user operates the operation unit 3 and selects to start parking assistance, or parking assistance may be started automatically when it is detected that the vehicle has entered a parking lot, when it is determined that the vehicle has arrived at the set destination, or when the vehicle approaches a parking lot.

[0039] If it is determined that parking assistance should be started (S1: YES), the process proceeds to S2. On the other hand, if it is determined that parking assistance should not be started (S1: NO), the parking assistance processing program is terminated.

[0040] In S2, CPU 31 identifies a parking space for parking the vehicle. The parking space may be identified by, for example, the user arbitrarily specifying a parking space where the user wants to park the vehicle in an image of the vehicle's surroundings displayed on the screen. Alternatively, the user may pre-register a parking space where the user wants to park (e.g., a home garage), and the registered parking space may be identified as the parking space for parking the vehicle. Alternatively, ultrasonic sensors 9A-9L or a camera may be used to detect vacant parking spaces around the vehicle in real time, and the detected vacant parking space may be identified as the parking space for parking the vehicle. When identifying a parking space using ultrasonic sensors 9A-9L or a camera, the parking space may be identified based on the detection results of obstacles such as walls or other vehicles, or based on the detection results of parking lane lines painted on the road surface.

[0041] In addition, the parking space identified in S2 also sets a target parking position where the vehicle will ultimately be positioned by parking assistance. However, when identifying a closed space such as a garage as a parking space, it is difficult to grasp the condition of the parking space from the outside. Therefore, the target parking position is set by detecting obstacles that form the parking space (for example, the wall of the garage) from the outside, as described below (S8, S9).

[0042] Then, in S3, the CPU 31 determines whether the ultrasonic sensors 9A-9L have detected an obstacle around the vehicle. As described above, the ultrasonic sensors 9A-9L transmit ultrasonic waves as search waves around the vehicle 2 and receive reflected waves from objects around the vehicle, thereby detecting the object that reflected the search wave (FIGS. 2 and 3). In this embodiment, multiple ultrasonic sensors are installed on the vehicle, but the range over which the search wave can be transmitted is limited. Furthermore, even if the search wave is reflected by an obstacle, the reflected wave can only be received if it is reflected by an obstacle located within a certain distance (e.g., within 5 m) from the ultrasonic sensors 9A-9L. Similarly, the ultrasonic sensors cannot receive reflected waves from obstacles located farther to the left or right of their detection axes (i.e., they have high directivity). Specifically, as shown in FIG. 6, the detection ranges 41-52 within which the ultrasonic sensors 9A-9L can detect obstacles are elongated, approximately elliptical areas. The detection ranges 41 to 52 are limited, and the obstacles that can be detected by the ultrasonic sensors 9A to 9L are basically limited to those located very close to the vehicle. In particular, to detect an obstacle on the side of the vehicle, the obstacle must be located within the detection ranges 45 to 48, and the vehicle must be located next to the obstacle to be detected.

[0043] If it is determined that the ultrasonic sensors 9A to 9L have detected an obstacle around the vehicle (S3: YES), the process proceeds to S9. On the other hand, if it is determined that the ultrasonic sensors 9A to 9L have not detected an obstacle around the vehicle (S3: NO), the process proceeds to S4.

[0044] In S4, the CPU 31 determines whether an obstacle has been detected around the vehicle (particularly between the vehicle and the parking space identified in S2) in the captured images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. The obstacle to be detected is an object that obstructs the vehicle's travel to the parking space, regardless of whether it is a moving or stationary object. Examples of such an obstacle include pedestrians, other vehicles, and walls. The process of detecting an obstacle from the captured image may involve, for example, performing brightness correction based on the brightness difference between the road surface and the obstacle on the road surface, followed by binarization processing to separate the obstacle from the image, geometric processing to correct distortion, and smoothing processing to remove noise from the image, thereby detecting the boundary between the road surface and the obstacle. The type of obstacle may also be detected using well-known template matching processing, feature point detection processing, or the like. The image recognition processing of the captured image is not limited to the above example, and may also be performed using, for example, machine learning. The detection range of obstacles by the camera is wider than that of the ultrasonic sensors 9A to 9L of S4, and even obstacles located far away from the vehicle can be detected as obstacles if they are included in the captured image. However, the detection accuracy is lower than that of the ultrasonic sensors 9A to 9L, and even if the presence of an obstacle can be detected, it is difficult to accurately identify its position.

[0045] If it is determined that an obstacle has been detected between the vehicle and the parking space based on the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B (S4: YES), the process proceeds to S8. On the other hand, if it is determined that no obstacle has been detected between the vehicle and the parking space based on the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B (S4: NO), the process proceeds to S5.

[0046] In S5, the CPU 31 calculates a parking trajectory, which is a travel trajectory for parking the host vehicle from the current position of the vehicle to the parking space identified in S2. In S5, since there are no obstacles around the vehicle that would affect the movement to the parking space, in the case of forward parking, a target parking position 56 (i.e., the position of the host vehicle when parking is completed) for parking the host vehicle in the parking space 55 is set as shown in FIG. 7, and a parking trajectory 57 for moving straight to the target parking position 56 is calculated. For example, the target parking position 56 is set to the center of the parking space 55 and to the position of the vehicle facing in the same direction as the orientation of the parking space. The orientation of the parking space 55 can be identified, for example, from parking stall lines or other parked vehicles. However, if the current orientation of the host vehicle does not match the vehicle orientation at the target parking position 56, the parking trajectory 57 is set to include a circular arc or a clothoid curve in part to adjust the vehicle's orientation. In the case of reverse parking, a parking trajectory is calculated in which the vehicle turns around before entering the parking space 55, and then the vehicle moves backward while turning to enter the parking space 55.

[0047] Thereafter, in S6, the CPU 31 starts parking assistance according to the parking trajectory calculated in S5. Specifically, the current position of the vehicle is detected at any time, and vehicle control such as steering, drive source, and brakes is automatically performed so that the vehicle travels at a specified speed along the generated parking trajectory. In addition, if a change in direction is required to enter a parking space, the shift position is also automatically switched.

[0048] However, when moving the vehicle along the parking path, only the steering operation may be performed automatically, and the drive source and brakes may be controlled manually. Alternatively, the vehicle may be moved manually rather than automatically. In this case, parking assistance may be provided by displaying the parking space and parking path on the LCD display 4, and providing voice guidance on when to turn the steering wheel.

[0049] Furthermore, even after it is determined that no obstacles exist when parking assistance begins and movement to the parking space begins, detection of surrounding obstacles continues using cameras and ultrasonic sensors, and if a new obstacle is detected (S3: YES, S4: YES), processing in S8 or S9, described below, is performed.

[0050] Next, in S7, the CPU 31 determines whether parking of the vehicle is complete. Specifically, when the vehicle is positioned at the target parking position 56 set in the parking space identified in S2 and the shift position of the vehicle is changed to "P," parking of the vehicle is determined to be complete.

[0051] If it is determined that the parking of the vehicle is complete (S7: YES), the parking assistance processing program is terminated. On the other hand, if it is determined that the parking of the vehicle is not complete (S7: NO), the process returns to S3.

[0052] On the other hand, in S8, which is executed when it is determined that an obstacle has been detected between the host vehicle and the parking space based on the images captured by the front camera 6, rear camera 7, and side cameras 8A, 8B (S4: YES), the CPU 31 calculates a parking trajectory, which is a driving trajectory for parking the host vehicle from the current position of the vehicle to the parking space identified in S2, after taking the detection result into consideration. Specifically, as will be described later, a target parking position, which is a target position for ultimately parking the vehicle, is set according to the detected obstacle, and the parking trajectory is calculated accordingly. Note that if a parking trajectory has already been generated, the parking trajectory is corrected as necessary.

[0053] However, because the detection accuracy of the camera is lower than that of the ultrasonic sensors 9A-9L, it is primarily used to confirm the parking space and its surroundings, and parking assistance is not terminated based on the camera detection results. For example, in this embodiment, the parking space is an independent parking space such as a home garage. However, if the camera detects an obstacle that defines the parking space in the vehicle width direction, i.e., the garage wall, it is also possible to calculate the distance between X1 and X2 on the left and right ends of the entrance, i.e., the width L of the garage entrance, as shown in FIG. 8. The entrance width L is used to set the target parking position, as described below. However, if it is determined that the entrance width L is insufficient compared to the vehicle width (e.g., entrance width < vehicle width + margin distance), it will be difficult for the vehicle to pass through the entrance (i.e., enter the parking space). Therefore, parking assistance should normally be terminated at that point, but parking assistance continues due to the possibility of erroneous detection. This makes it possible to prevent parking assistance from being terminated due to erroneous detection as much as possible. Furthermore, when the vehicle subsequently reaches the entrance, the width L of the entrance is detected again by the ultrasonic sensors 9A to 9L, and if the detection results of the ultrasonic sensors 9A to 9L also determine that the width L of the entrance is not sufficiently close to the vehicle width, the vehicle will stop and parking assistance will be discontinued (S11).

[0054] That is, in this embodiment, until it becomes possible for ultrasonic sensors 9A to 9L to detect the garage wall (more specifically, until ultrasonic sensors 9E and 9G enter the garage entrance), the garage wall is detected by the camera, and after it becomes possible for ultrasonic sensors 9A to 9L to detect the garage wall (more specifically, after ultrasonic sensors 9E and 9G enter the garage entrance), the garage wall is detected by ultrasonic sensors 9A to 9L. This makes it possible to grasp the general situation of the parking space and its surroundings in advance using the camera, while making important decisions at a stage when accurate information can be obtained by the ultrasonic sensors.

[0055] To explain the process of S8 using an example, when an enclosed space such as a garage is identified as a parking space, the target parking position 56 is first set based on the positions X1 and X2 at the left and right ends of the entrance identified by the camera detection results, and the angles α and β of the left and right walls (α and β are angles relative to the vehicle's orientation, and only one of them may be detected). Specifically, as shown in FIG. 8, the target parking position 56 is set to the position of the vehicle located a predetermined distance inside the entrance, equidistant from X1 and X2, and facing in the same direction as angles α and β. Then, as shown in FIG. 9, a parking trajectory 57 for moving to the target parking position 56 is calculated based on the identified target parking position 56. If the vehicle's current orientation does not match the vehicle orientation at the target parking position 56, the parking trajectory 57 is set to include a circular arc or a clothoid curve in part to adjust the vehicle's orientation. Furthermore, if it is determined that a turnaround is necessary, the turnaround trajectory is also included in the parking trajectory 57. However, since the detection results of the camera have low accuracy, when the vehicle reaches the entrance, the ultrasonic sensors 9A to 9L detect the positions X1 and X2 at the left and right ends of the entrance and the angles α and β of the left and right walls again, and the target parking position 56 and parking trajectory 57 are corrected based on the results. Therefore, in this embodiment, the target parking position and parking trajectory are temporarily set (provisionally set) based on approximate information obtained in advance by the camera, and then the target parking position and parking trajectory can be corrected to more appropriate ones at a stage when accurate information can be obtained by the ultrasonic sensors.

[0056] Thereafter, the process proceeds to S6, where the vehicle is controlled to travel along the parking path calculated in S8.

[0057] In addition, in S9, which is executed when it is determined that an obstacle has been detected around the vehicle by the ultrasonic sensors 9A to 9L (S3: YES), the CPU 31 takes the detection result into consideration and calculates a parking trajectory, which is a driving trajectory for parking the vehicle from the current position of the vehicle to the parking space identified in S2. Specifically, as will be described later, a target parking position, which is a target position for ultimately parking the vehicle, is set according to the detected obstacle, and the parking trajectory is calculated accordingly. Note that if a parking trajectory has already been generated, the parking trajectory is corrected as necessary.

[0058] Furthermore, because the detection accuracy of the ultrasonic sensors 9A-9L is high, they are used not only to confirm the status of the parking space and its surroundings, but also to determine whether parking assistance will be terminated. For example, in this embodiment, the parking space is an independent parking space such as a home garage. However, if the ultrasonic sensors 9A-9L detect an obstacle that defines the parking space in the vehicle width direction, i.e., the garage wall, they can also calculate the distance between X1 and X2 on the left and right ends of the entrance, i.e., the width L of the garage entrance, as shown in FIG. 8. In this case, if it is determined that the entrance width L is not sufficient compared to the vehicle width (e.g., entrance < vehicle width + margin distance) (S10: YES), it is difficult for the vehicle to pass through the entrance (i.e., enter the parking space), and parking assistance is terminated (S11). As a result, parking assistance can be terminated appropriately when it is difficult for the vehicle to park in the parking space.

[0059] As mentioned above, ultrasonic sensors 9E, 9F and ultrasonic sensors 9G, 9H are installed on the sides of the vehicle (Fig. 3). For these sensors to detect an obstacle, the obstacle must be located within detection ranges 45-48 (Fig. 6), and the vehicle must be located next to the obstacle to be detected. Therefore, when the parking space is located inside a garage, as shown in Fig. 10, ultrasonic sensors 9E and 9G are positioned at the entrance to the garage, and the distance to point X1 on the garage wall and the distance to point X2 on the garage wall can be obtained as distance measurements, respectively. From these distance measurements, the width L of the entrance can be calculated. Furthermore, as the vehicle then enters the garage, the distances to the left and right walls inside the garage can be obtained as distance measurements, and the angle α and angle β (relative angle) of the left and right walls can also be determined from the change in the distance measurements.

[0060] To explain the process of S9 using an example, if an enclosed space such as a garage is identified as a parking space, the target parking position 56 is first reset (corrected) based on the positions X1 and X2 at the left and right ends of the entrance identified by the detection results of the ultrasonic sensors 9E to 9H and the angles α and β of the left and right walls (only one of these may be detected). However, if there is no change in the detection results of the camera in S8, the current target parking position 56 is maintained. Specifically, as shown in FIG. 8, the target parking position 56 is determined to be the position of the vehicle located a predetermined distance inside the entrance, equidistant from X1 and X2, and facing in the same direction as angles α and β. Then, as shown in FIG. 9, a new parking trajectory 57 for moving to the target parking position 56 is calculated based on the identified target parking position 56. If the current orientation of the host vehicle does not match the vehicle orientation at the target parking position 56, the parking trajectory 57 is set to include a circular arc or a clothoid curve in order to adjust the vehicle's orientation. Furthermore, if it is determined that a turnaround is necessary, the turnaround trajectory is also included in the parking trajectory 57. The detection of X1, X2, angle α, and angle β by the ultrasonic sensors 9A to 9L is more accurate than that by a camera, so it is possible to correct the target parking position and parking trajectory to be more appropriate at a stage when accurate information can be obtained by the ultrasonic sensors.

[0061] After that, the process proceeds to S6, where the vehicle is controlled to travel along the parking trajectory calculated in S9. In addition, if a warning object such as a pedestrian is detected around the vehicle while parking assistance is being performed, a warning image indicating the presence of the warning object will be displayed at the position of the warning object in the overhead image.

[0062] 8 to 10 are examples where the parking space is located in a garage, but similar effects can be expected when the system is implemented in a case where there is an obstacle that defines the width of the parking space. For example, as shown in FIG. 11, a case where the width of the parking space 55 is defined by another parked vehicle will be described. First, until the host vehicle approaches the parking space 55, the target parking position 56 is set based on the positions X1 and X2 at the left and right ends of the parking space 55 identified by the camera detection results and the angles α and β of the other vehicles on the left and right (only one of these may be detected), and the parking trajectory 57 is also calculated. Then, when the host vehicle approaches the parking space 55 and the ultrasonic sensors 9A to 9L can detect the parked vehicles on the left and right, the ultrasonic sensors 9A to 9L again detect the positions X1 and X2 at the left and right ends and the angles α and β of the other vehicles on the left and right. Based on the results, the target parking position 56 and the parking trajectory 57 are corrected to be more appropriate.

[0063] As explained in detail above, according to the parking assistance device 1 and the computer program executed by the parking assistance device 1 of this embodiment, a parking space for parking a vehicle is identified (S2), and parking assistance is performed to park the vehicle in the identified parking space (S6). During the process of the vehicle entering the parking space, until it becomes possible to detect an obstacle using the ultrasonic sensors 9A to 9L, obstacles forming the vehicle width direction of the parking space are detected based on images captured by the front camera 6, rear camera 7, and side cameras 8A, 8B installed on the vehicle (S4, S8). In the system, parking assistance continues even when it is determined that it is difficult for the vehicle to enter the parking space, and once it becomes possible for the ultrasonic sensors 9A to 9L to detect an obstacle, obstacles forming the vehicle width direction of the parking space are detected based on the distance measurements detected by the ultrasonic sensors 9A to 9L installed in the vehicle (S3, S9), and if it is determined that it is difficult for the vehicle to enter the parking space at that time, parking assistance is stopped (S11), so it is possible to accurately determine whether to pass through an obstacle while preventing parking assistance from being stopped or the parking route being changed after assistance has started as much as possible.In particular, while the general situation of the parking space and its surroundings is grasped in advance using a camera, the important decision on whether to continue parking assistance can be made at a stage when accurate information can be obtained from the ultrasonic sensors. Furthermore, until it becomes possible for the ultrasonic sensors 9A to 9L to detect an obstacle, a target parking position is provisionally set in the parking space based on the result of obstacle detection by the ultrasonic sensors 9A to 9L, and parking assistance is performed according to the parking trajectory to the provisionally set target parking position. After it becomes possible for the ultrasonic sensors 9A to 9L to detect an obstacle, the target parking position is corrected based on the result of obstacle detection by the ultrasonic sensors 9A to 9L, and parking assistance is performed according to the parking trajectory to the corrected target parking position. Therefore, by gradually improving the accuracy of parking assistance in accordance with changes in the accuracy of the information that can be obtained, it becomes possible to ultimately perform parking assistance with high accuracy. Furthermore, since the parking space is a parking space within a garage, and the obstacles that are detected by the camera and ultrasonic sensor are the walls of the garage, even when parking a vehicle in a parking space that is difficult to identify from the outside, such as inside a garage, appropriate parking assistance can be provided by detecting the walls of the garage that form the width of the parking space. In addition, ultrasonic sensors 9E to 9H are distance measurement sensors that are installed on the sides of the vehicle and transmit detection waves to the sides of the vehicle.When the vehicle enters a parking space, the ultrasonic sensors 9E to 9H detect the walls of the garage using a camera until they enter the garage entrance, and after they enter the garage entrance, they detect the walls of the garage using an ultrasonic sensor.Therefore, it is possible to appropriately use the detection results using the imaging device and the detection results using the distance measurement sensor as the vehicle enters the garage.

[0064] The present invention is not limited to the above-described embodiment, and it goes without saying that various improvements and modifications are possible within the scope of the present invention. For example, in this embodiment, ultrasonic sensors 9E to 9H located on the sides of the vehicle are used to detect obstacles that define the width of the parking space, but ultrasonic sensors located on the front or rear of the vehicle may also be used to detect them. Also, while the garage wall has been used as an example of an obstacle that defines the width of the parking space, other vehicles, fences, guardrails, etc. are also possible obstacles.

[0065] In addition, in this embodiment, it is assumed that the vehicle is parking in a parking space by forward parking, but it may also be parking backward or parallel parking. When parallel parking, the obstacles that form the parking space to be detected are obstacles (for example, parked vehicles) located in front of or behind the parking space where parallel parking is performed, and are detected by ultrasonic sensors located on the front and rear of the vehicle.

[0066] In this embodiment, the parking assistance processing program (FIG. 5) is executed by the driving assistance ECU 10 of the parking assistance device 1, but the executing entity can be changed as appropriate. For example, the program may be executed by the control unit of the liquid crystal display 4, the vehicle control ECU, the control unit of the navigation device, or other in-vehicle device. [Explanation of symbols]

[0067] 1... parking assistance device, 2... vehicle, 3... operation unit, 4... liquid crystal display, 6... front camera (imaging device), 7... rear camera (imaging device), 8A, 8B... side cameras (imaging devices), 9A to 9L... ultrasonic sensors (distance measuring sensors), 10... driving assistance ECU, 55... parking space, 56... target parking position, 57... parking trajectory

Claims

1. a parking space specifying means for specifying a parking space for parking a vehicle; a parking assistance means for providing parking assistance for parking a vehicle in the parking space identified by the parking space identification means; a first obstacle detection means for detecting an obstacle that defines the parking space in a vehicle width direction based on an image of the surroundings of the vehicle captured by an imaging device installed in the vehicle; and a second obstacle detection means for detecting an obstacle that defines the parking space in the vehicle width direction based on a distance measurement value detected by a distance measurement sensor installed in the vehicle, The parking assistance means When the vehicle enters the parking space, detecting the obstacle by the first obstacle detection means until the obstacle can be detected by the distance measuring sensor; Continues parking assistance even when it is determined that it is difficult for the vehicle to enter the parking space based on the detection result of the first obstacle detection means, After the distance measuring sensor is able to detect the obstacle, the second obstacle detection means detects the obstacle; a parking assistance device that suspends parking assistance when it is determined that it is difficult for the vehicle to enter the parking space based on the detection result of the second obstacle detection means;

2. The parking assistance means Until the distance measuring sensor is able to detect the obstacle, a target parking position is provisionally set in the parking space based on the detection result of the obstacle by the first obstacle detection means, and parking assistance is performed according to a parking trajectory to the provisionally set target parking position; 2. The parking assistance device according to claim 1, wherein after the distance measuring sensor is able to detect the obstacle, the target parking position is corrected based on the detection result of the obstacle by the second obstacle detection means, and parking assistance is performed according to the parking trajectory to the corrected target parking position.

3. the parking space is a parking space in a garage, 3. The parking assistance device according to claim 1, wherein the obstacle to be detected by the first obstacle detection means and the second obstacle detection means is a wall of the garage.

4. The distance measurement sensor is installed on a side of a vehicle and transmits a search wave to the side of the vehicle, The parking assistance means When the vehicle enters the parking space, the first obstacle detection means detects the wall of the garage until the distance measurement sensor enters the garage entrance; 4. The parking assistance device according to claim 3, wherein after the distance measuring sensor has entered the entrance of the garage, the second obstacle detection means detects the wall of the garage.

Images