Parking support device

Abstract

To provide a parking support device which allows a parking space to be specified for properly parking a vehicle without being influenced by parking methods or an environment, by selectively using as required a detection result by a measuring sensor and a detection result using an imaging device.SOLUTION: A parking space for parking a vehicle is specified by using at least either a detection result by an ultrasonic sensor or a detection result by a camera. In particular for every type of parking method, setting is conducted as to whether to use a detection result by the ultrasonic sensor or a detection result by the camera, in specifying a parking space where parking is performed according to the parking method.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a parking assistance device that assists a vehicle in parking. [Background technology]

[0002] Conventionally, parking assistance devices have been proposed that provide parking assistance by displaying camera images of the vehicle's surrounding environment when the vehicle is parking, issuing warnings about people or obstacles in the vicinity, or performing some or all of the user's driving operations on the vehicle side.

[0003] Here, in order to properly perform the parking assistance described above, it is important that the device pre-specify the parking space in which the vehicle will be parked. Furthermore, there are various methods for parking a vehicle, such as parallel parking and perpendicular parking, and it is necessary to specify a parking space according to the parking method. For example, Japanese Patent Application Laid-Open No. 2016-16681 and Japanese Patent Application Laid-Open No. 2021-163066 propose technologies that use a camera mounted on a vehicle to detect parking frame lines painted on the road surface of a parking lot, and then specify a parking space in which to park the vehicle after pre-specifying a possible parking method on the device side based on the length and angle of the parking frame lines. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-16681 A (paragraph 0034, Figures 7 to 9) [Patent Document 2] JP 2021-163066 A (paragraphs 0043-0046) Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technologies described in Patent Documents 1 and 2 assume that parking frames are painted in parking lots, but there are many parking lots that do not have painted parking frames. Furthermore, the detection accuracy of cameras is greatly affected by the environment, and even if parking frames are painted, they may not be accurately detected at night or in bad weather. Therefore, there is a problem in that it is not possible to properly identify the parking space in which the vehicle will park.

[0006] Furthermore, in the above-mentioned Patent Document 2, an ultrasonic sensor is used in addition to a camera to detect surrounding obstacles, and a determination is made as to whether parking in a parking space is possible based on whether an obstacle is detected (for example, paragraph 0075). However, while ultrasonic sensors can detect surfaces facing the vehicle (surfaces perpendicular or nearly perpendicular to the detection axis of the sensor), it is difficult to detect surfaces in the depth direction as viewed from the vehicle (surfaces parallel or nearly parallel to the detection axis of the sensor), and so there have been cases where it has been determined that parking is not possible after the vehicle has actually entered the parking space.

[0007] The present invention has been made to solve the above-mentioned problems in the conventional technology, and aims to provide a parking assistance device that makes it possible to identify a parking space in which to properly park a vehicle, regardless of the parking method or environment, by appropriately using the detection results from a distance measurement sensor and the detection results from an imaging device. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the parking assistance device of the present invention comprises a first obstacle detection means that detects obstacles that form a parking space based on distance values ​​detected by a distance measurement sensor installed in the vehicle, a second obstacle detection means that detects obstacles that form a parking space based on images of the area around the vehicle captured by an imaging device installed in the vehicle, and a parking space identification means that identifies a parking space in which the vehicle is to be parked using at least one of the detection results of the first obstacle detection means and the second obstacle detection means, and there are multiple types of parking methods that the vehicle can select, and for each type of parking method, it is set as to whether the parking space identification means will use the detection results of the first obstacle detection means or the detection results of the second obstacle detection means, or both, when identifying a parking space in which to park using that parking method. "Parking method" includes, for example, parallel parking, where a vehicle is parked in the same straight line as other parked vehicles, and parallel parking, where a vehicle is parked parallel to other parked vehicles. Furthermore, parallel parking includes not only parking perpendicular to the aisle (the direction in which the parked vehicle is moving) but also diagonal parking, where a vehicle is parked at an angle. [Effects of the Invention]

[0009] According to the parking assistance device of the present invention having the above-described configuration, it is equipped with both a distance measuring sensor and an imaging device, and identifies a parking space using a means suitable for identifying a parking space for parking using each type of parking method, making it possible to identify a parking space for parking a vehicle appropriately regardless of the parking method or environment. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of a vehicle according to an embodiment of the present invention. [Figure 2] 1 is a diagram showing an example of the arrangement of ultrasonic sensors on the front of a vehicle; [Figure 3] 1 is a diagram showing an example of the arrangement of ultrasonic sensors on the side of a vehicle; [Figure 4] 1 is a block diagram showing the configuration of a parking assistance device according to an embodiment of the present invention. [Figure 5] 4 is a flowchart of a parking assistance processing program according to the present embodiment. [Figure 6] 10A and 10B are diagrams illustrating distance measurements calculated based on the detection results of ultrasonic sensors arranged on the sides of a vehicle. [Figure 7] 10A and 10B are diagrams illustrating a method for detecting an obstacle based on distance measurements of an ultrasonic sensor disposed on the side of a vehicle. [Figure 8] 1A and 1B are diagrams illustrating problems with obstacle detection using an ultrasonic sensor. [Figure 9] FIG. 10 is a diagram illustrating a method for specifying a parking space for parallel parking. [Figure 10] FIG. 10 is a diagram illustrating a method for specifying a parking space for performing double parking. [Figure 11] FIG. 10 is a diagram illustrating a parking trajectory. [Figure 12] FIG. 10 is a diagram showing an example of guidance for a specified parking space. DETAILED DESCRIPTION OF THE INVENTION

[0011] A specific embodiment of a parking assistance device according to the present invention will be described below in detail with reference to the drawings. First, a vehicle 2 equipped with a parking assistance device 1 according to this embodiment will be described below. Figure 1 is a schematic diagram of the vehicle 2 according to this embodiment.

[0012] Here, the vehicle 2 may be, for example, an automobile (internal combustion engine automobile) that uses an internal combustion engine (engine, etc.) as a drive source, an automobile (electric automobile, fuel cell automobile, etc.) that uses an electric motor (motor, etc.) as a drive source, or an automobile that uses both of these as a drive source (hybrid automobile). Furthermore, the vehicle type is not limited, and it may be a standard car, or a large commercial truck, bus, construction machinery, etc. Furthermore, although the following description will be of a four-wheeled automobile, it may also be a two-wheeled or three-wheeled vehicle.

[0013] However, vehicle 2 is a vehicle capable of manual driving, in which the vehicle drives based on the driving operation of the user, as well as assisted driving using automatic driving assistance, in which the vehicle drives automatically without the driving operation of the user.

[0014] Furthermore, autonomous driving assistance may be performed only under specific circumstances, such as when parking or leaving a parking lot, or may be performed for all road sections, or may be performed only while the vehicle is traveling on a specific road section (for example, a highway with a gate (manned or unmanned, toll or free) at the boundary). In the following description, the autonomous driving section in which autonomous driving assistance is performed includes all road sections, including general roads and highways, as well as parking lots, and is performed only when the user selects to perform autonomous driving assistance (for example, turns on the autonomous driving start button) and it is determined that autonomous driving assistance is possible. On the other hand, vehicle 2 may be a vehicle that is only capable of assisted driving with autonomous driving assistance. Alternatively, autonomous driving assistance may be performed only when the vehicle is traveling to a parking space when parking (i.e., parking assistance).

[0015] In the vehicle control in the automated driving assistance of this embodiment, for example, the current position of the vehicle, the lane the vehicle is traveling on, and the positions of surrounding obstacles are detected as needed, and vehicle control of the steering, drive source, brakes, etc. is automatically performed so that the vehicle travels along the generated travel trajectory at a speed in accordance with the generated speed plan. In particular, when performing parking assistance, candidate parking spaces for parking the vehicle are identified in advance using detection results from sensors and cameras, as described below, and the user is also prompted to select a parking method (e.g., parallel parking, perpendicular parking) if multiple parking methods are available. Then, the vehicle is automatically controlled to enter a parking space along a parking trajectory from the candidate parking spaces to a parking space selected as the final parking space, and to complete parking. However, it is also possible to automatically perform only the steering operation and manually control the drive source and brakes. Alternatively, it is also possible to provide only parking space guidance and have the user manually park the vehicle into the parking space. Furthermore, when providing the above-mentioned autonomous driving assistance, the scenery (actual scene) around the vehicle captured by a camera installed in the vehicle is displayed on the vehicle's display, and if there is a warning object such as a pedestrian around the vehicle, a warning image indicating the presence of the warning object is superimposed on the position of the warning object within the scenery.

[0016] 1, the vehicle 2 has an operation unit 3 that accepts operations from the occupant, a liquid crystal display 4 that displays bird's-eye and overhead images of the vehicle's surroundings and other driving assistance-related information to the occupant, a speaker 5 that outputs audio guidance related to the driving assistance, a front camera 6, a rear camera 7, and side cameras 8A and 8B for capturing images of the vehicle's surroundings, ultrasonic sensors 9A to 9L that detect obstacles around the vehicle, and a driving assistance ECU (electronic control unit) 10 that performs various calculations based on input information. The driving assistance ECU 10 and other components are collectively referred to as the parking assistance device 1.

[0017] Each component of the vehicle 2 will be described below. First, the operation unit 3 is arranged, for example, on the front of the steering wheel, and includes an operation button that is operated when starting automatic driving assistance and an operation button used to select a parking method during parking assistance. By operating the operation unit 3, the user can switch between manual driving, in which the vehicle drives based on the user's driving operation, and automatic driving assistance, in which the vehicle drives automatically without the user's driving operation, and can also select a parking method during parking assistance. The operation unit 3 may have a touch panel provided on the front of the liquid crystal display 4. It may also have a microphone and a voice recognition device.

[0018] The LCD display 4 is mounted on the instrument panel of the vehicle 2 and displays bird's-eye and overhead images of the vehicle's surroundings generated by performing viewpoint conversion and synthesis processing on images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B during autonomous driving assistance. When parking assistance begins, the LCD display 4 also displays potential parking spaces to guide the user. Furthermore, if there is a warning object such as a pedestrian near the vehicle 2, a warning image indicating the presence of the warning object at the position of the warning object in the bird's-eye and overhead images is also displayed. The LCD display 4 may also be used for a navigation device.

[0019] The speaker 5 is mounted on the instrument panel of the vehicle 2 and outputs voice guidance and warning sounds related to driving assistance. The speaker 5 may also be used for a navigation device.

[0020] The forward camera 6 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, with its optical axis facing forward in the direction of travel of the vehicle.

[0021] The rear camera 7 is an imaging device having a camera that also uses a solid-state imaging element such as a CCD, and is installed, for example, near the upper center of the license plate attached to the rear of the vehicle 2, with the optical axis facing toward the rear of the vehicle.

[0022] Furthermore, the side cameras 8A and 8B are imaging devices each having a camera using a solid-state imaging element such as a CCD, and are attached to the left and right side mirrors of the vehicle 2, for example, with their optical axes directed to the sides of the vehicle.

[0023] The driving assistance ECU 10 then performs viewpoint conversion and synthesis processing on the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B to generate bird's-eye and overhead images of the vehicle's surroundings. During autonomous driving assistance, the driving assistance ECU 10 also performs image recognition processing on the captured images to detect lane lines, parking frame lines, and obstacles (other vehicles, pedestrians, bicycles, walls, guardrails, and other structures) around the vehicle, and performs autonomous driving assistance based on the detection results. In particular, when performing parking assistance, the system also identifies potential parking spaces using the obstacle detection results from the cameras.

[0024] Meanwhile, the ultrasonic sensors 9A-9L are arranged at predetermined intervals on the front, rear, and sides of the vehicle 2. They transmit ultrasonic waves as search waves around the vehicle 2 and receive reflected waves from objects around the vehicle, thereby detecting the objects that reflect the search waves. Specifically, they are a type of distance measurement sensor that can measure the distance (measured distance) to the object that reflected the search wave by measuring the time from transmission to reception. The ultrasonic sensors 9A-9L are also configured to generate output signals (including the distance to the detected object) corresponding to the reception results of the received waves and output them to the control unit. Examples of objects that can be detected by the ultrasonic sensors 9A-9L include obstacles that the vehicle 2 must avoid when traveling, such as people, bicycles, other vehicles, and walls, as well as obstacles that form parking spaces. Instead of ultrasonic sensors, millimeter-wave sensors or laser sensors may be used as distance measurement sensors.

[0025] The installation position and installation direction of each ultrasonic sensor 9A-9L can be set as appropriate. In this embodiment, to detect objects in all directions (forward, backward, left, and right) of the vehicle 2, for example, ultrasonic sensors 9A-9D are installed on the front of the vehicle 2 facing the vehicle's traveling direction so that the transmission direction of the search wave is forward. Ultrasonic sensors 9E and 9F are installed on the left side of the vehicle 2 facing left so that the transmission direction of the search wave is to the left of the vehicle's traveling direction. Ultrasonic sensors 9G and 9H are installed on the right side of the vehicle 2 facing right so that the transmission direction of the search wave is to the right of the vehicle's traveling direction. Ultrasonic sensors 9I-9L are installed on the rear of the vehicle 2 facing the opposite direction to the vehicle's traveling direction so that the transmission direction of the search wave is to the rear of the vehicle. The ultrasonic sensors 9A-9L are all approximately the same height from the ground surface.

[0026] Taking ultrasonic sensors 9A to 9D as an example in particular, it is desirable that ultrasonic sensors 9A to 9D be installed at different positions around the front bumper or the front grille above the front bumper of vehicle 2, as shown in Figure 2, with equal spacing between them without bias in the left and right directions, so that they can transmit detection waves over a wider range in front of the vehicle (i.e., the range in which objects can be detected is wider).

[0027] Specifically, as shown in FIG. 2 , ultrasonic sensor 9A is installed near the left front corner of vehicle 2 with the transmission direction of the probe wave tilted slightly leftward from the direction of travel of vehicle 2 so that it transmits the probe wave to the left front of vehicle 2. Ultrasonic sensor 9B is installed slightly left of the center line of vehicle 2 with the transmission direction of the probe wave tilted toward the vehicle's direction of travel so that it transmits the probe wave mainly from the front, particularly the left side, of vehicle 2. Ultrasonic sensor 9C is installed slightly right of the center line of vehicle 2 with the transmission direction of the probe wave tilted toward the vehicle's direction of travel so that it transmits the probe wave mainly from the front, particularly the right side, of vehicle 2. Ultrasonic sensor 9D is installed near the right front corner of vehicle 2 with the transmission direction of the probe wave tilted slightly rightward from the direction of travel of vehicle 2 so that it transmits the probe wave to the right front of vehicle 2. Ultrasonic sensors 9A and 9D, and ultrasonic sensors 9B and 9C are each arranged symmetrically across the vehicle's center line in a plan view. Although not shown, the ultrasonic sensors 9I to 9L on the rear surface of the vehicle 2 are similarly arranged in a vertically symmetrical manner.

[0028] On the other hand, as shown in Figure 3, the ultrasonic sensors 9E and 9F on the sides are each installed so as to emit search waves in a direction that intersects at 90 degrees with the direction of travel of the vehicle 2. Since there are fewer sensors installed relative to the range on the sides compared to the front and rear of the vehicle as described above, there are areas where the ultrasonic sensors 9E and 9F cannot directly detect objects. However, in these areas, it is possible to estimate the presence or position of an object from the detection history of the object by the ultrasonic sensors 9A to 9L. Although not shown, the ultrasonic sensors 9G and 9H on the right side of the vehicle 2 are similarly symmetrical.

[0029] In this embodiment, among the ultrasonic sensors 9A-9L, the ultrasonic sensors 9A-9D on the front of the vehicle 2 and the ultrasonic sensors 9I-9L on the rear of the vehicle 2 are particularly installed in positions where they can receive reflected waves from adjacent sensors as indirect waves, so that by receiving direct and indirect waves, it is possible to determine not only the distance to an object but also the specific position of the object (its position relative to the vehicle) using triangulation. The ultrasonic sensors 9E-9H on the sides are installed at a distance from each other and cannot receive indirect waves, but as the vehicle moves, it is possible to determine the specific position of the object (its position relative to the vehicle) using triangulation (FIG. 6), which will be described later, using the measured distances at the previous and current positions and the distance traveled between them.

[0030] Meanwhile, the driving assistance ECU 10 is an electronic control unit that performs various processes related to autonomous driving assistance. For example, it constantly detects the vehicle's current position, the lane the vehicle is traveling on, and the positions of surrounding obstacles, and controls the vehicle, including steering, drive source, and braking, to drive the vehicle along the generated driving trajectory at a speed according to the generated speed plan. In particular, when performing parking assistance, it uses the detection results of the front camera 6, rear camera 7, side cameras 8A and 8B, and ultrasonic sensors 9A-9L to identify candidate parking spaces for the vehicle, and controls the vehicle to enter the parking space along the parking trajectory to the selected parking space from the candidate parking spaces, thereby completing the parking. The LCD display 4 also displays the scenery (real scene) around the vehicle, and if there are warning objects such as pedestrians around the vehicle, it superimposes a warning image indicating the presence of the warning object on the position of the warning object in the scenery. The driving assistance ECU 10 is connected to the operation unit 3, LCD display 4, speaker 5, front camera 6, rear camera 7, side cameras 8A and 8B, and ultrasonic sensors 9A to 9L via an in-vehicle network such as a CAN. The driving assistance ECU 10 is also connected to various sensors mounted on the vehicle 2, such as a vehicle speed sensor, an acceleration sensor, a gyro sensor, a steering sensor, and a shift position sensor, as well as to a navigation device or other in-vehicle device. The detailed configuration of the driving assistance ECU 10 will be described later.

[0031] In addition, vehicle 2 has basic components as vehicle 2 in addition to the components shown in Figure 1, but we will only explain the configuration related to the control of automatic driving assistance and the control related to that configuration.

[0032] Next, a detailed description will be given of the driving assistance ECU 10 in particular of the parking assistance device 1 provided in the vehicle 2. Fig. 4 is a block diagram showing the configuration of the parking assistance device 1 according to this embodiment.

[0033] As shown in FIG. 4, the driving assistance ECU (electronic control unit) 10 is an electronic control unit that controls the entire parking assistance device 1. It includes a CPU 31, which functions as a calculation device and a control device; a RAM 32, which serves as a working memory for the CPU 31 to perform various calculation processes and stores data such as travel path data calculated when the travel path is calculated; a ROM 33, which stores control programs and a parking assistance processing program (see FIG. 5) described below; and a flash memory 34, which stores programs read from the ROM 33. The driving assistance ECU 10 includes various processing algorithms. For example, the first obstacle detection means detects obstacles forming a parking space based on distance measurements detected by ultrasonic sensors 9A-9L installed on the vehicle. The second obstacle detection means detects obstacles forming a parking space based on images of the surroundings of the vehicle captured by the front camera 6, rear camera 7, and side cameras 8A and 8B installed on the vehicle. The parking space specifying means specifies a parking space for parking the vehicle using at least one of the detection result of the first obstacle detection means and the detection result of the second obstacle detection means.

[0034] The driving assistance ECU 10 is also connected to various sensors 36 for detecting vehicle behavior, such as a vehicle speed sensor, an acceleration sensor, a gyro sensor, a steering sensor, and a shift position sensor, as well as to each of the vehicle's drive units 37, such as the steering, brake, accelerator, and transmission, and detects the current vehicle behavior based on the detection results of these sensors 36, while controlling each of the drive units 37 to provide automatic driving assistance for the vehicle 2. Specific details of the automatic driving assistance include, for example, constantly detecting the current vehicle position, the lane the vehicle is traveling on, and the positions of surrounding obstacles, and controlling the vehicle, such as the steering, drive source, and brakes, so that the vehicle travels along a generated travel trajectory at a speed in accordance with a speed plan that is also generated. However, it is also possible to automatically perform only the steering operation, while manually controlling the drive source and brakes.

[0035] The flash memory 34 also includes a vehicle information DB 35, which stores various information related to the vehicle 2. For example, the vehicle information DB 35 stores the installation positions (height from the ground and left-right positions) of the cameras and ultrasonic sensors 9A-9L installed on the vehicle 2, the detection axes (optical axes for cameras), overall length, vehicle width, wheelbase, minimum turning radius, etc. This information is input in advance by the occupants or a person from the vehicle manufacturer.

[0036] Next, a parking assistance processing program executed by the driving assistance ECU 10 in the parking assistance device 1 having the above configuration will be described with reference to Fig. 5. Fig. 5 is a flowchart of the parking assistance processing program according to this embodiment. Here, the parking assistance processing program is executed after the ACC power supply (accessory power supply) of the vehicle 2 is turned on, and is a program that provides parking assistance, particularly when parking the vehicle, as one type of automatic driving assistance. The program shown in the flowchart in Fig. 5 below is stored in the RAM 32 and ROM 33 provided in the parking assistance device 1, and is executed by the CPU 31.

[0037] First, the CPU 31 determines whether to start parking assistance in step (hereinafter abbreviated as S) 1. In particular, the parking assistance of this embodiment involves automatically identifying candidate parking spaces for parking the vehicle and moving the vehicle to a parking space that is ultimately identified as the parking space from among the candidate parking spaces.

[0038] For example, parking assistance may be started when the user operates the operation unit 3 and selects to start parking assistance, or parking assistance may be started automatically when it is detected that the vehicle has entered a parking lot, when it is determined that the vehicle has arrived at the set destination, or when the vehicle approaches a parking lot.

[0039] If it is determined that parking assistance should be started (S1: YES), the process proceeds to S2. On the other hand, if it is determined that parking assistance should not be started (S1: NO), the parking assistance processing program is terminated.

[0040] In S2, the CPU 31 stores the vehicle information at the time when the ultrasonic sensors 9A-9L transmit a search wave and the vehicle information at the time when the ultrasonic sensors 9A-9L receive a wave reflected from an obstacle, along with the acquired distance measurement values. Note that the ultrasonic sensors 9A-9L continue to transmit search waves at regular intervals until the parking assistance ends, and the following steps S2 and S3 are repeatedly executed until the parking assistance ends.

[0041] Details of the processing in S2 will be explained below with reference to Fig. 6. Note that in the present embodiment, an example will be described in which a parking space candidate (hereinafter referred to as a parking space candidate) is identified using the detection results of the ultrasonic sensors 9E, 9F or the ultrasonic sensors 9G, 9H installed on the sides of the vehicle 2, but it is also possible to identify a parking space candidate using the detection results of the ultrasonic sensors 9A to 9D installed in the front of the vehicle 2 or the ultrasonic sensors 9I to 9L installed in the rear.

[0042] First, the CPU 31 acquires various vehicle information at the timing when reflected waves were received by the ultrasonic sensors 9E and 9F or the ultrasonic sensors 9G and 9H at the previous position, as shown in Fig. 6. Specific vehicle information to be acquired includes, for example, the vehicle speed v, the current position coordinates (x1, y1) of the vehicle, and the vehicle direction φ1. This information is identified using, for example, a vehicle speed sensor, a gyro sensor, a steering sensor, etc. The CPU 31 also acquires the count value t1 of a timer that identifies the timing when reflected waves were received at the previous position.

[0043] Next, the CPU 31 similarly acquires various vehicle information at the timing when reflected waves are received by the ultrasonic sensors 9E, 9F or the ultrasonic sensors 9G, 9H at the current position. Specifically, the acquired vehicle information includes, as with the previous position, the vehicle speed v (the vehicle speed is assumed to remain unchanged because the interval between probe wave transmissions is short), the vehicle's current position coordinates (x2, y2), the vehicle direction φ2, and the timer count value t2. If reflected waves are not received within a predetermined time after transmitting ultrasonic waves (for example, until the next ultrasonic wave transmission timing), it is determined that the target object was not detected. The previous position and the current position are basically determined to be times when consecutive probe waves are received in chronological order, but they may be spaced apart, for example.

[0044] Thereafter, the CPU 31 calculates distance values ​​based on the time interval between transmitting the search wave and receiving the reflected wave from the target object at each of the previous position and the current position, and stores the calculated distance values ​​in association with the acquired vehicle information. Note that the distance values ​​calculated by the ultrasonic sensor are, as shown in Figure 6, a distance L1 from the sensor position S of the vehicle 2 at the previous position to the reflection point P where the reflected wave is reflected, and a distance L2 from the sensor position S' of the vehicle 2 at the current position to the reflection point P where the reflected wave is reflected.

[0045] Thereafter, the CPU 31 calculates the distance Δy traveled by the host vehicle from the previous position to the current position based on the host vehicle's speed v during distance measurement. Then, taking into consideration the vehicle information for the previous position and the vehicle information for the current position, the CPU 31 calculates the coordinates (x3, y3) of the reflection point P where the reflected wave is reflected by triangulation using the measured distance L1 for the previous position, the measured distance L2 for the current position, and the distance traveled Δy therebetween. Note that the above calculation example is just one example, and it is also possible to calculate the coordinates (x3, y3) of the reflection point P using other methods.

[0046] The CPU 31 then stores the calculated coordinates (x3, y3) of the reflection point P as point sequence data in the flash memory 34 or the like. The coordinates (x3, y3) of the reflection point P identify the position of the obstacle (more specifically, the position of the surface that forms the obstacle's outline), and the shape (surface) of the obstacle is identified by connecting the point sequence data. Note that the above example is an example in which the coordinates of the reflection point P are calculated using distance measurements detected by the ultrasonic sensors 9E, 9F or the ultrasonic sensors 9G, 9H installed on the sides of the vehicle 2. However, when calculating the coordinates of the reflection point P using distance measurements detected by the ultrasonic sensors 9A-9D installed in front of the vehicle 2 or the ultrasonic sensors 9I-9L installed in the rear, triangulation can be used to calculate the coordinates of the reflection point P by receiving direct and indirect waves as received waves. The direct wave is the wave received when the ultrasonic sensor that transmitted the search wave is the same as the ultrasonic sensor that received the wave reflected by the object of the search wave as a received wave. In contrast, an indirect wave is a wave received when the ultrasonic sensor that transmitted the search wave is different from the ultrasonic sensor that received the wave reflected by the object of the search wave as a received wave. For example, if a search wave transmitted from ultrasonic sensor 9A is reflected at reflection point P and received as an indirect wave by ultrasonic sensor 9B, triangulation can be performed using the distance from ultrasonic sensor 9A to reflection point P detected using the direct wave, the distance from ultrasonic sensor 9B to reflection point P detected using the indirect wave, and the distance between ultrasonic sensor 9A and ultrasonic sensor 9B to calculate the position of reflection point P. Then, the process proceeds to S4.

[0047] In S4, the CPU 31 detects an obstacle from the point sequence data of the reflection point P calculated in S3. Specifically, the CPU 31 determines the endpoints of the obstacle from the slope of the multiple point sequences, the rate of change of the slope, and the like, and identifies the position and shape of the obstacle. For example, in a situation where other vehicles 41 and 42 are parallel parked along a road edge 40 as shown in FIG. 7, the point sequence data of the reflection point P will first identify the position and shape of the other vehicle 41 in the foreground, and then identify the position and shape of the other vehicle 42 in the background. Note that if there is a structure such as a guardrail on the road edge 40, the road edge 40 between the other vehicles 41 and 42 can also be detected.

[0048] However, when detecting an obstacle using an ultrasonic sensor, it is possible to detect an obstacle on a surface facing the host vehicle (a surface perpendicular or nearly perpendicular to the detection axis of the sensor) because a reflection point P exists there, as shown in Figure 7, but it is difficult to detect on a surface in the depth direction as viewed from the host vehicle (a surface parallel or nearly parallel to the detection axis of the sensor) because there is no reflection point P or even if there is, it is only a small reflection point P. Therefore, for example, when other vehicles are parked side-by-side as shown in Figure 8, the angle of the surface in the depth direction of the parked other vehicle cannot be identified, which poses a problem in that it is not possible to determine whether the vehicle is parked perpendicular to the aisle (the traveling direction of the host vehicle that is parking) or at an angle.

[0049] Therefore, in S5, the CPU 31 detects obstacles by performing image recognition on the captured images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. The process of detecting obstacles from the captured images may involve, for example, performing brightness correction based on the brightness difference between the road surface and obstacles on the road surface, followed by binarization processing to separate the obstacles from the image, geometric processing to correct distortion, and smoothing processing to remove noise from the image, thereby detecting the boundary between the road surface and the obstacles. The type of obstacle may also be detected using well-known template matching processing, feature point detection processing, or the like. The image recognition process for the captured images is not limited to the above examples, and may also be performed using, for example, machine learning.

[0050] However, the accuracy of obstacle detection by a camera is lower than that of an ultrasonic sensor, and it may not be possible to detect obstacles, especially at night or in bad weather.On the other hand, a camera can detect surfaces in the depth direction (surfaces parallel or nearly parallel to the detection axis of the sensor), which are difficult to detect with an ultrasonic sensor.

[0051] In the above example, the obstacle detection process by the ultrasonic sensors 9A-9L in S2-S4 is performed first, followed by the obstacle detection process by the camera in S5. However, in reality, after it is determined in S1 that parking assistance should be started, each process is executed independently and in parallel. However, they may be executed in a predetermined order instead of in parallel. Then, in S6, the CPU 31 integrates (combines) the obstacle detection results by the ultrasonic sensors 9A-9L in S2-S4 (detection results by the ultrasonic sensors, hereinafter referred to as sensor detection results) and the obstacle detection results by the camera in S5 (i.e., detection results by the camera, hereinafter referred to as camera detection results).

[0052] Next, in S7, the CPU 31 identifies parking space candidates that are candidates for parking the vehicle based on the obstacle detection results integrated in S6. In particular, in this embodiment, parking space candidates for parallel parking and parking space candidates for perpendicular parking are identified separately. However, if parking is not possible using only one of the parking methods, only parking space candidates for parking using the parking method that is possible are identified.

[0053] In this embodiment, for each type of parking method selectable by the vehicle, when identifying a parking space for parking using that parking method, it is preset whether to use the sensor detection results or the camera detection results among the obstacle detection results integrated in S6. More specifically, when identifying a parking space candidate for parallel parking, the sensor detection results are used but the camera detection results are not used. On the other hand, when identifying a parking space candidate for perpendicular parking (including diagonal parking), both the sensor detection results and the camera detection results are used to identify the parking space candidate.

[0054] Here, the parking space candidate 43 for parallel parking is specified as a rectangle with one long side located along an extension of the surface of the obstacle facing the vehicle, as shown in Figure 9. The width and height of the rectangle are lengths obtained by adding a predetermined margin distance to the length and width of the vehicle.

[0055] Therefore, when identifying a parking space candidate 43 for parallel parking, it is basically possible to identify an appropriate parking space as long as the orientation of the surface of the obstacle facing the host vehicle can be identified, regardless of the angle of the depth-direction surface of the surrounding obstacles that form the parking space candidate 43. For example, as shown in FIG. 9, if the other vehicle forming the parking space candidate 43 is parked at an angle, even if the angle α or angle β between the facing surface of the other vehicle and the depth-direction surface cannot be identified, the parking space candidate 43 can be accurately identified as long as the orientation γ of the surface of the other vehicle facing the host vehicle can be accurately identified. In particular, as shown in Example 2, even if the host vehicle is tilted relative to the parking space candidate, the parking space candidate 43 can be accurately identified as long as the orientation γ of the surface of the other vehicle facing the host vehicle can be accurately identified. As described above, the surface of the obstacle facing the host vehicle can be identified based on the obstacle detection results obtained by an ultrasonic sensor, and this detection accuracy is higher than when identifying the surface based on the obstacle detection results obtained by a camera. Therefore, when identifying a parking space candidate 43 for parallel parking, the results of obstacle detection by the camera are not used, and the parking space candidate 43 is identified using only the results of obstacle detection by the ultrasonic sensor.

[0056] Then, for the surface of the obstacle forming the parking space candidate 43 facing the vehicle, if the distance over which the surface of the obstacle facing the vehicle is continuously detected, i.e., the distance L in FIG. 9, is equal to or greater than a predetermined distance (e.g., the length of a typical vehicle), the reliability of the surface (or its orientation) is determined to be high (above a threshold). On the other hand, if the surface of the obstacle facing the vehicle is detected only within a predetermined distance, the reliability of the surface (or its orientation) is determined to be low (below a threshold). Note that the reliability of the surface of the opposing obstacle also corresponds to the reliability of the parking space candidate 43. Therefore, if the reliability of the surface of the opposing obstacle is determined to be low, the parking space candidate 43 is not confirmed but is set as a provisional setting, and is subject to revision depending on the detection results of new obstacles thereafter. Furthermore, if the parking space candidate 43 is to be used as is, it is updated (revised) depending on new detection results obtained after the vehicle starts moving toward the parking space candidate 43 (S13).

[0057] On the other hand, a parking space candidate 44 for parallel parking is specified as a rectangular shape in which one short side or a corner (if inclined) is located along an extension of the surface of the obstacle facing the host vehicle, as shown in FIG. 10, and the rectangular shape is located a predetermined parking distance from the surface of the obstacle in the depth direction and in the same direction as the angle of the surface. That is, the parking space candidate 44 is specified based on the "surface of the obstacle facing the host vehicle" and the "angle between the surface of the obstacle facing the host vehicle and the surface of the obstacle in the depth direction." However, if the "angle between the surface of the obstacle facing the host vehicle and the surface of the obstacle in the depth direction" cannot be specified, the parking space candidate 44 is provisionally set perpendicular to the surface of the obstacle facing the host vehicle. The width and height of the rectangle are the length and width of the host vehicle plus a predetermined margin distance.

[0058] Therefore, when identifying a parking space candidate 44 for parallel parking, it is necessary to determine not only the surface of the obstacle forming the parking space candidate 44 facing the vehicle, but also the angle of the surface in the depth direction relative to the facing surface, in order to identify an appropriate parking space. For example, as shown in FIG. 10 , when another vehicle forming the parking space candidate 44 is parked at an angle, the angle of the parking space candidate 44 to be identified changes depending on the angle α and angle β of the surface in the depth direction. Therefore, if the angles α and β can be accurately determined, the parking space candidate 44 can be accurately identified. As mentioned above, it is difficult to identify the surface in the depth direction using the obstacle detection results obtained by an ultrasonic sensor, and therefore it is identified based on the obstacle detection results obtained by a camera. Therefore, when identifying a parking space candidate 44 for parallel parking, the parking space candidate 44 is identified using the obstacle detection results obtained by the ultrasonic sensor as well as the obstacle detection results obtained by a camera. On the other hand, for a parking space candidate 44 for parallel parking, the orientation γ of the surface of the obstacle facing the vehicle does not require high accuracy compared to parallel parking.

[0059] Regarding the angles α and β between the surface of an obstacle forming the parking space candidate 44 facing the vehicle and its depth surface, if the depth surface can be detected by the camera, the reliability of the angle is determined to be high (above a threshold). On the other hand, if the depth surface cannot be detected by the camera due to factors such as nighttime or bad weather, the reliability of the angle is determined to be low (below a threshold). However, in exceptional cases where the depth surface cannot be detected by the camera but the ultrasonic sensor can detect it (such as when the slope is closer to parallel to the aisle even in double parking), the reliability is determined to be high (above a threshold). Note that the reliability of the angles α and β also corresponds to the reliability of the parking space candidate 44. Therefore, if the reliability of the angles α and β is determined to be low, the parking space candidate 44 is not confirmed but is set as a provisional setting and is subject to revision depending on the detection results of new obstacles thereafter. Furthermore, if the parking space candidate 44 is to be used as is, the reliability is updated (revised) depending on new detection results obtained after the vehicle starts moving toward the parking space candidate 44. Specifically, even if another vehicle is parked at an angle as shown in Fig. 10, it is assumed that the vehicle is parked parallel to the direction perpendicular to the direction of travel of the vehicle, and the parking space candidate 44 is also provisionally set in the vertical direction. Thereafter, as the vehicle approaches the parking space candidate 44, the parking space candidate 44 is corrected to an inclined parking space candidate 44 (S13).

[0060] However, even if the depth surface can be detected by a camera, the detection accuracy of a camera is lower than that of an ultrasonic sensor. Therefore, even if the reliability of the angles α and β is determined to be high, the parking space candidate 44 may be set as a provisional setting without being finalized, as in the case of a low reliability, and may be subject to correction based on the detection result of a new obstacle thereafter. If the parking space is to be used as is, the parking space candidate 44 may be updated (corrected) based on the new detection result obtained after starting movement toward the parking space candidate 44 (S13). In this embodiment, as described above, when identifying a parking space for double parking, the depth surface of the obstacle is detected by a camera. Therefore, even if the parking direction is inclined, it is possible to identify the parking space according to the inclination before entering the parking space (before parking starts). As a result, even if the parking space is corrected slightly after parking assistance has started, it is possible to avoid major corrections or the suspension of assistance as much as possible.

[0061] The number of parking space candidates identified in S7 is not limited to one for each of double parking and parallel parking. If there are multiple spaces available for each parking method, all available spaces will be identified as parking space candidates. However, in S8 (described later), parking space candidates that are not available for parking will be excluded.

[0062] In S8, the CPU 31 calculates a parking trajectory, which is a driving trajectory for parking the host vehicle into each of the parking space candidates identified in S7. Taking the calculation of the parking trajectory for parallel parking as an example, the CPU 31 first sets a target parking position 57 (i.e., the position of the host vehicle when parking is completed) when parking the host vehicle into the parking space candidate 56 selected by the user, as shown in FIG. 11. Furthermore, the CPU 31 sets a reverse start position 58 (i.e., the position of the host vehicle where reverse starts to park) when parking the host vehicle into the parking space candidate 56 according to the parking trajectory. The reverse start position 58 is determined relative to the target parking position 57, taking into consideration the length, width, minimum turning radius, etc. of the host vehicle.

[0063] Next, the CPU 31 calculates a turning trajectory 59 from the reverse start position 58 to the target parking position 57. Specifically, the turning trajectory 59 is a path that allows the host vehicle to enter the parking space candidate without contacting any obstacle and that turns at an optimal steering angle derived from the vehicle's turning characteristics. For example, the turning trajectory 59 is calculated as a traveling trajectory that turns with a minimum turning radius. The turning trajectory 59 may be a circular arc or a clothoid curve, and may include a straight trajectory in part. Furthermore, the CPU 31 calculates a preparatory trajectory 60 from the current position of the vehicle to the reverse start position 58. The preparatory trajectory 60 may be a straight trajectory as shown in Fig. 11, or may include a circular arc or a clothoid curve depending on the positional relationship between the current position of the vehicle and the reverse start position 58. Furthermore, if a turn is required to correct the vehicle heading to an appropriate direction (basically a direction parallel to the parking space candidate), the CPU 31 also calculates the trajectory of the turn.

[0064] As a result, the parking trajectory calculated in S8 as shown in Figure 11 is composed of a combination of a preparation trajectory 60 that moves forward from the current vehicle position to the reverse start position 58, a turning trajectory 59 that turns from the reverse start position 58 to the target parking position 57, and a turning trajectory that requires turning if necessary. Also, in the above example, the case where the vehicle is performing perpendicular parking has been described, but when performing parallel parking, for example, there will be a steering position in the middle of the turning trajectory where the turning direction is changed to the opposite direction.

[0065] In addition, among the parking space candidates identified in S7, there are some for which the parking trajectory cannot be calculated from the current position of the vehicle, i.e., there are parking space candidates for which parking cannot be performed from the current position of the vehicle. In S8, such parking space candidates for which the parking trajectory cannot be calculated are excluded from the parking space candidates.

[0066] Thereafter, in S9, the CPU 31 displays the parking space candidates identified in S7 and for which the parking trajectory was plotted in S8 on the liquid crystal display 4. Note that a bird's-eye view image of the vehicle's surroundings viewed vertically downward from the sky, which is generated in advance based on real-time captured images taken by the front camera 6, the rear camera 7, and the side cameras 8A and 8B, is displayed on the liquid crystal display 4. Note that a bird's-eye view image viewed diagonally downward may be displayed instead of the bird's-eye view image, or both the bird's-eye view image and the bird's-eye view image may be displayed.

[0067] FIG. 12 shows an overhead image 61 of the area around the vehicle that is displayed on the LCD display 4 when a parking space candidate has been identified. As shown in FIG. 12, an illustration image 62 that schematically shows the vehicle's current position within the overhead image 61 is displayed. Furthermore, parking space images 63 and 64 that show the position and shape of the parking space candidate are displayed at the position of the parking space candidate identified in S7 and for which the parking trajectory was plotted in S8. The example shown in FIG. 12 shows a case in which one parking space candidate for parallel parking and one parking space candidate for perpendicular parking have been identified. Parking space image 63 indicates the parking space candidate for parallel parking, and parking space image 64 indicates the parking space candidate for perpendicular parking. If only one of the parking space candidates for parallel parking and the parking space candidate for perpendicular parking has been identified, only one of the parking space images will be displayed. In addition, in the example shown in Figure 12, only one parking space candidate for parallel parking and one parking space candidate for perpendicular parking are identified, but if there are multiple parking space candidates for perpendicular parking, for example, a parking space image may be displayed for each of the multiple parking space candidates.

[0068] The user can then arbitrarily select a parking space candidate where he or she wishes to park from among the parking space images 63 and 64 displayed on the liquid crystal display 4. Note that selecting a parking space candidate also corresponds to selecting a parking method. That is, if the parking space image 63 is selected, it means that the user has selected to perform parallel parking, and if the parking space image 64 is selected, it means that the user has selected to perform perpendicular parking.

[0069] 12 shows an example in which a parking space image 64 is selected. When the parking space image 64 is selected, all parking space images other than the selected parking space image 64 are temporarily hidden, and a confirmation window 65 is displayed on the liquid crystal display 4, prompting the user to confirm whether or not to start parking in the selected parking space candidate. If the user finally selects to start parking in the designated parking space candidate (S10: YES), the process proceeds to S11. On the other hand, if the user does not select to start parking in the designated parking space candidate (S10: NO), the process waits until a selection is made. Note that in addition to parking space images 63 and 64 indicating the position and shape of the parking space candidate, a parking trajectory for parking in the parking space candidate may also be displayed on the overhead image 61.

[0070] Then, in S11, the CPU 31 starts parking assistance according to the parking trajectory calculated in S8 for the parking space candidate selected by the user in S10 (hereinafter simply referred to as the parking space). Specifically, the current position of the vehicle and the positions of surrounding obstacles are detected as needed, and vehicle control such as steering, drive source, and braking is automatically performed so that the vehicle travels at a specified speed along the generated parking trajectory. In addition, if a change in direction is required to enter the parking space, the shift position is also automatically switched. The parking assistance continues until parking into the parking space is completed.

[0071] Furthermore, even after the vehicle has started moving to the parking space, the detection of surrounding obstacles using cameras and ultrasonic sensors continues, and if a warning object such as a pedestrian is detected around the vehicle, a warning image indicating the presence of the warning object is displayed at the position of the warning object in the overhead image. Furthermore, if it is detected that an obstacle such as another vehicle or a wall is particularly close to the vehicle in addition to the warning object, the parking assistance will be temporarily suspended or stopped.

[0072] However, when moving the vehicle along the parking trajectory, only the steering operation may be performed automatically, and the drive source and brakes may be controlled manually. Alternatively, the vehicle may be moved manually rather than automatically. In this case, parking assistance may be provided by displaying the parking trajectory on the LCD display 4 and providing voice guidance on the timing to turn the steering wheel.

[0073] Next, in S12, the CPU 31 determines whether the reliability of the surface or angle of the obstacle used to identify the parking space that is the parking target (S7) is low (less than a threshold). As described above, for a parking space where parallel parking is performed, if the distance L over which the surface of the obstacle facing the vehicle is continuously detected is less than a predetermined distance, the reliability of the surface of the obstacle facing the vehicle is determined to be low (Fig. 9). On the other hand, for a parking space where perpendicular parking is performed, if the camera cannot detect the surface of the obstacle in the depth direction as viewed from the vehicle, the reliability of the angle between the surface of the obstacle facing the vehicle and the surface of the obstacle in the depth direction is determined to be low (Fig. 10).

[0074] If it is determined that the reliability of the surface or angle of the obstacle used to identify the parking space as the parking target (S7) is low (less than the threshold) (S12: YES), the process proceeds to S13 to correct the parking space as the parking target as necessary. On the other hand, if it is determined that the reliability of the surface or angle of the obstacle used to identify the parking space as the parking target (S7) is high (above the threshold) (S12: NO), the parking space as the parking target has been determined, so there is no need to correct the parking space and parking assistance continues.

[0075] In S13, if the CPU 31 determines that the parking space that is the parking target needs to be corrected, it corrects the parking space. For example, as shown in FIG. 11 , a case will be described in which, at the stage of initially identifying the parking space 56, the angle β between the surface of the obstacle facing the host vehicle and the surface of the obstacle in the depth direction cannot be determined, and the parking space 56 is provisionally set perpendicular to the aisle (the traveling direction of the vehicle being parked). Subsequently, if the surface of the obstacle in the depth direction can be detected by a camera or ultrasonic sensor while the host vehicle is entering the parking space 56 and the angle β of the obstacle surface can be determined, the CPU 31 corrects the parking space 56 so that it is in the same direction as the angle β. Since the parking space 56 is corrected, the parking trajectory calculated in S8 also needs to be corrected, so the process returns to S8 and a parking trajectory 71 for parking from the current host vehicle position into the corrected parking space 56 is calculated again. If a turn is required to correct the vehicle's orientation, the turn is also included in the parking trajectory 71. Furthermore, even if the reliability of the surface or angle of the obstacle used to identify the parking space as the parking target (S7) is low (less than the threshold), the parking space will not necessarily be corrected, and if there is no need to correct it, the processing of S13 will not be performed.

[0076] As explained in detail above, the parking assistance device 1 and the computer program executed by the parking assistance device 1 according to this embodiment detect obstacles that form a parking space based on distance measurements detected by the ultrasonic sensors 9A to 9L installed in the vehicle (S4), while also detecting obstacles that form a parking space based on captured images of the area around the vehicle taken by the front camera 6, rear camera 7, and side cameras 8A, 8B installed in the vehicle (S5). At least one of the detection results from the ultrasonic sensors and the detection results from the cameras is used to identify a parking space for the vehicle (S7). In particular, for each type of parking method, it is set as to whether or not to use the detection results from the ultrasonic sensors or the detection results from the cameras when identifying a parking space for parking using that parking method. Therefore, a parking space can be identified by a means suitable for identifying a parking space for parking using that parking method for each type of parking method, and it becomes possible to identify a parking space for parking the vehicle appropriately regardless of the parking method or the environment. Furthermore, the parking methods selectable by the vehicle include at least parallel parking and perpendicular parking, and when specifying a parking space for parallel parking, the detection results from the ultrasonic sensor are used but not the camera detection results, and when specifying a parking space for perpendicular parking, the parking space is specified using both the detection results from the ultrasonic sensor and the camera detection results (S7). This makes it possible to specify a parking space using a means appropriate for specifying a parking space for parking by that parking method for each type of parking method. In particular, when specifying a parking space for perpendicular parking, even if the parking direction is inclined, it is possible to specify the parking space in accordance with the inclination before entering the parking space. Furthermore, when identifying a parking space for parallel parking, the parking space is identified based on the surface of the obstacle facing the vehicle, detected by the ultrasonic sensors 9A-9L or the camera, and the angle between the surface of the obstacle facing the vehicle and the surface of the obstacle in the depth direction as seen from the vehicle (S7). The reliability of the angle is determined to be equal to or greater than a threshold value when the surface of the obstacle in the depth direction as seen from the vehicle is detected by the camera, making it possible to accurately determine the reliability of the information used to identify a parking space for parallel parking, in particular. Furthermore, when identifying a parking space for parallel parking, the parking space is identified based on the surface of the obstacle facing the vehicle detected by the ultrasonic sensors 9A to 9L (S7), and the reliability of the surface of the obstacle facing the vehicle is determined to be above a threshold if the surface of the obstacle facing the vehicle is detected continuously for a predetermined distance or more, making it possible to accurately determine the reliability of the information used to identify a parking space for parallel parking, in particular.

[0077] The present invention is not limited to the above-described embodiment, and it goes without saying that various improvements and modifications are possible within the scope of the present invention. For example, in this embodiment, when a candidate parking space for parking is identified in S6, the identified candidate parking space is guided to the user (S9), but parking assistance may be performed without providing guidance.

[0078] In addition, in this embodiment, an overhead image generated from images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B is displayed on the LCD display 4 as a landscape image of the area around the vehicle to guide the driver to the identified parking space candidate. However, the landscape around the vehicle displayed on the LCD display 4 may be a schematic virtual landscape image (for example, a three-dimensional map image) rather than an image captured by a camera.

[0079] In this embodiment, the parking assistance processing program (FIG. 5) is executed by the driving assistance ECU 10 of the parking assistance device 1, but the executing entity can be changed as appropriate. For example, the program may be executed by the control unit of the liquid crystal display 4, the vehicle control ECU, the control unit of the navigation device, or other in-vehicle device. [Explanation of symbols]

[0080] 1... parking assistance device, 2... vehicle, 3... operation unit, 4... liquid crystal display, 6... front camera (imaging device), 7... rear camera (imaging device), 8A, 8B... side cameras (imaging devices), 9A to 9L... ultrasonic sensors (distance measuring sensors), 10... driving assistance ECU, 43, 44... parking space

Claims

1. a first obstacle detection means for detecting an obstacle forming a parking space based on a distance measurement value detected by a distance measurement sensor installed in the vehicle; a second obstacle detection means for detecting an obstacle forming a parking space based on an image of the surroundings of the vehicle captured by an imaging device installed in the vehicle; a parking space specifying means for specifying a parking space for parking the vehicle by using at least one of the detection results of the first obstacle detection means and the second obstacle detection means, There are several parking methods that vehicles can choose from. A parking assistance device in which, for each type of parking method, the parking space identification means is set to use either the detection result of the first obstacle detection means or the detection result of the second obstacle detection means, or to use both, when identifying a parking space in which parking is to be performed using that parking method.

2. The parking methods available to the vehicle include at least parallel parking and perpendicular parking; The parking space identification means When specifying a parking space for parallel parking, the detection result of the first obstacle detection means is used while the detection result of the second obstacle detection means is not used to specify the parking space; 2. The parking assistance device according to claim 1, wherein when a parking space for double parking is identified, the parking space is identified using both the detection results of the first obstacle detection means and the detection results of the second obstacle detection means.

3. When identifying a parking space for parallel parking, when the second obstacle detection means detects a surface of the obstacle in the depth direction as seen from the vehicle, a parking space is specified based on the surface of the obstacle facing the vehicle detected by the first obstacle detection means and the second obstacle detection means, and the angle between the surface of the obstacle facing the vehicle and the surface of the obstacle in the depth direction as seen from the vehicle, 3. The parking assistance device according to claim 2, wherein if the angle cannot be specified, the parking space is provisionally set in a direction perpendicular to the surface of the obstacle facing the vehicle.

4. 3. The parking assistance device according to claim 2, wherein when a parking space for parallel parking is specified, the parking space is specified based on the orientation of the face of the obstacle facing the vehicle detected by the first obstacle detection means.

Images