Parking assistance system, parking assistance device, parking assistance method, and program

Abstract

To improve the estimation accuracy of the shape of an obstacle.SOLUTION: A parking support system includes: a distance measurement section for acquiring distance information indicating a distance from a movable body to an obstacle by using the reflection of ultrasonic waves; a detection section for detecting a division line showing a parking area; an estimation section for estimating the extension direction of the obstacle on the basis of the distance information and a predetermined direction when the division line is not detected, and for estimating the extension direction of the obstacle on the basis of the distance information and the division line when the division line is detected; and a generation section for generating the movement path of the movable body on the basis of the extension direction estimated by the estimation section.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a parking assistance system, a parking assistance device, a parking assistance method, and a program. [Background technology]

[0002] In a parking assistance system that assists a vehicle in parking a parking area, an ultrasonic sensor that measures the distance from the vehicle to an obstacle by using reflected ultrasonic waves may be used as a means for detecting obstacles present around the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6479130 Summary of the Invention [Problem to be solved by the invention]

[0004] When performing parking assistance, it is necessary to estimate the shape of obstacles (e.g., other vehicles parked in adjacent parking areas) that exist around the target parking area. In particular, accurately estimating the extension direction of the obstacle (e.g., the direction along the side of the other vehicle) is important for improving the reliability and efficiency of control. However, when using a sensor with a relatively narrow detection range, such as an ultrasonic sensor, it is difficult to accurately estimate the shape of parts far away from the vehicle.

[0005] Therefore, one object of the present invention is to provide a parking assistance system, a parking assistance device, a parking assistance method, and a program that can improve the accuracy of estimating the shape of an obstacle. [Means for solving the problem]

[0006] One aspect of the present invention is a parking assistance system mounted on a vehicle, comprising: a distance measurement unit that uses ultrasonic reflection to acquire distance information indicating the distance from the vehicle to an obstacle; a detection unit that detects demarcation lines indicating a parking area; an estimation unit that, if no demarcation line is detected, sets the extension direction of the obstacle that passes through the end of the obstacle closest to the parking area to be parallel to a predetermined vehicle width direction of the vehicle; and, if a demarcation line is detected, sets the extension direction to be parallel to the extension direction of the demarcation line; and a generation unit that generates a movement path for the vehicle based on the extension direction set by the estimation unit. an integration unit that, when a plurality of obstacles are detected based on the distance information and the lane marking is also detected, performs processing to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking; This is a parking assistance system equipped with the above.

[0007] According to the above configuration, when a lane marking is detected by the detection unit, the extension direction of the obstacle is estimated based on the lane marking. This makes it possible to improve the accuracy of estimating the extension direction of the obstacle even when detecting an obstacle using ultrasound, thereby improving the reliability and efficiency of parking assist control.

[0008] In addition, when a parking line is detected, the parking assistance system setting Based on the extension direction, vehicle The vehicle may further include a calculation unit that calculates a passable frontage distance when the vehicle enters the parking area.

[0009] This allows the frontage distance to be calculated accurately, improving the reliability and efficiency of parking assist control.

[0010] In addition, the parking assistance system may further include an integration unit that, when multiple obstacles are detected based on distance information and a demarcation line is detected, performs processing to integrate the multiple obstacles into a single obstacle based on the demarcation line.

[0011] This makes it possible to improve the accuracy of obstacle recognition, and to improve the reliability and efficiency of parking assist control.

[0012] Another aspect of the present invention is a parking assistance device that is mounted on a vehicle and performs processing to assist the movement of the vehicle when parking the vehicle in a parking area based on distance information indicating the distance from the vehicle to an obstacle obtained using reflected ultrasonic waves, the parking assistance device including: an estimation unit that, when a demarcation line indicating the parking area is not detected, sets the extension direction of the obstacle that passes through the end of the obstacle closest to the parking area to be parallel to a predetermined vehicle width direction of the vehicle; and, when a demarcation line is detected, sets the extension direction to be parallel to the extension direction of the demarcation line; and a generation unit that generates a movement path for the vehicle based on the extension direction set by the estimation unit. an integration unit that, when a plurality of obstacles are detected based on the distance information and the lane marking is also detected, performs processing to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking; It is equipped with the following.

[0013] Another aspect of the present invention includes a step of acquiring distance information indicating the distance from the vehicle to an obstacle using reflected ultrasonic waves, a step of detecting a parking area demarcation line, a step of setting, if no demarcation line is detected, an extension direction of the obstacle passing through the end of the obstacle closest to the parking area so as to be parallel to a predetermined vehicle width direction of the vehicle, a step of setting, if a demarcation line is detected, an extension direction so as to be parallel to the extension direction of the demarcation line, and a step of generating a movement path for the vehicle based on the set extension direction. When a plurality of obstacles are detected based on the distance information and the lane marking is also detected, a process is performed to integrate the plurality of obstacles into a single obstacle, based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking; The parking assistance method includes the steps of:

[0014] Another aspect of the present invention is a computer that is mounted on a vehicle and performs processing to assist the movement of the vehicle when the vehicle is parking in a parking area based on distance information indicating the distance from the vehicle to an obstacle obtained using reflection of ultrasonic waves, the computer comprising: a process for setting, when a demarcation line indicating the parking area is not detected, an extension direction of the obstacle that passes through the end of the obstacle closest to the parking area so that the extension direction is parallel to a predetermined vehicle width direction of the vehicle; a process for setting, when a demarcation line is detected, the extension direction so that the extension direction is parallel to the extension direction of the demarcation line; and a process for generating a movement path for the vehicle based on the set extension direction. When a plurality of obstacles are detected based on the distance information and the lane marking is also detected, a process is performed to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking; This is what causes the following to be executed. [Brief explanation of the drawings]

[0015] [Figure 1]FIG. 1 is a plan view showing the configuration of a vehicle equipped with a parking assistance system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of the parking assistance system according to the embodiment. [Figure 3] FIG. 3 is a block diagram showing the functional configuration of the parking assistance system according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a parking lot where the parking assistance system according to the embodiment is used. [Figure 5] FIG. 5 is a diagram showing an extension direction estimated when a lane marking is detected in the embodiment. [Figure 6] FIG. 6 is a diagram showing an extension direction estimated when no lane marking is detected in the embodiment. [Figure 7] FIG. 7 is a flowchart showing the processing in the parking assistance system according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating a method for calculating the frontage distance according to the embodiment. [Figure 9] FIG. 9 is a flowchart showing the processing in the frontage distance calculation unit according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a detection area that can be detected by an ultrasonic sensor when a vehicle passes through a parking area in the embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a detection area that can be detected by an ultrasonic sensor when a vehicle enters a parking area in the embodiment. [Figure 12] FIG. 12 is a diagram showing an example of a state in which one obstacle is recognized as two obstacles in the embodiment. [Figure 13] FIG. 13 is a diagram showing a process of integrating multiple recognized objects into one recognized object when a lane marking is detected in the embodiment. [Figure 14] FIG. 14 is a flowchart illustrating the processing in the obstacle integrating unit according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Exemplary embodiments of the present invention are disclosed below. The configurations of the embodiments described below, as well as the actions, results, and advantages brought about by the configurations, are merely examples. The present invention can be realized with configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of the various advantages based on the basic configurations and derivative advantages.

[0017] FIG. 1 is a plan view showing the configuration of a vehicle 10 equipped with a parking assistance system according to an embodiment.

[0018] Vehicle 10 is an example of a moving body. Vehicle 10 may be, for example, an automobile powered by an internal combustion engine (internal combustion engine automobile), an automobile powered by an electric motor (electric automobile, fuel cell automobile, etc.), or an automobile powered by both of these (hybrid automobile). Vehicle 10 may be equipped with various transmissions and various devices (systems, parts, etc.) required to drive the internal combustion engine or electric motor. The type, number, layout, etc. of devices related to driving wheels 13 of vehicle 10 may be set in various ways.

[0019] 1 , a vehicle 10 includes a vehicle body 12, four wheels 13, one or more (four in this embodiment) imaging devices 14a, 14b, 14c, and 14d, and one or more (eight in this embodiment) ultrasonic sensors 16a, 16b, 16c, 16d, 16e, 16f, 16g, 16h, 16i, 16j, 16k, and 16l. When it is not necessary to distinguish between the imaging devices 14a, 14b, 14c, and 14d, they will be referred to as imaging device 14. When it is not necessary to distinguish between the ultrasonic sensors 16a, 16b, 16c, 16d, 16e, 16f, 16g, 16h, 16i, 16j, 16k, and 16l, they will be referred to as ultrasonic sensor 16.

[0020] The vehicle body 12 forms a vehicle compartment where passengers ride in. The vehicle body 12 houses or holds wheels 13, an imaging device 14, an ultrasonic sensor 16, and the like.

[0021] Four wheels 13 are provided on the front, rear, left and right sides of the vehicle body 12. For example, the two front wheels 13 function as steering wheels, and the two rear wheels 13 function as drive wheels.

[0022] The imaging devices 14 are digital cameras incorporating imaging elements such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging devices 14 output, as imaging data, video data including a plurality of frame images generated at a predetermined frame rate or still image data. Each of the imaging devices 14 has a wide-angle lens or a fisheye lens and can capture, for example, a range of 140° to 190° in the horizontal direction. The optical axis of the imaging devices 14 is set diagonally downward. Therefore, the imaging devices 14 output imaging data capturing an image of the surroundings of the vehicle 10, including the surrounding road surface.

[0023] The imaging devices 14 are provided on the outer periphery of the vehicle body 12. For example, the imaging device 14a is provided in the center in the left-right direction of the front end of the vehicle body 12 (e.g., the front grill). The imaging device 14a generates an image of the surroundings in front of the vehicle 10. The imaging device 14b is provided in the center in the left-right direction of the rear end of the vehicle body 12 (e.g., around the back door switch). The imaging device 14b generates an image of the surroundings in the rear of the vehicle 10. The imaging device 14c is provided in the center in the front-to-rear direction of the left end of the vehicle body 12 (e.g., the left side mirror 12a). The imaging device 14c generates an image of the surroundings on the left side of the vehicle 10. The imaging device 14d is provided in the center in the front-to-rear direction of the right end of the vehicle body 12 (e.g., the right side mirror 12b). The imaging device 14d generates an image of the surroundings on the right side of the vehicle 10.

[0024] The ultrasonic sensor 16 is, for example, an ultrasonic sensor (sonar) provided on the outer periphery of the vehicle 10, which transmits ultrasonic waves as detection waves and receives reflected waves reflected by objects (obstacles) present around the vehicle 10. The ultrasonic sensor 16 acquires (generates) distance information indicating the distance from the vehicle 10 to obstacles present around the vehicle 10. For example, the ultrasonic sensor 16 acquires the time from transmitting the detection wave to receiving the reflected wave (TOF: Time Of Flight) as distance information for identifying the presence or absence, distance, position, movement, etc. of an obstacle.

[0025] The ultrasonic sensors 16a, 16b, 16c, and 16d are also called side sonars and are provided on the left and right sides of the vehicle 10. The ultrasonic sensors 16e and 16f are also called corner sonars and are provided further rearward (e.g., nearer the corners of the vehicle 10) than the ultrasonic sensors 16a, 16b, 16c, and 16d of the vehicle 10 and are directed rearward (e.g., toward the outside rearward) than the ultrasonic sensors 16a, 16b, 16c, and 16d. The ultrasonic sensors 16g and 16h are also called corner sonars and are provided further forward (e.g., toward the outside frontward) than the ultrasonic sensors 16a, 16b, 16c, and 16d of the vehicle 10. The ultrasonic sensors 16i and 16j are also called rear sonars and are provided at the rear end of the vehicle 10. The ultrasonic sensors 16k and 16l are also called front sonars and are provided at the front end of the vehicle 10.

[0026] The ultrasonic sensor 16a is provided at a front position on the left side of the vehicle 10. The ultrasonic sensor 16a is oriented toward the left. The ultrasonic sensor 16a acquires distance information regarding an obstacle present in a detection area on the left side of the front of the vehicle 10.

[0027] The ultrasonic sensor 16b is provided at a rear position on the left side of the vehicle 10. The ultrasonic sensor 16b is oriented toward the left. The ultrasonic sensor 16b acquires distance information regarding an obstacle present in a detection area on the left side of the rear of the vehicle 10.

[0028] The ultrasonic sensor 16c is provided at a position on the front side of the right side of the vehicle 10. The ultrasonic sensor 16c is oriented toward the right. The ultrasonic sensor 16c acquires distance information regarding an obstacle present in a detection area on the front right side of the vehicle 10.

[0029] The ultrasonic sensor 16d is provided at a rear position on the right side of the vehicle 10. The ultrasonic sensor 16d is oriented toward the right. The ultrasonic sensor 16d acquires distance information regarding an obstacle present in a detection area on the right side of the rear of the vehicle 10.

[0030] The ultrasonic sensor 16e is provided on the left side of the rear end of the vehicle 10. The ultrasonic sensor 16e is directed toward the left rear. The ultrasonic sensor 16e acquires distance information regarding an obstacle present in a detection area in the left rear of the vehicle 10.

[0031] The ultrasonic sensor 16f is provided at a position on the right side of the rear end of the vehicle 10. The ultrasonic sensor 16f is directed toward the right rear. The ultrasonic sensor 16f acquires distance information regarding an obstacle present in a detection area in the right rear of the vehicle 10.

[0032] The ultrasonic sensor 16g is provided on the left side of the front end of the vehicle 10. The ultrasonic sensor 16g is directed toward the left front. The ultrasonic sensor 16g acquires distance information regarding an obstacle present in a detection area in the left front of the vehicle 10.

[0033] The ultrasonic sensor 16h is provided at a position on the right side of the front end of the vehicle 10. The ultrasonic sensor 16h is directed toward the right front. The ultrasonic sensor 16h acquires distance information regarding an obstacle present in a detection area in the right front of the vehicle 10.

[0034] The ultrasonic sensors 16i and 16j are provided at the rear end of the vehicle 10 between the ultrasonic sensors 16e and 16f, spaced apart from each other in the left-right direction. The ultrasonic sensors 16i and 16j are oriented rearward. The ultrasonic sensors 16i and 16j acquire distance information regarding obstacles present in the detection areas behind the vehicle 10.

[0035] The ultrasonic sensors 16k and 16l are provided at the front end of the vehicle 10 between the ultrasonic sensors 16g and 16h and spaced apart from each other in the left-right direction. The ultrasonic sensors 16k and 16l are oriented forward. The ultrasonic sensors 16k and 16l acquire distance information regarding obstacles present in a detection area ahead of the vehicle 10.

[0036] FIG. 2 is a block diagram showing the configuration of the parking assistance system 20 according to the embodiment.

[0037] The parking assistance system 20 is mounted on the vehicle 10 and assists the movement of the vehicle 10 when parking the vehicle 10 in a predetermined parking area by performing automatic driving (including partial automatic driving).

[0038] As shown in FIG. 2, the parking assistance system 20 includes an imaging device 14, an ultrasonic sensor 16, a braking system 22, an acceleration system 24, a steering system 26, a gear shift system 28, a vehicle speed sensor 30, a monitoring device 32, a parking assistance device 34, and an in-vehicle network 36.

[0039] The braking system 22 controls the deceleration of the vehicle 10. The braking system 22 includes a braking unit 40, a braking control unit 42, and a braking unit sensor 44.

[0040] The braking unit 40 includes, for example, a brake, a brake pedal, etc., and is a device for decelerating the vehicle 10.

[0041] The braking control unit 42 is a computer such as a microcomputer having a hardware processor such as a CPU (Central Processing Unit). The braking control unit 42 controls the braking unit 40 based on instructions from the parking assistance device 34, and controls the deceleration of the vehicle 10.

[0042] The brake unit sensor 44 is, for example, a position sensor, and when the brake unit 40 is a brake pedal, detects the position of the brake unit 40. The brake unit sensor 44 outputs the detected position of the brake unit 40 to the in-vehicle network 36.

[0043] The acceleration system 24 controls the acceleration of the vehicle 10. The acceleration system 24 includes an acceleration unit 46, an acceleration control unit 48, and an acceleration unit sensor 50.

[0044] The acceleration unit 46 includes, for example, an accelerator pedal and is a device for accelerating the vehicle 10.

[0045] The acceleration control unit 48 is, for example, a computer such as a microcomputer having a hardware processor such as a CPU. The acceleration control unit 48 controls the acceleration unit 46 based on instructions from the parking assistance device 34, and controls the acceleration of the vehicle 10.

[0046] The acceleration sensor 50 is, for example, a position sensor, and when the acceleration unit 46 is an accelerator pedal, it detects the position of the acceleration unit 46. The acceleration sensor 50 outputs the detected position of the acceleration unit 46 to the in-vehicle network 36.

[0047] The steering system 26 controls the direction of travel of the vehicle 10. The steering system 26 includes a steering unit 52, a steering control unit 54, and a steering unit sensor 56.

[0048] The steering unit 52 includes, for example, a steering wheel and is a device that steers the steered wheels of the vehicle 10 to steer the traveling direction of the vehicle 10.

[0049] The steering control unit 54 is, for example, a computer such as a microcomputer having a hardware processor such as a CPU. The steering control unit 54 controls the steering unit 52 based on instructions from the parking assistance device 34, and controls the traveling direction of the vehicle 10.

[0050] The steering unit sensor 56 is an example of a third detection unit and is, for example, an angle sensor including a Hall element or the like, and detects the steering angle, which is the rotation angle of the steering unit 52. The steering unit sensor 56 outputs the detected steering angle of the steering unit 52 to the in-vehicle network 36.

[0051] The transmission system 28 controls the gear ratio of the vehicle 10. The transmission system 28 includes a transmission unit 58, a transmission control unit 60, and a transmission unit sensor 62.

[0052] The transmission unit 58 includes, for example, a shift lever and is a device that changes the gear ratio of the vehicle 10.

[0053] The transmission control unit 60 is, for example, a computer such as a microcomputer having a hardware processor such as a CPU. The transmission control unit 60 controls the transmission unit 58 based on instructions from the parking assistance device 34, and controls the gear ratio of the vehicle 10.

[0054] The transmission sensor 62 is, for example, a position sensor, and when the transmission 58 is a shift lever, detects the position of the transmission 58. The transmission sensor 62 outputs the detected position of the transmission 58 to the in-vehicle network 36.

[0055] The vehicle speed sensor 30 is a sensor that has, for example, a Hall element provided near the wheel 13 of the vehicle 10 and detects the amount of rotation or the number of rotations per unit time of the wheel 13. The vehicle speed sensor 30 outputs the number of wheel speed pulses indicating the detected amount of rotation or number of rotations to the in-vehicle network 36 as a sensor value for calculating the vehicle speed. The parking assistance device 34 can calculate the speed (vehicle speed), amount of movement, etc. of the vehicle 10 based on the sensor value acquired from the vehicle speed sensor 30.

[0056] The monitor device 32 is provided on a dashboard or the like inside the passenger compartment of the vehicle 10. The monitor device 32 has a display unit 64, an audio output unit 66, and an operation input unit 68.

[0057] The display unit 64 displays an image based on the image data transmitted by the parking assistance device 34. The display unit 64 is, for example, a display device such as a liquid crystal display (LCD) or an organic electroluminescent display (ELD). The display unit 64 displays, for example, an image that accepts an operation instruction to switch between automatic driving and manual driving.

[0058] The audio output unit 66 outputs audio based on the audio data transmitted by the parking assistance device 34. The audio output unit 66 is, for example, a speaker. The audio output unit 66 outputs audio related to an operation instruction to switch between automatic driving and manual driving, for example.

[0059] The operation input unit 68 accepts input from the occupant. The operation input unit 68 is, for example, a touch panel. The operation input unit 68 is provided on the display screen of the display unit 64. The operation input unit 68 is configured to be able to transmit images displayed by the display unit 64. This allows the occupant to view the images displayed on the display screen of the display unit 64. The operation input unit 68 accepts instructions input by the occupant touching a position corresponding to the image displayed on the display screen of the display unit 64, and transmits the instructions to the parking assistance device 34. Note that the operation input unit 68 is not limited to a touch panel, and may be a hardware switch such as a push button.

[0060] The parking assistance device 34 is a computer including a microcomputer such as an ECU (Electronic Control Unit), and assists the vehicle 10 in parking.

[0061] The parking assistance device 34 includes a CPU 34a, a ROM (Read Only Memory) 34b, a RAM (Random Access Memory) 34c, a display control unit 34d, a sound control unit 34e, and an SSD (Solid State Drive) 34f. The CPU 34a, the ROM 34b, and the RAM 34c may be integrated in the same package.

[0062] The CPU 34a is an example of a hardware processor, and reads out a program stored in a nonvolatile storage device such as the ROM 34b, and executes various types of arithmetic processing and control in accordance with the program. The CPU 34a executes, for example, parking assistance by automatic driving of the vehicle 10.

[0063] The ROM 34b stores programs, parameters necessary for executing the programs, etc. The RAM 34c temporarily stores various data used in the calculations by the CPU 34a. The display control unit 34d, among the calculation processes in the parking assistance device 34, mainly performs image processing of images obtained by the imaging device 14 and data conversion of images to be displayed on the display unit 64. The audio control unit 34e, among the calculation processes in the parking assistance device 34, mainly performs audio processing to be output to the audio output unit 66. The SSD 34f is a rewritable non-volatile storage device that maintains data even when the power to the parking assistance device 34 is turned off.

[0064] The in-vehicle network 36 includes, for example, a Controller Area Network (CAN), a Local Interconnect Network (LIN), etc. The in-vehicle network 36 connects the acceleration system 24, the braking system 22, the steering system 26, the gear shift system 28, the ultrasonic sensor 16, the vehicle speed sensor 30, the operation input unit 68 of the monitor device 32, and the parking assistance device 34 so that they can send and receive information to and from each other.

[0065] FIG. 3 is a block diagram showing the functional configuration of the parking assistance system 20 according to the embodiment.

[0066] The parking assistance system 20 includes a distance measurement unit 101, a demarcation line detection unit 102 (detection unit), an extension direction estimation unit 103 (estimation unit), a travel path generation unit 104 (generation unit), and a driving control unit 105.

[0067] The distance measuring unit 101 acquires distance information indicating the distance from the vehicle 10 to an obstacle by utilizing the reflection of ultrasonic waves. The distance measuring unit 101 is configured by the cooperation of the ultrasonic sensor 16, the parking assistance device 34, a program, etc.

[0068] The lane marking detection unit 102 detects lane markings that indicate parking areas. The lane marking detection unit 102 is configured by cooperation of the imaging device 14, the parking assistance device 34, a program, etc. The lane marking detection unit 102 detects lane markings that exist around the vehicle 10 by performing image recognition processing on the captured image acquired by the imaging device 14. Note that the method for detecting lane markings is not limited to this, and it is also possible to detect lane markings based on information acquired by a LIDAR (Light Detection And Ranging) sensor, millimeter wave radar, etc. instead of the imaging device 14, for example.

[0069] The extension direction estimation unit 103 estimates the extension direction of obstacles present around the vehicle 10, particularly obstacles present near the target parking area, based on the distance information acquired by the distance measurement unit 101 and the lane line information acquired by the lane line detection unit 102. The extension direction indicates part of the shape characteristics of the obstacle and is the direction in which a relatively long portion of the obstacle extends. For example, if the obstacle is a vehicle, the extension direction is the direction along the side of the vehicle. The lane line information includes the position, shape, etc. of the lane lines detected by the lane line detection unit 102. The extension direction estimation unit 103 is configured in cooperation with the parking assistance device 34, a program, etc.

[0070] When a lane marking is not detected, the extension direction estimation unit 103 according to this embodiment estimates the extension direction based on the distance information acquired by the distance measurement unit 101 and a predetermined direction (for example, the width direction of the vehicle 10). When a lane marking is detected by the lane marking detection unit 102, the extension direction estimation unit 103 estimates the extension direction based on the distance information and the lane marking.

[0071] The movement path generation unit 104 generates a movement path for the vehicle 10 based on the extension direction of the obstacle estimated by the extension direction estimation unit 103. The movement path generation unit 104 is configured by cooperation of the parking assistance device 34, a program, etc.

[0072] The travel path generation unit 104 according to this embodiment includes an entrance distance calculation unit 111 and an obstacle integration unit 112.

[0073] When a demarcation line is detected by the demarcation line detection unit 102, the frontage distance calculation unit 111 calculates the passable frontage distance when the vehicle 10 enters the parking area based on the extension direction of the obstacle estimated based on the demarcation line.

[0074] When multiple obstacles are detected based on distance information acquired by the distance measurement unit 101 and a lane marking is detected by the lane marking detection unit 102, the obstacle integration unit 112 performs processing to integrate the multiple obstacles into a single obstacle based on the lane marking.

[0075] The travel route generation unit 104 generates a travel route for the vehicle 10 based on the calculation result by the frontage distance calculation unit 111 and the processing result by the obstacle integration unit 112.

[0076] The driving control unit 105 performs processing for automatically driving the vehicle 10 based on the travel route generated by the travel route generation unit 104. The driving control unit 105 is configured by the cooperation of the parking assistance device 34, the braking system 22, the acceleration system 24, the steering system 26, the gear shift system 28, programs, etc.

[0077] FIG. 4 is a diagram showing an example of a parking lot where the parking assistance system 20 according to the embodiment is used.

[0078] The parking lot illustrated in FIG. 4 has multiple parking areas 201 arranged diagonally (in a stepped pattern). Each parking area 201 is divided by demarcation lines 211. In this example, other vehicles 200A and 200B are parked in parking areas adjacent to the parking area 201 where vehicle 10 (host vehicle) is attempting to park, and these other vehicles 200A and 200B become obstacles. FIG. 4 shows a situation in which vehicle 10 travels at a low speed in a traveling direction Df, passes through the target parking area 201 once, and then backs into the parking area. In this situation, the traveling direction Df of vehicle 10 and the extension direction E of the other vehicles 200A and 200B are not perpendicular to each other.

[0079] When vehicle 10 passes through parking area 201, ultrasonic sensor 16 (ultrasonic sensors 16c and 16d in this example) acquires distance information indicating the distance from vehicle 10 to other vehicles 200A and 200B, and imaging device 14 (imaging device 14d in this example) acquires imaging data (captured image) of the periphery of parking area 201. If demarcation lines 211 are detected (recognized) from the captured image, extension direction E of other vehicles 200A and 200B is estimated based on the distance information and demarcation lines 211. On the other hand, if demarcation lines 211 are not detected, extension direction E is estimated based on the distance information and a predetermined direction.

[0080] FIG. 5 is a diagram showing an extension direction E that is estimated when a demarcation line 211 is detected in the embodiment.

[0081] FIG. 5 illustrates a case in which four ends P1 to P4 of the other vehicle 200A are detected by the ultrasonic sensor 16, and a demarcation line 211 is detected from an image captured by the imaging device 14. In this case, the extension direction estimation unit 103 (see FIG. 3) estimates the extension direction E of the other vehicle 200A based on the positions of the ends P1 to P4 obtained from the distance information and the demarcation line information related to the demarcation line 211. Specifically, the extension direction E passing through the end (end P1 in this example) of the other vehicle 200A closest to the target parking area 201 is set to be parallel to the extension direction Es of the demarcation line 211. Note that while only the other vehicle 200A has been described above, the same applies to the other vehicle 200B. According to this estimation method, the extension direction E of the other vehicles 200A and 200B can be accurately estimated even in a special parking lot in which multiple parking areas 201 are arranged diagonally as shown in FIG. 4.

[0082] FIG. 6 is a diagram showing an extension direction E that is estimated when the demarcation line 211 is not detected in the embodiment.

[0083] FIG. 6 illustrates a case in which the ultrasonic sensor 16 detects four ends P1 to P4 of the other vehicle 200A, but the partition line 211 is not detected in the image captured by the imaging device 14. In such a case, the extension direction estimation unit 103 estimates the extension direction E of the other vehicle 200A based on the positions of the ends P1 to P4 obtained from the distance information and the vehicle width direction W, which is a predetermined direction. Specifically, the extension direction E passing through the end (end P1 in this example) of the other vehicle 200A closest to the target parking area 201 is set to be parallel to the vehicle width direction W. Note that while only the other vehicle 200A has been described above, the same applies to the other vehicle 200B. With this estimation method, the estimation accuracy of the extension directions of the other vehicles 200A and 200B decreases in a parking lot with a special structure as shown in FIG. 4. However, in a parking lot with a normal structure in which the traveling direction Df and the extension direction E form a right angle, the extension direction E can be estimated with substantially sufficient accuracy.

[0084] FIG. 7 is a flowchart showing the processing in the parking assistance system 20 according to the embodiment.

[0085] When the distance measurement unit 101 acquires distance information (S101), the extension direction estimation unit 103 determines whether or not the lane marking 211 has been detected by the lane marking detection unit 102 (S102). If the lane marking 211 has been detected (S102: Yes), the extension direction estimation unit 103 estimates the extension direction E of the obstacle (other vehicles 200A, 200B) based on the distance information and the lane marking 211 (S103). On the other hand, if the lane marking 211 has not been detected (S102: No), the extension direction estimation unit 103 estimates the extension direction E of the obstacle based on the distance information and the vehicle width direction W (S104).

[0086] The travel route generation unit 104 generates a travel route for the vehicle 10 based on the extension direction E estimated as described above (S105), and the driving control unit 105 controls the vehicle 10 according to the generated travel route (S106).

[0087] Furthermore, the travel route generation unit 104 generates a travel route based on the calculation result by the frontage distance calculation unit 111 and the processing result by the obstacle integration unit 112, as described above.

[0088] FIG. 8 is a diagram showing a method for calculating the frontage distance De according to the embodiment.

[0089] The frontage distance De indicates the width of the area through which the vehicle 10 can pass when entering the target parking area 201. As shown in FIG. 8, when a demarcation line 211 is detected, the frontage distance De is calculated based on the extension direction E estimated based on the demarcation line 211. This makes it possible to accurately calculate the frontage distance De even for parking lots with a special structure such as that shown in FIG. 4.

[0090] FIG. 9 is a flowchart showing the processing in the frontage distance calculation unit 111 according to the embodiment.

[0091] The frontage distance calculation unit 111 determines whether the lane marking 211 has been detected by the lane marking detection unit 102 (S201). If the lane marking 211 has been detected (S201: Yes), the frontage distance calculation unit 111 calculates the frontage distance De based on the extension direction E (see FIGS. 5 and 8) of the lane marking 211 (S202). On the other hand, if the lane marking 211 has not been detected (S201: No), the frontage distance calculation unit 111 calculates the frontage distance De based on the extension direction E of the obstacle based on the vehicle width direction W (see FIG. 6) (S203). According to the processing of step S203, the calculation accuracy of the frontage distance De is low in parking lots with a special structure such as that shown in FIG. 4. However, in parking lots with a normal parallel structure (a structure in which the traveling direction Df and the extension direction E form a right angle), the extension direction can be estimated with substantially sufficient accuracy.

[0092] The process of integrating a plurality of obstacles will be described below with reference to FIGS.

[0093] Fig. 10 is a diagram showing an example of a detection area A1 that can be detected by the ultrasonic sensor 16 when the vehicle 10 passes through the parking area 201 in the embodiment. Fig. 11 is a diagram showing an example of a detection area A2 that can be detected by the ultrasonic sensor 16 when the vehicle 10 enters the parking area 201 in the embodiment.

[0094] At the timing shown in Fig. 10, the detection area A1 of the ultrasonic sensor 16 (ultrasonic sensors 16c and 16d in this example) is the front end portion of the other vehicle 200A. At the timing shown in Fig. 11, the detection area A2 of the ultrasonic sensor 16 is the side portion of the other vehicle 200A on the vehicle 10 side. In this way, the detection areas A1 and A2 of the ultrasonic sensor 16 change as the vehicle 10 moves.

[0095] FIG. 12 is a diagram showing an example of a state in which one obstacle is recognized as two obstacles in the embodiment.

[0096] Fig. 12 shows a recognized object R1 corresponding to the detection area A1 shown in Fig. 10, and a recognized object R2 corresponding to the detection area A2 shown in Fig. 11. When the detection areas A1 and A2 of the ultrasonic sensor 16 change as the vehicle 10 moves, continuity is not recognized in the acquired distance information, and one obstacle (another vehicle 200A in this example) may be recognized as multiple obstacles.

[0097] FIG. 13 is a diagram showing a process of integrating a plurality of recognition objects R1 and R2 into one recognition object R when a demarcation line 211 is detected in this embodiment.

[0098] When a demarcation line 211 is detected, as shown in FIG. 13, multiple recognized objects R1 and R2 present within a predetermined area S are integrated into a single recognized object R based on the demarcation line 211. The predetermined area S should be set appropriately depending on the usage situation, etc., and may be, for example, an area determined based on the size of a typical vehicle. In this embodiment, the multiple recognized objects R1 and R2 are integrated into a single recognized object R based on an extension direction E that passes through a nearby portion Pn of the obstacle acquired by the ultrasonic sensor 16 that is closest to the vehicle 10 and is parallel to the extension direction Es of the demarcation line 211. This makes it possible to accurately estimate the shape of an obstacle even in a parking lot with a special structure such as that shown in FIG. 4.

[0099] FIG. 14 is a flowchart showing the processing in the obstacle integrating unit 112 according to the embodiment.

[0100] The obstacle integrating unit 112 determines whether multiple obstacles (recognized objects R1, R2) have been recognized within the predetermined area S (S301). If multiple obstacles have not been recognized (S301: No), this flow ends. If multiple obstacles have been recognized (S301: Yes), the obstacle integrating unit 112 determines whether the lane marking 211 has been detected by the lane marking detection unit 102 (S302).

[0101] If the lane markings 211 are detected (S302: Yes), the obstacle integrating unit 112 integrates the multiple obstacles (recognized objects R1, R2) into a single obstacle (recognized object R) based on the lane markings 211 (S303). On the other hand, if the lane markings 211 are not detected (S302: No), the obstacle integrating unit 112 integrates the multiple obstacles (recognized objects R1, R2) into a single obstacle (recognized object R) based on the vehicle width direction W (see FIG. 6) (S304). According to the processing of step S304, the accuracy of estimating the shape of the obstacle is low in parking lots with a special structure such as that shown in FIG. 4, but the shape of the obstacle can be estimated with substantially sufficient accuracy in parking lots with a normal parallel structure (a structure in which the traveling direction Df and the extension direction E are at right angles), etc.

[0102] The program that causes the parking assistance device 34 to execute the various processes described above may be provided as a computer program product by being stored in an installable or executable file on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD). Alternatively, the program may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the program may be provided or distributed via a network such as the Internet.

[0103] According to the above embodiment, even when detecting an obstacle using ultrasound or in a parking lot with a special structure, the accuracy of estimating the shape of the obstacle can be improved, thereby improving the reliability and efficiency of parking assistance control.

[0104] Although the embodiments of the present invention have been described above, the above embodiments are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as set forth in the claims. [Explanation of symbols]

[0105] 10...vehicle (moving body), 12...vehicle body, 13...wheel, 14 (14a to 14d)...imaging device, 16 (16a to 16l)...ultrasonic sensor, 20...parking assistance system, 34...parking assistance device, 101...distance measurement unit, 102...landing line detection unit (detection unit), 103...extension direction estimation unit (estimation unit), 104...movement path generation unit (generation unit), 105...travel control unit, 111...frontage distance calculation unit (calculation unit), 112...obstacle integration unit (integration unit), 200A, 200B...other vehicles (obstacles), 201...parking area, 211...landing line, A1, A2...detection area, Df...direction of travel, E...(obstacle) extension direction, Es...(landing line) extension direction, P1 to P8...edge, R, R1, R2...recognized object, S...predetermined area, W...vehicle width direction

Claims

1. A parking assistance system mounted on a vehicle, a distance measuring unit that acquires distance information indicating the distance from the vehicle to an obstacle by using reflected ultrasonic waves; a detection unit that detects parking area markings; an estimation unit that, when the lane marking is not detected, sets an extension direction of the obstacle that passes through an end of the obstacle obtained from the distance information to be parallel to a predetermined vehicle width direction of the vehicle, and, when the lane marking is detected, sets an extension direction of the obstacle that passes through an end of the obstacle that is closest to the parking area obtained from the distance information to be parallel to the extension direction of the lane marking; a generation unit that generates a travel path of the vehicle based on the extension direction set by the estimation unit; an integration unit that, when a plurality of obstacles are detected within a predetermined area based on the distance information and the lane marking is also detected, performs processing to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking; A parking assistance system equipped with

2. a calculation unit that calculates a passable frontage distance when the vehicle enters the parking area based on the set extension direction when the demarcation line is detected; The parking assistance system of claim 1 further comprising:

3. A parking assistance device that is mounted on a vehicle and performs processing to assist in movement of the vehicle when parking the vehicle in a parking area, based on distance information indicating a distance from the vehicle to an obstacle, the distance information being acquired by utilizing reflection of ultrasonic waves, an estimation unit that, when a demarcation line indicating the parking area is not detected, sets an extension direction of the obstacle that passes through an end of the obstacle obtained from the distance information to be parallel to a predetermined vehicle width direction of the vehicle, and, when the demarcation line is detected, sets an extension direction of the obstacle that passes through an end of the obstacle that is closest to the parking area obtained from the distance information to be parallel to the extension direction of the demarcation line; a generation unit that generates a travel path of the vehicle based on the extension direction set by the estimation unit; an integration unit that, when a plurality of obstacles are detected within a predetermined area based on the distance information and the lane marking is also detected, performs processing to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking; A parking assistance device comprising:

4. obtaining distance information indicating a distance from the vehicle to an obstacle using reflected ultrasonic waves; detecting parking zone lines; When the lane marking is not detected, setting an extension direction of the obstacle that passes through an end of the obstacle determined from the distance information so that the extension direction is parallel to a predetermined vehicle width direction of the vehicle; When the demarcation line is detected, setting an extension direction of the obstacle that passes through an end of the obstacle that is closest to the parking area, which is determined from the distance information, to be parallel to the extension direction of the demarcation line; generating a travel path for the vehicle based on the set extension direction; When a plurality of obstacles are detected within a predetermined area based on the distance information and the lane marking is also detected, a process is performed to integrate the plurality of obstacles into a single obstacle, based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking; A parking assistance method including:

5. a computer that is mounted on a vehicle and performs processing to assist the movement of the vehicle when the vehicle is parking in a parking area, based on distance information indicating a distance from the vehicle to an obstacle obtained by using reflection of ultrasonic waves; When a parking area marking line is not detected, a process of setting an extension direction of the obstacle passing through an end of the obstacle obtained from the distance information so that the extension direction is parallel to a predetermined vehicle width direction of the vehicle; When the demarcation line is detected, a process of setting an extension direction of the obstacle that passes through an end of the obstacle that is closest to the parking area, which is obtained from the distance information, to be parallel to the extension direction of the demarcation line; A process of generating a travel route of the vehicle based on the set extension direction; When a plurality of obstacles are detected within a predetermined area based on the distance information and the lane marking is also detected, a process is performed to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking; A program that executes the following.

Images