Parking assistance system, parking assistance device, parking assistance method, and recording medium

Abstract

The parking assistance system (20) includes: a ranging unit (101) configured to acquire distance information representing the distance from a moving body to an obstacle; a detection unit (102) for detecting lane lines representing a parking area; an inference unit for inferring the extension direction of the obstacle based on the distance information and a predetermined direction when the lane line is not detected, and for inferring the extension direction of the obstacle based on the distance information and the lane line when the lane line is detected; and a generation unit (104) for generating a movement path of the moving body based on the extension direction.

Description

Technical Field

[0001] This invention relates to parking assistance systems, parking assistance devices, parking assistance methods, and recording media. Background Technology

[0002] In parking assistance systems that assist in stopping a vehicle in a parking area, ultrasonic sensors are sometimes used as units to detect obstacles around the vehicle. These sensors use reflected ultrasonic waves to determine the distance from the vehicle to the obstacle.

[0003] When providing parking assistance, it is necessary to infer the shape of obstacles surrounding the target parking area (such as other vehicles parked in adjacent parking areas). In particular, accurately inferring the direction of extension of obstacles (such as along the side of other vehicles) is important for improving the reliability and efficiency of control. However, when using sensors with a narrow detection range, such as ultrasonic sensors, it is difficult to accurately infer the shape of parts far from the vehicle. Summary of the Invention

[0004] Therefore, the present invention provides a parking assistance system, parking assistance device, parking assistance method, and recording medium that can improve the accuracy of obstacle shape inference.

[0005] The first aspect of the present invention is a parking assistance system. The parking assistance system includes: a ranging unit configured to acquire distance information representing the distance from a moving body to an obstacle using the reflection of ultrasonic waves; a detection unit configured to detect lane lines representing a parking area; an inference unit configured to infer the extension direction of the obstacle based on the distance information and a predetermined direction when the lane lines are not detected, and configured to infer the extension direction of the obstacle based on the distance information and the lane lines when the lane lines are detected; and a generation unit configured to generate a movement path for the moving body based on the extension direction inferred by the inference unit.

[0006] Based on the above structure, when lane lines are detected by the detection unit, the direction of obstacle extension is inferred based on the lane lines. Therefore, even when using ultrasonic waves to detect obstacles, the accuracy of inferring the direction of obstacle extension can be improved, thereby enhancing the reliability and efficiency of parking assistance control.

[0007] Alternatively, the aforementioned parking assistance system may also include a calculation unit configured to calculate the lateral width distance that a moving body can travel when entering the parking area, based on the extension direction inferred from the lane line, upon detecting a lane line.

[0008] This allows for accurate calculation of lateral distance, thereby improving the reliability and efficiency of parking assistance control.

[0009] Alternatively, the aforementioned parking assistance system may also include an integration unit, which is configured to integrate multiple obstacles into one obstacle based on the lane lines when multiple obstacles are detected based on distance information and lane lines are detected.

[0010] Based on the above structure, the accuracy of obstacle recognition can be improved, thereby improving the reliability and efficiency of parking assistance control.

[0011] The second aspect of the present invention is a parking assistance device. This parking assistance device performs processing to assist the movement of the mobile body when it is parked in a parking area, based on distance information obtained by utilizing the reflection of ultrasonic waves, which represents the distance from the moving body to an obstacle. The parking assistance device includes: an inference unit configured to infer the extension direction of the obstacle based on the distance information and a predetermined direction when no lane line representing the parking area is detected, and configured to infer the extension direction of the obstacle based on the distance information and the lane line when the lane line is detected; and a generation unit that generates a movement path for the mobile body based on the extension direction inferred by the inference unit.

[0012] The third aspect of the present invention is a parking assistance method. This parking assistance method includes: acquiring distance information representing the distance from a moving body to an obstacle by utilizing the reflection of ultrasonic waves; detecting lane lines representing a parking area; if the lane lines are not detected, inferring the extension direction of the obstacle based on the distance information and a predetermined direction; if the lane lines are detected, inferring the extension direction of the obstacle based on the distance information and the lane lines; and generating a movement path for the moving body based on the inferred extension direction.

[0013] The fourth aspect of the present invention is a recording medium. This recording medium is a non-temporary recording medium that enables a processor to execute the following process for assisting the movement of the mobile body when parking in a parking area, based on distance information representing the distance from the moving body to an obstacle obtained using ultrasonic wave reflection: If no lane line representing the parking area is detected, the extension direction of the obstacle is inferred based on the distance information and a predetermined direction; if the lane line is detected, the extension direction of the obstacle is inferred based on the distance information and the lane line; and a movement path of the mobile body is generated based on the inferred extension direction. Attached Figure Description

[0014] The features, advantages, technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, wherein like reference numerals denote like elements, and wherein:

[0015] Figure 1 This is a top view showing the structure of a vehicle equipped with the parking assistance system according to the embodiment.

[0016] Figure 2 This is a block diagram illustrating the structure of the parking assistance system involved in the implementation method.

[0017] Figure 3 This is a block diagram illustrating the functional structure of the parking assistance system involved in the implementation method.

[0018] Figure 4 This diagram illustrates an example of a parking lot utilizing the parking assistance system described in the implementation method.

[0019] Figure 5 This is a diagram showing the direction of extension inferred when lane lines are detected in the implementation method.

[0020] Figure 6 This is a diagram showing the direction of extension inferred in the implementation where no lane lines are detected.

[0021] Figure 7 This is a flowchart illustrating the processing in the parking assistance system according to the implementation method.

[0022] Figure 8 This is a diagram illustrating the method for calculating the horizontal width distance involved in the implementation method.

[0023] Figure 9 This is a flowchart illustrating the processing in the width-to-distance calculation unit involved in the implementation method.

[0024] Figure 10 This is a diagram illustrating an example of the detection area that the ultrasonic sensor can detect when a vehicle passes through a parking area in an embodiment.

[0025] Figure 11 This is a diagram illustrating an example of the detection area that the ultrasonic sensor can detect when a vehicle enters the parking area in an embodiment.

[0026] Figure 12 This is a diagram illustrating an example of a state in an implementation where one obstacle is identified as two obstacles.

[0027] Figure 13 This diagram illustrates the process of integrating multiple identification objects into one identification object when lane lines are detected in an implementation.

[0028] Figure 14 This is a flowchart illustrating the processing in the obstacle integration section of the implementation method. Detailed Implementation

[0029] Hereinafter, embodiments and modifications of the present disclosure will be described based on the accompanying drawings. The structures of the embodiments and modifications described below, as well as the functions and effects brought about by such structures, are merely examples and are not limited to the following description.

[0030] Figure 1 This is a top view showing the structure of the vehicle 10 equipped with the parking assistance system according to the embodiment.

[0031] Vehicle 10 is an example of a moving body. Vehicle 10 can be, for example, a car powered by an internal combustion engine (internal combustion engine vehicle), a car powered by an electric motor (electric vehicle, fuel cell vehicle, etc.), or a car powered by both (hybrid vehicle). Vehicle 10 can be equipped with various transmission devices, as well as various devices (systems, components, etc.) required to drive the internal combustion engine and electric motor. The manner, number, and layout of the drive-related devices for the wheels 13 in vehicle 10 can be configured in various ways.

[0032] like Figure 1 As shown, the vehicle 10 includes a body 12, four wheels 13, one or more (four in this embodiment) camera devices 14a, 14b, 14c, 14d, and one or more (eight in this embodiment) ultrasonic sensors 16a, 16b, 16c, 16d, 16e, 16f, 16g, 16h, 16i, 16j, 16k, 16l. Where it is not necessary to distinguish between camera devices 14a, 14b, 14c, and 14d, they are referred to as camera device 14. Where it is not necessary to distinguish between ultrasonic sensors 16a, 16b, 16c, 16d, 16e, 16f, 16g, 16h, 16i, 16j, 16k, and 16l, they are referred to as ultrasonic sensors 16.

[0033] The vehicle body 12 constitutes the passenger compartment. The vehicle body 12 houses or retains the wheels 13, the camera device 14, the ultrasonic sensor 16, etc.

[0034] Four wheels 13 are arranged at the front, rear, left, and right of the vehicle body 12. For example, the two front wheels 13 function as steering wheels, and the two rear wheels 13 function as drive wheels.

[0035] The imaging device 14 is, for example, a digital camera with built-in imaging elements such as a CCD (Charge-Coupled Device) or a CIS (CMOS Image Sensor). The imaging device 14 outputs imaging data as data containing multiple frames of images generated at a specified frame rate, either dynamic or static. The imaging device 14 has a wide-angle lens or a fisheye lens, capable of capturing a horizontal range of 140° to 190°. The optical axis of the imaging device 14 is set obliquely downwards. Therefore, the imaging device 14 outputs imaging data capturing the surroundings of the vehicle 10, including the surrounding road surface.

[0036] The imaging device 14 is disposed on the outer periphery of the vehicle body 12. For example, the imaging device 14a is disposed at the center of the front end of the vehicle body 12 in the left-right direction (e.g., the front grille). The imaging device 14a generates an image of the periphery of the front of the vehicle 10. The imaging device 14b is disposed at the center of the rear end of the vehicle body 12 in the left-right direction (e.g., around the rear door switch). The imaging device 14b generates an image of the periphery of the rear of the vehicle 10. The imaging device 14c is disposed at the center of the left end of the vehicle body 12 in the front-rear direction (e.g., the left side mirror 12a). The imaging device 14c generates an image of the periphery of the left side of the vehicle 10. The imaging device 14d is disposed at the center of the right end of the vehicle body 12 in the front-rear direction (e.g., the right side mirror 12b). The imaging device 14d generates an image of the periphery of the right side of the vehicle 10.

[0037] The ultrasonic sensor 16 is, for example, disposed on the outer periphery of the vehicle 10. It transmits ultrasonic waves as detection waves and receives reflected waves from objects (obstacles) present around the vehicle 10. The ultrasonic sensor 16 acquires (generates) distance information representing the distance from the vehicle 10 to obstacles present around the vehicle 10. For example, the ultrasonic sensor 16 acquires the time (TOF) from transmitting the detection wave to receiving the reflected wave as distance information used to determine the presence, distance, position, movement, etc., of obstacles.

[0038] Ultrasonic sensors 16a, 16b, 16c, and 16d, also referred to as side sonars, are located on the left and right sides of vehicle 10. Ultrasonic sensors 16e and 16f, also referred to as corner sonars, are located further rearward than ultrasonic sensors 16a, 16b, 16c, and 16d (e.g., near the corners of vehicle 10) and facing rearward (e.g., rearward outer side) than ultrasonic sensors 16a, 16b, 16c, and 16d. Ultrasonic sensors 16g and 16h, also referred to as corner sonars, are located further forward than ultrasonic sensors 16a, 16b, 16c, and 16d (e.g., near the corners of vehicle 10) and facing forward (e.g., frontal outer side) than ultrasonic sensors 16a, 16b, 16c, and 16d. Ultrasonic sensors 16i and 16j, also referred to as rear sonars, are located at the rear end of vehicle 10. The ultrasonic sensors 16k and 16l, also known as front sonar, are located at the front end of the vehicle 10.

[0039] An ultrasonic sensor 16a is positioned at the front of the left side of the vehicle 10. The ultrasonic sensor 16a faces left. The ultrasonic sensor 16a acquires distance information related to obstacles present in the detection area on the left side of the front of the vehicle 10.

[0040] An ultrasonic sensor 16b is positioned at the rear left side of the vehicle 10. The ultrasonic sensor 16b faces left. The ultrasonic sensor 16b acquires distance information related to obstacles present in the detection area on the left rear side of the vehicle 10.

[0041] The ultrasonic sensor 16c is positioned at the front of the right side of the vehicle 10. The ultrasonic sensor 16c faces to the right. The ultrasonic sensor 16c acquires distance information related to obstacles present in the detection area on the right side of the front of the vehicle 10.

[0042] An ultrasonic sensor 16d is positioned on the rear right side of the vehicle 10. The ultrasonic sensor 16d faces to the right. The ultrasonic sensor 16d acquires distance information related to obstacles present in the detection area on the right rear side of the vehicle 10.

[0043] An ultrasonic sensor 16e is positioned on the left side of the rear end of the vehicle 10. The ultrasonic sensor 16e faces to the left rear. The ultrasonic sensor 16e acquires distance information related to obstacles present in the detection area to the left rear of the vehicle 10.

[0044] An ultrasonic sensor 16f is positioned on the right side of the rear end of the vehicle 10. The ultrasonic sensor 16f faces to the right rear. The ultrasonic sensor 16f acquires distance information related to obstacles present in the detection area to the right rear of the vehicle 10.

[0045] An ultrasonic sensor 16g is positioned on the left side of the front end of the vehicle 10. The ultrasonic sensor 16g faces forward to the left. The ultrasonic sensor 16g acquires distance information related to obstacles present in the detection area on the forward left side of the vehicle 10.

[0046] An ultrasonic sensor 16h is positioned on the right side of the front end of the vehicle 10. The ultrasonic sensor 16h faces forward to the right. The ultrasonic sensor 16h acquires distance information related to obstacles present in the detection area on the forward right side of the vehicle 10.

[0047] Ultrasonic sensors 16i and 16j are spaced apart from each other in the left-right direction at the rear end of vehicle 10 between ultrasonic sensors 16e and 16f. Ultrasonic sensors 16i and 16j face rearward. Ultrasonic sensors 16i and 16j acquire distance information related to obstacles present in the detection area behind vehicle 10.

[0048] Ultrasonic sensors 16k and 16l are spaced apart from each other in the left-right direction at the front end of vehicle 10 between ultrasonic sensors 16g and 16h. Ultrasonic sensors 16k and 16l face forward. Ultrasonic sensors 16k and 16l acquire distance information related to obstacles present in the detection area in front of vehicle 10.

[0049] Figure 2 This is a block diagram illustrating the structure of the parking assistance system 20 according to the embodiment.

[0050] The parking assistance system 20 is installed on the vehicle 10 and assists the vehicle 10 in moving when it is parked in a designated parking area by performing autonomous driving (including partial autonomous driving).

[0051] like Figure 2 As shown, the parking assistance system 20 includes a camera 14, an ultrasonic sensor 16, a braking system 22, an acceleration system 24, a steering control system 26, a transmission system 28, a vehicle speed sensor 30, a monitoring device 32, a parking assistance device 34, and an in-vehicle network 36.

[0052] The braking system 22 controls the deceleration of the vehicle 10. The braking system 22 includes a braking unit 40, a braking control unit 42, and a braking unit sensor 44.

[0053] The braking unit 40 is, for example, a device that includes a brake, a brake pedal, etc., for decelerating the vehicle 10.

[0054] The braking control unit 42 is, for example, a computer such as a microcomputer with a hardware processor such as a CPU (Central Processing Unit). Based on instructions from the parking assist device 34, the braking control unit 42 controls the braking unit 40 to decelerate the vehicle 10.

[0055] The brake sensor 44 is, for example, a position sensor, which detects the position of the brake unit 40 when the brake unit 40 is a brake pedal. The brake sensor 44 outputs the detected position of the brake unit 40 to the vehicle network 36.

[0056] The acceleration system 24 controls the acceleration of the vehicle 10. The acceleration system 24 includes an acceleration unit 46, an acceleration control unit 48, and an acceleration sensor 50.

[0057] Acceleration unit 46 is, for example, a device that includes an accelerator pedal or the like for accelerating the vehicle 10.

[0058] The acceleration control unit 48 is, for example, a computer such as a microcomputer with a hardware processor such as a CPU. The acceleration control unit 48 controls the acceleration unit 46 based on the instructions from the parking assistance device 34, thereby controlling the acceleration of the vehicle 10.

[0059] Acceleration sensor 50, for example, is a position sensor that detects the position of accelerator pedal 46. Acceleration sensor 50 outputs the detected position of accelerator pedal 46 to vehicle network 36.

[0060] The steering system 26 controls the direction of travel of the vehicle 10. The steering system 26 includes a steering control unit 52, a steering control unit 54, and a steering control unit sensor 56.

[0061] The steering control unit 52 is, for example, a device that includes a handle (steering wheel) to turn the steering wheels of the vehicle 10 and to control the direction of travel of the vehicle 10.

[0062] The steering control unit 54 is, for example, a computer such as a microcomputer with a hardware processor such as a CPU. The steering control unit 54 controls the steering unit 52 based on the instructions from the parking assist device 34, thereby controlling the direction of travel of the vehicle 10.

[0063] The steering control unit sensor 56 is an example of a third detection unit, such as an angle sensor containing a Hall element, which detects the rotation angle of the steering control unit 52, i.e., the steering angle. The steering control unit sensor 56 outputs the detected steering angle of the steering control unit 52 to the in-vehicle network 36.

[0064] The transmission system 28 controls the gear ratio of the vehicle 10. The transmission system 28 includes a transmission unit 58, a transmission control unit 60, and a transmission unit sensor 62.

[0065] The transmission unit 58 is, for example, a device that includes a gear lever or the like, to change the gear ratio of the vehicle 10.

[0066] The transmission control unit 60 is, for example, a computer such as a microcomputer with a CPU or other hardware processor. The transmission control unit 60 controls the transmission unit 58 based on the instructions from the parking assist device 34, and controls the transmission ratio of the vehicle 10.

[0067] The transmission sensor 62 is, for example, a position sensor, which detects the position of the transmission unit 58 when the transmission unit 58 is a gear lever. The transmission sensor 62 outputs the detected position of the transmission unit 58 to the vehicle network 36.

[0068] The vehicle speed sensor 30 is, for example, a sensor with a Hall element located near the wheel 13 of the vehicle 10, that detects the amount of rotation of the wheel 13 or the rotational speed per unit time. The vehicle speed sensor 30 outputs the detected amount of rotation or the number of wheel speed pulses representing the rotational speed as a sensor value for calculating the vehicle speed to the in-vehicle network 36. The parking assistance device 34 can calculate the speed (vehicle speed), amount of movement, etc. of the vehicle 10 based on the sensor value obtained from the vehicle speed sensor 30.

[0069] The monitoring device 32 is installed in the dashboard or other parts of the vehicle 10's interior. The monitoring device 32 includes a display unit 64, a sound output unit 66, and an operation input unit 68.

[0070] Display unit 64 displays images based on the image data sent by parking assistance device 34. Display unit 64 is, for example, a liquid crystal display (LCD) or an organic electroluminescent display (OELD). Display unit 64 displays, for example, an image showing an operation instruction indicating the switching between automatic and manual driving.

[0071] The sound output unit 66 outputs sound based on the sound data sent from the parking assistance device 34. The sound output unit 66 may be, for example, a speaker. The sound output unit 66 may output sound related to operation instructions indicating the switching between automatic and manual driving.

[0072] The operation input unit 68 accepts input from the passenger. The operation input unit 68 is, for example, a touch panel. The operation input unit 68 is provided on the display screen of the display unit 64. The operation input unit 68 is configured to display an image through the display unit 64. Thus, the operation input unit 68 allows the passenger to visually confirm the image displayed on the display screen of the display unit 64. The operation input unit 68 accepts instructions input by the passenger touching a position corresponding to the image displayed on the display screen of the display unit 64 and sends them to the parking assist device 34. Furthermore, the operation input unit 68 is not limited to a touch panel; it can also be a hard switch such as a button.

[0073] The parking assistance device 34 is a computer that includes a microcomputer such as an ECU (Electronic Control Unit) to assist the vehicle 10 in parking.

[0074] The parking assistance device 34 includes a CPU 34a, a ROM (Read-Only Memory) 34b, a RAM (Random Access Memory) 34c, a display control unit 34d, a sound control unit 34e, and an SSD (Solid State Drive) 34f. The CPU 34a, ROM 34b, and RAM 34c can also be integrated into the same package.

[0075] CPU34a is an example of a hardware processor that reads a program stored in a non-volatile memory device such as ROM34b and performs various arithmetic operations and controls according to the program. For example, CPU34a performs parking assistance based on the autonomous driving of vehicle 10.

[0076] ROM 34b stores the program and parameters required for program execution. RAM 34c temporarily stores various data used in the calculations performed by CPU 34a. The display control unit 34d, in the calculations performed by the parking assist device 34, mainly performs image processing of the image acquired by the capturing device 14 and data conversion of the image displayed on the display unit 64. The sound control unit 34e, in the calculations performed by the parking assist device 34, mainly performs sound processing of the sound output unit 66. SSD 34f is a rewritable non-volatile storage device that retains data even when the power to the parking assist device 34 is disconnected.

[0077] The in-vehicle network 36 includes, for example, CAN (Controller Area Network) and LIN (Local Interconnect Network). The in-vehicle network 36 connects the acceleration system 24, braking system 22, steering system 26, transmission system 28, ultrasonic sensor 16, vehicle speed sensor 30, operation input unit 68 of monitoring device 32, and parking assistance device 34 in a manner that enables them to send and receive information.

[0078] Figure 3 This is a block diagram illustrating the functional structure of the parking assistance system 20 according to the implementation method.

[0079] The parking assistance system 20 includes a distance measuring unit 101, a lane line detection unit 102 (detection unit), an extension direction inference unit 103 (inference unit), a movement path generation unit 104 (generation unit), and a driving control unit 105.

[0080] The ranging unit 101 uses the reflection of ultrasonic waves to obtain distance information representing the distance from the vehicle 10 to the obstacle. The ranging unit 101 is configured in conjunction with the ultrasonic sensor 16, the parking assistance device 34, and the program.

[0081] The lane line detection unit 102 detects lane lines indicating parking areas. The lane line detection unit 102 is configured in conjunction with the imaging device 14, the parking assistance device 34, and a program. The lane line detection unit 102 detects lane lines existing around the vehicle 10 by performing image recognition processing on images captured by the imaging device 14. Furthermore, the method for detecting lane lines is not limited to this; for example, lane lines can be detected based on information obtained from LIDAR (Light Detection and Range) sensors, millimeter-wave radar, etc., instead of the imaging device 14.

[0082] The extension direction inference unit 103 infers the extension direction of obstacles existing around the vehicle 10, particularly obstacles near the parking area that is the target, based on distance information obtained by the ranging unit 101 and lane line information obtained by the lane line detection unit 102. The extension direction represents a part of the shape characteristics of the obstacle, and is the direction in which the longer portion of the obstacle extends. For example, in the case of a vehicle, the extension direction is along the side of the vehicle. The lane line information includes the position, shape, etc., of the lane lines detected by the lane line detection unit 102. The extension direction inference unit 103 is configured in conjunction with the parking assistance device 34, a program, etc.

[0083] The extension direction inference unit 103 of this embodiment infers the extension direction based on distance information obtained by the ranging unit 101 and a predetermined direction (e.g., the vehicle width direction of the vehicle 10) when no lane line is detected. Furthermore, when a lane line is detected by the lane line detection unit 102, the extension direction inference unit 103 infers the extension direction based on distance information and the lane line.

[0084] The movement path generation unit 104 generates the movement path of the vehicle 10 based on the extension direction of the obstacle inferred by the extension direction inference unit 103. The movement path generation unit 104 is configured in conjunction with the parking assistance device 34, a program, etc.

[0085] The movement path generation unit 104 in this embodiment includes a horizontal width distance calculation unit 111 and an obstacle integration unit 112.

[0086] When the lane line detection unit 102 detects a lane line, the lateral width distance calculation unit 111 calculates the lateral width distance that the vehicle 10 can pass through when entering the parking area based on the extension direction of the obstacle inferred from the lane line.

[0087] When the obstacle integration unit 112 detects multiple obstacles based on the distance information obtained by the ranging unit 101 and the lane line detection unit 102 detects a lane line, it performs a process to integrate multiple obstacles into one obstacle based on the lane line.

[0088] The movement path generation unit 104 generates the movement path of the vehicle 10 based on the calculation results of the lateral width distance calculation unit 111 and the processing results of the obstacle integration unit 112.

[0089] The driving control unit 105 performs processing to enable the vehicle 10 to drive automatically based on the movement path generated by the movement path generation unit 104. The driving control unit 105 is configured by the cooperation of the parking assist device 34, the braking system 22, the acceleration system 24, the steering control system 26, the transmission system 28, and the program.

[0090] Figure 4 This is a diagram illustrating an example of a parking lot utilizing the parking assistance system 20 according to the embodiment.

[0091] Figure 4 The illustrated parking lot is a parking lot with multiple parking areas 201 arranged at an angle (stepped). Each parking area 201 is divided by lane lines 211. In this example, other vehicles 200A and 200B are parked in the parking area adjacent to the parking area 201 where vehicle 10 (this vehicle) is to park, and these other vehicles 200A and 200B constitute obstacles. Figure 4The image shows a vehicle 10 traveling at low speed along the direction of travel Df, passing through the target parking area 201 once, and then reversing into the parking space. In this situation, the direction of travel Df of vehicle 10 is not orthogonal to the extension directions E of other vehicles 200A and 200B.

[0092] When vehicle 10 passes through parking area 201, ultrasonic sensors 16 (ultrasonic sensors 16c and 16d in this example) acquire distance information indicating the distance from vehicle 10 to other vehicles 200A and 200B, and imaging device 14 (imaging device 14d in this example) acquires imaging data (images) of the surrounding area of ​​parking area 201. Furthermore, if lane line 211 is detected (recognized) from the captured images, the extension direction E of other vehicles 200A and 200B is inferred based on the distance information and lane line 211. On the other hand, if lane line 211 is not detected, the extension direction E is inferred based on the distance information and a predetermined direction.

[0093] Figure 5 This is a diagram showing the extension direction E inferred when lane line 211 is detected in the implementation.

[0094] exist Figure 5 The diagram shows a scenario where ultrasonic sensors 16 detect other vehicles 200A at four ends P1 to P4, and lane lines 211 are detected from images captured by the imaging device 14. In this case, the direction inference unit 103 is extended (see reference 14). Figure 3 Based on the positions of the ends P1 to P4 determined from distance information and the lane line information related to lane line 211, the extension direction E of other vehicles 200A is inferred. Specifically, the extension direction E of the end (end P1 in this example) of the other vehicle 200A closest to the target parking area 201 is set to be parallel to the extension direction Es of lane line 211. Furthermore, while only other vehicles 200A have been described above, the same applies to other vehicles 200B. According to this inference method, even in situations such as... Figure 4 In a special parking lot where multiple parking areas 201 are arranged at an angle, as shown, the extension direction E of other vehicles 200A and 200B can also be accurately deduced.

[0095] Figure 6 This is a diagram showing the extension direction E inferred in the implementation where lane line 211 is not detected.

[0096] exist Figure 6The diagram illustrates a scenario where ultrasonic sensors 16 detect four ends P1 to P4 of another vehicle 200A, but the image captured by the imaging device 14 does not show lane lines 211. In this case, the extension direction inference unit 103 infers the extension direction E of the other vehicle 200A based on the positions of the ends P1 to P4 determined from distance information and a predetermined direction, i.e., the vehicle width direction W. Specifically, the extension direction E of the end of the other vehicle 200A closest to the target parking area 201 (end P1 in this example) is set to be parallel to the vehicle width direction W. Furthermore, while only other vehicle 200A has been described above, the same applies to other vehicle 200B. Based on this inference method, in Figure 4 In a specially constructed parking lot like the one shown, the accuracy of inferring the extension direction of other vehicles 200A and 200B is lower. However, in a normally constructed parking lot where the direction of travel Df is perpendicular to the extension direction E, the extension direction E can be inferred with sufficient accuracy.

[0097] Figure 7 This is a flowchart illustrating the processing in the parking assistance system 20 according to the embodiment.

[0098] If the ranging unit 101 acquires distance information (S101), the extension direction inference unit 103 determines whether the lane line detection unit 102 has detected the lane line 211 (S102). If the lane line 211 is detected (S102: Yes), the extension direction inference unit 103 infers the extension direction E of the obstacle (other vehicles 200A, 200B) based on the distance information and the lane line 211 (S103). On the other hand, if the lane line 211 is not detected (S102: No), the extension direction inference unit 103 infers the extension direction E of the obstacle based on the distance information and the vehicle width direction W (S104).

[0099] The movement path generation unit 104 generates the movement path of the vehicle 10 based on the extension direction E inferred as described above (S105), and the driving control unit 105 controls the vehicle 10 according to the generated movement path (S106).

[0100] Furthermore, as described above, the movement path generation unit 104 generates a movement path based on the calculation results of the horizontal width distance calculation unit 111 and the processing results of the obstacle integration unit 112.

[0101] Figure 8 This is a diagram illustrating the method for calculating the horizontal width distance De in the implementation method.

[0102] The lateral width distance De represents the width of the area that vehicle 10 can pass through when entering the parking area 201, which is its target direction. For example... Figure 8As shown, when lane line 211 is detected, the lateral width distance De is calculated based on the extension direction E inferred from lane line 211. Therefore, even... Figure 4 Even for specially constructed parking lots like the one shown, the horizontal width distance De can be accurately calculated.

[0103] Figure 9 This is a flowchart illustrating the processing in the width-to-distance calculation unit 111 according to the embodiment.

[0104] The lateral width distance calculation unit 111 determines whether the lane line detection unit 102 has detected the lane line 211 (S201). If the lane line 211 is detected (S201: Yes), the lateral width distance calculation unit 111 calculates the lateral width distance based on the extension direction E of the lane line 211 (refer to...). Figure 5 and Figure 8 The lateral width distance De is calculated (S202). On the other hand, if lane line 211 is not detected (S201: No), the lateral width distance calculation unit 111 calculates the lateral width distance based on the extension direction E of the obstacle in the vehicle width direction W (refer to...). Figure 6 To calculate the horizontal width distance De(S203), the following steps are performed: Figure 4 In parking lots with special structures like the one shown, the accuracy of calculating the lateral width distance De is lower, but in parking lots with typical parallel structures (where the direction of travel Df is perpendicular to the direction of extension E), the direction of extension can be inferred with sufficient accuracy.

[0105] The following is for reference Figures 10-13 This section explains the process of integrating multiple obstacles.

[0106] Figure 10 This is a diagram illustrating an example of a detection area A1 that the ultrasonic sensor 16 can detect when the vehicle 10 passes through the parking area 201 in the embodiment. Figure 11 This is a diagram illustrating an example of a detection area A2 that the ultrasonic sensor 16 can detect when the vehicle 10 enters the parking area 201 in an embodiment.

[0107] exist Figure 10 In the indicated timing, the detection area A1 of the ultrasonic sensor 16 (in this example, ultrasonic sensors 16c and 16d) is the front end of another vehicle 200A. Figure 11 In the indicated timing, the detection area A2 of the ultrasonic sensor 16 is the side portion of the vehicle 10 side of the other vehicle 200A. Thus, as the vehicle 10 moves, the detection areas A1 and A2 of the ultrasonic sensor 16 change.

[0108] Figure 12This is a diagram illustrating an example of a state in an implementation where one obstacle is identified as two obstacles.

[0109] exist Figure 12 In the middle, it is shown that... Figure 10 The detection area A1 shown corresponds to the identification object R1, and the object R1 and the detection area A1 shown. Figure 11 The detection area A2 shown corresponds to the identified object R2. As such, if the detection areas A1 and A2 of the ultrasonic sensor 16 change as the vehicle 10 moves, there is a possibility that the acquired distance information may not be recognized as continuous, and one obstacle (in this example, another vehicle 200A) may be identified as multiple obstacles.

[0110] Figure 13 This diagram illustrates the process of integrating multiple identification objects R1 and R2 into a single identification object R when lane line 211 is detected in the implementation.

[0111] If lane line 211 is detected, such as Figure 13 As shown, multiple identification objects R1 and R2 existing within a designated area S are integrated into one identification object R based on lane line 211. The designated area S should be appropriately set according to usage conditions, etc., but may be, for example, an area determined based on the size of a typical vehicle. In this embodiment, multiple identification objects R1 and R2 are integrated into one identification object R based on the approaching portion Pn closest to the vehicle 10 in the portion of the obstacle obtained by the ultrasonic sensor 16 and the extension direction E parallel to the extension direction Es of lane line 211. Thus, even if Figure 4 The specially constructed parking lot shown can also accurately deduce the shape of obstacles.

[0112] Figure 14 This is a flowchart illustrating the processing in the obstacle integration unit 112 according to the embodiment.

[0113] The obstacle integration unit 112 determines whether multiple obstacles (identification objects R1, R2) are detected within the designated area S (S301). If no multiple obstacles are detected (S301: No), the process ends. If multiple obstacles are detected (S301: Yes), the obstacle integration unit 112 determines whether lane line 211 has been detected by the lane line detection unit 102 (S302).

[0114] If lane line 211 is detected (S302: Yes), the obstacle integration unit 112 integrates multiple obstacles (identification objects R1, R2) into one obstacle (identification object R) based on lane line 211 (S303). On the other hand, if lane line 211 is not detected (S302: No), the obstacle integration unit 112 integrates multiple obstacles (identification objects R1, R2) into one obstacle (identification object R) based on the vehicle width direction W (refer to...). Figure 6 This process integrates multiple obstacles (identification objects R1, R2) into a single obstacle (identification object R) (S304). Based on the processing in step S304, in... Figure 4 In parking lots with special structures like those shown, the accuracy of inferring the shape of obstacles is low. However, in parking lots with typical parallel structures (where the direction of travel Df is perpendicular to the extension direction E), the shape of obstacles can be inferred with sufficient accuracy.

[0115] The program that enables the parking assistance device 34 to perform the various processes described above can also be provided as a computer program product by storing it as an installable or executable file on a computer- and processor-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or floppy disk (FD). Alternatively, it can be provided by storing the program on a computer connected to a network such as the Internet and downloading it via the network. Furthermore, the program can also be provided or distributed via a network such as the Internet.

[0116] According to the above embodiments, when using ultrasonic waves to detect obstacles, even in specially constructed parking lots, the accuracy of inferring the shape of obstacles can be improved, thereby enhancing the reliability and efficiency of parking assistance control.

[0117] The embodiments and modifications of this disclosure have been described above. However, the above embodiments and modifications are merely examples and do not limit the scope of the invention. The above-described new embodiments and modifications can be implemented in various ways, and various omissions, substitutions, and changes can be made without departing from the spirit of the invention. The above-described embodiments and modifications are included in the scope and spirit of the invention, and are included in the scope of the invention as described in the claims and its equivalents.

Claims

1. A parking assistance system (20) installed on a vehicle, the parking assistance system comprising: The ranging unit (101) is configured to acquire distance information representing the distance from the vehicle to the obstacle by utilizing the reflection of ultrasonic waves; The detection unit (102) is configured to detect lane lines indicating the parking area; The inference unit (103) is configured to, when the lane line is not detected, set the extension direction of the obstacle at the end of the obstacle, which is determined by the distance information, to be parallel to the predetermined vehicle width direction; and when the lane line is detected, set the extension direction of the obstacle at the end of the obstacle closest to the parking area, which is determined by the distance information, to be parallel to the extension direction of the lane line. The generation unit (104) is configured to generate the movement path of the vehicle based on the extension direction set by the inference unit (103); as well as The integration unit (112) is configured to perform a process of integrating multiple obstacles into one obstacle when multiple obstacles are detected in a specified area based on the distance information and the lane line is detected, using the extension direction parallel to the extension direction of the lane line as a reference for the approaching portion of the detected obstacle that is closest to the vehicle.

2. The parking assistance system (20) according to claim 1, wherein, It also includes a calculation unit (111) configured to calculate the lateral width distance that the vehicle can travel when entering the parking area based on the set extension direction when the lane line is detected.

3. A parking assistance device (34) installed on a vehicle, the parking assistance device performing processing to assist the movement of the vehicle when parking in a parking area based on distance information representing the distance from the vehicle to an obstacle obtained by reflecting ultrasonic waves. The parking assistance device (34) includes: The inference unit (103) is configured to, when no lane line representing the parking area is detected, set the extension direction of the obstacle at the end of the obstacle, which is determined by the distance information, to be parallel to the predetermined vehicle width direction; and when the lane line is detected, set the extension direction of the obstacle at the end of the obstacle closest to the parking area, which is determined by the distance information, to be parallel to the extension direction of the lane line. The generation unit (104) generates the movement path of the vehicle based on the extension direction set by the inference unit (103); as well as The integration unit (112) is configured to perform a process of integrating multiple obstacles into one obstacle when multiple obstacles are detected in a specified area based on the distance information and the lane line is detected, using the extension direction parallel to the extension direction of the lane line as a reference for the approaching portion of the detected obstacle that is closest to the vehicle.

4. A parking assistance method, comprising the following steps: Distance information representing the distance from a vehicle to an obstacle is obtained by using the reflection of ultrasonic waves; Detect lane lines indicating parking areas; If the lane line is not detected, the extension direction of the obstacle at its end, determined based on the distance information, is set to be parallel to the predetermined vehicle width direction. When the lane line is detected, the extension direction of the obstacle, which is the end of the obstacle closest to the parking area as determined by the distance information, is set to be parallel to the extension direction of the lane line. The vehicle's movement path is generated based on the defined extension direction; as well as In the case where multiple obstacles are detected within a specified area based on the distance information, and the lane line is also detected, the multiple obstacles are integrated into one obstacle based on the extension direction of the closest approaching portion of the detected obstacle to the vehicle and parallel to the extension direction of the lane line.

5. A non-transitory recording medium installed in a vehicle, the non-transitory recording medium causing a processor that performs processing for assisting the movement of the vehicle when parking in a parking area based on distance information representing the distance from the vehicle to an obstacle obtained by utilizing the reflection of ultrasonic waves to perform the following processing: If no lane lines representing the parking area are detected, the extension direction of the obstacle at its end, determined based on the distance information, is set to be parallel to the predetermined vehicle width direction. When the lane line is detected, the extension direction of the obstacle, which is the end of the obstacle closest to the parking area as determined by the distance information, is set to be parallel to the extension direction of the lane line. as well as The vehicle's movement path is generated based on the defined extension direction; as well as In the case where multiple obstacles are detected within a specified area based on the distance information, and the lane line is also detected, the multiple obstacles are integrated into one obstacle based on the extension direction of the closest approaching portion of the detected obstacle to the vehicle and parallel to the extension direction of the lane line.

Images