Parking assistance device

Abstract

A parking assistance device according to an embodiment assists in parking a vehicle in accordance with an instruction from a terminal carried by a user of the vehicle which has a sensor for collecting periphery information. The parking assistance device comprises: a detection unit that detects, on the basis of the periphery information from the sensor, a parking region and an obstacle in the parking region or around the parking region; and a determination unit that makes a determination about the obstacle. If a parking region and an obstacle are detected together, the determination unit determines that the vehicle can be parked in the parking region upon determining that the state of the obstacle can be changed so as to not hinder the parking of the vehicle.

Description

Parking assistance device

[0001] The present invention relates to a parking assistance device.

[0002] There are parking assistance devices that provide assistance when parking a vehicle. When a parking assistance device detects a parking area where the vehicle can be parked, it starts parking assistance in accordance with the driver's instructions. For example, in the technology disclosed in Patent Document 1, when an obstacle is detected on the vehicle's movement path including the parking area, the parking assistance is stopped.

[0003] International Publication No. 2018 / 139289

[0004] In recent years, parking assistance systems have been developed that remotely park a vehicle while the driver is out of the vehicle. There is a need for such parking assistance systems to continue providing parking assistance after the driver has dismounted and removed obstacles, such as traffic cones, placed within the parking area. However, the technology disclosed in Patent Document 1 described above has difficulty meeting such needs.

[0005] The present invention has been made in consideration of the above, and provides a parking assistance device that can give the driver the option of continuing parking assistance even when an obstacle is detected within a parking area, etc.

[0006] In order to solve the above-mentioned problems and achieve the objective, the parking assistance device of the embodiment is a parking assistance device that provides parking assistance to a vehicle having a sensor that collects surrounding information in accordance with instructions from a terminal carried by the vehicle's user, and is equipped with a detection unit that detects a parking area and obstacles within or around the parking area based on the surrounding information from the sensor, and a judgment unit that makes a judgment about the obstacles, and when the judgment unit detects the obstacle along with the parking area, if it determines that the state of the obstacle can be changed so that it does not interfere with parking of the vehicle, it determines that the vehicle can be parked in the parking area.

[0007] The parking assistance device of the embodiment can give the driver the option of continuing parking assistance even when an obstacle is detected in the parking area or the like.

[0008] FIG. 1 is a top view of a vehicle equipped with a parking assistance device according to an embodiment. FIG. 2 is a block diagram showing an example of the overall configuration of a parking assistance system according to an embodiment. FIG. 3 is a block diagram showing an example of the functional configuration of the parking assistance device according to an embodiment. FIG. 4 is a schematic diagram showing several example methods for determining an obstacle by the parking assistance device according to an embodiment. FIG. 5 is a schematic diagram showing several example methods for determining an obstacle by the parking assistance device according to an embodiment. FIG. 6 is a schematic diagram showing several example methods for determining an obstacle by the parking assistance device according to an embodiment. FIG. 7 is a schematic diagram showing several example methods for determining an obstacle by the parking assistance device according to an embodiment. FIG. 8 is a schematic diagram illustrating an example of a parking assistance operation by the parking assistance device according to an embodiment. FIG. 9 is a schematic diagram illustrating an example of a parking assistance operation by the parking assistance device according to an embodiment. FIG. 10 is a schematic diagram illustrating an example of a parking assistance operation by the parking assistance device according to an embodiment. FIG. 11 is a schematic diagram illustrating an example of a parking assistance operation by the parking assistance device according to an embodiment. FIG. 12 is a schematic diagram illustrating an example of a parking assistance operation by the parking assistance device according to an embodiment. Fig. 13 is a schematic diagram illustrating an example of an operation when the parking assistance device according to the embodiment cancels parking assistance. Fig. 14 is a schematic diagram illustrating an example of an operation when the parking assistance device according to the embodiment cancels parking assistance. Fig. 15 is a flow diagram illustrating an example of a procedure for parking assistance processing by the parking assistance device according to the embodiment. Fig. 16 is a flow diagram illustrating an example of a procedure for parking assistance processing by the parking assistance device according to a modified embodiment.

[0009] Similar components in the following exemplary embodiments will be given the same reference numerals, and duplicated descriptions will be omitted as appropriate.

[0010] (Example of Vehicle Configuration) Fig. 1 is a top view of a vehicle 10 equipped with a parking assistance device 20 according to an embodiment. The front, rear, left, and right of the vehicle 10 in Fig. 1 indicate directions when viewed from the driver's seat of the vehicle 10.

[0011] The vehicle 10 of the embodiment may be, for example, an internal combustion engine vehicle using an internal combustion engine as a power source, an electric vehicle or fuel cell vehicle using an electric motor as a power source, or a hybrid vehicle using both of these as a power source.

[0012] The vehicle 10 can be equipped with various transmissions and various systems, parts, and other devices required to drive the internal combustion engine or electric motor. The type, number, and layout of the devices related to driving the wheels 13 of the vehicle 10 can be set in various ways.

[0013] 1, a vehicle 10 includes a vehicle body 12, a plurality of wheels 13, a plurality of distance measuring units 14a-14l, and a plurality of image capturing units 16a-16d. When it is not necessary to distinguish between the distance measuring units 14a-14l, they will be simply referred to as distance measuring unit 14. When it is not necessary to distinguish between the image capturing units 16a-16d, they will be simply referred to as image capturing unit 16. These distance measuring units 14a-14l and image capturing units 16a-16d are examples of sensors that collect information about the surroundings of the vehicle 10.

[0014] The vehicle body 12 constitutes a vehicle cabin in which passengers ride. The vehicle body 12 is equipped with a plurality of wheels 13, a plurality of distance measuring units 14, and a plurality of image capturing units 16. In the example of Fig. 1, the vehicle body 12 is equipped with four wheels 13, twelve distance measuring units 14, and four image capturing units 16. However, the number of distance measuring units 14 and image capturing units 16 attached to the vehicle body 12 is arbitrary.

[0015] The four wheels 13 are provided on the front, rear, left and right sides of the vehicle body 12. The two front wheels 13 function as steering wheels, for example, and the two rear wheels 13 function as driving wheels, for example.

[0016] The distance measuring unit 14 is, for example, a sonar provided on the outer periphery of the vehicle 10, which transmits sound waves such as ultrasonic waves as detection waves and captures the detection waves reflected by objects such as obstacles present around the vehicle 10. Note that the distance measuring unit 14 may also be a radar that transmits detection waves such as laser light, a millimeter wave radar, or the like.

[0017] The distance measuring unit 14 collects surrounding information, which is information about the surroundings of the vehicle 10, and outputs it to the parking assistance device 20. The distance measuring unit 14 collects, for example, a response time, which is the time from transmitting a detection wave to receiving it, as surrounding information for determining the distance between an object and the vehicle 10. Based on the surrounding information collected by the distance measuring unit 14, the parking assistance device 20 can detect the presence or absence of obstacles, etc., around the vehicle 10 and the distance to those obstacles.

[0018] In addition, if the ranging unit 14 receives multiple detection waves reflected from multiple points on the object in response to a single detection wave transmission, it may include only the response time of the earliest received detection wave in the surrounding information.

[0019] The distance measuring units 14a to 14d are provided at the front of the vehicle body 12. Of these distance measuring units 14a to 14d, the distance measuring units 14b and 14c are also called front sonars and are provided at the front end of the vehicle 10. The distance measuring units 14b and 14c detect objects ahead of the vehicle 10 and collect information about the surroundings ahead of the vehicle 10. The distance measuring units 14a and 14d are also called corner sonars and are provided at the corners of the front of the vehicle 10. The distance measuring units 14a and 14d detect objects ahead of the vehicle 10 and collect information about the surroundings ahead of the vehicle 10.

[0020] The distance measuring units 14e to 14h are provided at the rear of the vehicle body 12. Of these distance measuring units 14e to 14h, distance measuring units 14f and 14g are also called rear sonars and are provided at the rear end of the vehicle 10. The distance measuring units 14f and 14g detect objects behind the vehicle 10 and collect information about the surroundings behind the vehicle 10. The distance measuring units 14e and 14h are also called corner sonars and are provided at the corners of the rear of the vehicle 10. The distance measuring units 14e and 14h detect objects on the outside behind the vehicle 10 and collect information about the surroundings behind the vehicle 10.

[0021] The distance measuring units 14i to 14l are also called side sonars and are provided on the sides of the vehicle body 12. Of the distance measuring units 14i to 14l, the distance measuring units 14i and 14j are provided on the front sides of the vehicle 10. The distance measuring units 14k and 14l are provided on the rear sides of the vehicle 10. The distance measuring units 14i to 14l detect objects on the sides of the vehicle 10 and collect peripheral information on the sides of the vehicle 10.

[0022] The imaging unit 16 is a digital camera incorporating an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor), etc. The imaging unit 16 generates a moving image including a plurality of frame images captured at a predetermined frame rate, or a still image.

[0023] The imaging units 16 are provided on the outer periphery of the vehicle body 12, and each has a wide-angle lens or a fisheye lens, and can capture images in a range of, for example, 140° to 190° in the horizontal direction. The optical axis of the imaging units 16 is set to face diagonally downward.

[0024] As a result, the imaging unit 16 collects surrounding information by capturing an image of the surroundings of the vehicle 10, including the road surface, and outputs the collected information to the parking assistance device 20. The parking assistance device 20 can detect the presence or absence of obstacles, etc., around the vehicle 10 and the positions of those obstacles based on the surrounding information collected by the imaging unit 16. Furthermore, the parking assistance device 20 can detect parking areas around the vehicle 10 and the positions of those parking areas based on the surrounding information collected by the imaging unit 16.

[0025] The imaging unit 16a is provided in the center in the left-right direction of the front end of the vehicle body 12, for example, on the front bumper. The imaging unit 16a collects captured images of the area in front of the vehicle 10 as peripheral information. The imaging unit 16b is provided in the center in the left-right direction of the rear end of the vehicle body 12, for example, on the rear bumper. The imaging unit 16b collects captured images of the area behind the vehicle 10 as peripheral information.

[0026] The imaging unit 16c is provided in the center in the longitudinal direction of the left end of the vehicle body 12, for example, on the left side mirror. The imaging unit 16c collects captured images of the left side of the vehicle 10 as peripheral information. The imaging unit 16d is provided in the center in the longitudinal direction of the right end of the vehicle body 12, for example, on the right side mirror. The imaging unit 16d collects captured images of the surroundings to the right of the vehicle 10 as peripheral information.

[0027] 2 is a block diagram showing an example of the overall configuration of a parking assistance system 200 according to an embodiment. The parking assistance system 200 is mounted on, for example, the vehicle 10, and remotely assists the driver in parking the vehicle 10 in a parking area after the driver has exited the vehicle 10.

[0028] 2, the parking assistance system 200 includes a parking assistance device 20, a monitor device 30, a braking system 40, an acceleration system 50, a steering system 60, a gear shift system 70, a vehicle speed sensor 83, a distance measurement unit 14, and an imaging unit 16. These components are connected via an in-vehicle network NT so that they can send and receive information to and from each other.

[0029] The in-vehicle network NT includes, for example, a controller area network (CAN) and a local interconnect network (LIN), etc. The in-vehicle network NT may be included as part of the parking assistance system 200.

[0030] The parking assistance system 200 is also connected to a mobile terminal 100 carried by a user such as a driver of the vehicle 10 via a wireless line such as the Internet so that information can be transmitted and received between them.

[0031] The parking assistance device 20 is configured as a microcomputer such as an ECU (Electronic Control Unit), and assists the vehicle 10 in parking.

[0032] The parking assistance device 20 includes a CPU (Central Processing Unit) 21, a communication control circuit 22, a display control circuit 23, a solid state drive (SSD) 24, a read only memory (ROM) 25, and a random access memory (RAM) 26. The CPU 21, the ROM 25, and the RAM 26 may be integrated in the same package.

[0033] The CPU 21 is an example of a hardware processor, and reads out a program stored in a nonvolatile storage device such as the ROM 25, and executes various types of arithmetic processing and control in accordance with the program.

[0034] The ROM 25 stores various programs and parameters necessary for executing the programs. The RAM 26 temporarily stores various data used in calculations by the CPU 21. The SSD 24 is a rewritable non-volatile storage device that maintains data even when the parking assistance device 20 is powered off.

[0035] Of the arithmetic processing performed by the parking assistance device 20, the communication control circuit 22 mainly executes processing related to sending and receiving information to and from the mobile terminal 100. Of the arithmetic processing performed by the parking assistance device 20, the display control circuit 23 mainly executes image processing of images obtained by the imaging unit 16, data conversion of images to be displayed on a display unit 31 (described later) provided in the monitor device 30, and the like.

[0036] The braking system 40 includes a braking unit 41 , a braking control unit 42 , and a braking unit sensor 43 , and controls the deceleration of the vehicle 10 .

[0037] The braking unit 41 is a device including, for example, a brake and a brake pedal, and decelerates the vehicle 10. The braking control unit 42 is, for example, a microcomputer having a hardware processor such as a CPU. The braking control unit 42 controls the braking unit 41 based on instructions from the parking assistance device 20 to control the deceleration of the vehicle 10. The braking unit sensor 43 is, for example, a position sensor, and detects the position of the brake pedal included in the braking unit 41. The braking unit sensor 43 outputs the detected brake pedal position to the in-vehicle network NT.

[0038] The acceleration system 50 includes an acceleration unit 51 , an acceleration control unit 52 , and an acceleration sensor 53 , and controls the acceleration of the vehicle 10 .

[0039] The acceleration unit 51 is a device including, for example, an accelerator pedal and accelerates the vehicle 10. The acceleration control unit 52 is, for example, a microcomputer having a hardware processor such as a CPU. The acceleration control unit 52 controls the acceleration unit 51 based on instructions from the parking assistance device 20 to control the acceleration of the vehicle 10. The acceleration sensor 53 is, for example, a position sensor and detects the position of the accelerator pedal included in the acceleration unit 51. The acceleration sensor 53 outputs the detected accelerator pedal position to the in-vehicle network NT.

[0040] The steering system 60 has a steering unit 61 , a steering control unit 62 , and a steering unit sensor 63 , and controls the direction of travel of the vehicle 10 .

[0041] The steering unit 61 is a device including, for example, a steering wheel, and steers the steered wheels of the vehicle 10 to steer the traveling direction of the vehicle 10. The steering control unit 62 is, for example, a microcomputer having a hardware processor such as a CPU. The steering control unit 62 controls the steering unit 61 based on instructions from the parking assistance device 20 to control the traveling direction of the vehicle 10. The steering unit sensor 63 is, for example, an angle sensor including a Hall element, and detects the steering angle, which is the rotation angle of the steering unit 61. The steering unit sensor 63 outputs the detected steering angle of the steering unit 61 to the in-vehicle network NT.

[0042] The transmission system 70 has a transmission unit 71 , a transmission control unit 72 , and a transmission unit sensor 73 , and controls the gear ratio of the vehicle 10 .

[0043] The transmission unit 71 is a device that includes, for example, a shift lever and changes the gear ratio of the vehicle 10. The transmission control unit 72 is a microcomputer that includes, for example, a hardware processor such as a CPU. The transmission control unit 72 controls the transmission unit 71 based on instructions from the parking assistance device 20 to control the gear ratio of the vehicle 10. The transmission sensor 73 is, for example, a position sensor that detects the position of a shift lever included in the transmission unit 71. The transmission sensor 73 outputs the detected position of the shift lever to the in-vehicle network NT.

[0044] The vehicle speed sensor 83 has, for example, a Hall element provided near the wheel 13 of the vehicle 10, and detects the amount of rotation or the number of rotations per unit time of the wheel 13. The vehicle speed sensor 83 outputs the number of wheel speed pulses indicating the detected amount of rotation or number of rotations to the in-vehicle network NT as a sensor value for calculating the speed (vehicle speed) of the vehicle 10. The parking assistance device 20 can calculate the speed, movement amount, etc. of the vehicle 10 based on the sensor value acquired from the vehicle speed sensor 83.

[0045] The monitor device 30 is provided on a dashboard or the like inside the vehicle 10 and includes a display unit 31 and an input unit 32 .

[0046] The display unit 31 is a display device such as a liquid crystal display (LCD) or an organic electroluminescent display (OELD). The display unit 31 displays, for example, an image based on image data transmitted by the parking assistance device 20, an image for receiving an operation instruction to switch between automatic driving and manual driving, and the like.

[0047] The input unit 32 is, for example, a touch panel provided on the display screen of the display unit 31. The input unit 32 is configured to be transparent to the content displayed on the display screen of the display unit 31. This allows the input unit 32 to allow the occupant to visually recognize the display content of the display unit 31.

[0048] The input unit 32 receives instructions input by the occupant touching a position corresponding to the display content of the display unit 31, and transmits the instructions via the in-vehicle network NT to the parking assistance device 20. Note that the input unit 32 is not limited to a touch panel, and may be a hard switch such as a push button.

[0049] The mobile terminal 100 is a computer including a CPU 111, a communication control circuit 112, an input / output control circuit 113, an SSD 114, a ROM 115, and a RAM 116. The mobile terminal 100 also includes an output unit 117 and an input unit 118. The mobile terminal 100 is held by a user such as a driver of the vehicle 10 and is configured to be portable by the user. As an example, the mobile terminal 100 may be a smartphone, a mobile phone, a tablet terminal, or the like.

[0050] The CPU 111 is an example of a hardware processor, and reads out a program stored in a nonvolatile storage device such as the ROM 115, and executes various types of arithmetic processing and control in accordance with the program.

[0051] The ROM 115 stores various programs and parameters necessary for executing the programs. The RAM 116 temporarily stores various data used in calculations by the CPU 11. The SSD 114 is a rewritable non-volatile storage device that maintains data even when the mobile terminal 100 is powered off.

[0052] Of the arithmetic processing performed by the mobile terminal 100, the communication control circuit 112 mainly executes processing related to sending and receiving information to and from the parking assistance device 20. The input / output control circuit 113 outputs information and notifications related to parking assistance received by the communication control circuit 112 from the parking assistance device 20 to the output unit 117. The input / output control circuit 113 also accepts operation details input from the input unit 118 and transmits them from the communication control circuit 112 to the parking assistance device 20.

[0053] The output unit 117 is a display device such as a liquid crystal display or an organic EL display. The output unit 117 displays, for example, an image based on image data transmitted by the parking assistance device 20 and an image for accepting operation instructions related to parking assistance. The output unit 117 may include a speaker or the like capable of outputting voice, an alarm, or the like.

[0054] The input unit 118 is, for example, a touch panel or the like provided on the output unit 117, which is a display device or the like. The input unit 118 accepts instructions input by the user by touching a position corresponding to the display content of the display device. Note that the input unit 118 may also include push-button type hard switches such as a keyboard.

[0055] 3 is a block diagram showing an example of the functional configuration of the parking assistance device 20 according to the embodiment. As shown in FIG. 3, the parking assistance device 20 includes a display control unit 201, a notification generation unit 202, a transmission / reception unit 203, an acquisition unit 204, a detection unit 205, a path calculation unit 206, a movement control unit 207, and a determination unit 208 as functional units.

[0056] These functional units are realized, for example, by the CPU 21 reading and executing a program stored in a storage device such as the ROM 25. Alternatively, they are realized by the operation of the communication control circuit 22, the display control circuit 23, the SSD 24, etc. under the control of the CPU 21 in accordance with the program.

[0057] Note that some or all of these functional units may be configured by hardware such as a circuit including an ASIC (Application Specific Integrated Circuit).

[0058] The display control unit 201 generates content to be displayed on the display unit 31 of the monitor device 30 and causes it to be displayed on the display unit 31. That is, the display control unit 201 causes the display unit 31 to display, for example, a notification screen that a parking area in which the vehicle 10 can be parked has been detected, a selection screen for starting, pausing, or canceling parking assistance, an image of the surroundings of the vehicle 10 during parking assistance, and the like.

[0059] The notification generation unit 202 generates a notification regarding parking assistance to be sent to the mobile terminal 100. That is, the notification generation unit 202 generates notification content for the user, such as the start, interruption, or cancellation of parking assistance.

[0060] The transmitting / receiving unit 203 transmits and receives information to and from the mobile terminal 100. That is, the transmitting / receiving unit 203 receives, for example, location information of the mobile terminal 100 from the mobile terminal 100. The location information of the mobile terminal 100 is information indicating the location of the mobile terminal 100, such as radio waves emitted by the mobile terminal 100 or an operation signal of the mobile terminal 100 by the user. The transmitting / receiving unit 203 also transmits the notification generated by the notification generation unit 202 to the mobile terminal 100. The transmitting / receiving unit 203 also receives, from the mobile terminal 100, operation instructions from the user in response to the transmitted notification, etc.

[0061] The acquisition unit 204 acquires, as information about the surroundings of the vehicle 10 , information about the transmission and reception of sound waves from the distance measurement unit 14 and captured images of the surroundings of the vehicle 10 from the imaging unit 16 .

[0062] The detection unit 205 detects obstacles, parking areas, etc. around the vehicle 10 based on the surrounding information acquired by the acquisition unit 204.

[0063] Obstacles include various objects such as other vehicles, walls, pillars, fences, protrusions, steps, wheel chocks, etc. Obstacles may also be, for example, people walking in a parking lot.

[0064] A parking area is an area for parking the vehicle 10, and is an area for parking the vehicle 10 that is demarcated by, for example, a demarcation line, a frame line, a straight line, a strip, a step, etc., or an area for parking the vehicle 10 that is separated by structures such as spaces between vehicles or pillars or walls. A parking area that can be parked is a parking area where the vehicle 10 can be parked, that is, a parking area that does not, in principle, have other vehicles or obstacles that would hinder the parking of the vehicle 10.

[0065] The detection unit 205 detects the presence or absence of an obstacle, the distance from the vehicle 10 to the obstacle, etc., based on, for example, the detection result of the distance measurement unit 14. Furthermore, the detection unit 205 detects the presence or absence of an obstacle, the parking area, their positions (orientations) relative to the vehicle 10, as well as their shapes, sizes, heights, etc., by, for example, image processing based on the image captured by the imaging unit 16.

[0066] The determination unit 208 combines the various detection results of the detection unit 205 to extract information such as whether there is an obstacle around the vehicle 10, whether the obstacle may interfere with the driving of the vehicle 10, whether there is a parking area around the vehicle 10, and whether it is possible to park the vehicle 10 in that parking area.

[0067] Here, even if an obstacle is detected in or around a parking area, the obstacle may be a traffic cone, bicycle, or other movable object placed in the parking area, or it may be an object that can be removed or dismantled by a person who is legitimately authorized to use the parking area, such as a chain strung across the entrance to the parking area, a built-in pole erected at the entrance to the parking area, or a parking barrier installed in the parking area.

[0068] The embedded pole is a device that prevents unauthorized parking by being pulled out from the ground when the parking space is not in use. A legitimate vehicle can retract the pole into the ground by performing a predetermined operation, thereby releasing the unused status of the parking space and allowing the vehicle 10 to be parked there.

[0069] The parking prevention plate is a device with a flip-up plate to prevent unauthorized parking. When the parking area is unused, the parking prevention plate can be kept upright to prevent unauthorized parking. When parking in the parking area, a legitimate vehicle performs a predetermined operation to lower the parking prevention plate, thereby releasing the unused state of the parking area and allowing the vehicle 10 to park.

[0070] When an obstacle is detected in or around the parking area, the determination unit 208 determines whether the obstacle can be moved, removed, or released to change its state so that it does not interfere with parking of the vehicle 10.

[0071] The determination unit 208 determines whether the obstacle can change its state as described above, etc. If the determination unit 208 determines that the detected obstacle is an obstacle that can change its state so as not to interfere with parking of the vehicle 10, the determination unit 208 recognizes the parking area as a parking area in which the vehicle 10 can be parked.

[0072] The display control unit 201 displays on the display unit 31 of the monitor device 30 that a parking area has been detected, and waits for an instruction from the driver to start parking assistance.

[0073] When an instruction to start parking assistance is received, the route calculation unit 206 calculates a target position to which the vehicle 10 will be guided based on the detection result by the detection unit 205, and calculates a movement route for moving the vehicle 10 to the target position within the parking area.

[0074] In addition, the route calculation unit 206 may calculate a turning position as necessary. One or more turning positions are set when parking is difficult after a single back or forward movement.

[0075] The route calculation unit 206 calculates a travel route of the vehicle 10 from the current position of the vehicle 10 to the target position so as to guide the vehicle 10 to the target position calculated as described above. The travel route may include a turning point as described above. In this case, the route calculation unit 206 calculates a travel route from the current position of the vehicle 10 to the target position via the turning point.

[0076] The route calculation unit 206 calculates a movement route for parking, such as perpendicular parking by backing up, perpendicular parking by forwarding, or parallel parking.

[0077] When assisting parking of the vehicle 10, the movement control unit 207 controls each unit of the vehicle 10 to move the vehicle 10. Specifically, the movement control unit 207 controls all or part of the braking system 40, acceleration system 50, steering system 60, and transmission system 70 to move the vehicle 10 to a target position along a movement route. At this time, the movement control unit 207 moves the vehicle 10 while receiving feedback from each sensor, such as the braking unit sensor 43, acceleration unit sensor 53, steering unit sensor 63, transmission unit sensor 73, and vehicle speed sensor 83.

[0078] Even after the movement control unit 207 starts moving the vehicle 10, the detection unit 205 continues to acquire surrounding information from the distance measurement unit 14 and the imaging unit 16. The above-mentioned route calculation unit 206 appropriately recalculates the route of movement based on the detection result by the detection unit 205. When the route of movement is recalculated, the movement control unit 207 continues the movement of the vehicle 10 so as to fine-tune the position of the vehicle 10 accordingly.

[0079] Here, the above-mentioned obstacle determination method of the determination unit 208 will be described in detail using several examples.

[0080] The determination unit 208 can make the above determination based on, for example, the size of the obstacle relative to the parking area, its positional relationship (front, back, left, right, etc.), whether there is a gap between the obstacle and the partition lines, frame lines, straight lines, stripes, steps, etc. that define the parking area. Alternatively, these determinations may be made in comparison with other vehicles parked in parking areas adjacent to the parking area. In other words, the determination of whether an obstacle is capable of changing its state can be made based on, for example, the size of the obstacle relative to adjacent vehicles, its positional relationship (front, back, left, right, etc.), whether there is a gap between the adjacent vehicles and the obstacle.

[0081] 4 to 7 are schematic diagrams showing some examples of the method of determining obstacles 95 to 98 by the parking assistance device 20 according to the embodiment.

[0082] As shown in FIG. 4, in a parking lot or the like where a plurality of parking areas are arranged, a vehicle 91 is sandwiched between parking areas in which another vehicle 91 is parked, and the central parking area is empty.

[0083] The detection unit 205 of the parking assistance device 20 mounted on the vehicle 10 detects the parking area PA, which is an empty parking area in the center, from among a plurality of parking areas. However, in the example of Fig. 4, an obstacle 95 such as a traffic cone is placed within this parking area PA. The detection unit 205 also detects the obstacle 95 within the parking area PA.

[0084] The determination unit 208 determines whether the detected obstacle 95 is an obstacle whose state can be changed, such as by being moved, removed, or released, based on the size of the detected obstacle 95 relative to the parking area PA, its positional relationship in the front, back, left, right, etc., and whether there is a gap between the obstacle and the partitioning of the parking area PA.

[0085] The size of an obstacle 95 such as a triangular cone shown in Figure 4 is sufficiently small relative to the parking area PA, and it is placed approximately in the center of the parking area PA so that it fits within the parking area PA, and there is also a gap between it and the dividing lines that divide the parking area PA.

[0086] Furthermore, when viewed in relation to other vehicles 91 parked in parking areas adjacent to the central parking area PA, the size of the obstacle 95 is sufficiently small compared to the other vehicles 91, and the obstacle 95 is placed at a distance from the other vehicles 91 with a gap between them.

[0087] Therefore, the determination unit 208 determines that the obstacle 95 is an obstacle whose state can be changed so that it does not interfere with parking of the vehicle 10.

[0088] 5, the detection unit 205 of the parking assistance device 20 detects the central parking area PA and detects a chain or the like placed at the entrance of this parking area PA as an obstacle 96. The determination unit 208 determines, based on the above-described determination method, whether or not the obstacle 96 is an obstacle whose state can be changed.

[0089] The obstacle 96, such as a chain, shown in Figure 5 is small enough in size relative to the parking area PA, and is placed at the entrance to the parking area PA so that it fits within the width of the parking area PA, and there is also a gap between it and the dividing lines that define the parking area PA.

[0090] Furthermore, when viewed in relation to other vehicles 91 parked in parking areas adjacent to the central parking area PA, the size of the obstacle 96 is sufficiently small compared to the other vehicles 91, and the obstacle 96 is positioned at a distance from the other vehicles 91 with a gap between them.

[0091] Therefore, the determination unit 208 determines that the obstacle 96 is an obstacle whose state can be changed so that it does not interfere with parking of the vehicle 10.

[0092] In the example of Fig. 6, the detection unit 205 of the parking assistance device 20 detects the central parking area PA and detects a bicycle or the like placed in this parking area PA as an obstacle 97. The determination unit 208 determines, based on the above-mentioned determination method, whether or not the obstacle 97 is an obstacle whose state can be changed.

[0093] An obstacle 97 such as a bicycle shown in Figure 6 is sufficiently small in size relative to the parking area PA, and is placed approximately in the center of the parking area PA so as to fit within the parking area PA, with a gap being visible between it and the dividing lines that define the parking area PA.

[0094] Furthermore, when viewed in relation to other vehicles 91 parked in parking areas adjacent to the central parking area PA, the size of the obstacle 97 is sufficiently small compared to the other vehicles 91, and the obstacle 97 is placed at a distance from the other vehicles 91 with a gap between them.

[0095] Therefore, the determination unit 208 determines that the obstacle 97 is an obstacle whose state can be changed so that it does not interfere with parking of the vehicle 10.

[0096] 7, the detection unit 205 of the parking assistance device 20 detects the central parking area PA and detects other vehicles 91 and the like parked outside the parking area PA as obstacles 98. The determination unit 208 determines, based on the above-described determination method, whether or not the obstacle 98 is an obstacle whose state can be changed.

[0097] The obstacle 98 shown in Figure 7 is too large for the parking area PA, and is located at the edge of the parking area PA, straddling the dividing lines that define the parking area PA, with no gap between it and the dividing lines.

[0098] Also, when viewed in relation to another vehicle 91 parked in a parking area adjacent to the central parking area PA, a part of the other vehicle 91 is protruding into the parking area PA, and the obstacle 98 is connected to the other vehicle 91.

[0099] Therefore, the determination unit 208 determines that the obstacle 98 is an obstacle that cannot be changed in state so as not to hinder parking of the vehicle 10 .

[0100] Note that instead of or in addition to the above method, the determination unit 208 may be able to make the above determination of an obstacle based on the results of learning by machine learning, etc. In this case, the obstacles 95 to 98 in Figures 4 to 7 may be determined to be a traffic cone, a chain, a bicycle, an adjacent vehicle 91 protruding into the parking area PA, etc.

[0101] (Example of Operation of Parking Assistance Device) Next, an example of operation of the parking assistance device 20 according to the embodiment will be described with reference to Fig. 8 to Fig. 14. Fig. 8 to Fig. 14 are schematic diagrams illustrating an example of the operation of parking assistance by the parking assistance device 20 according to the embodiment. Fig. 8 to Fig. 14 describe an example in which the vehicle 10 is reverse-parked.

[0102] As shown in FIG. 8, in a parking lot or the like where a plurality of parking areas are arranged, it is assumed that one parking area is empty, sandwiched between parking areas in which another vehicle 91 is parked.

[0103] The detection unit 205 of the parking assistance device 20 mounted on the vehicle 10 detects the parking area PA, which is an empty parking area in the center, from among a plurality of parking areas. However, in the example of Fig. 8, an obstacle 95 such as a traffic cone is placed within this parking area PA, as shown in Fig. 4 above. The detection unit 205 also detects the obstacle 95 within the parking area PA.

[0104] The determination unit 208 determines whether it is possible to change the state of the detected obstacle 95 so that it does not interfere with parking, based on at least one of the following methods: the size and positional relationship of the detected obstacle 95 with respect to the parking area PA, the presence or absence of a gap between the obstacle 95 and the parking area PA, and the learning results of machine learning. In the example of Fig. 8, it is assumed that it is determined that it is possible to change the obstacle 95 so that it does not interfere with parking. The determination unit 208 recognizes the parking area PA as a parking area PA in which the vehicle 10 can be parked.

[0105] The display control unit 201 notifies the display unit 31 of the monitor device 30 that an obstacle 95 that needs to be removed has been detected in the parking area PA, and also displays a selection screen for starting parking assistance, etc. An example of such a selection screen is shown in FIG.

[0106] In the example shown in Figure 9, a button that allows the user to select whether or not to start parking assistance is displayed superimposed on the camera image showing the parking area PA, along with a mark 95M indicating that there is an obstacle 95 that needs to be removed within the parking area PA.

[0107] 10 , it is assumed that the driver has selected the start of parking assistance on the monitor device 30. Note that, in cases where an obstacle detected in the parking area PA is actually impossible to remove, the determination by the determination unit 208 that the obstacle can be removed may be an erroneous determination. In this case, the driver can eliminate the erroneous determination by the determination unit 208 by not selecting the start of parking assistance.

[0108] As shown in FIG. 11, the driver gets off the vehicle 10 while carrying the mobile terminal 100, and also moves the obstacle 95 in the parking area PA out of the parking area PA.

[0109] 12 , the parking assistance device 20 starts parking assistance when the driver selects to start parking assistance and when it detects that the mobile terminal 100 has moved outside the vehicle 10. At this time, the notification generation unit 202 generates a notification such as "Parking assistance started," and the transmission / reception unit 203 transmits the notification to the mobile terminal 100, which may be displayed on the display device of the mobile terminal 100.

[0110] As shown in FIG. 13, the path calculation unit 206 of the parking assistance device 20 calculates a movement path TR from the current position PP of the vehicle 10, through a turning point SP in the forward direction of the vehicle 10, to a target position DP in the backward direction of the vehicle 10 from the turning point SP and located within the parking area PA.

[0111] The movement control unit 207 of the parking assistance device 20 moves the vehicle 10 on the movement route TR in accordance with the calculation result of the route calculation unit 206. At this time, the notification generation unit 202 may generate a notification such as "Parking assistance in progress" indicating that the vehicle 10 is currently receiving parking assistance, and the transmission / reception unit 203 may transmit the notification to the mobile terminal 100, which may be displayed on the display device of the mobile terminal 100.

[0112] 14 , the movement control unit 207 continues the movement of the vehicle 10 and stops it at the desired target position DP. This ends the parking assistance. At this time, the notification generation unit 202 may generate a notification such as "Parking assistance ended," and the transmission / reception unit 203 may transmit the notification to the mobile terminal 100, causing it to be displayed on the display device of the mobile terminal 100.

[0113] 8 to 14, the parking assistance device 20 performs parking assistance for backward parking. However, the parking assistance device 20 also performs parking assistance for the vehicle 10 in the same manner in forward parking, parallel parking, or other similar situations.

[0114] (Processing Example of Parking Assistance Apparatus) Next, an example of parking assistance processing by the parking assistance apparatus 20 according to the embodiment will be described with reference to Fig. 15. Fig. 15 is a flow diagram showing an example of the procedure of parking assistance processing by the parking assistance apparatus 20 according to the embodiment.

[0115] 15, the detection unit 205 of the parking assistance device 20 detects parking areas PA that are candidates for the parking area PA, such as vacant parking areas PA (step S101). The determination unit 208 determines whether or not an obstacle has been detected in or around the parking area PA, along with the detected parking area PA (step S102).

[0116] When the detection unit 205 detects an obstacle (step S102: Yes), the determination unit 208 determines whether the obstacle can be removed by changing its state so that it does not interfere with parking, such as by moving, removing, or canceling the obstacle (step S103). When the determination unit 208 determines that the obstacle cannot be removed (step S103: No), the determination unit 208 determines that the vehicle 10 cannot be parked in the parking area PA, and ends the parking assistance process with an error (step S104).

[0117] If the judgment unit 208 judges that the obstacle can be removed (step S103: Yes), or if no obstacle is detected in the parking area PA in the first place (step S102: No), the display control unit 201 displays a selection screen for starting parking assistance, etc., on the display unit 31 of the monitor device 30.

[0118] In addition, if a removable obstacle is detected, the display control unit 201 displays the selection screen for starting parking assistance, as well as a display indicating that an obstacle 95 that needs to be removed has been detected within the parking area PA.

[0119] The driver can choose not to start parking assistance if he / she does not wish to park in the detected parking area PA, or if an obstacle in the parking area PA cannot actually be removed due to an erroneous recognition by the determination unit 208. When a selection is made from the input unit 32 of the monitor device 30 not to perform parking assistance (step S106: No), the parking assistance device 20 cancels and ends the parking assistance process (step S107).

[0120] When a selection is made to perform parking assistance from the input unit 32 of the monitor device 30 (step S106: Yes), parking assistance is started after it is detected that the mobile terminal 100 has moved outside the vehicle 10 based on the location information from the mobile terminal 100 (step S108).

[0121] The route calculation unit 206 calculates a travel route TR from the current position PP of the vehicle 10 to the target position DP within the parking area PA, possibly via the turning position SP (step S109).

[0122] The movement control unit 207 controls the movement of the vehicle 10 according to the movement route TR calculated by the route calculation unit 206 (step S110). The movement control unit 207 also determines whether the parking assistance process has ended, for example, when the vehicle 10 reaches the target position DP (step S111). If the parking assistance process is ongoing (step S111: No), the process of step S110 is repeated until the parking assistance process ends (step S111: Yes).

[0123] This completes the parking assistance process by the parking assistance device 20 of the embodiment.

[0124] (Summary) A parking assistance device assists a vehicle in parking based on information about the surroundings of the vehicle obtained from various sensors such as a distance measurement unit and an imaging unit. In recent years, there has been progress in the development of parking assistance devices that remotely assist a vehicle in parking a parking area while the driver is out of the vehicle.

[0125] In remote parking assistance, there is a need to continue parking assistance processing even if an obstacle is detected in the parking area, etc., after the driver has dismounted the vehicle and removed the obstacle. Conventional parking assistance devices have not been able to meet such needs because the parking assistance ends with an error when an obstacle is detected in the parking area, or the parking area is not recognized as an area where parking is possible in the first place.

[0126] According to the parking assistance device 20 of the embodiment, when an obstacle is detected along with a parking area, if it is determined that the state of the obstacle can be changed so that it does not interfere with parking of the vehicle 10, it is determined that parking of the vehicle 10 is possible in that parking area. As a result, even if an obstacle is detected in a parking area, etc., if the obstacle can be removed, the driver can be given the option of continuing parking assistance.

[0127] According to the parking assistance device 20 of the embodiment, when it is determined that parking is possible in the parking area, the driver of the vehicle 10 is notified that an obstacle has been detected. In this way, for example, the display control unit 201 can notify the driver that there is an obstacle in the parking area, thereby urging the driver to remove the obstacle.

[0128] (Modification) Next, a parking assistance device according to a modification of the embodiment will be described with reference to Fig. 16. The parking assistance device according to the modification differs from the embodiment described above in that it starts parking assistance control after receiving a notification from the driver's mobile terminal that the obstacle has been removed.

[0129] In the following description, the configurations of the modified examples corresponding to the configurations of the above-described embodiment will be referred to in FIGS. 1 to 3 and will be denoted by the same reference numerals.

[0130] FIG. 16 is a flowchart showing an example of a procedure of parking assistance processing by a parking assistance device according to a modified example of the embodiment.

[0131] The processing from steps S201 to S207 shown in Fig. 16 is the same as the processing from steps S101 to S107 of the above-described embodiment shown in Fig. 15. That is, when the detection unit 205 detects a parking area (step S201), if an obstacle is detected along with the parking area (step S202: Yes), if the obstacle cannot be removed (step S203: No), the processing is terminated with an error (step S204), if the obstacle can be removed (step S203: Yes), a selection screen for starting parking assistance is displayed (step S205), and if the start of parking processing is not selected (step S206: No), the processing is canceled and terminated (step S207).

[0132] In the modified parking assistance device, when the start of parking processing is selected from the selection screen (step S206: Yes), the notification generation unit 202 generates a notification requesting the removal of the obstacle, and the transmission / reception unit 203 transmits the notification to the driver's mobile terminal 100 (step S211).

[0133] The determination unit 208 waits for a notification that the obstacle has been removed from the driver's mobile terminal 100 in response to the notification transmitted from the transmission / reception unit 203 (step S212). The determination unit 208 waits until the notification of removal completion is received from the mobile terminal 100 (step S212: No).

[0134] When a removal completion notification is received from the mobile terminal 100 (step S212: Yes), the determination unit 208 causes the detection unit 205 to detect the obstacle again, and determines whether the obstacle has been removed (step S213). That is, the determination unit 208 determines whether the state of the obstacle has changed, such as by being moved, removed, or released, so that the obstacle no longer interferes with parking.

[0135] If the obstacle has not been completely removed (step S213: No), the determination unit 208 causes the notification of step S211 to be repeatedly retransmitted until the number of notifications to the mobile terminal 100 reaches a predetermined number (step S214: No), thereby urging the driver to remove the obstacle. If the number of notifications to the mobile terminal 100 exceeds the predetermined number (step S214: Yes), the parking assistance process is ended with an error (step S204).

[0136] If the obstacle has been completely removed (step S213: Yes), the determination unit 208 starts parking assistance (step S228).

[0137] The subsequent processes, including the process of step S228, are the same as the processes of steps S108 to S111 of the above-described embodiment shown in Fig. 15. That is, the route calculation unit 206 calculates a movement route TR of the vehicle 10 (step S229), and the movement control unit 207 repeats movement control of the vehicle 10 along the movement route TR (step S230) until the parking assistance ends (step S231).

[0138] If no obstacle is detected in the parking area detected by the detection unit 205 (step S202: No), the parking assistance process is executed while skipping the process of step S203 and steps S211 to S213.

[0139] That is, if no obstacle is detected (step S202: No), a selection screen for starting parking assistance is displayed (step S205s), and if the start of parking processing is not selected (step S206s: No), the processing is canceled and terminated (step S207).

[0140] If the start of parking processing is selected from the selection screen (step S206s: Yes), the route calculation unit 206 calculates the movement route TR of the vehicle 10 (step S229), and the movement control unit 207 repeats the movement control of the vehicle 10 along the movement route TR (step S230) until the parking assistance is completed (step S231).

[0141] This completes the parking assistance process by the parking assistance device of the modified example.

[0142] According to the parking assistance device of the modified example, when it is determined that parking is possible in the parking area, the driver of the vehicle 10 is notified that an obstacle has been detected. In this way, for example, by notifying the driver's mobile terminal 100 that an obstacle is present in the parking area, the driver can be prompted to remove the obstacle.

[0143] According to the parking assistance device of the modified example, after notifying the driver's mobile terminal 100, it is determined whether the state of the obstacle has changed so that it no longer interferes with parking of the vehicle 10, and parking assistance is initiated if it is determined that the state of the obstacle has changed. This makes it possible to more reliably provide parking assistance in a state where the obstacle has been removed.

[0144] In the above-described modified example, the parking assistance device detects the state of the obstacle again after receiving the notification of completion of obstacle removal from the driver's mobile device 100. However, the parking assistance device may start the parking assistance process in response to the notification of completion of obstacle removal from the mobile device 100 without redetecting the obstacle. In other words, the processes of steps S213 and S214 in FIG. 15 described above may not be performed.

[0145] In this way, by following the instructions of the driver, who is ultimately responsible for the safe driving of the vehicle 10, it is possible to perform parking assistance without obstacles interfering with parking. Also, it is possible that an obstacle that does not initially interfere with parking, such as a bicycle parked at the back of a parking area, may be erroneously detected. By following the driver's instructions, it is possible to continue parking assistance processing even in cases such as when the determination unit 208 makes an erroneous determination.

[0146] In this case, the obstacle does not necessarily have to be removed by the driver, and after the driver confirms the obstacle, the driver may be able to send a notification instructing the continuation of parking assistance from the mobile device 100. After sending a notification of a request to remove the obstacle to the mobile device 100, the determination unit starts parking assistance when it receives a notification instructing the continuation of parking assistance from the mobile device 100.

[0147] Furthermore, after the driver has confirmed the obstacle, the driver may be able to send a notification from the mobile device 100 to cancel the continuation of the parking assistance. If the detected obstacle is actually impossible to remove due to an erroneous recognition by the determination unit 208, it is assumed that the driver will select not to start the parking assistance on the selection screen displayed on the monitor device 30 inside the vehicle cabin by the display control unit 201, as described above. However, if it is found that the obstacle cannot be removed by checking after getting out of the vehicle, the driver can cancel the continuation of the parking assistance from the mobile device 100, thereby further improving convenience for the driver.

[0148] In the above-described embodiment and modified example, the display control unit 201 displays a selection screen for starting parking assistance on the monitor device 30 in the vehicle cabin, and the driver instructs the start of parking assistance before getting out of the vehicle. However, by displaying a selection screen on the driver's mobile terminal 100, the driver may be able to instruct the start of parking assistance after getting out of the vehicle.

[0149] In this case, it is preferable to display the presence of an obstacle in the parking area or the like on the mobile terminal 100 using graphics, text information, or the like. This can prevent the communication speed with the mobile terminal 100 from decreasing.

[0150] Furthermore, in the above-described embodiment and modified example, the parking assistance device 20 is mounted on the vehicle 10, but the parking assistance device 20 does not necessarily have to be mounted on the vehicle 10. For example, the parking assistance device may be located at a location away from the vehicle, acquire information about the surroundings of the vehicle from various sensors, and remotely assist the vehicle in parking.

[0151] Summary of the embodiment The parking assistance device (20) of the embodiment includes at least the following components.

[0152] A parking assistance device (20) according to an embodiment is a parking assistance device (20) that provides parking assistance to a vehicle (10) having sensors (14a to 14l, 16a to 16d) that collect surrounding information in accordance with an instruction from a terminal (100) carried by a user of the vehicle (10), and includes a detection unit (205) that detects a parking area (PA) and obstacles (95 to 98) in or around the parking area (PA) based on the surrounding information from the sensors (14a to 14l, 16a to 16d), and a determination unit (208) that makes a determination about the obstacles (95 to 98), and the determination unit (208) When the obstacles (95 to 98) are detected together with the parking area (PA), if it is determined that the state of the obstacles (95 to 98) can be changed so that they do not interfere with parking of the vehicle (109), it is determined that the vehicle (10) can be parked in the parking area (PA).

[0153] As a result, even if an obstacle is detected within a parking area, etc., if the obstacle can be removed, the driver can be given the option of continuing parking assistance.

[0154] When the determination unit (208) determines that the state of the obstacle (95-98) can be changed, it is preferable to notify the driver of the vehicle (10) that the obstacle (95-98) whose state can be changed has been detected.

[0155] In this way, for example, the display control unit 201 can notify the driver that there is an obstacle in the parking area or the like, thereby urging the driver to remove the obstacle.

[0156] After the notification, it is preferable that the parking assistance be started when a notification instructing continuation of the parking assistance is received from the terminal (100).

[0157] In this way, by following the driver's instructions, it is possible to continue the parking assistance process even if, for example, the determination unit 208 makes an erroneous determination.

[0158] After the notification, the determination unit (208) determines whether the state of the obstacles (95 to 98) has changed so that they do not interfere with parking of the vehicle (10), and preferably starts the parking assistance when the determination unit (208) determines that the state of the obstacles (95 to 98) has changed so that they do not interfere with parking of the vehicle (10).

[0159] This allows parking assistance to be performed with the obstacles more reliably removed.

[0160] The effects of the dependent claims and embodiments are additional effects that are different from the effects of the independent claims.

[0161] DESCRIPTION OF SYMBOLS 10 Vehicle 14 Distance measurement unit 16 Imaging unit 20 Parking assistance device 100 Portable terminal 201 Display control unit 202 Notification generation unit 203 Transmitting / receiving unit 204 Acquisition unit 205 Detection unit 206 Route calculation unit 207 Movement control unit 208 Determination unit

Claims

1. A parking assistance device that provides parking assistance to a vehicle having a sensor that collects surrounding information in accordance with instructions from a terminal carried by a user of the vehicle, the parking assistance device comprising: a detection unit that detects a parking area and an obstacle within or around the parking area based on the surrounding information from the sensor; and a judgment unit that makes a judgment about the obstacle, wherein when the obstacle is detected along with the parking area, the judgment unit determines that it is possible to change the state of the obstacle so that it does not hinder parking of the vehicle, and then determines that it is possible to park the vehicle in the parking area.

2. The parking assistance device according to claim 1, wherein, when the determination unit determines that the state of the obstacle can be changed, the determination unit notifies the driver of the vehicle that an obstacle whose state can be changed has been detected.

3. The parking assistance device according to claim 2, wherein the parking assistance is started when, after the notification, a notification instructing the continuation of the parking assistance is received from the terminal.

4. The parking assistance device of claim 2, wherein after the notification, the determination unit determines whether the state of the obstacle has changed so that it does not hinder parking of the vehicle, and starts the parking assistance when the determination unit determines that the state of the obstacle has changed so that it does not hinder parking of the vehicle.

Images