Parking assistance device and parking assistance method

Abstract

To provide a parking support device and a parking support method each enabling travelling by automatic drive taking into consideration to passengers riding on a vehicle when the vehicle automatically travels to a parking position.SOLUTION: A parking support device includes a storage part and a notification part. The storage part works as the parking support device for performing automatic travel of a vehicle on the basis of teaching travel, and stores open / close information for determining whether or not to open / close the door of the vehicle with respect to a parking position of the vehicle in the teaching travel. The notification part performs notification on the basis of the open / close information when making the vehicle automatically park at the parking position.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a parking assistance device and a parking assistance method. [Background technology]

[0002] Conventionally, parking assistance technologies are known that move a vehicle by automatic driving or driving assistance when parking the vehicle. One such parking assistance technology is a technology that performs parking assistance based on a teacher driving by the driver. This technology is used, for example, when the driver repeatedly parks the vehicle in a fixed parking position, such as a parking lot at the driver's home or workplace. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-26091 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the parking position of a vehicle in which a parking assistance device is operating does not necessarily ensure that there is a place for passengers in the vehicle to get off.

[0005] The present disclosure provides a parking assistance device and a parking assistance method that are capable of driving the vehicle automatically or with driving assistance while taking into account passengers in the vehicle when parking the vehicle. [Means for solving the problem]

[0006] The parking assistance device according to the present disclosure includes a storage unit and a notification unit. The storage unit is a parking assistance device that automatically drives a vehicle based on teacher driving, and stores opening / closing information for determining whether or not to open / close the doors of the vehicle with respect to the parking position of the vehicle during the teacher driving. The notification unit issues a notification based on the opening / closing information when automatically parking the vehicle at the parking position. When the door of the vehicle cannot be opened or closed during the teacher driving, the notification unit issues a notification before the automatic parking based on the teacher driving starts. Effect of the Invention

[0007] According to the parking assistance device and parking assistance method disclosed herein, when a vehicle automatically drives to a parking position, the vehicle can be driven automatically while taking into account passengers in the vehicle. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a vehicle equipped with a parking assistance device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a configuration in the vicinity of a driver's seat of a vehicle according to an embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of the parking assistance device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of parking assistance according to the embodiment. [Diagram 5] FIG. 5 is a block diagram showing an example of functions included in the parking assistance device according to the embodiment. [Figure 6] FIG. 6 is a table illustrating an example of opening / closing information according to the embodiment. [Figure 7] FIG. 7 is a table illustrating an example of opening / closing information according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a notification according to the embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the flow of a driving route learning process executed by the parking assistance device according to the embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of the flow of the parking assistance process executed by the parking assistance device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of a parking assistance device and a parking assistance method according to the present disclosure will be described with reference to the drawings.

[0010] (Vehicle configuration example) Fig. 1 is a diagram showing an example of a vehicle 1 equipped with a parking assistance device 100 according to this embodiment. As shown in Fig. 1, the vehicle 1 includes a vehicle body 12 and two pairs of wheels 13 arranged along a predetermined direction on the vehicle body 12. The two pairs of wheels 13 include a pair of front tires 13f and a pair of rear tires 13r.

[0011] The vehicle body 12 has a front end F, which is the end on the front tire 13f side, and a rear end R, which is the end on the rear tire 13r side. The vehicle body 12 has a substantially rectangular shape when viewed from above, and the four corners of the substantially rectangular shape are sometimes called ends. In addition, although not shown in FIG. 1, the vehicle 1 is equipped with a display device, a speaker, and an operation unit.

[0012] A pair of bumpers 14 are provided at the front end F and rear end R of the vehicle body 12 near the lower end of the vehicle body 12. Of the pair of bumpers 14, the front bumper 14f covers the entire front surface and part of the side surface near the lower end of the vehicle body 12. Of the pair of bumpers 14, the rear bumper 14r covers the entire rear surface and part of the side surface near the lower end of the vehicle body 12.

[0013] At predetermined ends of the vehicle body 12, a wave transmitting / receiving unit 15f and a wave transmitting / receiving unit 15r for transmitting and receiving sound waves such as ultrasonic waves are disposed. For example, one or more wave transmitting / receiving units 15f are disposed on the front bumper 14f, and one or more wave transmitting / receiving units 15r are disposed on the rear bumper 14r. Hereinafter, when the wave transmitting / receiving units 15f and the wave transmitting / receiving units 15r are not particularly limited, they are simply referred to as the wave transmitting / receiving unit 15. In addition, the number and positions of the wave transmitting / receiving units 15 are not limited to the example shown in FIG. 1. For example, the vehicle 1 may be provided with the wave transmitting / receiving units 15 on the left and right sides.

[0014] In this embodiment, sonar using ultrasonic waves will be described as an example of the wave transmitting and receiving unit 15, but it is not limited to sonar, and the wave transmitting and receiving unit 15 may be a sensor capable of measuring the distance to a three-dimensional object, such as a radar that transmits and receives electromagnetic waves. It is also possible to measure the distance to the three-dimensional object from an image captured by a camera mounted on the vehicle body 12. Alternatively, the vehicle 1 may be equipped with both sonar and radar. Also, the wave transmitting and receiving unit 15 may simply be called a sensor.

[0015] More specifically, the wave transmitting / receiving unit 15 includes a wave transmitting unit that transmits sound waves such as ultrasonic waves or electromagnetic waves, and a wave receiving unit that receives reflected sound waves that are the sound waves or electromagnetic waves transmitted from the wave transmitting unit and reflected by an object. The wave transmitting / receiving unit 15 detects obstacles around the vehicle 1 based on the results of transmitting and receiving the sound waves or electromagnetic waves. The wave transmitting / receiving unit 15 also measures the distance between the vehicle 1 and obstacles around the vehicle 1 based on the results of transmitting and receiving the sound waves or electromagnetic waves.

[0016] The vehicle 1 also includes a first imaging device 16a that images the area in front of the vehicle 1, a second imaging device 16b that images the area behind the vehicle 1, a third imaging device 16c that images the area to the left of the vehicle 1, and a fourth imaging device that images the area to the right of the vehicle 1. The fourth imaging device is not shown in the figure.

[0017] Hereinafter, when there is no need to distinguish between the first imaging device 16a, the second imaging device 16b, the third imaging device 16c, and the fourth imaging device, they will simply be referred to as imaging devices 16. The positions and the number of imaging devices 16 are not limited to the example shown in Fig. 1. Alternatively, the vehicle 1 may further include other imaging devices in addition to the above-mentioned examples.

[0018] The imaging device 16 is capable of capturing images of the surroundings of the vehicle 1, and is, for example, a camera that captures color images. The captured images captured by the imaging device 16 may be moving images or still images. The imaging device 16 may be a camera built into the vehicle 1, or may be a camera that is attached to the vehicle 1 at a later date.

[0019] The vehicle 1 is also equipped with a parking assistance device 100. The parking assistance device 100 performs automatic driving or driving assistance of the vehicle 1 to the parking position. The parking assistance device 100 is an information processing device that can be mounted on the vehicle 1, and is, for example, an ECU (Electronic Control Unit) or an OBU (On Board Unit) provided inside the vehicle 1. Alternatively, the parking assistance device 100 may be an external device installed near the dashboard 26 of the vehicle 1.

[0020] The vehicle 1 is provided with doors 17 that can open and close the vehicle 1. For example, the doors 17 include a pair of front doors 17f, a pair of rear doors 17r, and a back door 17b. The front door 17f shown in FIG. 1 is an example of a first door in this embodiment. The rear door 17r is an example of a second door in this embodiment. Furthermore, the back door 17b is an example of a third door in this embodiment. Note that the vehicle 1 shown in FIG. 1 is provided with five doors 17, but the number of doors 17 is not limited to this.

[0021] Next, a configuration of the vicinity of the driver's seat of the vehicle 1 according to this embodiment will be described. Fig. 2 is a diagram showing an example of the configuration of the vicinity of the driver's seat of the vehicle 1 according to this embodiment.

[0022] 2, the vehicle 1 has a driver's seat 21 and a passenger seat 22. In front of the driver's seat 21, a display device 20, a steering wheel 23, operation buttons 24, a windshield 25, and a dashboard 26 are provided.

[0023] The display device 20 is a display provided on the dashboard 26 of the vehicle 1. As an example, the display device 20 is located in the center of the dashboard 26 as shown in FIG. 2. The display device 20 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display device 20 may also serve as a touch panel. The steering wheel 23 is provided in front of the driver's seat 21 and can be operated by the driver.

[0024] The operation button 24 is a button that can accept an operation by a user. In this embodiment, the user is, for example, the driver of the vehicle 1. When the operation button 24 is pressed by the driver, the operation button 24 accepts, for example, an operation to start parking assistance from the driver. The position of the operation button 24 is not limited to the example shown in FIG. 2, and may be provided, for example, near the steering wheel 23 or a central shift lever.

[0025] The operation button 24 is an example of an operation unit in this embodiment. In addition, when the display device 20 also functions as a touch panel, the display device 20 may be an example of the operation unit. In addition, an operation terminal capable of transmitting a signal to the vehicle 1 from outside the vehicle 1, such as a remote controller or an electronic key (not shown), may be an example of the operation unit.

[0026] Further, the windshield 25 becomes a head-up display capable of displaying an image by projecting an image by a projection device (not shown). The display device 20 and the head-up display are an example of a display unit in this embodiment.

[0027] (Example of hardware configuration for parking assistance device) Fig. 3 is a diagram showing an example of a hardware configuration of the parking assistance device 100 according to this embodiment. As shown in Fig. 3, the parking assistance device 100 has a CPU (Central Processing Unit) 31, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, an I / F (Interface) 34, an auxiliary storage device 35, and the like, which are interconnected via a bus 36, and has a hardware configuration using a normal computer.

[0028] The CPU 31 is a calculation device that controls the entire ECU. The CPU 31 is an example of a processor in the parking assistance device 100 of the present embodiment, and another processor or processing circuit may be provided instead of the CPU 31. The ROM 32 stores programs and the like that realize various processes by the CPU 31.

[0029] The RAM 33 is, for example, a main storage device of the parking assistance device 100, and stores data necessary for various processes by the CPU 31. The I / F 34 is an interface for transmitting and receiving data. The I / F 34 may also transmit and receive information to and from other ECUs mounted on the vehicle 1 via a Controller Area Network (CAN) or the like within the vehicle 1. The auxiliary storage device 35 is, for example, configured with a Hard Disc Drive (HDD) or a Solid State Drive (SSD).

[0030] Next, functions of the vehicle 1 of this embodiment will be described. The vehicle 1 of this embodiment learns a driving route based on teacher driving, and performs parking assistance using the learning result. Such parking assistance is effective for reducing the driver's parking effort when repeatedly parking in a fixed parking position, such as the garage of the driver's home, a contracted parking position of an apartment building, or a specified parking position in a parking lot of a workplace, for example.

[0031] 4 is a diagram illustrating an example of the parking assistance according to the present embodiment. A travel route 50 shown in FIG.

[0032] The parking position 52 is, for example, located in a garage 53 at the home 44 of the driver of the vehicle 1, but is not limited thereto. In the example shown in Fig. 4, the travel start position 51 is on the roadway 40. The roadway 40 is assumed to be a public road. In addition, the roadway 40 includes a wall 41 arranged along the roadway 40, and an obstacle 42 and a utility pole 43 adjacent to the wall 41.

[0033] Home 44 faces wall 45. Furthermore, obstacles 46 and 47 are located in garage 53. Garage 53 is located between home 44 and residence 48. Residence 48 faces wall 49. Vehicle 1 moves from roadway 40 to garage 53, passing between walls 45 and 49 and avoiding obstacles 46 and 47.

[0034] The parking assistance device 100 of this embodiment learns a driving route 50 and performs parking assistance to park the vehicle 1 at a parking position 52 based on the learned driving route 50. The parking position 52 is the goal of the driving route 50 by the parking assistance, and is also called a target parking position. Note that the driving route 50 shown in FIG. 4 is an example, and the environment in which the parking assistance of this embodiment can be applied is not limited to this.

[0035] (Example of parking assistance system functions) Fig. 5 is a block diagram showing an example of functions of the parking assistance device 100 according to this embodiment. As shown in Fig. 5, the parking assistance device 100 according to this embodiment includes an acquisition unit 101, an extraction unit 102, a learning unit 103, an estimation unit 104, a generation unit 105, a storage unit 106, a vehicle control unit 107, a setting unit 108, a detection unit 109, a determination unit 110, and a notification unit 111.

[0036] The acquisition unit 101 acquires vehicle information and environmental information from the wave transmitting / receiving unit 15, the imaging device 16, various sensors, or other ECUs mounted on the vehicle 1.

[0037] The vehicle information is information related to the behavior of the vehicle 1. The vehicle information includes, for example, information related to the speed of the vehicle 1, the wheel speed pulse, the steering angle, the acceleration, the deceleration, and the shift information.

[0038] The environmental information is information about the environment around the vehicle 1. For example, the environmental information includes the presence or absence of an obstacle around the vehicle 1 and the distance between the vehicle 1 and the obstacle. More specifically, the acquisition unit 101 acquires the presence or absence of an obstacle detected by the wave transmitting and receiving unit 15 and the distance between the vehicle 1 and the obstacle around the vehicle 1 measured by the wave transmitting and receiving unit 15.

[0039] The environmental information may further include other information. For example, an image of the surroundings of the vehicle 1 may also be included in the environmental information. The acquisition unit 101 acquires an image of the surroundings of the vehicle 1 from the imaging device 16. Hereinafter, the image is referred to as a surrounding image. More specifically, the acquisition unit 101 acquires a plurality of surrounding images when the vehicle 1 learns the driving route 50 based on the teacher driving and when the vehicle 1 performs automatic driving based on the learned driving route 50. The acquisition unit 101 may detect an obstacle from the surrounding image and acquire the presence or absence of the obstacle, its position from the vehicle 1, and its distance.

[0040] The extraction unit 102 extracts feature points from the peripheral image. The method of extracting feature points by the extraction unit 102 is not particularly limited, and a known method may be applied. For example, the extraction unit 102 extracts feature points by a method such as FAST (Features from Accelerated Segment Test) or ORB (Oriented FAST and Rotated BRIEF).

[0041] The learning unit 103 learns a driving route 50 for parking the vehicle 1 in the parking position 52 based on the teacher driving by the driver. In addition, the learning unit 103 sets a coordinate system for a parking area including the driving start position 51, the parking position 52, and the driving route 50. When learning the driving route 50, the learning unit 103 creates map information for the parking area by utilizing a SLAM (Simultaneous Localization and Mapping) technique for feature points extracted from a plurality of vehicle surrounding images captured by the imaging device 16 during teacher driving, detects coordinates indicating the positions of the feature points, and stores the coordinates in the storage unit 106.

[0042] In addition, when learning the driving route 50, the learning unit 103 represents the driving route 50 as a collection of points that exist at approximately equal distance intervals, and stores each point in the memory unit 106 as route point cloud data that holds coordinates indicating a position, a direction indicating the direction of the vehicle 1 at that position, and the driving speed as parameters.

[0043] The teacher driving in this embodiment is a driving in which the driver manually drives the vehicle 1 to move from the driving start position 51 outside the parking position 52 to the parking position 52. In this embodiment, the learning unit 103 learns the driving start position 51 in the teacher driving as the start position of the driving route 50. Note that the teacher driving includes not only driving by the driver, but also driving in which the vehicle automatically drives without the driver's operation and moves the vehicle 1 from the driving start position 51 outside the parking position 52 to the parking position 52.

[0044] In addition, the driver starts the teacher driving from a state in which the vehicle 1 is stopped at a position different from the driving start position 51, for example, a desired position outside the garage 53, and drives the vehicle 1 in reverse to the parking position 52 in the garage 53. Note that the teacher driving is not limited to reverse, and may be forward.

[0045] During the teacher driving, the above-mentioned acquisition unit 101 acquires surrounding images and vehicle information. Also, during the teacher driving, the above-mentioned extraction unit 102 extracts feature points from the surrounding images acquired by the acquisition unit 101. The learning unit 103 generates and records map information and a driving route 50 using SLAM technology based on the extracted feature points and vehicle information.

[0046] Specifically, when the driver performs teacher driving in which the vehicle 1 is parked in parking position 52 by reversing, the driving route 50 also becomes a route in which the vehicle 1 is parked in parking position 52 by reversing. In addition, in this embodiment, the position where the teacher driving is started becomes the driving start position 51 of the driving route 50. The learning unit 103 stores the map information that defines the recorded driving route 50 in the memory unit 106. Note that the method of learning the driving route 50 and the manner of defining the driving route 50 are not limited to this example.

[0047] The estimation unit 104 estimates the position of the vehicle 1 based on the surrounding image and vehicle information acquired by the acquisition unit 101. More specifically, when the driver sets an operation to start parking assistance by using the setting unit 108 described later, the estimation unit 104 reads map information represented by a set of feature points from the storage unit 106. Then, the estimation unit 104 estimates the position of the vehicle 1 based on the map information, the feature points extracted from the surrounding image by the extraction unit 102, and the amount of movement estimated from the vehicle information.

[0048] The estimation unit 104 compares the feature points of the map information at the time of learning with the feature points of the surrounding image at the time when the operation to start parking assistance was accepted. The estimation unit 104 estimates the position and orientation of the vehicle 1 at the time when the operation to start parking assistance was accepted based on the difference between these feature points.

[0049] It should be noted that the estimation unit 104 does not need to specify the travel start position 51 and the current position of the vehicle 1 as absolute positions, but only needs to specify the relative positional relationship between the travel start position 51 and the current position of the vehicle 1. It should be noted that the method of estimating the position of the vehicle 1 and the travel start position 51 by the estimation unit 104 is not limited to this example.

[0050] The estimation unit 104 estimates the position of the vehicle 1 based on the characteristic points included in the map information, the characteristic points extracted from the surrounding image acquired during the automatic driving, and the amount of movement estimated from the vehicle information, even while the vehicle 1 is being automatically driven by the vehicle control unit 107 described later. The estimation unit 104 sends the estimated position of the vehicle 1 to the vehicle control unit 107.

[0051] The generating unit 105 generates opening / closing information for determining whether the door 17 of the vehicle 1 can be opened or closed for the parking position 52. The generating unit 105 also generates the opening / closing information when learning the teacher driving.

[0052] Here, the opening / closing information will be described. Fig. 6 is a table showing an example of the opening / closing information according to this embodiment. As shown in Fig. 6, the table shows opening / closing information for determining whether the door 17, excluding the driver's seat 21, can be opened or closed for each parking position 52.

[0053] The contents of the opening / closing information include opening / closing availability information indicating whether the door 17 can be opened or closed when the vehicle is parked in the parking position 52. It is assumed that the parking position A stored in the table in Fig. 6 is the parking position 52 in Fig. 4. It is also assumed that the vehicle 1 is facing the roadway 40 with its front facing the roadway 40.

[0054] For example, when vehicle 1 is parked in parking position 52, acquisition unit 101 acquires information on home 44, house 48, and obstacles 46 and 47 located in garage 53, which are detected by wave transmitting / receiving unit 15. Based on the obstacle information acquired by acquisition unit 101, generation unit 105 determines that there is an obstacle 47 outside passenger seat 22 and that door 17 cannot be opened or closed, and generates "not possible" as information on door 17 corresponding to parking position A.

[0055] Similarly to the above, the generating unit 105 determines that there is no obstacle outside the rear right seat and that the door 17 can be opened and closed, and generates "OK", determines that there is an obstacle 46 outside the rear left seat and that the door 17 cannot be opened and closed, and generates "NOT OK", and determines that there is no obstacle outside the back door 17b and that the door 17 can be opened and closed, and generates "OK". Note that the detection of the distance and positional relationship to the surrounding obstacles at the parking position 52, which are necessary for determining whether the door is opened or closed, may be performed not only when the vehicle is stopped at the parking position 52, but also when the vehicle is parked and driven to the parking position 52. Also, the surrounding obstacles may be detected from the image captured by the imaging device 16, and the distance to the surrounding obstacles may be measured.

[0056] In addition, the driver may stop the vehicle 1 at the parking position 52 and fine-tune the opening / closing information by using the whole surrounding image of the vehicle displayed on the display device 20 and the operation buttons 24, etc. for the parking target position. The generating unit 105 generates the opening / closing information taking into account the moving distance due to the fine adjustment.

[0057] The open / close information generated by the generating unit 105 is stored in the storage unit 106. Note that the storage medium for storing the open / close information is not limited to this. Note that the open / close information may include open / close information of the door 17 corresponding to the driver's seat 21. Note that the information in the generated table is not limited to this.

[0058] The opening / closing information also includes distance information indicating the distance between the door 17 of the vehicle 1 when the vehicle 1 is parked in the parking position 52 and an obstacle present around the parking position 52. FIG. 7 is a table showing an example of the opening / closing information according to this embodiment. The table shown in FIG. 7 shows distance information between the door 17 excluding the parking position 52 and the driver's seat 21, and an obstacle. As in FIG. 6, it is assumed that the parking position A stored in the table is the parking position 52 in FIG. 4. It is also assumed that the direction of the vehicle 1 is such that the front of the vehicle 1 faces the roadway 40.

[0059] For example, when vehicle 1 is parked in parking position 52, acquisition unit 101 acquires distance information between home 44, house 48, and obstacles 46 and 47 located in garage 53, which are detected by wave transmitting / receiving unit 15. Based on the distance information acquired by acquisition unit 101, when vehicle 1 is parked in parking position A, generation unit 105 generates distance information between door 17 corresponding to parking position A and the obstacle, such that obstacle 47 is outside passenger seat 22 and the opening / closing distance of door 17 is "20 cm."

[0060] Similarly to the above, the generating unit 105 generates that the opening and closing distance of the door 17 for the rear right seat is "100 cm", that the rear left seat has an obstacle 46 and the opening and closing distance of the door 17 is "20 cm", and that the back door 17b has an opening and closing distance of "90 cm". The distance information generated by the generating unit 105 is stored in the storage unit 106. Note that the distance information of the door 17 corresponding to the driver's seat 21 may be included. Note that the information in the table to be generated is not limited to this.

[0061] Returning to Fig. 5, the storage unit 106 is configured by, for example, the ROM 32, the RAM 33, or the auxiliary storage device 35. Although Fig. 5 illustrates one storage unit 106 being included in the parking assistance device 100, a plurality of storage media may function as the storage unit 106. The storage unit 106 stores programs and data used in various processes executed by the parking assistance device 100.

[0062] For example, the program executed by the parking assistance device 100 of this embodiment has a modular configuration including each of the above-mentioned units (acquisition unit 101, extraction unit 102, learning unit 103, estimation unit 104, generation unit 105, memory unit 106, vehicle control unit 107, setting unit 108, detection unit 109, judgment unit 110, and notification unit 111).

[0063] In terms of actual hardware, the CPU 31 reads and executes a program from the memory unit 106, thereby loading the above-mentioned units onto the RAM 33, and the acquisition unit 101, extraction unit 102, learning unit 103, estimation unit 104, generation unit 105, memory unit 106, vehicle control unit 107, setting unit 108, detection unit 109, judgment unit 110 and notification unit 111 are generated on the RAM 33.

[0064] The vehicle control unit 107 automatically drives the vehicle 1 to the parking position 52 based on the learning result by the learning unit 103. The vehicle control unit 107 also moves the vehicle 1 to the parking position 52 by automatic driving based on the driving route 50 and the vehicle position estimation by the estimation unit 104. The vehicle control unit 107 controls the steering, braking, and acceleration / deceleration of the vehicle 1 to automatically drive the vehicle 1.

[0065] In this embodiment, the vehicle control unit 107 automatically drives the vehicle 1 along the learned driving route 50. Note that in this embodiment, the driver may be seated in the driver's seat 21 of the vehicle 1 or may be alight from the vehicle 1 while the vehicle control unit 107 is automatically driving.

[0066] Furthermore, when the vehicle 1 deviates from the travel route 50, the vehicle control unit 107 uses feedback control to move the vehicle 1 back to the travel route 50. For example, the vehicle control unit 107 estimates a difference between the position of the vehicle 1 and the travel route 50 based on the position of the vehicle 1 estimated by the estimation unit 104, and drives the vehicle 1 so as to reduce the difference.

[0067] In addition, while FIG. 5 illustrates the parking assistance device 100 as including the vehicle control unit 107, the vehicle control unit 107 may be realized by another ECU outside the parking assistance device 100.

[0068] The setting unit 108 sets various operations from the driver. For example, when the operation button 24 is pressed, the setting unit 108 sets the operation to start parking assistance. When the setting unit 108 sets the operation to start operation assistance, it sets a target parking position 52 for the vehicle 1. Furthermore, when the display device 20 is a touch panel, the setting unit 108 may accept the operation to start parking assistance when an image button on the touch panel is pressed. Furthermore, when there are multiple pieces of setting information related to parking assistance on the touch panel, the setting unit 108 may accept an operation to select which setting information to use from the driver.

[0069] Furthermore, the setting unit 108 sets opening / closing information corresponding to the target parking position 52. More specifically, the setting unit 108 acquires the opening / closing information generated by the generation unit 105 from the storage unit 106, and sets the opening / closing information corresponding to the target parking position 52.

[0070] The detection unit 109 detects a passenger riding in the vehicle 1. For example, the detection unit 109 is detected by a sensor including at least one of a seating sensor mounted on a seat in which the passenger rides, a seat belt sensor mounted on a seat belt worn by the passenger, and an image sensor mounted in the vehicle 1 and capturing images of the inside and outside of the vehicle. The image sensor is, for example, a camera built into the vehicle 1.

[0071] 5 illustrates the parking assistance device 100 as including the detection unit 109, the detection unit 109 may be realized by another ECU outside the parking assistance device 100. In that case, the parking assistance device 100 acquires the result of detecting a passenger riding in the vehicle 1.

[0072] When automatically parking vehicle 1 at a target parking position, the judgment unit 110 judges whether a passenger is present in vehicle 1 and whether the door 17 of vehicle 1 can be opened or closed based on the opening / closing information corresponding to the target parking position set by the setting unit 108.

[0073] More specifically, the determination unit 110 determines in which seat a passenger is detected by the detection unit 109. The determination unit 110 also refers to the opening / closing information set by the setting unit 108. Furthermore, when there is a passenger detected by the detection unit 109, the determination unit 110 determines whether or not the door 17 of the vehicle 1 can be opened or closed based on the opening / closing information specified by the setting unit 108. When the passenger's seat does not allow the door 17 to be opened or closed, the determination unit 110 sends a notification to the notification unit 111 that the door 17 cannot be opened or closed. Note that the automatic parking in this embodiment includes parking control in which some of the accelerator, brake, shift, and steering operations are performed automatically, or parking control in which all of the operations are performed automatically.

[0074] The notification unit 111 notifies passengers who need to get out of the vehicle before the parking operation based on the determination of the determination unit 110. For example, when the vehicle 1 sets parking position A shown in Fig. 6 as the target parking position, the notification unit 111 notifies passengers who are present in the passenger seat 22 and the rear left seat.

[0075] Here, the contents of the notification made to the passengers by the notification unit 111 will be described. Fig. 8 is a diagram showing an example of a notification according to this embodiment. As shown in Fig. 8, when the vehicle 1 moves to the travel start position 51, the notification unit 111 causes the display device 20 to display a message M1 such as "Occupants in the passenger seat and the rear left seat, please get out of the vehicle before parking."

[0076] The notification unit 111 may also notify the passengers of the vehicle 1 that the door 17 is not open or closed by displaying a symbol or a figure or the like encouraging the passengers to get off on the head-up display. By the notification unit 111 notifying the passengers of the guidance encouraging the passengers to get off, the passengers can get off the vehicle 1 before the vehicle 1 is moved to the parking position 52.

[0077] The method of notification by the notification unit 111 is not limited to display, and the notification unit 111 may output a message by voice from a speaker of the vehicle 1. In the present embodiment, the determination unit 110 and the notification unit 111 are described separately, but these functions may be provided in a single functional unit.

[0078] Next, a flow of the parking assistance process executed by the parking assistance device 100 of this embodiment configured as above will be described.

[0079] (Example of learning process for parking assistance device) 9 is a flowchart showing an example of the flow of the learning process of the driving route 50 executed by the parking assistance device 100 according to this embodiment. The process of this flowchart is started, for example, when the driver stops the vehicle at the parking start position and starts the registration of the map information.

[0080] First, the acquisition unit 101 acquires vehicle information and environmental information from the wave transmitting / receiving unit 15, the imaging device 16, various sensors, or other ECUs mounted on the vehicle 1 (step S1).

[0081] Next, the acquisition unit 101 acquires an image of the surroundings of the vehicle 1 captured by the imaging device 16 (step S2).

[0082] Then, the extraction unit 102 extracts feature points from the peripheral image acquired by the acquisition unit 101 (step S3).

[0083] The learning unit 103 sets a coordinate system for the parking area, detects coordinates indicating the positions of each feature point extracted during teacher driving, creates map information for the parking area, and stores the map information in the storage unit 106 (step S4). The learning unit 103 also learns a driving route 50 for parking the vehicle 1 in the parking position 52 based on the driving trajectory that accumulates the position coordinates of the vehicle 1 during teacher driving (step S5). The estimation unit 104 may be used to detect the position coordinates of the vehicle 1.

[0084] The setting unit 108 also determines whether or not an operation to end teacher driving has been set from the steering button 24 or the like (step S6). The vehicle is parked at the parking position 52, and the processes from step S1 to step S5 are repeated until an operation to end teacher driving is set (step S6: No).

[0085] When an operation to end the teacher driving is accepted (step S6: Yes), the generating unit 105 generates open / close information from the detection result of the distance between each door and the obstacle acquired by the acquiring unit 101 (step S7). The generating unit 105 stores the generated open / close information in the storage unit 106, and the process ends.

[0086] (Example of parking assistance processing by a parking assistance device) FIG. 10 is a flowchart showing an example of the flow of parking assistance processing executed by the parking assistance device 100 according to this embodiment. In FIG. 10, processing up to the start of automatic driving or driving assistance will be described. The processing of this flowchart starts, for example, when the user stops the vehicle near the driving start position 51 for parking and the parking assistance start operation is set by the setting unit 108. Note that it is assumed that the driving route 50 and map information based on the teacher driving are learned by the learning unit 103 before the processing of this flowchart is executed.

[0087] First, the acquisition unit 101 acquires vehicle information and environmental information from the wave transmitting / receiving unit 15, the imaging device 16, various sensors, or other ECUs mounted on the vehicle 1 (step S21).

[0088] Next, the acquisition unit 101 acquires an image of the surroundings of the vehicle 1 captured by the imaging device 16 (step S22).

[0089] Then, the extraction unit 102 extracts feature points from the peripheral image acquired by the acquisition unit 101 (step S23).

[0090] The estimation unit 104 compares the feature points extracted by the extraction unit 102 in step S23 with the feature points that are the map information at the time of learning, performs a matching process, and uses the result together with the vehicle information to estimate the coordinate position of the vehicle 1 in the map information (step S24).

[0091] Then, the vehicle control unit 107 determines whether the vehicle 1 has stopped at the travel start position 51 based on the difference between the coordinate position of the vehicle 1 estimated by the estimation unit 104 in step S24 and the coordinate position of the travel start position 51 of the learned travel route 50 (step S25). If the vehicle 1 has not reached the travel start position 51 (step S25: No), the vehicle control unit 107 waits until the vehicle 1 reaches the travel start position 51 and stops.

[0092] Then, when the vehicle 1 reaches the travel start position 51 and stops there due to driving by the user (step S25: Yes), the process proceeds to step S26.

[0093] The setting unit 108 sets the opening / closing information corresponding to the target parking position 52 (step S26).

[0094] Next, determination unit 110 determines whether or not there is a fellow passenger detected by detection unit 109 (step S27). If no fellow passenger is detected (step S27: No), the process proceeds to step S30.

[0095] Furthermore, the determination unit 110 reads out the opening / closing information set by the setting unit 108. Furthermore, when there is a fellow passenger detected by the detection unit 109, the determination unit 110 determines whether or not the door 17 of the vehicle 1 can be opened or closed when the fellow passenger gets on or off, based on the opening / closing information set by the setting unit 108 (step S28).

[0096] If the door 17 can be opened and closed when passengers get in and out of the vehicle 1 at the target parking position (step S28: Yes), proceed to step S30. If the door 17 cannot be opened and closed when passengers get in and out of the vehicle 1 at the target parking position (step S28: No), a signal is sent to the notification unit 112 that the door 17 cannot be opened or closed.

[0097] The notification unit 112 notifies the passenger of the exit guidance (step S29).

[0098] Next, the vehicle control unit 107 starts automatic driving (step S30). The vehicle control unit 107 controls the vehicle 1 based on the map information and the driving route 50 stored in the storage unit 106. When the vehicle control unit 107 moves the vehicle 1 to the target parking position, this process ends.

[0099] As described above, the parking assistance device 100 of this embodiment stores opening / closing information for determining whether or not to open / close the vehicle door 17 at the parking position 52 of the vehicle 1 during teacher driving. In addition, when automatically parking the vehicle 1 at the parking position 52, a notification is given based on the opening / closing information.

[0100] As a result, when the vehicle 1 moves to the target parking position and the door 17 cannot be opened or closed, a guide is given to get out of the vehicle, so that the passengers can know in advance that the door 17 cannot be opened or closed and can get out of the vehicle before it moves to the target parking position.

[0101] (Variation 1) In each of the above-described embodiments, an example has been described in which the notification unit 112 notifies the driver of disembarking guidance when there is a passenger in the vehicle 1 and the passenger is unable to disembark when the vehicle 1 is moved to the parking position 52, but the content of the notification is not limited to content related to the passenger disembarking. For example, when the back door 17b cannot be opened or closed when the vehicle 1 is parked in the target parking position, the notification unit 112 notifies the driver that the back door 17b cannot be opened or closed.

[0102] In order to determine whether the back door 17b can be opened or closed, for example, the determination unit 110 determines whether the door 17 of the vehicle 1 cannot be opened or closed based on the opening / closing information set by the setting unit 108 when the vehicle 1 is automatically parked at the target parking position, and sends a message to the notification unit 112 that the back door 17b cannot be opened or closed. This allows the driver to take out luggage from the luggage compartment before moving the vehicle 1 to the target parking position.

[0103] (Variation 2) Furthermore, for example, the generating unit 105 may generate information on whether the door 17 can be opened or closed by using the distance information. For example, the distance at which the door 17 can be opened or closed may be set to 50 cm, and if the distance between an obstacle and the door 17 exceeds 50 cm, it may be determined that the door 17 can be opened or closed. Furthermore, the distance at which the door 17 can be opened or closed may be set as a threshold value, and the threshold value may be made changeable. By setting the threshold value, the generating unit 105 can generate information on whether the door 17 can be opened or closed according to the user who gets on the vehicle 1, for example, the physique of the user.

[0104] The program executed by the parking assistance device 100 of this embodiment is provided as a file in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0105] The program executed by the parking assistance device 100 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. The program executed by the parking assistance device 100 of this embodiment may be provided or distributed via a network such as the Internet. The program executed by the parking assistance device 100 of this embodiment may be provided by being pre-installed in the ROM 32 or the like.

[0106] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0107] 1 vehicle 20 Display device 50 Travel route 51 Starting position 52 Parking Location 53 Garage 100 Parking assistance device 101 Acquisition Department 102 Extraction part 103 Learning Department 104 Estimation part 105 Generation part 106 Storage section 107 Vehicle control unit 108 Setting section 109 Detection unit 110 Judgment section 111 Notification Department

Claims

1. A parking assistance device that automatically drives a vehicle based on a teacher driving method, a storage unit that stores opening / closing information for determining whether or not the doors of the vehicle can be opened / closed with respect to the parking position of the vehicle during the teacher travel; a notification unit that performs a notification based on the opening / closing information when the vehicle is automatically parked at the parking position; Equipped with The notification unit issues a notification before the automatic parking based on the teacher driving starts when a door of the vehicle cannot be opened or closed during the teacher driving. Parking assistance device.

2. The notification unit issues a notification when it is determined that the door of the vehicle cannot be opened or closed based on the opening / closing information.

2. The parking assistance device according to claim 1.

3. The notification unit receives a detection result of detecting a fellow passenger riding in the vehicle, and issues a notification when a door through which the fellow passenger gets on or off cannot be opened or closed based on the opening / closing information corresponding to the door through which the fellow passenger gets on or off.

3. A parking assistance device according to claim 1 or 2.

4. a detection unit for detecting a passenger riding in the vehicle, When the passenger is detected by the detection unit, the notification unit notifies the passenger when the passenger is unable to exit the vehicle based on the opening / closing information corresponding to the door through which the passenger gets on and off. A parking assistance device according to any one of claims 1 to 3.

5. The opening / closing information includes opening / closing availability information indicating whether the door can be opened or closed when the vehicle is parked at the parking position. A parking assistance device according to any one of claims 1 to 4.

6. The opening / closing information includes distance information indicating a distance between the door and an obstacle when the vehicle is parked at the parking position. A parking assistance device according to any one of claims 1 to 5.

7. The storage unit stores the opening / closing information associated with each of the one or more parking positions, A parking assistance device according to any one of claims 1 to 6.

8. A parking assistance method executed in a parking assistance device that performs automatic driving of a vehicle based on teacher driving, comprising: a storage step of storing opening / closing information for determining whether or not the doors of the vehicle can be opened / closed with respect to a parking position of the vehicle during the teacher running; a notification step of performing a notification based on the opening / closing information when the vehicle is automatically parked at the parking position, The notification step includes, when a door of the vehicle cannot be opened or closed during the teacher driving, performing a notification before the automatic parking based on the teacher driving is started. Parking assistance methods.

Images