Parking support device and parking support method

Abstract

To improve convenience of a parking support device.SOLUTION: A parking support device includes storage means, detection means, travel control means, notification means, and support control means. The storage means stores a parking route to a predetermined parking position. The detection means performs detection at least around a vehicle to obtain detection information. The travel control means controls traveling in automatic parking. The notification means makes a notification to an occupant of the vehicle. The support control means suspends the traveling in the automatic parking on the basis of at least one of the detection information and a motion of the occupant. In at least one of a case where the detection means detects an obstacle which interferes during the automatic parking and a case where the occupant stops the vehicle, the support control means suspends the traveling in the automatic parking, and the notification means makes a notification about suspension of the traveling in the automatic parking to the occupant.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a parking assistance device and a parking assistance method. [Background technology]

[0002] Conventionally, there is known a technology that automatically applies the brakes and halts automatic parking when an obstacle is detected close to the vehicle during automatic parking. In such a case, with the conventional technology, even if the vehicle occupant gets out of the vehicle, removes the obstacle, and then returns to the vehicle, automatic parking cannot be resumed, which is inconvenient. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-160476 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide a parking assistance device with improved convenience. [Means for solving the problem]

[0005] In order to solve the above problems, a parking assistance device according to the present disclosure includes a storage means, a detection means, a driving control means, a notification means, and an assistance control means. The storage means stores a parking route to a predetermined parking position. The detection means detects at least the surroundings of the vehicle and obtains detection information. The driving control means controls the driving for automatic parking. The notification means notifies the vehicle occupant. The assistance control means suspends the driving for automatic parking based on at least one of the detection information and the behavior of the occupant. When the detection means detects an obstacle that will get in the way during automatic parking or when the occupant stops the vehicle, the assistance control means suspends the driving for automatic parking, and the notification means notifies the occupant about the suspension of the driving for automatic parking. [Effects of the Invention]

[0006] According to the present disclosure, the convenience of a parking assistance device can be improved. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of a vehicle to which a parking assist ECU according to an embodiment can be applied. [Figure 2] FIG. 2 is a block diagram showing an example of a hardware configuration of the parking assistance ECU according to the embodiment. [Figure 3] FIG. 3 is a block diagram showing an example of the configuration of the parking assistance system according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of processing in which the spatial recognition unit according to the embodiment identifies the positions of feature points. [Figure 5] FIG. 5 is a diagram illustrating an example of a vehicle disembarking obstruction range according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a vehicle disembarking obstruction range according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a parking interference range according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a vehicle disembarking obstruction range and a parking obstruction range according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of assistance in putting things away in a position that does not get in the way according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of assistance in putting things away in a position that does not get in the way according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of notification of a non-obstructive range according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of notification of a non-obstructive range according to the embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of notification of whether or not there is a disturbance according to the embodiment. [Figure 14]FIG. 14 is a diagram illustrating an example of notification of whether or not there is a disturbance according to the embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of assistance in stopping the vehicle at a position suitable for tidying up according to the embodiment. [Figure 16] FIG. 16 is a flowchart illustrating an example of processing executed by the parking assistance ECU according to the embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of automatic parking performed by the parking assistance ECU according to the embodiment. [Figure 18] FIG. 18 is a diagram illustrating an example of automatic parking performed by the parking assistance ECU according to the embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of automatic parking performed by the parking assistance ECU according to the embodiment. [Figure 20] FIG. 20 is a diagram illustrating an example of automatic parking performed by the parking assistance ECU according to the embodiment. [Figure 21] FIG. 21 is a diagram illustrating an example of automatic parking by the parking assistance ECU according to the embodiment. [Figure 22] FIG. 22 is a diagram illustrating an example of automatic parking by the parking assistance ECU according to the embodiment. [Figure 23] FIG. 23 is a diagram illustrating an example of automatic parking by the parking assistance ECU according to the embodiment. [Figure 24] FIG. 24 is a diagram illustrating an example of automatic parking by the parking assistance ECU according to the embodiment. [Figure 25] FIG. 25 is a diagram illustrating an example of automatic parking by the parking assistance ECU according to the embodiment. [Figure 26] FIG. 26 is a diagram illustrating an example of automatic parking by the parking assistance ECU according to the embodiment. [Figure 27] FIG. 27 is a diagram illustrating an example of automatic parking by the parking assistance ECU according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. Therefore, the components, the arrangement and connection of each component, and each step and the order of each step shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0009] Furthermore, each drawing is a schematic diagram and is not necessarily an exact illustration. In each drawing, substantially the same components are denoted by the same reference numerals, and redundant explanations will be omitted or simplified.

[0010] The parking assistance device according to the present embodiment, which will be described below, suspends the automatic parking operation when it detects an obstacle that obstructs automatic parking, and the notification means issues a notification regarding the suspension of the automatic parking operation. This notification assists in removing the obstructing obstacle, thereby enabling the suspension of the automatic parking operation to be released early. Furthermore, the method of the present application determines when the vehicle has stopped during automatic parking, and suppresses assistance if the occupant has not exited the vehicle and is not removing the obstacle, thereby avoiding unnecessary inconvenience to the occupant.

[0011] Furthermore, although there is a conventional technology for interrupting automatic parking after starting the automatic parking operation, the application of the parking assistance device according to this embodiment is not limited to the case where automatic parking is interrupted after starting the automatic parking operation. In other words, the parking assistance device according to this embodiment can also be applied to cases where automatic parking is not started.

[0012] For example, after a driver stops in front of an available parking space and activates the automatic parking function of the parking assistance system, the driver may notice an obstacle in the way and begin to move to get out of the vehicle. In such a case, the parking assistance system according to this embodiment starts a notification that is useful for removing the obstacle in the way. In other words, the parking assistance system according to this embodiment functions even if the driver has not started driving for automatic parking. The same applies when the start of driving for automatic parking is postponed because an obstacle in the way is detected.

[0013] Furthermore, the parking assistance system according to this embodiment, like the conventional technology, also functions when the automatic brakes are activated after the start of automatic parking and the automatic parking is suspended. For example, a three-dimensional object with thin components, such as an aluminum shopping cart, may not be detected as an obstacle or its position may be misidentified at the start position of automatic parking. Therefore, the automatic brakes may be activated after the start of automatic parking, or the occupant may notice the obstacle first and apply the brakes. The parking assistance system according to this embodiment functions in the same way in both cases. In other words, it functions regardless of the context or causal relationship between the detection of an obstructing obstacle and the suspension of automatic parking.

[0014] In this specification, not starting driving for automatic parking and suspending driving for automatic parking are called "suspending driving for automatic parking" and no distinction is made between the two. On the other hand, in this specification, a distinction is made between automatic parking and driving for automatic parking as necessary.

[0015] This is because automatic parking is a process in which the parking assistance device sets a parking route and drives the vehicle along the parking route, so automatic parking begins before the automatic parking drive begins. In other words, although the automatic parking drive appears to be automatic parking itself from the outside, the two are different. However, because the automatic parking drive appears to be automatic parking itself to the occupants, for the sake of convenience, the automatic parking drive is sometimes simply referred to as automatic parking.

[0016] As an embodiment of the present application, an example of parking using learning-type automatic parking and an example of parking in a parking space detected by parking space detection are shown, but the scope of application of the present application is not limited to a specific parking method. For example, it can also be applied to a case where an occupant visually finds an obstacle when a parking space is detected based on sonar detection.

[0017] This embodiment is particularly effective when parking using learning-based automatic parking. Learning-based automatic parking is a method suitable for repeatedly parking in the same location, such as in a home garage. Learning-based automatic parking involves manually parking the vehicle in a garage in advance, memorizing the parking route and parking location, and then automatically driving along the memorized route when parking automatically. Therefore, since automatic driving can be started from a location far away from the garage, there may be cases where the vehicle encounters obstacles that get in the way while driving.

[0018] Furthermore, even in a system in which the driver searches for an empty parking space, parks in front of the space, and then activates the parking assistance device, if the device is equipped with the functions of the present invention, the occupant can receive useful assistance when removing an obstacle from the empty parking space simply by activating the parking assistance device.

[0019] The parking assistance system of this embodiment is characterized by its ability to deal with obstacles that get in the way, so the automatic parking method is not specified. In this embodiment, the explanation will be given on the assumption that the parking assistance system is activated and a parking route is set. The route calculation method before that and the automatic driving method thereafter can be selected and implemented as desired, so detailed explanations will be omitted.

[0020] (Vehicle configuration) First, a vehicle to which the parking assistance device of the embodiment can be applied will be described. FIG. 1 is a diagram showing a vehicle 1 to which the parking assistance device of the embodiment can be applied. As shown in FIG. 1, the vehicle 1 is equipped with a parking assistance system 1S. The parking assistance system 1S includes an operation device 10, an HMI (Human Machine Interface) device 20, a vehicle control device 30, a navigation device 40, a sonar ECU 50, and a parking assistance ECU 100. The parking assistance ECU 100 is an example of a parking assistance device.

[0021] Note that other devices may also be installed in the vehicle 1. Also, in Fig. 1, the operation device 10, the HMI device 20, the vehicle control device 30, the navigation device 40, the sonar ECU 50, and the parking assistance ECU 100 are illustrated as separate devices, but some or all of these devices may be integrated.

[0022] The operation device 10, the HMI device 20, the vehicle control device 30, the navigation device 40, the sonar ECU 50, and the parking assistance ECU 100 will be described later.

[0023] Cameras 2a, 2b, 2c, and 2d are installed at four locations on the front, rear, left, and right sides of the body of vehicle 1. Hereinafter, when no distinction is made between cameras 2a, 2b, 2c, and 2d, they will be simply referred to as cameras 2. Each camera 2 is equipped with a fisheye lens and has a horizontal field of view of 180 degrees or more.

[0024] Each camera 2 is mounted at a depression angle to capture the road surface, so when the range of the road surface captured is converted into a horizontal field of view, a range of about 240 degrees (see dashed lines) of the road surface is captured by one camera 2. For example, the front and rear wheels and the sides of the vehicle 1 are captured in the images captured by side cameras 2a and 2b installed on the left and right sides of the vehicle 1.

[0025] The locations and number of cameras 2 are not limited to the example shown in FIG.

[0026] Furthermore, as shown in Fig. 1, twelve sonar sensors 3a to 3l are installed on the vehicle 1. Hereinafter, when the sonar sensors 3a to 3l are not individually distinguished, they will be simply referred to as sonar sensors 3. When the sonar sensors are individually referred to, they will be named according to their installation positions. For example, on the left side of the vehicle 1, sonar sensor 3a is installed on the front side (FLS: Front Left Side) of the vehicle 1, and sonar sensor 3b is installed on the rear side (BLS: Back Left Side) of the vehicle 1.

[0027] Furthermore, on the right side of the vehicle 1, a sonar sensor 3c is installed on the front side (FRS: Front Right Side) of the vehicle 1, and a sonar sensor 3d is installed on the rear side (BRS: Back Right Side) of the vehicle 1. These four sonar sensors are also called side sonars because they detect obstacles on the sides of the vehicle.

[0028] In addition, in front of the vehicle 1, sonar sensor 3e (FLC: Front Left Corner), sonar sensor 3f (FL: Front Left), sonar sensor 3g (FR: Front Right), and sonar sensor 3h (FRC: Front Right Corner) are installed in that order from the left side in the forward direction of the vehicle 1.

[0029] Sonar sensors 3f and 3g, which are provided on the inside, detect obstacles in the direction of travel when the vehicle 1 is traveling straight. Also, sonar sensors 3e and 13h, which are provided on the outside, detect obstacles in the direction of the turn when the vehicle 1 is turning. Sonar sensors 3e and 3h are also called corner sonars. While the detection ranges of the four sonar sensors 3e, 3f, 3g, and 3h are shown as triangles in FIG. 1, the detection ranges are not limited to the triangular range and are capable of detection up to approximately 10 meters from the vehicle. Also, the detection ranges of adjacent sonar sensors are installed so as to overlap with each other.

[0030] In addition, at the rear of vehicle 1, sonar sensor 3i (BLC: Back Left Corner), sonar sensor 3j (BL: Back Left), sonar sensor 3k (BR: Back Right), and sonar sensor 3l (BRC: Back Right Corner) are installed in that order from the left side in the forward direction of vehicle 1.

[0031] The sonar sensors 3j and 3k provided on the inside detect obstacles in the direction of travel when the vehicle 1 is backing up. The sonar sensors 3i and 3l provided on the outside detect obstacles in the direction of the turn when the vehicle 1 is backing up and turning. The sonar sensors 3i and 3l are also called corner sonars. In FIG. 1, the detection directions of the four sonar sensors 3i, 3j, 3k, and 3l and their fan-shaped spread are illustrated as triangles, but their detection ranges are not limited to the inside of the triangles in the figure and extend beyond. For example, adjacent sonar sensors are installed so that their detection ranges overlap each other. The same applies to the four sonar sensors 3e, 3f, 3g, and 3h at the front of the vehicle.

[0032] The detection ranges of the sonar sensors 3a, 3b, 3c, and 3d installed on the sides of the vehicle 1 are set to be narrower than the detection ranges of the sonar sensors installed in front and behind the vehicle 1. This is to improve the position resolution when the parking assist ECU 100 detects parking spaces on the sides of the vehicle 1 by minimizing overlap of the side sonar detection ranges when the vehicle 1 moves.

[0033] Furthermore, each sonar sensor is installed at a height and depression angle that allows it to easily detect surrounding obstacles when parking the vehicle 1. The installation locations and number of sonar sensors 3a to 3l are not limited to the example shown in FIG.

[0034] In this embodiment, sonar refers to a sonar system made up of the above-mentioned sonar sensors 3a to 3l and the sonar ECU 50. The sonar ECU 50 is a control device that performs overall control of the sonar system.

[0035] The sonar sensor 3 emits directional sound waves and receives reflected waves. The sonar ECU 50 detects the distance to an obstacle based on the time between when the sonar sensor 3 emits the sound waves and when it receives the reflected waves. The sonar ECU 50 detects the surroundings of the vehicle using multiple sonar sensors and identifies the position of the obstacle based on the distance from the multiple sonar sensors 3 that detect it.

[0036] The sonar system detects obstacles in the traveling direction of the vehicle 1 using a sonar sensor 3 provided on the bumper of the vehicle 1. When the sonar ECU 50 detects an obstacle, it determines whether the vehicle 1 will collide with the obstacle. If the sonar ECU 50 determines that the vehicle 1 will collide with the obstacle within a predetermined time, it instructs the vehicle control device 30 to activate the automatic brakes.

[0037] This ensures that vehicle 1 will not collide with obstacles in front or behind it during automatic parking. In addition, by detecting with side sonar equipped on the left and right sides of vehicle 1, it is possible to predict when the side of the vehicle will approach an obstacle due to the difference in inner wheel rotation.

[0038] Note that sonar is not an essential component of the parking assistance system 1S. In other words, the vehicle 1 may be configured without sonar. In this case, for example, the parking assistance ECU 100 may detect the distance to an obstacle by processing camera images captured by the camera 2. For example, the parking assistance ECU 100 may calculate the time until a collision between the vehicle 1 and the obstacle by processing the camera images. As a result, if a collision occurs within a predetermined time, the parking assistance ECU 100 may instruct the vehicle control device 30 to activate the automatic brakes.

[0039] (Parking Assist ECU hardware configuration) Next, the hardware configuration of the parking assist ECU 100 will be described. The parking assist ECU 100 is an example of a parking assist device. FIG. 2 is a diagram showing an example of the hardware configuration of the parking assist ECU 100 according to the embodiment. The functions of the parking assist ECU 100, which will be described later, may be implemented in the hardware shown in FIG. 2.

[0040] The parking assist ECU 100 may be a computer including a CPU 101, a ROM 102, a RAM 103, an I / O (input / output interface) 104, an IMP (Image Processor) 105, and a communication I / F (Interface) 106, with each element connected via a bus.

[0041] The parking assistance ECU 100 may accommodate multiple elements on a single chip. Also, the parking assistance ECU 100 may have one element configured with multiple chips. The bus does not have to be a single bus, but multiple types of buses may be combined.

[0042] For example, the CPU 101, ROM 102, RAM 103, IMP 105, and communication I / F 106 may be housed on a single chip and connected via a parallel bus, while the I / O 104 may be made up of multiple chips and connected to the chip housing the CPU 101 via a serial bus.

[0043] The parking assistance ECU 100 performs automatic parking by obtaining information from other devices (e.g., navigation device 40) and giving instructions to other devices (e.g., vehicle control device 30) via the communication I / F and in-vehicle LAN.

[0044] The CPU 101 controls the entire parking assistance ECU 100. The functions of each part of the parking assistance ECU 100 may be implemented in the form of a program executed by the CPU 101. The ROM 102 and RAM 103 correspond to storage parts, with the ROM 102 corresponding to a non-volatile area. The RAM 103 is used for temporary storage as a work area for the CPU 101. For example, the RAM 103 temporarily stores camera images such as display images and detection images, as well as information on targets and feature points detected from the images.

[0045] The IMP 105 is a processor that is specialized for image processing and parallel processing and has improved processing performance. Some of the functions of the parking assistance ECU 100 (for example, the image processing unit 120, the space recognition unit 130, etc.) described later may be executed by the IMP 105.

[0046] (Parking Assist ECU function) Next, the functions of the parking assistance ECU 100 will be described. FIG. 3 is a block diagram showing an example of the functional block configuration of the parking assistance system 1S according to the embodiment. In FIG. 3, the functions of the parking assistance ECU 100 are divided into blocks: a state management unit 110, an image processing unit 120, a space recognition unit 130, a driving control unit 140, a storage unit 150, and an alarm unit 160. As shown in FIG. 3, the cameras 2 (2c(F), 2d(B), 2a(L), 2b(R)) output captured camera images. The image processing unit 120 of the parking assistance ECU 100 receives the camera images and performs operations such as generating a display image. The display image is output from the alarm unit 160 to the HMI device 20.

[0047] The notification unit 160 superimposes a message on a display image or outputs a voice message in response to an instruction from the state management unit 110. The notification unit 160 is an example of a notification means, and the HMI device 20 to which the message is output and the state management unit 110 that instructs the output of the message may also be included in the notification means. In other words, the notification means is a functional element that conveys information to the occupant. Since the state management unit 110, the notification unit 160, and the HMI device 20 are involved in the notification, these may be referred to as the HMI device 20, etc.

[0048] The state management unit 110 receives user operations via the operation device 10, and controls the functions of the parking assist ECU 100 in accordance with the user operations. Here, the touch panel of the navigation device 40 is included in the operation device 10.

[0049] The state management unit 110 receives position information from the main body (not shown) of the navigation device 40. The memory unit 150 stores a map including a route for automatic parking, and when the vehicle 1 stops near the start point of the route, the state management unit 110 compares the position information of the vehicle indicated by the navigation device 40 with the position information of the start point marked on the map, and starts automatic parking if they roughly match. Furthermore, during automatic parking, the state management unit 110 determines whether to suspend driving depending on the situation. In other words, the state management unit 110 is an example of an assistance control means that determines whether to start or suspend driving for automatic parking.

[0050] The image processing unit 120 generates a display image and a detection image. The detection image is, for example, an image in which changes in brightness and color, that is, contrast, are emphasized.

[0051] The space recognition unit 130 detects targets and feature points from the detection image and compares them with the positions of the targets and feature points stored in the map to estimate the self-location. The method for self-location estimation can be selected from the proposed methods, so detailed explanations will be omitted.

[0052] Furthermore, the space recognition unit 130 detects the position of an obstacle and a space without the obstacle from the detection image. FIG. 4 is a diagram illustrating an example of a process in which the space recognition unit 130 identifies the position of an obstacle. As shown in FIG. 4, the space recognition unit 130 identifies the position of a target (obstacle) P captured by the camera 2 of the vehicle 1 using motion parallax. Specifically, the space recognition unit 130 first converts the position of the target P on the camera image into the angle (azimuth angle) of the target P relative to the vehicle 1. FIG. 4 shows a case in which the camera 2 of the vehicle 1 moves from point A to point B on the Y axis as the vehicle 1 moves. In this case, the image of the target moves on the camera image, and the azimuth angle of the target P captured by the camera 2 changes from θ1 to θ2. This change in azimuth angle (θ1 → θ2) is called motion parallax.

[0053] The distance between points A and B (the length of line segment AB) is the amount of movement of vehicle 1, and can be determined from the number of wheel rotations. Using the principle of triangulation, the spatial recognition unit 130 can determine the XY coordinates (x, y) of target P from the coordinates of points A and B, the length of line segment AB, and θ1 and θ2. The vertical position of target P captured by camera 2 in the image corresponds to the height of target P. For example, if target P is the tip of a pole standing vertically on the ground, it will appear higher than the base of the pole in the image. Therefore, once the spatial recognition unit 130 determines the XY coordinates, it can determine the Z coordinate based on the vertical position in the image. In other words, the spatial recognition unit 130 can determine the three-dimensional coordinates of target P by applying the motion parallax that appears in camera images captured at different points in time to the principle of triangulation.

[0054] The space recognition unit 130 also analyzes the distribution of motion parallax in the detection image. In the range where the road surface is captured in the detection image, the motion parallax gradually decreases as the distance from the vehicle increases, but where a three-dimensional object is captured, the parallax changes discontinuously. Utilizing this, the space recognition unit 130 identifies a range where there is no discontinuity in parallax from the analysis results of the motion parallax distribution, and by matching this range to a range on the road surface, it is possible to detect a range where there are no three-dimensional objects (free space).

[0055] The space recognition unit 130 also receives information about obstacle detection by sonar, etc., and reinforces the recognition of the obstacle's position and the space. For example, since the error in estimating the position based on motion parallax increases if an obstacle is in the traveling direction, the space recognition unit 130 may supplement distance information obtained by sonar to reduce the error in position estimation.

[0056] When performing automatic parking, the driving control unit 140 controls the steering angle and vehicle speed to drive the vehicle so as to follow the route stored in the memory unit 150. Specifically, the driving control unit 140 controls the steering angle and vehicle speed in cooperation with the memory unit 150 and the space recognition unit 130. When performing automatic parking, the space recognition unit 130 estimates the vehicle position based on the position of a target P captured by the camera 2. The driving control unit 140 compares the vehicle position estimated by the space recognition unit 130 with the route stored in the memory unit 150 to calculate the deviation from the route.

[0057] When the position of the vehicle 1 deviates from the route, the driving control unit 140 controls the steering angle to change the steering angle in a direction in which the path of the vehicle 1 intersects with the route, and when the vehicle 1 overlaps with the route, changes the steering angle in a tangential direction of the route. In other words, the driving control unit 140 feedback-controls the steering angle so that the position estimated by the space recognition unit 130 follows the route stored in the memory unit 150.

[0058] Since the greater the vehicle speed and steering angle of vehicle 1, the more likely it is that the vehicle will deviate from the route, the driving control unit 140 may, for example, maintain the vehicle speed at 5 km / h during automatic parking, or may adjust the vehicle speed according to the steering angle. When the vehicle approaches the end point of the route, the driving control unit 140 reduces the vehicle speed of vehicle 1, and when the vehicle is stopped at the end point of the route, automatic parking of vehicle 1 is completed at that point. Vehicle control for automatic parking is not a feature of the present application, so further explanation will be omitted.

[0059] The parking assistance device is not limited to the configuration exemplified as the parking assistance ECU 100 described above, and may be configured arbitrarily. The configuration example of the parking assistance ECU 100 disclosed in FIG. 3 corresponds to learning-type automatic parking. In the learning type, the driving route during learning driving is stored in the storage unit 150 as the parking route, and therefore no route calculation unit is provided. In the case of a parking assistance device that performs parking space detection type parking assistance, which does not perform learning driving but detects a parking space from a camera image and calculates a parking route from the vehicle to the parking space, a parking space detection unit may be provided instead of the space recognition unit 130, and a route calculation unit that calculates the parking route may be further provided.

[0060] The feature of the present disclosure, assistance regarding obstructive obstacles, is mainly performed by the spatial recognition unit 130 detecting the obstacle, the state management unit 110 making a decision, and the notification unit 160 notifying the occupant. However, some of these functions may be performed by a separate device such as the navigation device 40. Furthermore, the separate device may be included in the parking assistance device. Furthermore, it is also optional to refer to the functional blocks by different names.

[0061] (Range of obstructions) The parking assist ECU 100 according to this embodiment suspends automatic parking travel and issues a notification regarding the suspension if an obstructive obstacle is detected. Obstructive obstacles are obstacles around the vehicle 1 that are within a range that will obstruct parking or that will obstruct occupants from getting out of the vehicle once it has been parked. Hereinafter, the range in which an obstacle will obstruct parking or occupants from getting out of the vehicle will be referred to as the obstruction range. In other words, an obstructive obstacle is an obstacle that is within the obstruction range.

[0062] When vehicle 1 is traveling along the parking path, the range in which the distance to vehicle 1 is below the margin threshold (also called the clearance threshold) is the range in which parking is hindered (if an obstacle is present). This range is called the parking obstruction range. The parking obstruction range is an example of the obstruction range. The margin threshold is, for example, 15 cm.

[0063] When vehicle 1 is in a parking position, the range where the distance from the door is equal to or less than the alighting room threshold is the range where (if an obstacle is present) alighting is prevented. This range is called the alighting obstruction range. The alighting obstruction range is an example of an obstruction range. The alighting room threshold is, for example, 30 cm.

[0064] Obstacles in the parking interference range and obstacles in the dismount interference range are obstructive obstacles. However, the state management unit 110 does not set a dismount interference range around a door where the occupant does not dismount. For example, when the occupant dismounts from the driver's seat and automatic parking is resumed remotely (i.e., when the smart key that remotely commands automatic parking is outside the vehicle), the state management unit 110 does not set a dismount interference range for the driver's seat door. In this case, even if an obstacle is located 20 cm away from the driver's seat door when the vehicle 1 is parked, the state management unit 110 may determine that there is no obstructive obstacle and start automatic parking. In other words, as long as automatic parking can be resumed, the state management unit 110 may assist in removing the obstacle that obstructs dismounting, or may have the occupant dismount in advance to prevent the obstacle from obstructing the vehicle.

[0065] (Parking and disembarking obstruction ranges) 5 to 8 are diagrams showing examples of alighting obstruction ranges and parking obstruction ranges, and FIGS. 5 and 6 show alighting obstruction ranges. In all of the diagrams, the vehicle 1 is assumed to be in a parking position. As shown in FIG. 5, when an occupant C is present in all seats, the state management unit 110 sets alighting obstruction range DR around the doors of all seats. Furthermore, the state management unit 110 does not set alighting obstruction range DR around the doors of seats where no occupant C is present, such as the rear seats in FIG. 6.

[0066] The vehicle 1 detects the state of seat belts and doors using known technology and shares the detected information with various devices via an in-vehicle LAN. The state management unit 110 of the parking assistance ECU 100 determines that an occupant C is present if the seat belt is fastened, and predicts that the occupant will exit the vehicle if the door is opened. For example, when the seat belt is unfastened and the door is not open, the notification unit 160, under the control of the state management unit 110, may blink the exit obstruction range DR corresponding to the seat where the seat belt is unfastened, to indicate that the exit obstruction range DR will change when the occupant exits the vehicle.

[0067] FIG. 7 shows a portion of the parking interference area PR when vehicle 1 backs up straight to a parking position. The parking interference area PR is the area where the distance to vehicle 1 becomes equal to or less than the margin threshold while vehicle 1 moves from the parking start position to the parking position, so the area shown in FIG. 7 is only a small portion of the parking interference area PR. FIG. 7 shows vehicle 1 when parked. The parking interference area PR is determined based on the distance from the body of vehicle 1, so the parking interference area PR is determined based on the width of the side mirror area in the area forward of the side mirror area, while the width of the vehicle body is used as the basis for the area behind the side mirror area. Furthermore, when backing up and parking with the side mirrors closed as shown in FIG. 8, the width of the parking interference area PR narrows in the area forward of the side mirror area.

[0068] When there is no occupant C in the vehicle, the obstructive range is only the parking obstruction range PR, as shown in Figure 7. When there is an occupant C in the vehicle, the obstructive range is the combination of the parking obstruction range PR and the disembarkation obstruction range DR, as shown in Figure 8. The range that is the disembarkation obstruction range DR but not the parking obstruction range PR, as shown in the region DR in Figure 8, may be called the disembarkation obstruction range in the narrow sense.

[0069] (Notification when passengers are seated) The state management unit 110 sets a dismounting obstruction range DR that obstructs dismounting of the occupant C if an obstacle is present, in response to the seating of the occupant C. Furthermore, when the state management unit 110 determines that the obstacle detected by the space recognition unit 130 is within the dismounting obstruction range DR, the notification unit 160 issues a notification regarding dismounting of the occupant C based on the determination of the state management unit 110.

[0070] For example, the state management unit 110 sets the dismounting obstruction range DR to a range that is within 30 cm of the passenger door when the vehicle 1 is parked, as a range that will interfere when the passenger C in the passenger seat dismounts. If the state management unit 110 determines that the detected obstacle is not in the parking obstruction range PR but is in the dismounting obstruction range DR, the notification unit 160 may issue a notification, for example, stating, "There is an obstacle in a position that will prevent dismounting from the passenger seat." In response to this, when the passenger C in the passenger seat dismounts, the state management unit 110 may cause the notification unit 160 to issue a notification, "There is no obstacle that will prevent parking. Automatic parking can be resumed."

[0071] Furthermore, the state management unit 110 may change the range in which the obstacle is required to be moved, depending on whether the occupant C has dismounted. For example, if the seatbelt buckle of the passenger seat is unbuckled while the notification unit 160 is displaying the parking obstruction range PR and the dismounting obstruction range DR, the notification unit 160 may blink the dismounting obstruction range corresponding to the passenger seat with a transmittance that alternates between 80% and 20%, so that the user can see that the dismounting obstruction range DR will change when the passenger in the passenger seat dismounts.

[0072] The notification unit 160 may also issue a notification in response to the movement of an obstacle. For example, when an obstacle is moved and placed within the dismounting prevention range DR corresponding to the driver's seat, the notification unit 160 may issue a notification that "an obstacle is located in a position that prevents dismounting from the driver's seat." In response to this, when the driver's seat belt buckle is released, the notification unit 160 may issue a notification that "automatic parking can be resumed from outside the vehicle using the smart key," thereby encouraging remote parking.

[0073] (Environmental Dependence of Room and Slack) Next, the clearance and margin (safety margin) will be explained. The status management unit 110 sets the clearance threshold to, for example, 15 cm and the exit clearance threshold to, for example, 30 cm, but the clearance and margin may be increased or decreased depending on the situation. Furthermore, the status management unit 110 may change the starting point of the clearance depending on the state of the vehicle 1.

[0074] Since the starting point of the clearance is the outermost part of the vehicle body of vehicle 1, the tip of the side mirror is the starting point of the clearance for the side of the vehicle body when traveling straight. If it is determined that the side mirrors are to be closed when reversing straight in the final section of automatic parking, the state management unit 110 may set the starting point of the clearance for the side of the vehicle body to the side of the vehicle body instead of the tip of the side mirror, or may set it to the maximum protruding position of the closed side mirror. Also, for example, the state management unit 110 may set the clearance threshold to 10 cm only for the final section where the side mirrors are closed and the vehicle travels straight and backs up. This makes it possible to prevent occupant C from being bothered by unnecessary assistance in a small garage.

[0075] Furthermore, if the road surface is icy, there is a risk that the wheels of the vehicle 1 may slip and deviate from the parking path. Therefore, for example, if the temperature is below 5°C and it is early morning, the state management unit 110 may set the margin threshold to 30 cm. In this way, if obstacles are kept further away than usual, the possibility of contacting an obstacle is reduced even if the wheels skid on an icy road surface.

[0076] Furthermore, when the road is steep or the wind is strong, the status management unit 110 may increase the exit room threshold. Slopes and strong winds can both be detected by an acceleration sensor. By doing so, if obstacles are kept away from the door, the possibility of the door coming into contact with an obstacle is reduced even if the door is wide open due to a slope or wind.

[0077] (Help put things away in a place that won't get in the way) When there is an obstructing obstacle, the notification unit 160, under the control of the state management unit 110, performs any one of the following: notification of the range (obstruction range) where the obstruction will be present; notification indicating the obstructing obstacle; notification indicating the position where the obstructing obstacle should be removed; notification indicating the distance to be traveled until the obstructing obstacle is no longer an obstruction; and notification that the obstacle is no longer an obstruction.

[0078] 9 and 10 are diagrams illustrating an example of assistance in putting the vehicle out of the way. When illustrating the parking path for vehicle 1, the midpoint of the rear axle of vehicle 1 is illustrated as a representative point representing the position of vehicle 1, and the path through which the representative point passes during the parking process is illustrated as the parking path. As shown in FIG. 9, vehicle 1 drives straight to turning start position TS, starts turning, turns at turning position TP, finishes turning at turning end position TE, and performs automatic parking by parking at parking position PS in garage G. If space recognition unit 130 detects an abandoned bicycle as obstacle O while vehicle 1 is driving straight toward turning start position TS, notification unit 160 issues a notification, for example, saying, "Please move the obstacle on the front right side 4 meters ahead."

[0079] Although not shown, the notification unit 160 may also display an overhead view of an area within 1.2 m on both sides of the parking path (a range corresponding to the parking interference area PR) and indicate the area where an obstacle O would be an obstacle (an interference area). As shown in FIG. 10, the notification unit 160 may issue a notification by illuminating the left headlight LF and right headlight RF of the vehicle 1 when the obstacle O leaves the interference area, or by issuing an audio notification saying, "The obstacle O has left the interference area." This prevents the occupant C from moving the obstacle O more than necessary and wasting time.

[0080] (Notification within a non-disturbing range) Next, notification of a range that will not get in the way will be described. Figures 11 and 12 are diagrams that explain an example of notification of a range that will not get in the way. As shown in Figure 11, when an obstacle O is detected in a garage G, the notification unit 160 issues a notification indicating a position CP where the obstacle O should be removed. The state management unit 110 sets the position CP where the obstacle O should be removed based on the parking route (→TS→TP→TE→PS) and the size of the obstacle O.

[0081] As shown in Fig. 11, if the obstacle O is a two-wheeled vehicle, the status management unit 110 may estimate that the obstacle O will fit into an occupied area CP that is, for example, 80 cm wide and 2 m long. Alternatively, as shown in Fig. 12, the status management unit 110 may estimate a position to put the obstacle O away that is appropriate for the shape of the two-wheeled vehicle. The status management unit 110 determines whether the occupied area CP and the position to put the obstacle away that is appropriate for the shape of the two-wheeled vehicle can fit into a space within the garage G that will not get in the way, and if so, causes the notification unit 160 to notify the user of the occupied area CP and the position to put the obstacle away that is appropriate for the shape of the two-wheeled vehicle.

[0082] The non-obstructive range is a range that is not included in the parking interference range PR and the dismounting interference range DR. The state management unit 110 sets the dismounting interference range DR based on the seat where the occupant C sits, but for the driver's seat, if the driver gets off the vehicle with the key in hand, the dismounting interference range DR is not set for the driver's seat, and if the driver gets off the vehicle leaving the key behind, the dismounting interference range DR is set for the driver's seat.

[0083] (Notification of whether it is a nuisance or not) Next, notification of whether or not a motorcycle is in the way will be described. Figures 13 and 14 are diagrams illustrating an example of notification of whether or not a motorcycle is in the way. For example, as shown in Figure 13, when the wheels or handle of a motorcycle are within the dismounting obstruction range DR, a notification is made that the motorcycle is in the way. As shown in Figure 14, when the motorcycle body is not within the dismounting obstruction range DR, a notification may be made that the motorcycle is not in the way, or a notification may not be made that the motorcycle is in the way. Furthermore, when the motorcycle leaves the obstructing range, a notification may be made that no further movement is necessary.

[0084] The position of the obstacle may be estimated based on the position of the occupant. For example, if the state management unit 110 does not detect any obstructive obstacle when the occupant C gets off the vehicle, it may have the space recognition unit 130 detect the occupant C from the camera image and identify the occupant C's standing position. This is because if the obstacle O is small, it may be hidden by the occupant C and the obstacle O may not be detected. In this case, the state management unit 110 may have the space recognition unit 130 detect the occupant C's standing position, and may determine whether the occupant C's position is obstructive by regarding the occupant's position as the position of the obstacle O, and may have the notification unit 160 notify the result.

[0085] As described above, the notification unit 160 may issue a notification when the obstacle O leaves the range where it is an obstacle, or may issue a notification when the occupant C engages in an inquiry behavior. An inquiry behavior is, for example, turning one's face toward the vehicle 1, stopping, returning to the vehicle 1, etc. The space recognition unit 130 may detect a facial image based on a camera image, and when the face of the occupant C is detected as a result, the state management unit 110 may determine that an inquiry behavior has occurred. In this way, when the occupant C wants to know whether the obstacle has been moved sufficiently, he or she can simply turn toward the vehicle 1, and the vehicle 1 will notify him or her whether the obstacle has been moved sufficiently.

[0086] When a querying behavior is detected, the notification unit 160 notifies the driver whether or not there is an obstacle O in the way. The notification may be by audio output or by controlling lights. For example, the notification unit 160 may blink the sidelights once if the obstacle O is in the way, as shown in FIG. 13, or twice if the obstacle O is not in the way, as shown in FIG. 14. In this case, when the occupant C turns back toward the vehicle 1, if the sidelights blink once, the occupant C should continue moving, and if the blinking occurs twice, the occupant C should remove the obstacle O.

[0087] The notification unit 160 may make different notifications depending on whether the obstacle O is in the parking interference range PR or only in the dismount interference range DR. Since the state management unit 110 sets the dismount interference range DR for the driver's seat when the key is inside the vehicle, if the obstacle O is only in the dismount interference range DR for the driver's seat, the notification unit 160 may make a voice notification that automatic parking can be resumed remotely by taking the key out.

[0088] (Assistance in stopping the vehicle in a suitable location for tidying up) Up to this point, we have explained support for tidying up. Now, we will explain support before tidying up begins, using parking in Fig. 15 as an example. Fig. 15 is a diagram showing an example of support for parking a vehicle in a position suitable for tidying up. In the example of Fig. 15, garage G faces the road, and its long side is perpendicular to the road. The road is wide enough for cars to pass each other.

[0089] The parking assistance ECU 100 mounted on the vehicle 1 in FIG. 15 is capable of learning-type automatic parking and has a parking path stored in advance. The parking path begins at a start position SP on the road, just before the entrance of the garage G. The vehicle 1 starts automatic parking at the start position SP and travels straight on the left side of the road to a turning start position TS. The vehicle 1 starts turning at the turning start position TS in front of the garage, turns 45 degrees, and stops temporarily by the time of the turning position TP. The vehicle 1 turns stationary at the turning position TP and starts backing up, and ends the turn by turning 45 degrees by the time of the turning end position TE. The vehicle 1 sets the steering angle to zero at the turning end position TE, backs up straight into the garage G, and parks at a parking position PS. In the example of FIG. 15, the parking position is positioned to the left side of the garage G so that the driver can get in and out of the garage G.

[0090] The navigation screen displayed on the HMI device 20 of the vehicle 1 displays a rear view when the gear is in reverse at the turning point TP. If there is a bicycle (obstacle O) in the garage G as shown in Figure 15, the driver may notice it when passing in front of the garage. However, since the driver's basic rule while moving forward is to look in the direction of travel to ensure safety, the driver may only notice the obstructing obstacle when the driver stops at the turning point TP and looks behind.

[0091] In this case, the driver can suspend automatic parking at the turning point TP, get out of the vehicle, and remove the obstacle O. However, if the driver realizes that vehicle 1 is blocking the road and that getting out of the vehicle to remove the obstacle O may obstruct the passage of other vehicles, it becomes psychologically difficult to remove the obstacle. Also, if the driver manually drives vehicle 1 to a position where it does not obstruct the passage of other vehicles, vehicle 1 will leave the route, and automatic parking will not be able to resume.

[0092] Therefore, it is preferable that the state management unit 110 be able to stop the vehicle 1 in a position that does not obstruct the passage of other vehicles when an obstructing obstacle O is present. Specifically, when the vehicle 1 is parallel to the road before the turning start position TS, the state management unit 110 instructs the notification unit 160 to ask the occupant C whether or not there is an obstructing obstacle O, or instructs the space recognition unit 130 to detect the obstructing obstacle O. In the latter case, if an obstructing obstacle O is detected, the state management unit 110 instructs the driving control unit 140 to stop the vehicle 1 before the turning start position TS. In this way, if an obstructing obstacle O is present, the vehicle 1 is stopped manually or automatically before starting to turn. Therefore, even if another vehicle passes by while the occupant is getting off the vehicle to clean up, the vehicle will not be obstructed, and the occupant can continue cleaning up.

[0093] (Control to slow down / stop at the appropriate position for tidying up) In some cases, the vehicle 1 does not have a sonar on the front side, and the spatial recognition unit 130 processes the portion of the front camera image that shows the front to detect obstacles, and performs emergency braking based on the detection results. In this case, the obstacle detection range is limited to the road, and obstacles inside the garage G are not detected. Furthermore, if processing of the side camera image showing the inside of the garage G is interrupted with processing of the front camera image, the detection of an obstacle in front may be delayed, and emergency braking may not be possible in time. Therefore, the driving control unit 140 may decelerate the vehicle so that the vehicle speed is low when the side camera reaches a position where it can capture the inside of the garage G. By decelerating the vehicle in advance, emergency braking can be performed in time even if the detection of an obstacle in front is delayed, so the state management unit 110 can interrupt the detection of an obstacle inside the garage between detections of obstacles in front.

[0094] Furthermore, the driving control unit 140 may decelerate the vehicle at the same time as prompting the occupant to check the parking location (within the garage). Since occupants notice the acceleration of deceleration, if the notification is given at the same time as deceleration, the occupant will be more likely to notice the notification. Furthermore, if the vehicle is decelerating, there will be more time to check, and if an obstacle is discovered, the occupant can stop the vehicle in a short time.

[0095] The state management unit 110 may instruct the driving control unit 140 to temporarily slow down or stop the vehicle 1, and perform detection inside the garage G during that time. Since obstacle detection ahead is not necessary while the vehicle is stopped, the space recognition unit 130 may only perform obstacle detection inside the garage G. Furthermore, the state management unit 110 may instruct the driving control unit 140 to decelerate the vehicle 1 to a position where the inside of the garage G can be seen, and have the occupant C check the inside of the garage G before causing the space recognition unit 130 to detect the inside of the garage G.

[0096] In other words, the parking assistance ECU 100 does not detect the interior of the garage G when passing in front of the garage G, but instead has the occupant C visually check the interior of the garage G, and when the occupant C then applies the brakes to stop the vehicle 1, the space recognition unit 130 detects the interior of the garage G. Alternatively, when the occupant decelerates the vehicle in front of the garage G, the space recognition unit 130 may detect the interior of the garage G, and as a result, may stop the vehicle 1 when an obstacle O is detected. In other words, when it is believed that the occupant has discovered an obstacle, the detection means detects the obstacle. Either a chronological relationship or a causal relationship is acceptable, but it is preferable that both detection by the detection means and recognition by the occupant occur.

[0097] Currently, there are performance limitations to obstacle detection using cameras and sonar. Therefore, an obstacle O that is determined to be an obstacle during detection may actually be in a position that does not get in the way, or an obstacle O that is determined to be in a position that does not get in the way may actually be in a position that does get in the way. Therefore, when the state management unit 110 detects an obstacle O near the parking interference area PR or the disembarking interference area DR, the state management unit 110 may inquire with the occupant C to determine whether the obstacle O will get in the way.

[0098] Alternatively, when occupant C stops the vehicle, the detection means may perform detection and notify the result. There is no problem if the detection result and the occupant's recognition match, but if they do not match, it can be left to the occupant C's judgment. For example, when the occupant stops vehicle 1, an undetected object such as a small tool may be left lying on the ground, or when the automatic brake is activated, there may be no obstacle and a false detection of a step may occur. Therefore, the occupant can receive a notification of the detection result by the detection means and ignore the detection if they are confident in their recognition, or get off the vehicle to check if they are not confident.

[0099] (Need for support according to the situation) On the other hand, even if an obstacle O that is clearly in the way is detected, there are many cases where assistance in removing it is not required. For example, if the garage gate is designed to open automatically when it detects the approach of vehicle 1, the driver can simply stop and wait for the gate to open, so the notification about the obstacle O is merely annoying. When a pedestrian or another vehicle approaches, the driver does not need to get out of the vehicle because they can simply wait until the obstacle O is no longer detected. Even when the garage gate needs to be opened manually, it is annoying to be notified of the same thing every time.

[0100] In this way, whether or not assistance is necessary is not determined solely by the presence of an obstructing obstacle O, so it is also possible to have the occupant C decide whether or not to provide assistance corresponding to the obstructing obstacle O. In this way, it is possible to avoid providing assistance when it is unnecessary and causing inconvenience to the occupant C.

[0101] However, being asked to make a decision can be annoying. In the majority of cases where the vehicle stops during automatic parking, assistance to remove obstacles O is not required, and the problem can be resolved by waiting for pedestrians or approaching vehicles to leave. Therefore, as the driver is repeatedly asked to make a decision, the operation becomes increasingly annoying. Eventually, the driver may decide that the assistance function itself is unnecessary and turn it off. This means that the assistance function will not work when assistance is needed.

[0102] Therefore, it is preferable that the state management unit 110 distinguishes as accurately as possible between cases where assistance is required and cases where it is not, and when it determines that assistance is not required, it does not provide assistance or provides assistance in a restrained manner so as not to bother the occupants, and when it determines that assistance is required, it provides assistance proactively. For example, the state management unit 110 also functions as a scene determination means that determines that a scene requires assistance.

[0103] Here, restrained support means, for example, a notification that is simply displayed on the screen without any sound or lighting, or a query as to whether or not support is needed in a manner that can be ignored, with a notification being issued if the answer is yes, and no action being taken if the answer is ignored.

[0104] (Suppression of support based on history) Depending on the facilities of the parking lot, vehicle 1 may necessarily stop temporarily. For example, if garage G has a shutter that opens and closes with a remote control, occupant C can stop vehicle 1, operate the remote control, and wait for the shutter to open before resuming automatic parking. Therefore, it is recommended that memory unit 150 store, as parking history, the location where vehicle 1 was parked and the fact that occupant C did not get out of the vehicle for a certain period of time or more during automatic parking. Then, if the behavior of vehicle 1 and occupant C substantially matches the parking history, state management unit 110 may suppress assistance or not provide assistance at all.

[0105] Here, suppressing assistance means that, for example, the notification unit 160 displays a text message on the screen asking, "Do you want to be notified about obstacles?", but does not make a voice inquiry. In this way, occupant C can receive the notification only when he presses the Yes button. If assistance is not required, he can simply ignore the message, and since no voice is emitted, it is not annoying.

[0106] It should be noted that when the door of the garage G is opened using the remote control, it may be discovered that there is an obstacle O inside the garage G. Therefore, even if it is determined based on the history that support should not be provided, the state management unit 110 may continue to perform scene determination, and if behavior that differs from the history or an obstacle is detected, a notification about the obstacle may be proactively given.

[0107] For example, if the occupant does not get out of the vehicle when the vehicle stops at a stored position, this is in accordance with the parking history, so only a suppressed notification displayed on the screen is issued. However, when the occupant C performs an action to prepare for getting out of the vehicle that is not recorded in the history, such as turning on the hazard lights or blinkers or buckling the belt, the state management unit 110 may instruct the notification unit 160 to start an active notification by voice.

[0108] Here, the dismounting preparation action refers to an action leading to dismounting, such as stopping the vehicle with the brake pedal, applying the parking brake (PB), releasing the brake pedal, shifting the gear to P (parking) or N (neutral), unbuckling the seat belt, opening the door, etc. Even if the location where the vehicle stopped is the same as a location in the history where the vehicle was stopped but not dismounted, when the state management unit 110 detects a dismounting preparation action that is not in the history, the state management unit 110 may infer that an unusual situation exists outside the vehicle and start an alert regarding an obstacle O. Note that if the vehicle is stopped at a location different from a location in the history where the vehicle was stopped but not dismounted and the parking brake is activated, an audio alert may start without waiting for any further dismounting preparation action.

[0109] (Detection-based assistance suppression) Furthermore, the state management unit 110 may determine whether assistance is necessary based on the detection results. For example, the state management unit 110 may perform face detection or human detection on the image of the obstacle O, and determine that assistance is unnecessary if the obstructing obstacle is a person. This is because if it is a person, it is expected that the person will notice that the vehicle is in the middle of parking and leave. Furthermore, the state management unit 110 may monitor the position of the obstacle O, and if the position of the obstacle O is moving, determine that assistance is unnecessary because the obstacle O will no longer be an obstruction over time.

[0110] Furthermore, the state management unit 110 may determine the time point at which to make a determination based on detection based on history. For example, even if the space recognition unit 130 detects when the shutter of the garage G is closed, it cannot detect the parking position PS. Therefore, the state management unit 110 may not provide notification or assistance until the parking time recorded in the history has elapsed, and may detect when the recorded parking time has elapsed and determine whether to provide notification or assistance. Therefore, when the space recognition unit 130 detects an obstacle O at the parking position, the space recognition unit 130 may notify the detection of the obstacle O and suggest assistance by voice.

[0111] Furthermore, the state management unit 110 may compare the detection result with the history and make a judgment. For example, in a mechanical parking lot or a parking lot with a turntable, the occupant C always gets out of the vehicle and operates the operation panel. Even in such parking lots, there are times when assistance is required and times when it is not, so the state management unit 110 may compare the detection result with the history and make a judgment.

[0112] For example, the state management unit 110 may track the position of the occupant C outside the vehicle and compare the moving direction of the occupant C with the history. In this case, the notification unit 160 may not issue a notification if the occupant C is heading in a direction recorded in the history (e.g., toward the control panel), but may start an audio notification if the occupant C is heading in a direction different from the history (e.g., toward the turning position TP). Furthermore, if the state management unit 110 detects a dismounting preparation action (e.g., turning on the hazard lights) after stopping at a position not recorded in the history, the state management unit 110 may start active assistance before the occupant C gets out of the vehicle.

[0113] (Restriction of support based on probability) The state management unit 110 may determine whether or not support is needed based on the occurrence of such a small number of events or a combination thereof, but it may also comprehensively evaluate the probability of needing support based on a large number of events and select whether or not to issue a notification and the mode of notification based on the probability. For example, a notification may be issued if the probability is high and not issued if the probability is low, or an active notification may be issued if the probability is high and a restrained notification may be issued if the probability is low. Alternatively, a combination of the two options may be used, where an active notification is issued if the probability is high, no notification is issued if the probability is low, and a restrained notification is issued if the probability is somewhere between the two.

[0114] As a comprehensive evaluation, the state management unit 110, for example, sums up the evaluation values ​​of the likelihood and compares the sum with a first threshold and a second threshold. Then, the state management unit 110 does not provide support if the sum is less than the first threshold, provides support in a restrained manner if the sum is equal to or greater than the first threshold and less than the second threshold, and provides active support if the sum is equal to or greater than the second threshold.

[0115] Furthermore, the state management unit 110 may continue to evaluate the probability even after starting assistance, and may stop assistance or switch to suppressed assistance when the evaluation value of the probability falls below a threshold. For example, the state management unit 110 may determine to perform suppressed assistance when the vehicle 1 stops at a position that matches the history, and may notify the occupant C of obstacles in the direction of the parking position PS using only an image before the occupant C gets out of the vehicle. The notification may then be stopped when the occupant C stops at a position that matches the history (e.g., in front of a control panel for a mechanical parking lot, etc.), or may be stopped when an obstacle (e.g., a garage door) in the direction of the parking position PS is no longer detected.

[0116] Furthermore, the state management unit 110 may continue to evaluate the probability even after stopping the assistance, and may resume the assistance when the evaluation value of the probability exceeds a threshold. For example, after stopping the assistance when the occupant C stops in front of the operation panel, the state management unit 110 may determine that active assistance will be provided when the occupant C does not return to the vehicle 1 but heads toward the parking position (inside the garage), and may issue a warning with audio according to the position of an obstacle O in the garage G.

[0117] (Examples of criteria for evaluating probability) The following provides an example of the criteria for evaluating the probability. The state management unit 110 may start evaluating the probability when an obstacle O is detected, when the occupant starts decelerating, or when automatic parking starts. For example, if the time lag between asking the occupant C to confirm the parking position and detecting the brake operation is short, the state management unit 110 will evaluate the probability as high. This is because if the time lag is short, it can be assumed that the occupant C found the obstacle O when checking inside the garage G.

[0118] The state management unit 110 evaluates the probability highly if the time between stopping and getting-off preparation is short. This is because if the time difference is short, it can be estimated that the occupant C is certain that tidying up is necessary.

[0119] The state management unit 110 adds the probability each time a vehicle disembarking preparation action is performed, such as operating the brake pedal, applying the parking brake, releasing the brake pedal, shifting the gear to P or N, unbuckling the seat belt, or opening a door. This is because if the occupant does not disembark, no assistance is required, and the higher the probability that the occupant will disembark, the higher the probability that assistance will be required. Therefore, the closer the occupant is to disembark, the higher the probability becomes.

[0120] The state management unit 110 evaluates the probability lower when many occupants C are preparing to get off, especially when all occupants get off, than when only one occupant gets off. This is because it is reasonable to assume that when many occupants get off, they get off before parking in the garage and choose to park remotely.

[0121] The state management unit 110 evaluates the probability as high when the direction in which the obstacle O was detected matches the seat where the passenger C prepared to dismount. This is because it is reasonable to assume that the passenger C closest to the obstacle O dismounts and puts away the obstacle.

[0122] The space recognition unit 130 identifies the position of the obstacle O based on the camera image, and the state management unit 110 calculates the closest approach distance when the obstacle O comes closest to the vehicle body based on the positional relationship between the parking path and the obstacle O, and evaluates the probability higher as the closest approach distance becomes shorter. Note that if the obstacle O hits the vehicle body, the state management unit 110 evaluates the closest approach distance as zero and sets the evaluation value of the probability to an upper limit value.

[0123] Alternatively, the spatial recognition unit 130 may estimate the size of the obstacle O based on the camera image, and the state management unit 110 may evaluate the probability based on the size of the obstacle O. Specifically, the spatial recognition unit 130 evaluates the length and width of the obstacle O, and the state management unit 110 evaluates the probability based on the larger of the length and width. For example, if the length is less than 1.5 meters, the probability is evaluated as high, and if it is 1.5 meters or more, the probability is evaluated as low. This is because small objects such as bicycles and trash cans can be moved by the occupant, whereas large objects such as garage gates often have a set method of movement and do not require assistance. Also, if the obstacle O is another vehicle and the spatial recognition unit 130 detects its license plate from the camera image, it is not something that the occupant would need to move and is therefore not eligible for assistance.

[0124] Furthermore, the space recognition unit 130 may detect people based on camera images. The state management unit 110 evaluates the detection results, and if the accuracy of human detection (probability that it is a person) is high, it evaluates the probability low. This is because if what is detected as the obstacle O is a person, it can be expected that the person will leave the obstructing area. The space recognition unit 130 may also detect a face. When a face is detected, it is highly likely that the person will recognize the vehicle 1 and leave the obstructing area, so it may evaluate the probability even lower.

[0125] The state management unit 110 evaluates the probability lower when the vehicle is parked at home (garage G is the vehicle's own garage) than when the vehicle is not parked at home. This is because when the vehicle is parked at home, even if there is an obstacle O, such as a gate to the garage G, the method of moving is already determined and assistance is often not required.

[0126] The state management unit 110 may store in the memory unit 150 the locations where it has detected that the obstacle O is moving up and down, and may evaluate the probability at those locations as low. Objects that move up and down are objects that do not need to be put away, such as gates in mechanical parking lots or shutters or doors in garages G, so it is advisable to make it difficult to provide unnecessary assistance at such locations. Also, at locations where the vehicle dismounts to operate machinery, the probability that increases with dismounting needs to be discounted before evaluation.

[0127] The state management unit 110 stores the location where it has detected that the body of the vehicle 1 is turning in the storage unit 150, and evaluates the probability at that location as low. This is because it is necessary to discount the probability that is added when the vehicle gets off to operate the turntable, and because it is easy to misidentify the parking position at a location where the body of the vehicle turns, it is desirable to make it difficult to provide unnecessary assistance.

[0128] The state management unit 110 evaluates the probability that the occupant C stopped at a position that matches a stopping position in the history as low. This is because if the occupant C stopped at a position in the history, it can be assumed that the purpose was not to tidy up, so it is better to make it difficult to provide assistance.

[0129] When an obstacle O is not detected, the state management unit 110 evaluates the probability lower than when an obstacle O is detected. For example, if an obstacle is detected, there is a high probability that the occupant has gotten off the vehicle to tidy up, whereas if an obstacle is not detected, it is doubtful that the occupant has gotten off the vehicle to tidy up. However, even if an obstacle O is not detected, if the probability is added due to other factors such as the occupant's actions and exceeds a threshold, assistance for tidying up will be initiated. Therefore, if the detection means does not detect an obstructing obstacle, the start of the notification may be delayed compared to when an obstructing obstacle is detected.

[0130] Furthermore, if no obstacle is detected, it is not possible to issue a notification according to the position of the obstacle, so the start of notification may be postponed, the content of the notification may be limited, or a suppressed notification may be issued. This can also be rephrased as: when the detection means has not detected an obstructing obstacle, it is more difficult to issue a notification to assist in tidying up, or it is easier to select a suppressed notification, than when the detection means has detected an obstructing obstacle.

[0131] (Example of the flow of evaluation based on probability) Below, we will explain the flow of evaluation based on probability and the suspension of the start of assistance as an example. Here, we consider a case where a thin object such as a bicycle air pump has fallen onto the floor of garage G. After vehicle 1 begins automatic parking, the spatial recognition unit 130 detects a three-dimensional object at a position where the front camera can capture images of the interior of garage G. In addition, the side sonar detects an obstacle at a position where it can detect the interior of the garage. However, since the spatial recognition unit 130 has difficulty detecting low objects and sonar has difficulty detecting thin objects, we will assume that neither the spatial recognition unit 130 nor the sonar detected the air pump at this time.

[0132] The driving control unit 140 of the parking assistance ECU 100 decelerates the vehicle 1 when the vehicle approaches a position where the occupant C can see inside the garage G. When the vehicle reaches that position, the notification unit 160 issues a notification urging the occupant C to check inside the garage G. When the driver applies the brakes to stop the vehicle 1, the state management unit 110 evaluates the probability and determines whether assistance is required.

[0133] For example, the state management unit 110 adds a probability if an obstacle is detected, and does not add a probability if no obstacle is detected. Furthermore, if the garage G is the home garage, the probability is added, but not added if it is outside the home. Furthermore, if the time between the notification prompting the driver to check and the braking operation is short, the probability is added. Furthermore, the memory unit 150 refers to the past parking history stored therein, and subtracts the probability if the driver has previously stopped near the current parking position a predetermined number of times or more, and does not subtract the probability otherwise. In this case, if the vehicle is parked at home and there is no history of the vehicle being parked in a nearby location, and if the time between the notification prompting the driver to check and the braking operation is short, and the time between the vehicle 1 stopping and the first preparation for getting out of the vehicle is also short, the addition of the probability may overlap and exceed the threshold for determining the start of assistance.

[0134] When assistance is started before the occupant dismounts, if an obstructing obstacle is detected, a notification based on the position of the obstacle should be made, and if no obstructing obstacle is detected, a notification indicating the range of the obstruction should be made. Of course, when an obstacle is detected, the obstructing range and the location of the obstacle may be notified together. The notification is made, for example, by displaying on the navigation screen. This allows the occupant to know where to move the obstacle before dismounting. The notification via the navigation screen should be continued even after the occupant dismounts. If it is displayed on the navigation screen, the occupants remaining in the vehicle can see it and notify the occupants who have dismounted.

[0135] If no obstacle is detected when it is decided to start assistance, possible assistance may be started even if the position of the obstacle is unknown, but it is also possible to only notify the user that assistance will be provided, or to postpone the start of notification. The notification of assistance may be a suppressed notification that only displays a message on the screen.

[0136] However, if no obstacle is detected, the state management unit 110 may detect an occupant who has exited the vehicle and provide assistance based on the detection result. For example, the state management unit 110 may instruct the spatial recognition unit 130 to track occupant C who has exited the vehicle. If the location where the occupant C stops is within a parking or disembarking obstruction range, it may infer that an obstacle is present within the obstruction range. Therefore, if the location where occupant C stops is within the obstruction range, the probability may be further increased, and if the probability exceeds a second threshold, assistance with audio may be initiated. For example, when occupant C stops to pick up an air pump, an audio notification may be issued saying, "You are in an obstruction range," and when the occupant C leaves the obstruction range, an audio notification may be issued saying, "You have left the obstruction range." Furthermore, if occupant C places the air pump upright and it is detected as a three-dimensional object, the target for determining whether or not it is an obstruction may be switched from occupant C to the three-dimensional object. Furthermore, the occupant's face may be continuously detected, and when the occupant's face is detected, that is, when the occupant turns their face toward the vehicle 1, it may be determined whether the position of the object to be judged (air pump) is in a position that will interfere with parking or disembarking, and an alert may be sent to the occupant C.

[0137] In other words, even if no obstacle is detected, if the probability evaluated from other evaluation factors is high, the state management unit 110 may presume that the occupant has gotten off the vehicle to tidy up, and may provide assistance to the occupant C. By evaluating the probability in this way and determining whether or not to provide assistance, the state management unit 110 can increase the probability of providing assistance when assistance is needed and the probability of not providing assistance when assistance is not needed.

[0138] (Parking assist ECU processing) Next, the processing executed by the parking assist ECU 100 will be described with reference to the flowchart of Fig. 16. Fig. 16 shows an example of the processing executed by the parking assist ECU 100.

[0139] 17 to 27 are diagrams illustrating an example of automatic parking by the parking assistance ECU 100 according to the embodiment. In the initial state (step S000) of the flowchart in Fig. 16, the vehicle 1A is automatically traveling on an aisle in a parking lot under the control of the traveling control unit 140, and the space recognition unit 130 is executing a parking space detection process. In other words, the parking assistance function of the parking assistance ECU 100 of the vehicle 1A is operating in an automatic traveling mode in which the vehicle 1A travels autonomously to search for a parking space.

[0140] Specifically, in the initial state, the vehicle 1 is located at the position of the vehicle 1A in Fig. 17, the space recognition unit 130 detects an empty parking space while detecting the angle and distance of the vehicle body relative to the passage (white line), and the driving control unit 140 automatically drives the vehicle 1 so as to maintain the distance from the white line (step S000). Here, the result of the parking space detection is evaluated (step S001), and if an empty parking space is not detected (step S001: No), the process returns to step S001.

[0141] When an available parking space is detected (step S001: Yes), the state management unit 110 sets a parking route as shown by the arrow in FIG. 17 (a route that passes through OP, TP, and TE in that order and stops at PS), and also sets a confirmation point OP and a stopping point PP2 (shown in FIG. 20), which will be described later (step S002). Although not shown, in step S002, the notification unit 160 displays the parking route and makes an inquiry to request approval for automatic parking. Next, the state management unit 110 determines whether the driver has approved the parking (step S003). If the driver has not approved the parking (step S003: No), the process returns to step S001.

[0142] If the driver approves (step S003: Yes), the state management unit 110 starts automatic parking of the vehicle 1A. That is, the automatic driving mode, in which the vehicle 1A travels along an aisle, is switched to an automatic parking mode, in which the vehicle 1A travels along a parking path (step S004). Although not shown in the figure, in response to the mode change, the space recognition unit 130 stops detecting white lines and parking spaces, and changes the obstacle detection range from the range along the aisle to the range along the parking path.

[0143] In Figure 17, there are parking spaces (PS1, PS2, PS3, PS5) in which other vehicles (1B-1E) are parked, but since they are unrelated to vehicle 1A, parking space PS4 in which vehicle 1A is parked will also be referred to simply as a parking space. Assuming there are abandoned carts CT1 and CT2 inside and outside the parking space. The illustration of the carts is a schematic diagram, and they are not lying down, but are standing on their wheels. In other words, carts CT1 and CT2 are abandoned in a position that touches the white line. Vehicle 1A has only one occupant, the driver (occupant C), and there are no other people around.

[0144] In this example, the parking assistance ECU 100 captures an image of the vicinity of the entrance of the parking stall PS4 from the position of the vehicle 1A shown in FIG. 17 using the front camera, determines that an empty parking stall has been detected, and starts automatic parking. In other words, automatic parking is started at a position where it is not possible to check whether there are any obstacles in or around the parking stall PS4. Therefore, the state management unit 110 sets a confirmation point OP on the parking route to check whether there are any obstacles. As shown in FIG. 18, the confirmation point OP is located on an extension of the long side of the front side of the parking stall, where the front camera of the vehicle 1A can capture the entire parking stall. In addition, the position of the obstacle CT2 around the parking stall can be more accurately identified by detecting it from a closer distance.

[0145] Returning to the description of Fig. 16, the state management unit 110 determines whether the vehicle 1A has reached the check point OP (step S005). If the vehicle 1A has not reached the check point OP (step S005: No), the process returns to step S005.

[0146] 18, when the vehicle 1A reaches the confirmation point OP (step S005: Yes), the state management unit 110 determines whether or not there is an obstacle in the way based on the image captured by the front camera (step S006). If there is no obstacle in the way (step S006: No), the process proceeds to step S020.

[0147] If an obstructing obstacle is detected (step S006: Yes), the notification unit 160 issues a notification that "there is an obstacle, so the vehicle will stop" (step S007).

[0148] Next, the state management unit 110 determines whether the vehicle 1A has reached the stopping point PP2 (step S008). The stopping position PP2 is a position where the head of the occupant C is positioned on an extension of the long side of the front side of the parking space, and the entire parking space can be seen, as shown in Figure 20. If the vehicle 1A has not reached the stopping point PP2 (step S008: No), the process returns to step S008.

[0149] When the vehicle 1A reaches the stopping point PP2 (step S008: Yes), the traveling control unit 140 stops the vehicle 1A at the stopping point PP2 (step S009). That is, the vehicle does not stop when an obstacle is detected, but travels to a position where the occupant C can see the entire parking space, and then stops. Then, the notification unit 160 notifies the occupant C of the movement of the obstacle.

[0150] If the vehicle 1 is made to stop when an obstructing obstacle is detected, the vehicle will stop at the point where an obstacle CT2 in the passage is detected, and therefore, for example, as shown in Figure 19, the vehicle may stop at a position PP1 where an obstacle CT1 in the parking space cannot be detected.

[0151] In this case, since the parking assistance ECU 100 detects only the obstacle CT2 on the passage, the occupant C cannot receive assistance based on the detection of the obstacle CT1 inside the parking space. As a result, the occupant C may overlook the obstacle CT1 inside the parking space, return to the vehicle 1A, and then detect the obstacle CT1 again after restarting automatic parking. In other words, the obstacle may not be the one initially detected, so the vehicle 1 should be stopped after it has reached a position where all obstacles can be detected.

[0152] 18, there is also a problem with a position where the front camera can detect the inside of the parking space, but the occupant C cannot see the obstacle CT1 inside the parking space. In this position, even if the notification unit 160 notifies the occupant C about the obstacles CT2 and CT1, the occupant C may only listen to part of the notification before getting out of the vehicle, remove only the obstacle CT2 that was visible from the driver's seat, and return to the vehicle 1, which may then notify the occupant C that the obstacle CT2 has not been removed. Rather than such a position, it is more efficient and takes less time to stop the vehicle 1A in a position where the occupant C can see the entire parking path, as shown in FIG. 20, and notify the occupant C about all obstacles, and have the occupant C remove all obstacles in one go.

[0153] Therefore, in this embodiment, the state management unit 110 sets the parking position PP2 at a position where the inside of the parking space can be seen, as shown in FIG.

[0154] In normal parking that is not automatic, the driver visually checks the inside of the parking space, then starts turning the vehicle and backs into the parking space. In other words, if the stopping position PP2 is set under the above conditions, the stopping position PP2 will be located before the turning start point TS.

[0155] Furthermore, the state management unit 110 may set the parking position PP2 to any position between the position where the head of the occupant C is on the extension of the long side of the near side of the parking space and the turning start point TS. In this case, the vehicle 1A has not started turning when it stops, and does not obstruct the passage of other vehicles, so the occupant C can start tidying up without hesitation.

[0156] In other words, the conditions for a parking position suitable for removing an obstructing obstacle are that the vehicle has not started turning, the vehicle's long side is approximately parallel to the direction of the road or aisle, the detection means is able to detect the parking position, and the occupant is able to see the parking position. A position that meets all three conditions is optimal, but a position that meets at least one of the conditions is more suitable for removal than a position that does not. For example, even if the occupant cannot see the parking position, if the detection means can detect it, there is a high possibility that the occupant can be guided to the obstacle. Even if the obstacle cannot be detected from the parked position, the location of the obstacle can be memorized, and if the occupant can see the obstacle in the parking position, there is a high possibility that the obstacle can be removed. Even if the vehicle is blocking the road diagonally, if the occupant can be guided to the obstacle and removed in a short time, there is a high possibility that the obstacle will not be obstructed by passing vehicles. As such, none of the conditions are essential, and it is better to satisfy as many conditions as possible.

[0157] Returning to the point in time when an obstacle is detected, the notification will be explained. When the space recognition unit 130 detects an obstacle, the state management unit 110 determines whether the obstacle is within an obstruction range that would prevent parking or passengers from getting out of the vehicle. If the determination is Yes (step S006: Yes), the notification unit 160 issues a notification, for example, saying, "There is an obstacle, so the vehicle will stop." However, as described above, the vehicle does not stop immediately after receiving the notification, but instead proceeds to the stopping position PP2 and stops there.

[0158] When the vehicle 1A stops, the notification unit 160 may display the detected obstacles (carts CT1 and CT2) in a frame to notify the driver that the obstacles are obstructing the vehicle, as shown in Fig. 21. The notification unit 160 may also display a boundary line BL to indicate the obstructing range, as shown in Fig. 22. At this time, the notification unit 160 may change the display method for the parking obstruction range and the disembarking obstruction range so that they can be distinguished.

[0159] 22 illustrates the range that would be an obstacle when the driver gets out of the vehicle, corresponding to the presence of occupant C in the driver's seat. When the state management unit 110 inquires of the driver about switching to remote parking, the notification unit 160 may blink the get-out obstruction range DR, and may turn off the display of the get-out obstruction range DR if the answer is Yes.

[0160] The parking interference area TR may be displayed as a surface as shown by hatching in FIG. 22, or may be displayed as a boundary line BL between the interference area and the non-interference area as shown by a dotted line in FIG. 22, or both may be used.

[0161] Furthermore, when displaying the boundary line BL, the notification unit 160 may display only the boundary line that overlaps with an obstacle or the boundary line that is close to the obstacle. Furthermore, the notification unit 160 may notify the destination and distance of the obstacle. In this case, since the space recognition unit 130 detects the parking space, the notification unit 160 may notify based on the parking space and may notify, for example, "Please move the obstacle out of the parking space."

[0162] 16, the description will be continued. After step S009, the state management unit 110 determines whether the occupant C has dismounted (step S010). If the occupant C has not dismounted (step S010: No), the process returns to step S010.

[0163] When the occupant C gets out of the vehicle 1A (step S010: Yes), the space recognition unit 130 starts assisting the occupant C in tidying up. The space recognition unit 130 tracks the occupant C and the obstacles (carts CT1 and CT2) (step S011).

[0164] Next, the state management unit 110 determines whether both the occupant C and the obstacle have left the parking interference area PR (step S012). If at least one of the occupant C and the obstacle has not left the parking interference area PR (step S012: No), the process returns to step S011.

[0165] Although not shown, a step of determining the direction of the occupant C's face and a step of notifying whether the occupant has left the parking interference area PR may be inserted into the loop that repeats step S012. For example, if the occupant turns his / her face toward the vehicle and the occupant is in the parking interference area PR, a negative response is notified, and if the occupant is outside the area PR, a positive response is notified. In this way, the occupant can know whether he / she is inside or outside the parking interference area PR simply by turning his / her face toward the vehicle.

[0166] When the parking assist ECU 100 assists the occupant C who has left the vehicle, the state management unit 110 may provide assistance by communication using lights or voice. For example, the state management unit 110 may control the notification unit 160 to notify the occupant C in advance of a rule (code) that indicates "Yes" (positive response) when both left and right lamps (LF, RF, LB, RB) of the vehicle 1A are lit, and "No" (negative response) when only one of the left and right lamps is lit, and respond to the action of the occupant C with lights. Alternatively, the state management unit 110 may notify the occupant C in advance of a code for voice notification, and respond by voice, such as "Yes" (positive response) when a mechanical sound "fan fan" is heard twice, or "No" (negative response) when a mechanical sound "fan" is heard once.

[0167] When the occupant C leaves the parking interference area PR together with the obstacle (cart CT2) (step S012: Yes), the notification unit 160 notifies the occupant C that the obstacle has left the parking interference area PR by lighting up both lamps (LF, RF, LB, RB) as shown in Fig. 23 (step S013). At this time, it is preferable to simultaneously notify the occupant C by audio (for example, outputting a mechanical sound "fan fan" twice) so that the notification can be heard even if the occupant C has their back to the vehicle 1A.

[0168] Next, the state management unit 110 determines whether the occupant C is approaching the vehicle 1A (step S014). If the occupant C is not approaching the vehicle 1A (step S014: No), the process returns to step S014.

[0169] If the occupant C is approaching the vehicle 1A (step S014: Yes), the state management unit 110 determines whether there is an obstacle in the way (step S015). As shown in FIG. 26, if there is no obstacle in the way (step S015: No), the process proceeds to step S017.

[0170] The reason why the judgment conditions are different in step 12 and step 15 is because the purpose of the notification is different. The detection means detects the direction of the occupant's face or head, or the direction of the occupant's movement, and if the direction is away from the vehicle, it is sufficient to notify whether the occupant has moved sufficiently, so the notification means notifies whether the occupant has left the interference range. If the occupant turns toward the vehicle after leaving the interference range, it is sufficient to notify whether automatic parking can be resumed, so the notification means notifies whether there is an obstacle within the interference range.

[0171] The explanation of step S015 continues. As shown in FIG. 24, if an obstructing obstacle is detected when occupant C approaches vehicle 1A, the process proceeds to step S016 (step S015: Yes). For example, as shown in FIG. 24, if the obstructing obstacle (cart CT1) is located to the right of vehicle 1A, the notification unit 160 notifies the driver of a negative response by turning on only the right lamps (RF, RB) and notifies the driver that automatic parking cannot be resumed (step S016). At this time, the notification unit 160 notifies the driver that the obstructing obstacle (cart CT1) is located to the right of vehicle 1A by turning on the right lamps (RF, RB). Then, the process returns to step S011. In this negative response, a mechanical "fan" sound may be used in combination with a negative response, or a voice announcement such as "There is an obstacle in the parking space" may be made.

[0172] After returning to step S011, as shown in FIG. 25, when the obstacle (cart CT1 that was in the parking lot) leaves the parking interference area PR (step S012: Yes), the notification unit 160 issues a positive notification (step S013). Subsequently, as shown in FIG. 26, when occupant C approaches vehicle 1A without any obstructing obstacles (step S014: Yes, step S015: No), the notification unit 160 issues a positive notification by turning on both lamps, indicating that automatic parking can be resumed (step S017). Then, the process proceeds to step S018. The positive response at this time may be a mechanical "flash flash" sound twice, or a voice announcement such as "Automatic parking can be resumed."

[0173] Here, we have introduced examples of notifications using lamps and speakers on vehicle 1, but notifications can also be made by vibrating or making sounds on a device (for example, a smart key or a mobile phone) carried by occupant C. In this case, misunderstandings can be avoided by using a common code rule (for example, two times for Yes, one time for No) regardless of the device.

[0174] When assisting an occupant who has left the vehicle, the parking assist ECU 100 may use a detection means to detect the direction of the face or head of the occupant who has left the vehicle or the movement of the occupant, and change the notification means depending on the direction. For example, the notification means may be different depending on whether the direction of the occupant's face or head or the direction of movement of the occupant is away from the vehicle, or whether the direction of the occupant's face or head or the direction of movement of the occupant is toward the vehicle, or whether the movement has stopped. If the direction of the occupant's face or head or the direction of movement of the occupant is away from the vehicle, the notification may be made by means including at least one of a sound or vibration of a terminal carried by the occupant, and if the direction of the occupant's face or head or the direction of movement of the occupant is toward the vehicle, or whether the movement has stopped, the notification may be made by means including a light on the vehicle.

[0175] Returning to FIG. 16, the explanation will continue. If all obstructing obstacles have been removed (step S015: No), the notification unit 160 issues a positive response to notify that automatic parking can be resumed (step S017). Although not shown in the figure, the positive response also issues a notification inquiring about the resumption of automatic parking. For example, the notification unit 160 may display a message inquiring about the resumption of automatic parking on the HMI device 20 (e.g., a navigation screen), or may send a signal to a terminal carried by the occupant C to output a message inquiring about the resumption of automatic parking. The message inquiring about the resumption may be a code such as three vibrations. When issuing the notification via a portable terminal, the positive response may be omitted and only the message inquiring about the resumption may be output.

[0176] After the inquiry, the state management unit 110 determines whether the resumption has been approved by the occupant C (step S018). For example, the occupant C may return to the driver's seat and instruct the automatic parking to be resumed by inputting Yes to the question "Do you want to resume automatic parking?" displayed on the navigation screen. Alternatively, the occupant C may instruct the automatic parking to be resumed by operating a mobile terminal without returning to the driver's seat. If the resumption has not been approved (step S018: No), the process returns to step S018.

[0177] When an instruction to resume automatic parking is received (S018: Yes), the state management unit 110 resumes automatic parking (step S019), as shown in Fig. 27. Fig. 27 shows a case where the occupant C approves the resumption of automatic parking without returning to the driver's seat. The driving control unit 140 drives the vehicle 1A toward the parking position PS, and the state management unit 110 determines whether the vehicle 1A has reached the parking position PS (step S020).

[0178] If the vehicle 1A has not reached the parking position PS (step S020: No), the process returns to step S020. If the vehicle 1A has reached the parking position PS (step S020: Yes), the driving control unit 140 parks the vehicle 1A at the parking position PS, and after the notification unit 160 notifies the completion of parking, the state management unit 110 terminates all functions of the parking assistance device (step S021).

[0179] Although the embodiments of the present invention have been described above, the above-described embodiments are presented as examples and are not intended to limit the scope of the present invention. This novel embodiment can be embodied in various other forms. Furthermore, various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. Furthermore, this embodiment is included within the scope and spirit of the invention, and is also included in the inventions and their equivalents described in the claims. [Explanation of symbols]

[0180] 1 vehicle 2 Cameras 10 Operating device 20 HMI device 30 Vehicle control device 40 Navigation Devices 50 Sonar ECU 100 Parking Assist ECU 101 CPU 102 ROM 103 RAM 104 I / O 105 IMP 106 Communication I / F 110 Status Management Unit 120 Image processing unit 130 Spatial recognition section 140 Travel control unit 150 Storage section 160 Information Department

Claims

1. A parking assistance device that automatically parks a vehicle in a predetermined parking position, a storage means for storing a parking route to the parking position; a detection means for detecting at least the surroundings of the vehicle and obtaining detection information; A driving control means for controlling the driving of the automatic parking vehicle; a notification means for notifying an occupant of the vehicle; an assistance control means for suspending the automatic parking based on at least one of the detection information and the occupant's action; Equipped with When the detection means detects an obstacle that gets in the way during the automatic parking or when the occupant stops the vehicle, the assistance control means suspends the automatic parking, The notification means notifies the occupant about the suspension of the automatic parking. Parking assistance device.

2. The assistance control means When the detection means detects an obstacle that is in the way, the automatic parking movement is suspended at a predetermined position, The predetermined position is a position where the long side of the vehicle is approximately parallel to the direction of the road or passage; a position where the detection means can detect the parking position; a position where the occupant can see the parking position; The position corresponds to at least one of The parking assistance device according to claim 1.

3. The assistance control means The vehicle is oriented substantially parallel to the road, and, When the occupant can see the parking position, At least one of causing the notification means to issue a notification requesting confirmation of the parking position and causing the driving control means to decelerate the vehicle is performed. The parking assistance device according to claim 1.

4. When the occupant stops the vehicle, when the detection means detects the obstruction, or When the occupant performs a dismount preparation operation, Suspending the automatic parking The parking assistance device according to claim 1.

5. the obstructing obstacle is an obstacle within a blocking range, The interference range corresponds to at least one of a parking interference range in which the closest approach distance to the vehicle body is equal to or less than a margin threshold during automatic parking, and a disembarking interference range in which the distance to the door is equal to or less than a disembarking room threshold when the occupant disembarks, The assistance control means determines whether the obstacle is an obstacle that will get in the way based on a position of the obstacle detected by the detection means and a positional relationship with the interference range. The parking assistance device according to claim 1.

6. The vehicle further includes an occupant detection means for detecting whether the occupant is seated, the assistance control means sets the alighting obstruction range corresponding to a seat in which the occupant is seated, and does not set the alighting obstruction range for a seat in which the occupant is not seated, 6. The parking assistance device according to claim 5.

7. The smart key detection means detects the location of the smart key. the assistance control means sets an alighting prevention range corresponding to the driver's seat when the smart key is inside the vehicle, and does not set an alighting prevention range corresponding to the driver's seat when the smart key is outside the vehicle; 6. The parking assistance device according to claim 5.

8. The assistance control means At least one of the margin threshold and the room-to-get-out threshold is set in accordance with at least one of opening / closing of a side mirror and an environmental condition; The environmental conditions include at least one of a vehicle body tilt, a vehicle body vibration, an outside temperature, and an outside wind speed.

6. The parking assistance device according to claim 5.

9. When an obstacle is present in the dismounting obstruction range and no obstacle is present in the parking obstruction range, the notification means Notification of an occupant who is prevented from disembarking by the obstacle; Notifying the driver that automatic parking can begin when the driver gets off the vehicle; A notification suggesting that the occupant disembark; Notification of means for starting automatic parking from outside the vehicle; At least one of the following is notified:

6. The parking assistance device according to claim 5.

10. The notification means When the automatic parking or passengers get off the vehicle, a notification indicating the range of obstacles that will be an obstacle is provided. a notification indicating the obstruction; Notification indicating a position where the obstructing obstacle should be removed; a notification indicating a distance that the obstructing obstacle will move until it is no longer obstructing the vehicle; Notification of the presence or absence of the obstruction; Notification of the direction of the obstacle in the way; a notification that the obstacle is no longer in the way; perform at least one of the following: The parking assistance device according to claim 1.

11. The assistance control means If the detection means does not detect the obstruction, than when the detection means detects the obstructing obstacle, Making it difficult to make the notification or making it easy to select a suppressed notification; The parking assistance device according to claim 1.

12. The assistance control means If the detection means does not detect the obstruction, causing the detection means to detect an occupant who has left the vehicle; The position of the occupant is regarded as the position of the obstructing obstacle, and the notification means is caused to make the notification. The parking assistance device according to claim 1.

13. The notification means If the detection means does not detect the obstructing obstacle, a notification is given indicating the extent of the obstruction when the automatic parking or the occupant gets out of the vehicle, When the detection means detects the obstructing obstacle, a notification is made based on the position of the obstacle. The parking assistance device according to claim 1.

14. the assistance control means evaluates the probability that the notification is required; The assistance control means performs at least one of selecting whether to notify or not and selecting a mode of notification based on the probability; The selection of whether to notify is a selection of making a notification if the probability is high, and not making a notification if the probability is low, The selection of the notification mode is a selection of making an active notification if the probability is high, and making a suppressed notification if the probability is low. The parking assistance device according to claim 1.

15. the assistance control means evaluates the probability based on at least one of the occupant's behavior, the detection result of the detection means, the position information, and the history of past parking; The assistance control means If the time from stopping the vehicle to the act of preparing to get off is short, the evaluation that the probability is higher than when the time is long; evaluating the probability higher if the obstacle is closer to the parking path than if the obstacle is farther from the parking path; If the obstacle is detected, the probability is evaluated to be higher than if the obstacle is not detected; If the obstacle is a person, the probability is lowered compared to when the obstacle is not a person; If the parking position is a home garage, the evaluation lowers the probability compared to when the parking position is not a home garage; determining whether the location where the occupant stopped outside the vehicle substantially matches the location where the occupant stopped recorded in the history, and if the location substantially matches, evaluating the probability to be lower than if the location does not substantially match; evaluate at least one of the following:

15. A parking assistance device according to claim 14.

16. The detection means detects the direction of the face of the occupant who has left the vehicle or the position of the occupant, When the position of the occupant changes from within a range where an obstacle would be in the way to outside that range, Or, when the speed at which the occupant's position is moving away from the vehicle decreases, Or, when the occupant's face is turned toward the vehicle, When either of the above occurs, a notification based on the position of the occupant is made. The parking assistance device according to claim 1.

17. The detection means detects at least one of the direction of the face of the occupant who has left the vehicle or the direction of movement of the occupant, the notification means changes the manner of notification depending on whether the orientation of the face or the direction of movement is a direction away from the vehicle or whether the orientation of the face or the direction of movement is a direction toward the vehicle; a notification mode when the orientation of the face or the moving direction is a direction away from the vehicle includes at least one of a sound and a vibration of a terminal carried by the occupant; When the face orientation or the movement direction is a direction toward the vehicle, the notification mode includes controlling lights of the vehicle. The parking assistance device according to claim 1.

18. The detection means detects at least one of a face direction of the occupant who has left the vehicle or a moving direction of the occupant, The notification means The content of the notification is different depending on whether the orientation of the face or the direction of movement is a direction away from the vehicle or whether the orientation of the face or the direction of movement is a direction toward the vehicle. The parking assistance device according to claim 1.

19. A parking assistance method using a parking assistance device that includes a storage means for storing a parking route to a predetermined parking position, and automatically parks a vehicle at the parking position, comprising: a detection step of detecting at least the surroundings of the vehicle and obtaining detection information; a driving control step for controlling driving of the automatic parking vehicle; a notification step of notifying an occupant of the vehicle; an assistance control step of suspending the automatic parking based on at least one of the detection information and the movement of the occupant; Including, When the detection step detects an obstacle that will hinder the automatic parking or when the occupant stops the vehicle, the assistance control step suspends the automatic parking, The notifying step notifies the occupant about suspension of the automatic parking. Parking assistance methods.

20. The assistance control step includes: When an obstructing obstacle is detected in the detection step, the automatic parking is suspended at a predetermined position; The predetermined position is a position where the long side of the vehicle is approximately parallel to the direction of the road or passage; a position where the parking position can be detected in the detection step; a position where the occupant can see the parking position; The position corresponds to at least one of 20. The parking assistance method of claim 19.

Images