Parking assistance device and parking assistance method

Abstract

To improve convenience of a parking support device.SOLUTION: A parking support device includes storage means, detection means, travel control means, notification means, and support control means. The storage means stores a parking route to a predetermined parking position. The detection means performs detection at least around a vehicle to obtain detection information. The travel control means controls traveling in automatic parking. The notification means makes a notification to an occupant of the vehicle. The support control means suspends the traveling in the automatic parking on the basis of at least one of the detection information and a motion of the occupant. In at least one of a case where the detection means detects an obstacle which interferes during the automatic parking and a case where the occupant stops the vehicle, the support control means suspends the traveling in the automatic parking, and the notification means makes a notification about suspension of the traveling in the automatic parking to the occupant.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to a parking support device and a parking support method.

Background Art

[0002] Conventionally, when an obstacle in the vicinity of a vehicle is detected during automatic parking, a technique of automatically activating the brakes and stopping the automatic parking is known. In such a case, with the conventional technique, even if the vehicle occupant gets out of the vehicle, clears the obstacle, and then returns to the vehicle interior, it is inconvenient because the automatic parking cannot be restarted.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present disclosure is to provide a parking support device with improved convenience.

Means for Solving the Problems

[0005] In order to solve the above problems, a parking support device according to the present disclosure includes a storage means, a detection means, a travel control means, a notification means, and a support control means. The storage means stores a parking route to a predetermined parking position. The detection means detects at least the vicinity of the vehicle to obtain detection information. The travel control means controls the travel of automatic parking. The notification means notifies the vehicle occupant. The support control means suspends the travel of automatic parking based on at least one of the detection information and the occupant's operation. When at least one of the detection means detecting an obstacle that interferes during automatic parking or the occupant stopping the vehicle occurs, the support control means suspends the travel of automatic parking, and the notification means notifies the occupant about the suspension of the travel of automatic parking. [Effects of the Invention]

[0006] According to this disclosure, the convenience of parking assistance devices can be improved. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of a vehicle to which the parking assistance ECU according to the embodiment can be applied. [Figure 2] Figure 2 is a block diagram showing an example of the hardware configuration of a parking assistance ECU according to the present invention. [Figure 3] Figure 3 is a block diagram showing an example of the configuration of a parking assistance system according to the embodiment. [Figure 4] Figure 4 illustrates an example of the process by which the spatial recognition unit according to this embodiment identifies the location of feature points. [Figure 5] Figure 5 shows an example of the area that obstructs disembarking according to the embodiment. [Figure 6] Figure 6 shows an example of the range of obstruction to disembarking according to the embodiment. [Figure 7] Figure 7 shows an example of the parking obstruction range according to the embodiment. [Figure 8] Figure 8 shows an example of the area obstructing vehicle disembarkation and the area obstructing parking according to the embodiment. [Figure 9] Figure 9 illustrates an example of support for tidying up items in a non-obtrusive location according to the embodiment. [Figure 10] Figure 10 illustrates an example of support for tidying up items in a non-obtrusive location according to the embodiment. [Figure 11] Figure 11 illustrates an example of a notification within a non-intrusive range according to the embodiment. [Figure 12] Figure 12 illustrates an example of a non-intrusive notification within the embodiment. [Figure 13] Figure 13 illustrates an example of notification of whether or not something is interfering according to the embodiment. [Figure 14]FIG. 14 is a diagram for explaining an example of notification of whether or not there is an obstruction according to the embodiment. [Figure 15] FIG. 15 is a diagram showing an example of assistance for parking at a position suitable for storage according to the embodiment. [Figure 16] FIG. 16 is a flowchart showing an example of processing executed by the parking support ECU according to the embodiment. [Figure 17] FIG. 17 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 18] FIG. 18 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 19] FIG. 19 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 20] FIG. 20 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 21] FIG. 21 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 22] FIG. 22 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 23] FIG. 23 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 24] FIG. 24 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 25] FIG. 25 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 26] FIG. 26 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment. [Figure 27] FIG. 27 is a diagram for explaining an example of automatic parking by the parking support ECU according to the embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0008] The embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are all specific examples of this disclosure. Therefore, the components, their arrangement and connection configurations, and the steps and their order shown in the following embodiments are examples and are not intended to limit this disclosure. Furthermore, components in the following embodiments that are not described in an independent claim will be described as optional components.

[0009] Furthermore, each figure is a schematic diagram and not necessarily a strictly accurate representation. In each figure, substantially identical components are denoted by the same reference numerals, and redundant explanations are omitted or simplified.

[0010] The parking assistance device according to this embodiment, described below, will suspend the automatic parking process if it detects an obstacle that is getting in the way during automatic parking, and the notification means will provide notification regarding the suspension of the automatic parking process. This notification is intended to assist in clearing the obstructing obstacle, thereby enabling the suspension of the automatic parking process to be released earlier. Furthermore, the method of this application determines when the vehicle has stopped during automatic parking and suppresses assistance when it is not a scene in which an occupant should get out of the vehicle to clear an obstacle, so as not to unnecessarily bother the occupant.

[0011] Furthermore, while there are conventional technologies for interrupting automatic parking after it has started, the parking assistance device according to this embodiment is not limited to cases where it is interrupted after it has started. In other words, the parking assistance device according to this embodiment can also be applied when automatic parking is not started.

[0012] For example, an occupant may stop in front of an empty parking space, activate the automatic parking function of the parking assist system, and then notice an obstruction and begin to exit the vehicle. In such a case, the parking assist system according to this embodiment will start providing a notification useful for clearing the obstruction. In other words, the parking assist system according to this embodiment will function even if the automatic parking drive has not been started. The same applies if the start of the automatic parking drive is suspended because an obstruction has been detected.

[0013] Furthermore, the parking assistance device according to this embodiment also functions when the automatic brake is activated after the automatic parking process has started, causing the automatic parking to be suspended, similar to the conventional technology. For example, thin, three-dimensional objects such as aluminum shopping carts may not be detected as obstacles at the starting position of the automatic parking process, or their position may be misidentified. Therefore, the automatic brake may be activated after the automatic parking process has started, or the occupant may notice the obstacle first and apply the brakes, but the parking assistance device according to this embodiment functions similarly in both cases. In other words, it functions regardless of the sequence or causal relationship between the detection of an obstructing obstacle and the suspension of the automatic parking process.

[0014] In this specification, not initiating an automatic parking maneuver and interrupting an automatic parking maneuver are both referred to as "suspending the automatic parking maneuver," and the two are not distinguished. On the other hand, in this specification, automatic parking and automatic parking maneuver are distinguished as needed.

[0015] This is because automatic parking is a process in which the parking assist system sets a parking path and drives the vehicle along that path; therefore, automatic parking begins before the automatic parking drive is initiated. In other words, outwardly, the automatic parking drive may look like automatic parking itself, but the two are different. However, to the occupants, the automatic parking drive appears to be automatic parking itself, so for the sake of explanation, the automatic parking drive is sometimes simply referred to as automatic parking.

[0016] As embodiments of this invention, examples of parking by learning-type automatic parking and examples of parking in a parking space detected by parking space detection are shown, but the scope of application of this invention is not limited to a specific parking method. For example, it can also be applied when a parking space is detected based on sonar detection and an obstacle is visually detected by the occupant.

[0017] This embodiment is particularly effective when parking using a learning-type automatic parking system. A learning-type automatic parking system is suitable for situations where the vehicle is repeatedly parked in the same location, such as in a home garage. In a learning-type automatic parking system, the vehicle performs a learning drive by manually parking in the garage beforehand, memorizing the parking route and parking position. When performing automatic parking, the vehicle automatically drives along the memorized route. Therefore, since it is possible to start automatic driving from a position far away from the garage, there is a possibility that the vehicle may encounter obstacles that get in the way during the drive.

[0018] Furthermore, even in a system where the driver searches for an empty parking space and activates the parking assist device after stopping in front of it, if the device is equipped with the functions of the present invention, the occupants can receive useful assistance simply by activating the parking assist device when clearing obstacles within the empty parking space.

[0019] The parking assistance device of this embodiment is characterized by its ability to deal with obstructing obstacles, so the method of automatic parking is not specified. In this embodiment, the explanation is given on the premise that the parking assistance device is activated and a parking path has been set. The method of calculating the path before that and the method of automatic driving after that can be arbitrarily selected and implemented, so a detailed explanation is omitted.

[0020] (Vehicle configuration) First, a vehicle to which the parking assist device in the embodiment can be applied will be described. Figure 1 shows a vehicle 1 to which the parking assist device in the embodiment can be applied. As shown in Figure 1, vehicle 1 is equipped with a parking assist system 1S. The parking assist system 1S comprises an operating device 10, an HMI (Human Machine Interface) device 20, a vehicle control device 30, a navigation device 40, a sonar ECU 50, and a parking assist ECU 100. The parking assist ECU 100 is an example of a parking assist device.

[0021] Furthermore, other devices may be installed on vehicle 1. Also, although Figure 1 shows the operating device 10, HMI device 20, vehicle control device 30, navigation device 40, sonar ECU 50, and parking assist ECU 100 as separate devices, some or all of these devices may be integrated.

[0022] The operating device 10, HMI device 20, vehicle control device 30, navigation device 40, sonar ECU 50, and parking assist ECU 100 will be described later.

[0023] Cameras 2a, 2b, 2c, and 2d are installed at four locations on the front, rear, left, and right sides of the vehicle body 1. Hereafter, unless otherwise specified, cameras 2a, 2b, 2c, and 2d will simply be referred to as camera 2. Each camera 2 is equipped with a fisheye lens and has a field of view of 180 degrees or more in the horizontal direction.

[0024] Each camera 2 is mounted at a downward angle to capture the road surface, so when the range of the road surface captured is converted to a horizontal field of view, a range of approximately 240 degrees of the road surface (see dashed line) is captured by one camera 2. For example, the images captured by the side cameras 2a and 2b, which are installed on the left and right sides of the vehicle 1, will include the front and rear wheels and the sides of the vehicle 1.

[0025] Note that the installation locations and number of cameras 2 are not limited to the example shown in Figure 1.

[0026] Furthermore, as shown in Figure 1, vehicle 1 is equipped with 12 sonar sensors 3a to 3l. Hereafter, when sonar sensors 3a to 3l are not individually distinguished, they will simply be referred to as sonar sensor 3. When referring to sonar sensors individually, they will be named according to their installation location. For example, on the left side of vehicle 1, sonar sensor 3a is installed on the front side (FLS: Front Left Side) of vehicle 1, and sonar sensor 3b is installed on the rear side (BLS: Back Left Side) of vehicle 1.

[0027] Additionally, on the right side of vehicle 1, sonar sensor 3c is installed on the front side (FRS: Front Right Side) and sonar sensor 3d is installed on the rear side (BRS: Back Right Side). These four sonar sensors are also called side sonars because they detect obstacles on the sides of the vehicle.

[0028] Furthermore, the following sonar sensors are installed in front of vehicle 1, in order from left to right in the direction of travel: sonar sensor 3e (FLC: Front Left Corner), sonar sensor 3f (FL: Front Left), sonar sensor 3g (FR: Front Right), and sonar sensor 3h (FRC: Front Right Corner).

[0029] The sonar sensors 3f and 3g, located on the inside, detect obstacles in the direction of travel when vehicle 1 is moving straight. The sonar sensors 3e and 13h, located on the outside, detect obstacles in the direction of the turn when vehicle 1 is turning. Sonar sensors 3e and 3h are also called corner sonars. In Figure 1, the detection range of the four sonar sensors 3e, 3f, 3g, and 3h is illustrated by a triangle, but the detection range is not limited to the triangle and can detect objects up to approximately 10m from the vehicle. In addition, the detection ranges of adjacent sonars are installed to overlap.

[0030] Furthermore, at the rear of vehicle 1, the following sonar sensors are installed in order from left to right in the direction of vehicle 1's forward movement: sonar sensor 3i (BLC: Back Left Corner), sonar sensor 3j (BL: Back Left), sonar sensor 3k (BR: Back Right), and sonar sensor 3l (BRC: Back Right Corner).

[0031] The sonar sensors 3j and 3k, located on the inside, detect obstacles in the direction of travel when vehicle 1 is reversing. The sonar sensors 3i and 3l, located on the outside, detect obstacles in the direction of turning when vehicle 1 is reversing and turning. Sonar sensors 3i and 3l are also called corner sonars. Figure 1 illustrates the detection directions and their fan-shaped spread by the four sonar sensors 3i, 3j, 3k, and 3l using triangles; however, their detection range is not limited to the inside of the triangles in the figure but extends further. For example, adjacent sonars are installed so that their detection ranges overlap. The same applies to the four sonar sensors 3e, 3f, 3g, and 3h on the front of the vehicle.

[0032] Furthermore, the detection range of the sonar sensors 3a, 3b, 3c, and 3d installed on the sides of vehicle 1 is set to be narrower than the detection range of the sonar sensors installed in front of and behind vehicle 1. This is to improve the positional resolution of the parking assist ECU 100 when detecting parking spaces to the sides of vehicle 1 by minimizing the overlap of the detection ranges of the side sonars when vehicle 1 moves.

[0033] Furthermore, each sonar sensor is installed at a height and angle that makes it easy to detect surrounding obstacles when vehicle 1 is parking. Note that the installation locations and number of sonar sensors 3a to 3l are not limited to the example shown in Figure 1.

[0034] In this embodiment, sonar refers to the sonar system consisting of the sonar sensors 3a to 3l and the sonar ECU 50 described above. The sonar ECU 50 is a control device that comprehensively controls the sonar system.

[0035] The sonar sensor 3 emits directional sound waves and receives the reflected waves. The sonar ECU 50 detects the distance to the obstacle based on the time from when the sonar sensor 3 emits sound waves until the reflected waves are received. The sonar ECU 50 detects the area around the vehicle using multiple sonars and identifies the location of the obstacle based on the distance from the multiple sonar sensors 3 detected.

[0036] The sonar system detects obstacles in the direction of travel of vehicle 1 using sonar sensors 3 mounted on the bumper of vehicle 1. When the sonar ECU 50 detects an obstacle, it determines whether vehicle 1 will collide with the obstacle. If the sonar ECU 50 determines that vehicle 1 will collide with the obstacle within a predetermined time, it instructs the vehicle control device 30 to activate the automatic brakes.

[0037] This ensures that vehicle 1 does not collide with obstacles in front of or behind it, even during automatic parking. Furthermore, by detecting obstacles with side sonars located on the left and right sides of vehicle 1's body, it is possible to predict when the side of the vehicle body will approach an obstacle due to the inner wheel difference.

[0038] Note that sonar is not an essential component of the parking assist system 1S. In other words, vehicle 1 may be configured without sonar. In this case, for example, the parking assist ECU 100 may detect the distance to an obstacle by processing the camera image captured by camera 2. For example, the parking assist ECU 100 may calculate the time until vehicle 1 collides with an obstacle by processing the camera image. If a collision occurs within a predetermined time, the parking assist ECU 100 may instruct the vehicle control device 30 to activate the automatic brakes.

[0039] (Hardware configuration of the parking assist ECU) Next, the hardware configuration of the parking assistance ECU 100 will be described. The parking assistance ECU 100 is an example of a parking assistance device. Figure 2 is a diagram showing an example of the hardware configuration of the parking assistance ECU 100 according to this embodiment. The functions of the parking assistance ECU 100, which will be described later, may be implemented in the hardware shown in Figure 2.

[0040] The parking assist ECU 100 may also be a computer equipped with a CPU 101, ROM 102, RAM 103, I / O (input / output interface) 104, IMP (image processor) 105, and communication I / F (interface) 106, with each element connected by a bus.

[0041] The parking assist ECU 100 may house multiple elements on a single chip. Alternatively, the parking assist ECU 100 may consist of multiple chips for a single element. The bus may not be a single unit, but rather a combination of multiple types of buses.

[0042] For example, the CPU 101, ROM 102, RAM 103, IMP 105, and communication I / F 106 could be housed on a single chip and connected via a parallel bus, while the I / O 104 could be composed of multiple chips connected to the chip housing the CPU 101 and a serial bus.

[0043] The parking assist ECU 100 performs automatic parking by obtaining information from other devices (e.g., navigation system 40) or giving instructions to other devices (e.g., vehicle control device 30) via a communication interface and in-vehicle LAN.

[0044] The CPU 101 controls the entire parking assist ECU 100. The functions of each part of the parking assist ECU 100 may also be implemented in the form of programs executed by the CPU 101. The ROM 102 and RAM 103 correspond to memory units, with the ROM 102 corresponding to a non-volatile area. The RAM 103 is used for temporary storage as the CPU 101's work area. For example, camera images such as display images and detection images, and information on targets and feature points detected from the images, are temporarily stored in the RAM 103.

[0045] The IMP105 is a processor with enhanced processing performance specifically designed for image processing and parallel processing. Some of the functions of the parking assistance ECU100, described later (for example, the image processing unit 120, the spatial recognition unit 130, etc.) may be performed by the IMP105.

[0046] (Parking Assist ECU Functions) Next, the functions of the parking assistance ECU 100 will be described. Figure 3 is a block diagram showing an example of the configuration of a functional block of the parking assistance system 1S according to the embodiment. In Figure 3, the functions of the parking assistance ECU 100 are divided into blocks: a state management unit 110, an image processing unit 120, a spatial recognition unit 130, a driving control unit 140, a storage unit 150, and a notification unit 160. As shown in Figure 3, the cameras 2 (2c(F), 2d(B), 2a(L), 2b(R)) output captured camera images. The image processing unit 120 of the parking assistance ECU 100 receives the camera images and generates display images, etc. The display images are output from the notification unit 160 to the HMI device 20.

[0047] The notification unit 160 superimposes messages onto the display image or outputs audio messages in accordance with instructions from the status management unit 110. The notification unit 160 is just one example of a notification means; the HMI device 20, which is the destination of the message output, and the status management unit 110, which instructs the output of the message, may also be included as notification means. In other words, a notification means is a functional element that conveys information to the crew. Since the status management unit 110, notification unit 160, and HMI device 20 are involved in notification, these are sometimes referred to as the HMI device 20, etc.

[0048] The status management unit 110 receives user input from the operating device 10 and controls the functions of the parking assist ECU 100 according to the user's input. In this case, the touch panel of the navigation device 40 is included in the operating device 10.

[0049] The state management unit 110 receives location information from the main unit (not shown) of the navigation device 40. The storage unit 150 stores a map including the automatic parking route. When vehicle 1 stops near the starting point of the route, the state management unit 110 compares the vehicle's location information shown by the navigation device 40 with the starting point location information marked on the map. If they roughly match, it starts automatic parking. The state management unit 110 also decides whether to suspend driving during automatic parking, depending on the situation. In other words, the state management unit 110 is an example of a support control means that determines whether to start or suspend driving for automatic parking.

[0050] The image processing unit 120 generates a display image and a detection image. The detection image is, for example, an image that emphasizes changes in brightness and color, i.e., contrast.

[0051] The spatial recognition unit 130 detects target objects and feature points from the detection image and estimates its own position by comparing them with the positions of target objects and feature points stored in the map. The method for self-position estimation can be any of the proposed methods, so a detailed explanation is omitted.

[0052] Furthermore, the spatial recognition unit 130 detects the location of obstacles and the space without obstacles from the detection image. Figure 4 is a diagram illustrating an example of the process by which the spatial recognition unit 130 identifies the location of an obstacle. As shown in Figure 4, the spatial recognition unit 130 identifies the location of the target object (obstacle) P as seen by the camera 2 of the vehicle 1 using motion parallax. Specifically, first, the spatial recognition unit 130 converts the position of the target object P on the camera image into the angle (azimuth angle) of the target object P relative to the vehicle 1. Figure 4 shows the case where the camera 2 of the vehicle 1 moves from point A to point B on the Y axis as the vehicle 1 moves. In this case, the image of the target object moves on the camera image, and the azimuth angle of the target object P seen by the camera 2 changes from θ1 to θ2. This change in azimuth angle (θ1→θ2) is called motion parallax.

[0053] The distance between point A and point B (the length of line segment AB) is the amount of movement of vehicle 1, and can be determined from the number of wheel rotations. The spatial recognition unit 130 can determine the XY coordinates (x, y) of target object P from the coordinates of points A and B, and the length of line segment AB, θ1 and θ2, using the principle of triangulation. The vertical position of target object P in the image captured by camera 2 corresponds to the height of target object P. For example, if target object P is the tip of a pole standing perpendicular to the ground, it will appear above the base of the pole in the image. Therefore, once the spatial recognition unit 130 determines the XY coordinates, it can determine the Z coordinate based on the vertical position in the image. In other words, the spatial recognition unit 130 can determine the three-dimensional coordinates of target object P by applying the principle of triangulation to the motion parallax that appears in camera images taken at different times.

[0054] Furthermore, the spatial recognition unit 130 analyzes the distribution of motion parallax in the detection image. In the area where the road surface is visible in the detection image, the motion parallax gradually decreases as the vehicle moves away, but where a three-dimensional object is visible, the parallax changes discontinuously. Utilizing this, the spatial recognition unit 130 identifies areas where there are no discontinuities in parallax based on the analysis of the motion parallax distribution, and by corresponding these areas to the area on the road surface, it can detect areas without three-dimensional objects (free space).

[0055] Furthermore, the spatial recognition unit 130 receives obstacle detection information from sonar or the like, and reinforces the recognition of the location of obstacles and the surrounding space. For example, since the error in position estimation using motion parallax increases when obstacles are in the direction of travel, the spatial recognition unit 130 may supplement it with distance information obtained from sonar to reduce the error in position estimation.

[0056] The driving control unit 140 controls the steering angle and vehicle speed when automatically parking, causing the vehicle to travel along the path stored in the memory unit 150. Specifically, the driving control unit 140 works in conjunction with the memory unit 150 and the spatial recognition unit 130 to control the steering angle and vehicle speed. When automatically parking, the spatial recognition unit 130 estimates the vehicle's position based on the position of the target object P displayed on the camera 2. The driving control unit 140 calculates the deviation from the path by comparing the vehicle's position estimated by the spatial recognition unit 130 with the path stored in the memory unit 150.

[0057] If the position of vehicle 1 deviates from the path, the driving control unit 140 controls the steering angle to change the steering angle so that the path of vehicle 1 intersects with the path, and when vehicle 1 overlaps with the path, it changes the steering angle in the tangential direction of the path. In other words, the driving control unit 140 provides feedback control of the steering angle so that the position estimated by the spatial recognition unit 130 follows the path stored in the memory unit 150.

[0058] The greater the vehicle speed and steering angle of vehicle 1, the more likely it is to deviate from the path. Therefore, when performing automatic parking, the driving control unit 140 may, for example, maintain the vehicle speed at 5 km / h, or adjust the vehicle speed according to the steering angle. When approaching the end of the path, the driving control unit 140 decelerates the vehicle speed of vehicle 1 and stops it at the end of the path, at which point the automatic parking of vehicle 1 is completed. Since the vehicle control for automatic parking is not a feature of this invention, further explanation will be omitted.

[0059] The parking assistance device is not limited to the configuration exemplified as the parking assistance ECU 100 described above, but can be configured as desired. The example configuration of the parking assistance ECU 100 disclosed in Figure 3 supports learning-type automatic parking. In the learning type, the driving path during the learning drive is stored as the parking path in the memory unit 150, so there is no path calculation unit. In the case of a parking assistance device that does not perform a learning drive, but detects a parking space from a camera image and calculates the parking path from the vehicle to the parking space, a parking space detection unit may be provided instead of the spatial recognition unit 130, and a path calculation unit for calculating the parking path may be further provided.

[0060] The assistance provided by this disclosure regarding obstructive obstacles largely involves the spatial recognition unit 130 detecting the obstacle, the state management unit 110 making a decision, and the notification unit 160 notifying the occupant. However, some of these functions may be performed by another device, such as the navigation device 40. Furthermore, that other device may be included in the parking assistance system. It is also optional to refer to the functional blocks by different names.

[0061] (Range of obstructing obstacles) The parking assistance ECU 100 according to this embodiment suspends the automatic parking process and provides notification regarding the suspension if there is an obstructing obstacle. An obstructing obstacle is an obstacle around the vehicle 1 that is within the range that hinders parking or hinders the occupants from getting out of the vehicle when parked. Hereinafter, the range in which the presence of an obstacle hinders parking or the occupants from getting out will be referred to as the obstruction range. In other words, an obstructing obstacle is an obstacle within the obstruction range.

[0062] When vehicle 1 travels along a parking path, the range in which the distance to vehicle 1 falls below a margin threshold (also called a clearance threshold) is the range in which parking is obstructed (if there is an obstacle). This is called the parking obstruction range. The parking obstruction range is just one example of an obstruction range. The margin threshold is, for example, 15 cm.

[0063] When vehicle 1 is parked, the area where the distance from the door is below the exit clearance threshold is the area where exiting is hindered (if there is an obstacle). This is called the exit obstruction area. The exit obstruction area is just one example of an obstruction area. The exit clearance threshold is, for example, 30 cm.

[0064] Obstacles within the parking obstruction range and obstacles within the exit obstruction range are considered obstructive obstacles. However, the state management unit 110 does not set an exit obstruction range around doors where occupants do not exit. For example, if an occupant exits from the driver's seat and the automatic parking is restarted remotely (i.e., the smart key that remotely instructs automatic parking is outside the vehicle), the state management unit 110 does not set an exit obstruction range around the driver's door. In that case, even if there is an obstacle at a position where the distance from the driver's door will be 20 cm when vehicle 1 is parked, the state management unit 110 may determine that there are no obstructive obstacles and start automatic parking. In other words, as long as automatic parking can be restarted, the state management unit 110 may assist in clearing away obstacles that obstruct exit, or it may have the occupant exit beforehand so that the obstacles are not in the way.

[0065] (Area of ​​obstruction to parking and area of ​​obstruction to dropping off passengers) Figures 5 to 8 show examples of areas that obstruct passenger exit and areas that obstruct parking, with Figures 5 and 6 showing areas that obstruct passenger exit. All vehicles 1 in the figures are assumed to be in a parked position. As shown in Figure 5, when there are passengers C in all seats, the state management unit 110 sets an area that obstructs passenger exit DR around the doors of all seats. Also, as shown in the rear seats in Figure 6, the state management unit 110 does not set an area that obstructs passenger exit DR around the doors of seats where there are no passengers C.

[0066] Vehicle 1 detects the status of seat belts and doors using known technology and shares detection information with various devices via the in-vehicle LAN. The status management unit 110 of the parking assist ECU 100 determines that occupant C is present if the seat belt is fastened and anticipates that the occupant will exit if the door is open. For example, when the seat belt is unbuckled and the door is closed, the notification unit 160, under the management of the status management unit 110, may flash the exit obstruction range DR corresponding to the seat where the seat belt was unbuckled to indicate that the exit obstruction range DR will change when the occupant exits.

[0067] Figure 7 shows a portion of the parking obstruction area PR when vehicle 1 reverses in a straight line to the parking position. The parking obstruction area PR is the range in which the distance to vehicle 1 is below the safety threshold as vehicle 1 moves from the starting position to the parking position. Therefore, the range shown in Figure 7 is only a small portion of the parking obstruction area PR. Figure 7 shows vehicle 1 when parked. The parking obstruction area PR is determined by the distance from the vehicle body of vehicle 1. In front of the side mirrors, the parking obstruction area PR is determined based on the width of the side mirrors, while behind the side mirrors, it is determined based on the width of the vehicle body. Also, as shown in Figure 8, when reversing and parking with the side mirrors folded, the width of the parking obstruction area PR narrows in the range in front of the side mirrors.

[0068] If there is no occupant C inside the vehicle, as shown in Figure 7, the obstructive area is only the parking obstruction area PR. If there is an occupant C inside the vehicle, as shown in Figure 8, the obstructive area is the combined area of ​​the parking obstruction area PR and the exit obstruction area DR. As shown in area DR in Figure 8, the area that is the exit obstruction area DR but not the parking obstruction area PR can be called the exit obstruction area in the narrow sense.

[0069] (Notification based on crew seating) The state management unit 110 sets a disembarkation obstruction range DR in response to passenger C being seated, where obstacles would prevent disembarkation. Furthermore, if the state management unit 110 determines that an obstacle detected by the spatial recognition unit 130 is within the disembarkation obstruction range DR, the notification unit 160 provides notification regarding passenger C's disembarkation based on the determination of the state management unit 110.

[0070] For example, the state management unit 110 sets the area within 30 cm of the passenger side door when the vehicle 1 is parked as the exit obstruction area DR, which is the area that may interfere when passenger C gets out of the passenger seat. If the state management unit 110 determines that the detected obstacle is not in the parking obstruction area PR but is in the exit obstruction area DR, the notification unit 160 will notify, for example, "There is an obstacle in a position that obstructs exiting from the passenger seat." In response, when passenger C gets out of the passenger seat, the state management unit 110 may have the notification unit 160 notify, "There are no obstacles obstructing parking. Automatic parking can be resumed."

[0071] Furthermore, the status management unit 110 may change the range in which obstacles are requested to move in response to occupant C getting out of the vehicle. For example, when the notification unit 160 is displaying the parking obstruction range PR and the exit obstruction range DR, if the passenger seat belt buckle is undone, the exit obstruction range corresponding to the passenger seat may be displayed with a blinking light that alternates between a transmittance of 80% and 20%, so that it is clear that the exit obstruction range DR changes when the passenger gets out of the vehicle.

[0072] Furthermore, the notification unit 160 may also provide notifications in response to the movement of obstacles. For example, when an obstacle moves and is placed in the exit obstruction range DR corresponding to the driver's seat, the notification unit 160 may provide a notification stating, "There is an obstacle in a position that obstructs exiting from the driver's seat." In response, when the driver's seat belt buckle is unbuckled, the notification unit 160 may provide a notification stating, "You can resume automatic parking from outside the vehicle using the smart key," to encourage remote parking.

[0073] (Environmental dependence of room for maneuver and leeway) Next, we will explain the margin and safety margin. The state management unit 110 sets the safety threshold to, for example, 15 cm and the disembarkation threshold to, for example, 30 cm, but the margin and safety may be adjusted depending on the situation. The state management unit 110 may also change the starting point of the safety margin according to the state of vehicle 1.

[0074] The starting point of the margin is the outermost part of the vehicle body 1; therefore, when moving straight, the tip of the side mirror becomes the starting point of the margin on the side of the vehicle body. If the control of vehicle 1 is determined to fold the side mirrors when reversing straight in the final section of automatic parking, the state management unit 110 may set the starting point of the margin on the side of the vehicle body to the side of the vehicle body instead of the tip of the side mirror, or to the maximum protruding position of the folded side mirror. Also, for example, the state management unit 110 may set the margin threshold to 10 cm only in the final section when reversing straight with the side mirrors folded. This would prevent occupant C from being bothered by unnecessary assistance in narrow garages.

[0075] Furthermore, if the road surface is icy, there is a risk that the wheels of vehicle 1 may slip and deviate from the parking path. For example, if the temperature is 5 degrees Celsius or lower and the time is early morning, the condition management unit 110 may set the safety threshold to 30 cm. In this way, by keeping obstacles further away than usual, the possibility of contact with obstacles is reduced even if the wheels skid on an icy road surface.

[0076] Furthermore, when the road surface has a steep incline or when the wind is strong, the state management unit 110 may increase the disembarkation clearance threshold. Both the incline and strong winds can be detected by the acceleration sensor. By keeping obstacles away from the area around the door in this way, the possibility of the door hitting an obstacle even if it opens wide due to the incline or wind is reduced.

[0077] (Support in getting them to put things away in a place where they won't get in the way) If there is an obstruction that is getting in the way, the notification unit 160, under the management of the status management unit 110, will notify the area where the obstruction is getting in the way (obstruction area), notify the obstruction itself, notify the location where the obstruction should be removed, notify the distance to move until the obstruction is no longer getting in the way, or notify that the obstruction is no longer getting in the way.

[0078] Here, Figures 9 and 10 illustrate an example of assistance in moving an object to a non-obstructive location. When illustrating the parking path of vehicle 1, the midpoint of the rear axle of vehicle 1 is shown as a representative point representing the position of vehicle 1, and the path that the representative point passes through during the parking process is shown as the parking path. Vehicle 1 performs automatic parking, as shown in Figure 9, by driving straight to the turning start position TS, turning at the turning return position TP, turning at the turning end position TE, and parking at the parking position PS in the garage G. When vehicle 1 is driving straight towards the turning start position TS, if the spatial recognition unit 130 detects an abandoned bicycle as an obstacle O, the notification unit 160 will notify, for example, "Please move the obstacle on the front right 4m ahead."

[0079] Furthermore, although not shown in the diagram, the notification unit 160 may also illustrate the area within 1.2m to the left and right of the parking path (corresponding to the parking obstruction area PR) in an overhead view, and display it as the area where the obstacle O would be an obstruction (obstruction area). Also, as shown in Figure 10, the notification unit 160 notifies the driver when the obstacle O leaves the obstruction area by illuminating the left headlight LF and right headlight RF of the vehicle 1. Alternatively, it notifies the driver by voice saying, "The obstacle has left the obstruction area." This prevents the occupant C from moving the obstacle O unnecessarily and wasting time.

[0080] (Notifications that do not interfere) Next, we will explain notification within a non-obtrusive range. Figures 11 and 12 illustrate an example of notification within a non-obtrusive range. As shown in Figure 11, when an obstacle O is detected inside the garage G, the notification unit 160 provides notification indicating the location CP where the obstacle O should be removed. The status management unit 110 sets the location CP where the obstacle O should be removed based on the parking route (→TS→TP→TE→PS) and the size of the obstacle O.

[0081] As shown in Figure 11, the status management unit 110 may estimate that if the obstacle O is a motorcycle, for example, that the obstacle O will fit within an occupied area CP that is 80 cm wide and 2 m long. Alternatively, as shown in Figure 12, it may estimate a storage location that corresponds to the shape of the motorcycle. The status management unit 110 determines whether the occupied area CP and the storage location corresponding to the shape of the motorcycle fit within the garage G without obstructing it, and if so, it notifies the notification unit 160 of the occupied area CP and the storage location corresponding to the shape of the motorcycle.

[0082] The area that does not obstruct is the area not included in the parking obstruction area PR and the exit obstruction area DR. The state management unit 110 sets the exit obstruction area DR based on the seat where occupant C is sitting, but for the driver's seat, if the driver gets out with the keys, the exit obstruction area DR is not set for the driver's seat, but if the driver leaves the keys and gets out, the exit obstruction area DR is set for the driver's seat.

[0083] (Notification of whether it is a nuisance or not) Next, we will explain how to notify whether something is an obstruction or not. Figures 13 and 14 illustrate examples of how to notify whether something is an obstruction or not. For example, as shown in Figure 13, if the wheels or handlebars of a motorcycle are within the obstruction zone (DR), a notification will be issued that it is an obstruction. As shown in Figure 14, if the body of the motorcycle is not within the obstruction zone (DR), a notification may be issued that it is not an obstruction, or no notification may be issued at all. Additionally, a notification may be issued when the vehicle has left the obstruction zone, indicating that further movement is not necessary.

[0084] The location of an obstacle may be estimated based on the position of the occupant. For example, if the state management unit 110 has not detected any obstructing obstacles when occupant C disembarks, it may have the spatial recognition unit 130 detect occupant C from the camera image and determine their standing position. This is because, if the obstacle O is small, it may be obscured by occupant C and become undetectable. In that case, the state management unit 110 may have the spatial recognition unit 130 detect occupant C's standing position, consider the occupant's position as the position of obstacle O, determine whether it is obstructing or not, and then have the notification unit 160 report the result.

[0085] As mentioned above, the notification unit 160 may notify when the obstacle O moves out of the obstructing area, or it may notify when occupant C takes an inquiry action. Inquiry actions include, for example, turning one's face towards vehicle 1, stopping, or returning to vehicle 1. The spatial recognition unit 130 detects a face image based on the camera image, and when it detects occupant C's face as a result, the state management unit 110 may determine that an inquiry action has occurred. In this case, when occupant C wants to know if the obstacle has moved sufficiently, they can turn towards the vehicle, and vehicle 1 will be notified whether it has moved sufficiently.

[0086] The notification unit 160 notifies the driver of the presence or absence of an obstructing object O when an inquiry action is made. The notification can be made by outputting sound or by controlling lights. For example, as shown in Figure 13, the notification unit 160 may flash the side marker lights once if the obstructing object O is in the way, or as shown in Figure 14, flash them twice if the obstructing object O is not in the way. In this case, when the occupant C turns to face the vehicle 1, if the side marker lights flash once, the occupant C should continue moving, and if they flash twice, the occupant should remove the obstructing object O.

[0087] The notification unit 160 may provide different notifications depending on whether the obstacle O is in the parking obstruction area PR or only in the exit obstruction area DR. The status management unit 110 sets the exit obstruction area DR to the driver's seat when the key is inside the vehicle, so if the obstacle O is only in the exit obstruction area DR at the driver's seat, it may provide an audible notification that automatic parking can be resumed remotely by taking the key out.

[0088] (Assistance in stopping the vehicle in a suitable position for tidying up) Up to this point, we have explained support for tidying up, but now we will explain support before tidying up begins, using the parking situation in Figure 15 as an example. Figure 15 shows an example of support for parking in a suitable position for tidying up. In the example in Figure 15, garage G faces the road, and its long side is perpendicular to the road. The road is wide enough for two cars to pass each other.

[0089] The parking assist ECU 100 installed in vehicle 1 in Figure 15 supports learning-type automatic parking and has a parking route pre-programmed into its memory. The parking route begins at starting position SP on the road, in front of the entrance to garage G. Vehicle 1 starts automatic parking at starting position SP and travels straight along the left side of the road to turning start position TS. Vehicle 1 starts turning at turning start position TS in front of the garage and turns 45 degrees to stop temporarily by the reversing position TP. After turning at reversing position TP, vehicle 1 starts reversing and turns 45 degrees to complete the turn by the turning end position TE. At turning end position TE, vehicle 1 sets the steering angle to zero and reverses straight into garage G, parking at parking position PS. In the example in Figure 15, the parking position is positioned to the left of garage G so that the driver can get in and out of the vehicle.

[0090] The navigation screen displayed on the HMI device 20 of vehicle 1 shows the rear view when the gear is reversed at the turning position TP. As shown in Figure 15, if there is a bicycle (obstacle O) in the garage G, the driver may notice it when passing in front of the garage. However, since the driver should check for safety by looking in the direction of travel while moving forward, they may only notice an obstructing obstacle when they stop at the turning position TP and check behind them.

[0091] In that case, the driver can pause automatic parking at the reversal position TP, get out of the vehicle, and clear away the obstacle O. However, if the driver realizes that vehicle 1 is blocking the road and that getting out to clear away the obstacle might obstruct the passage of other vehicles, they may find it psychologically difficult to do so. Also, if the driver manually moves vehicle 1 to a position that does not obstruct the passage of other vehicles, vehicle 1 will deviate from the path, and automatic parking will not be able to resume.

[0092] Therefore, the status management unit 110 should be able to stop vehicle 1 in a position that does not obstruct the passage of other vehicles if there is an obstructing obstacle O. Specifically, when vehicle 1 is parallel to the road before the turning start position TS, the status management unit 110 should instruct the notification unit 160 to check with the occupant C for the presence of an obstructing obstacle O, or instruct the spatial recognition unit 130 to detect an obstructing obstacle O. In the latter case, once an obstructing obstacle O is detected, the unit instructs the driving control unit 140 to stop vehicle 1 before the turning start position TS. This way, if there is an obstructing obstacle O, the vehicle will stop manually or automatically before the turning starts, so even if other vehicles pass by while the occupant is getting out to tidy up, their passage will not be obstructed, and the occupant can continue tidying up.

[0093] (Control that slows down / stops at a position suitable for tidying up) In some cases, vehicle 1 may not be equipped with sonar on the front side, and the spatial recognition unit 130 processes the portion of the front camera image that shows the area in front to detect obstacles, and then applies emergency braking according to the detection result. In this case, the range of obstacle detection is limited to the road, and obstacles inside the garage G will not be detected. Also, if the processing of the side camera image showing the inside of the garage G is interrupted from the processing of the front camera image, the detection of obstacles ahead may be delayed, and emergency braking may not be possible in time. Therefore, the driving control unit 140 may decelerate the vehicle so that the vehicle speed is reduced when the side camera reaches a position where it can photograph the inside of the garage G. If deceleration is performed in advance, emergency braking will be possible even if the detection of obstacles ahead is delayed, so the state management unit 110 can interrupt the detection of obstacles inside the garage in between the detection of obstacles ahead.

[0094] Furthermore, the driving control unit 140 may also decelerate at the same time as prompting the occupant to confirm the parking position (inside the garage). Since the occupant will notice the acceleration of deceleration, if the notification is given at the same time as the deceleration, they will be more likely to notice that a notification has been given. In addition, if the vehicle is decelerating, there will be more time to confirm, and if an obstacle is detected, it will be able to stop in a short time.

[0095] The state management unit 110 may instruct the driving control unit 140 to temporarily slow down or stop vehicle 1, and have it perform detection within the garage G during that time. Since obstacle detection ahead is unnecessary while stopped, the spatial recognition unit 130 may only perform obstacle detection within the garage G. Alternatively, the state management unit 110 may instruct the driving control unit 140 to slow down vehicle 1 to a position where the garage G can be visually inspected, and have occupant C check the garage G before the spatial recognition unit 130 detects the garage G.

[0096] In other words, the parking assistance ECU 100 does not detect the inside of garage G when passing in front of it, but instead allows occupant C to visually confirm the inside of garage G, and when occupant C applies the brakes and stops vehicle 1, the spatial recognition unit 130 detects the inside of garage G. Alternatively, when the occupant slows down the vehicle in front of garage G, the spatial recognition unit 130 may detect the inside of garage G, and if it detects an obstacle O as a result, it may stop vehicle 1. In other words, the detection means detects the obstacle when it is thought that the occupant has spotted an obstacle. The sequence of events and causal relationships do not matter, but it is preferable to have both detection by the detection means and recognition by the occupant occur.

[0097] Currently, there are performance limitations to obstacle detection using cameras and sonar. As a result, an obstacle O that is detected as an obstruction may actually be in a position where it is not an obstruction, and an obstacle O that is detected as not an obstruction may actually be in a position where it is an obstruction. Therefore, when the state management unit 110 detects an obstacle O near the parking obstruction area PR or the exit obstruction area DR, it may consult with the occupant C to determine whether or not it is an obstruction.

[0098] Alternatively, the system could be configured to detect an object using a detection device when occupant C stops the vehicle, and then notify the occupant of the result. If the detection result matches the occupant's perception, there is no problem, but if they do not match, the decision can be left to occupant C. For example, when an occupant stops vehicle 1, there may be an undetectable object, such as a small tool, lying on the ground, or the automatic brake may activate when there is no obstacle but it is a false detection of a step. Therefore, the occupant should receive notification of the detection result from the detection device, ignore the detection if they are confident in their perception, or get out of the vehicle to confirm if they are not confident.

[0099] (The need for support tailored to the situation) On the other hand, even if an obstacle O is detected in an obviously inconvenient location, assistance in clearing it is often not required. For example, if the garage gate is designed to open automatically when it detects the approach of vehicle 1, one can simply stop and wait for the gate to open, so the notification about the obstacle O is merely annoying. When a pedestrian or other vehicle approaches, one can wait until the detection of obstacle O is gone, so there is no need to get out of the vehicle. Even if the garage gate needs to be opened manually, being notified the same thing every time is annoying.

[0100] Thus, the need for assistance is not determined solely by the presence of an obstructing obstacle O. Therefore, it is acceptable to allow crew member C to decide whether or not to provide assistance in response to the obstructing obstacle O. This would avoid providing unnecessary assistance and thus troubling crew member C.

[0101] However, the very act of being asked to make a decision can sometimes be bothersome. In most cases where the automatic parking system stops midway, the problem can be solved by simply waiting for pedestrians or approaching vehicles to leave, without needing assistance to clear obstacles. As a result, repeated requests for decisions can become tedious. Eventually, the user may decide that the assistance function itself is unnecessary and turn it off. This means that the assistance function will not work when it is actually needed.

[0102] Therefore, the state management unit 110 should determine with the highest possible accuracy whether support is needed or not, and when it determines that support is not needed, it should either not provide support or provide support sparingly so as not to bother the crew, and when it determines that support is needed, it should provide support actively. For example, the state management unit 110 can also function as a scene determination means to determine whether a scene requires support.

[0103] In this context, restrained support refers to methods such as providing notifications only through screen displays without accompanying sounds or lights, or inquiring about the need for support in a way that can be ignored, providing notification if the answer is yes, and doing nothing if ignored.

[0104] (Restriction of support based on history) Depending on the parking lot facilities, vehicle 1 may always have to stop temporarily. For example, if garage G has a shutter that can be opened and closed with a remote control, occupant C may temporarily stop vehicle 1, operate the remote control, wait for the shutter to open, and then resume automatic parking. In this case, it is good for the memory unit 150 to store as parking history the position where the vehicle stopped for a certain period of time or longer, and whether occupant C did not get out of the vehicle during automatic parking. Then, if the behavior of vehicle 1 and occupant C closely matches the parking history, the state management unit 110 may suppress or not provide assistance.

[0105] Here, suppressing assistance means, for example, that the notification unit 160 displays the message "Do you wish to be notified about obstacles?" as text on the screen, without making an audible inquiry. In this way, crew member C will only receive notification when they press the Yes button. If assistance is not needed, they can simply ignore the message, and since there is no voice notification, it is not noisy.

[0106] Furthermore, when the door to garage G is opened with the remote control, it may be detected that there is an obstacle O inside garage G. Therefore, even if it is determined that no assistance should be provided based on the history, the state management unit 110 may continue to perform scene determination, and if there is behavior or detection of an obstacle that differs from the history, it may be made to proactively provide notification about the obstacle.

[0107] For example, if the occupant does not get out when the vehicle stops at a memorized location, the system will only provide a restraining notification displayed on the screen, as this is consistent with the parking history. However, if occupant C performs an action not recorded in the history, such as turning on the hazard lights or turn signals, or fastening their seatbelt buckle, the status management unit 110 may instruct the notification unit 160 to initiate a proactive voice notification.

[0108] Here, disembarking preparation actions refer to actions that lead to disembarking, such as stopping the vehicle with the brake pedal, engaging the parking brake (PB), releasing the brake pedal, shifting the gear to P (parking) or N (neutral), unbuckling the seat belt, and opening the door. The state management unit 110 may, even if the stopping location is the same as a location in the history where the vehicle stopped but the driver did not disembark, if it detects disembarking preparation actions that are not in the history, estimate that there is an unusual situation outside the vehicle and initiate notification regarding the obstacle O. Furthermore, if the vehicle stops at a location different from a location in the history where the vehicle stopped but the driver did not disembark and the parking brake is engaged, the unit may initiate an audible notification without waiting for any further disembarking preparation actions.

[0109] (Suppression of support based on detection) Furthermore, the state management unit 110 may determine whether assistance is necessary based on the detection results. For example, the state management unit 110 may perform face detection or person detection on the image of the obstacle O, and if the obstructing obstacle is a person, it may determine that assistance is unnecessary. This is because if it is a person, it can be expected that they will notice that the vehicle is in the process of parking and leave. Alternatively, the state management unit 110 may monitor the position of the obstacle O, and if the position of the obstacle O is moving, it may determine that assistance is unnecessary because it will no longer be an obstruction after some time has passed.

[0110] Furthermore, the state management unit 110 may determine the timing of its decision based on detection, based on the history. For example, the spatial recognition unit 130 cannot detect the parking position PS even if it detects that the garage G shutter is closed. Therefore, the state management unit 110 may, for example, refrain from providing notification or assistance until the parking time recorded in the history has elapsed, and then detect when the recorded parking time has elapsed and decide whether to provide notification or assistance. In this case, if the spatial recognition unit 130 detects an obstacle O at the parking position, it may also provide an audible notification of the detection of obstacle O and a suggestion for assistance.

[0111] Furthermore, the status management unit 110 may also compare the detection result with the history and make a determination. For example, in a mechanical parking garage or a parking garage with a turntable, occupant C must always get out of the vehicle and operate the control panel. Even in such parking garages, there are times when assistance is needed and times when it is not, so it is advisable for the status management unit 110 to compare the detection result with the history and make a determination.

[0112] For example, the status management unit 110 may track the position of occupant C outside the vehicle and compare the direction of occupant C's movement with the history. In this case, the notification unit 160 may not notify if occupant C is moving in a direction recorded in the history (for example, towards the control panel), but may start an audible notification if occupant C is moving in a different direction from the history (for example, towards the turning position TP). Also, if the status management unit 110 detects an action to prepare to disembark (for example, turning on the hazard lights) after stopping at a location not recorded in the history, it may start proactive assistance before occupant C disembarks.

[0113] (Restriction of support based on probability) The status management unit 110 may determine whether support is needed based on the occurrence of a small number of events or combinations thereof, but it may also comprehensively evaluate the probability of needing support based on a large number of events and, based on the probability, select whether or not to notify and the manner of notification. For example, it may notify if the probability is high and not notify if the probability is low, or it may notify actively if the probability is high and notify in a restrained manner if the probability is low. Alternatively, it may notify actively if the probability is high and not notify if the probability is low, and notify in a restrained manner if the probability is in between.

[0114] The state management unit 110 performs an overall evaluation, for example, by summing up the probability evaluation values ​​and comparing the sum to the first threshold and the second threshold. The state management unit 110 then decides not to provide support if the value is below the first threshold, to provide restrained support if the value is above the first threshold but below the second threshold, and to provide active support if the value is above the second threshold.

[0115] Furthermore, the status management unit 110 may continue to evaluate the probability even after starting support, and may stop support or switch to restrictive support when the evaluation value of the probability falls below a threshold. For example, the status management unit 110 may determine to provide restrictive support when vehicle 1 stops at a location corresponding to the history, and may notify the occupant C of any obstacles in the direction of the parking position using only images before the occupant C gets out of the vehicle. Subsequently, the notification may be stopped when the occupant C stops at a location corresponding to the history (for example, in front of the control panel of a mechanical parking system), or when no obstacles in the direction of the parking position PS (for example, a garage door) are detected.

[0116] Furthermore, the status management unit 110 may continue to evaluate the probability even after stopping the support, and may resume support when the evaluation value of the probability exceeds a threshold. For example, after stopping support when occupant C stops in front of the control panel, the status management unit 110 may decide to actively provide support when occupant C does not return to vehicle 1 but heads towards the parking position (inside the garage), and may provide an audible notification corresponding to the position of the obstacle O inside the garage G.

[0117] (Examples of criteria for evaluating probability) The following are examples of criteria for evaluating probability. The state management unit 110 may start evaluating probability when an obstacle O is detected, when the occupant starts to decelerate, or when automatic parking is initiated. For example, the state management unit 110 evaluates probability highly if the time difference between requesting occupant C to confirm the parking position and detecting brake operation is short. This is because if the time difference is short, it can be presumed that occupant C discovered obstacle O when checking inside the garage G.

[0118] The status management unit 110 highly values ​​the probability if the time between stopping and preparing to disembark is short. This is because a short time difference suggests that occupant C is confident that tidying up is necessary.

[0119] The state management unit 110 adds a probability each time an action is taken to prepare for disembarking, such as operating the brake pedal, engaging the parking brake, releasing the brake pedal, shifting the gear to P or N, releasing the seat belt buckle, or opening the door. This is because if the occupant does not disembark, no assistance is needed, and as the probability of the occupant disembarking increases, the probability of needing assistance also increases. Therefore, the probability increases as the occupant's disembarkation approaches.

[0120] The status management unit 110 evaluates the probability of many occupants C preparing to disembark as lower, especially when all occupants disembark, compared to when only one occupant disembarks. This is because it is reasonable to assume that when many people disembark, they have chosen to disembark before parking and perform remote parking.

[0121] The status management unit 110 highly evaluates the probability when the direction from which the obstacle O was detected coincides with the seat where the passenger was preparing to disembark. This is because it is reasonable to assume that the passenger C on the side closer to the obstacle O disembarks and clears it away.

[0122] The spatial recognition unit 130 identifies the position of the obstacle O based on the camera image, and the state management unit 110 calculates the closest approach distance when the obstacle O comes closest to the vehicle body based on the positional relationship between the parking path and the obstacle O, and evaluates the probability higher the shorter the closest approach distance. If the obstacle O hits the vehicle body, the state management unit 110 evaluates the closest approach distance as zero and sets the probability evaluation value to the upper limit.

[0123] Furthermore, the spatial recognition unit 130 may estimate the size of the obstacle O based on the camera image, and the state management unit 110 may evaluate the probability based on the size of the obstacle O. Specifically, the spatial recognition unit 130 evaluates the length and width of the obstacle O, and the state management unit 110 evaluates the probability based on the larger of the length and width. For example, if the length is less than 1.5 meters, the probability is evaluated as high, and if it is 1.5 meters or more, the probability is evaluated as low. This is because small objects such as bicycles and trash cans can be moved and put away by the occupants, whereas large objects such as garage gates often have a predetermined method of movement and therefore do not require assistance. Also, if the obstacle O is another vehicle and the spatial recognition unit 130 detects a license plate from the camera image, it is not something that the occupants can put away, so it is not subject to assistance.

[0124] Furthermore, the spatial recognition unit 130 may also detect people based on the camera image. The state management unit 110 evaluates the detection result, and if the accuracy of person detection (probability that it is a person) is high, it evaluates the probability as low. This is because if what is detected as an obstacle O is a person, it can be expected that the person will move away from the obstructing area. The spatial recognition unit 130 may also detect faces. When a face is detected, there is a high probability that the person will recognize vehicle 1 and move away from the obstructing area, so the probability may be evaluated as even lower.

[0125] The status management unit 110 evaluates the probability of the vehicle being parked at home (garage G being the home's garage) as lower than when it is not parked at home. This is because, when parking at home, even if there is an obstacle O, such as the gate of garage G, the method of movement is usually predetermined and assistance is not required.

[0126] The state management unit 110 stores in the memory unit 150 the locations where it has detected that the obstacle O is moving up and down, and the probability may be evaluated as low at those locations. Objects that move up and down are things that do not need to be put away, such as the gates of a mechanical parking system or the shutters and doors of a garage G, so it is good to make it difficult to provide unnecessary assistance in such places. Also, at places where people get out of their vehicles to operate machinery, the probability that is added due to getting out of the vehicle should be discounted before evaluation.

[0127] The status management unit 110 stores the location where it detected the vehicle body of vehicle 1 rotating in the storage unit 150, and evaluates the probability of that location as low. This is because, at locations where the driver disembarks to operate the turntable, the probability added by disembarking needs to be discounted, and at locations where the vehicle body rotates, it is easy to misidentify the parking position, making it desirable to avoid providing unnecessary assistance.

[0128] The status management unit 110 evaluates the probability as low if the position where crew member C stopped matches a stopping position recorded in the history. This is because if the crew member stops at a location recorded in the history, it can be presumed that the purpose is not to tidy up, making it difficult to provide assistance.

[0129] The status management unit 110 evaluates the probability lower when it has not detected an obstacle O than when it has detected one. For example, if an obstacle is detected, there is a high probability that the passenger has disembarked to clear it away, whereas if no obstacle is detected, it is doubtful that the passenger has disembarked to clear it away. However, even if obstacle O is not detected, if the probability is increased by other factors such as the passenger's actions and exceeds a threshold, assistance for clearing the obstacle will be initiated. Therefore, if the detection means has not detected an obstacle that would interfere with the detection, the notification may be delayed compared to when an obstacle is detected.

[0130] Furthermore, if no obstacle is detected, notification cannot be provided according to the location of the obstacle. Therefore, the notification may be delayed, the content of the notification may be limited, or the notification may be suppressive. This can also be rephrased as: if the detection means does not detect an obstructing obstacle, it becomes more difficult to provide notification to assist in clearing the obstacle, or it becomes easier to select a suppressive notification, compared to when the detection means has detected an obstructing obstacle.

[0131] (Example of an evaluation process based on probability) The following illustrates the evaluation process based on probability and the postponement of support. Here, we consider the case where a slender object, such as a bicycle air pump, is lying on the floor of garage G. After vehicle 1 begins automatic parking, the spatial recognition unit 130 detects a three-dimensional object at a position where the front camera can capture images of the garage G. The side sonar also detects an obstacle at a position where it can detect the garage. However, since the spatial recognition unit 130 has difficulty detecting low-height objects and the sonar has difficulty detecting thin objects, we assume that in this case neither the spatial recognition unit 130 nor the sonar detected the air pump.

[0132] The driving control unit 140 of the parking assistance ECU 100 slows down the vehicle 1 when it approaches a position where the occupant C can see inside the garage G. The notification unit 160 notifies the occupant to check inside the garage G when it reaches that position. If the driver then applies the brakes and stops the vehicle 1, the state management unit 110 evaluates the probability and determines whether assistance is necessary.

[0133] For example, the status management unit 110 adds to the probability if an obstacle is detected, and does not add to it if there is no obstacle. Also, if garage G is the home garage, it adds to the probability, and does not add to it if it is outside the home. Furthermore, if the time from the notification prompting confirmation to the braking operation is short, it adds to the probability. In addition, the memory unit 150 refers to the past parking history it stores and subtracts from the probability if the vehicle has been parked near the current stopping position more than a predetermined number of times in the past, and does not subtract otherwise. In this case, if the vehicle is parked at home, there is no history of parking in a nearby location, the time from the notification prompting occupant C to confirm to the braking operation is short, and the time from vehicle 1 stopping to the first disembarking preparation is also short, the addition of probability may accumulate and exceed the threshold for determining the start of assistance.

[0134] When starting assistance before the occupants disembark, if an obstructing obstacle is detected, it is advisable to provide notification based on the location of the obstacle. If no obstructing obstacle is detected, it is advisable to provide notification indicating the area of ​​the obstruction. Of course, when an obstacle is detected, it is also acceptable to provide notification of both the area of ​​the obstruction and the location of the obstacle. Notifications can be provided, for example, by displaying them on the navigation screen. This allows the occupants to know where to move to avoid the obstacle before disembarking. It is also advisable to continue providing notifications on the navigation screen even after the occupants have disembarked. This is because if the information is displayed on the navigation screen, occupants remaining in the vehicle can see it and inform those who have disembarked.

[0135] If no obstacles are detected when it is decided to begin assistance, assistance may be initiated even if the location of the obstacles is unknown. Alternatively, only a notice of assistance may be issued, or the start of the notice may be withheld. The notice of assistance may be a restrained notification, such as simply displaying a message on the screen.

[0136] If assistance is provided but no obstacle is detected, it is advisable to detect the occupant who has left the vehicle and provide assistance according to the detection result. For example, the state management unit 110 can instruct the spatial recognition unit 130 to track occupant C who has left the vehicle, and if the position where the occupant stopped is within the parking obstruction range or the exit obstruction range, it may be estimated that there was an obstacle in the obstruction range. In that case, if the position where occupant C stopped is within the obstruction range, the probability is further increased, and if the probability exceeds the second threshold, assistance with voice may be initiated. For example, when occupant C stops and picks up the air pump, it may be announced with the voice, "You are in the obstruction range," and when occupant C leaves the obstruction range, it may be announced with the voice, "You have left the obstruction range." Also, if occupant C places the air pump upright and it is detected as a three-dimensional object, the target for determining whether or not it is an obstruction should be switched from occupant C to the three-dimensional object. Furthermore, the system may continuously detect the occupant's face, and when the occupant's face is detected, that is, when the occupant turns their face towards vehicle 1, it may determine whether the position of the object being judged (air pump) is in a position that would obstruct parking or alighting, and notify occupant C accordingly.

[0137] In other words, even without detecting an obstacle, the state management unit 110 may assume that occupant C has disembarked to tidy up if the probability evaluated from other evaluation factors is high, and may provide assistance. By evaluating the probability in this way and deciding whether or not to provide assistance, the state management unit 110 can increase the probability of providing assistance when it is needed and the probability of not providing assistance when it is not needed.

[0138] (Processing by the parking assist ECU) Next, the processes performed by the parking assistance ECU 100 will be explained using the flowchart in Figure 16. Figure 16 shows an example of the processes performed by the parking assistance ECU 100.

[0139] Figures 17 to 27 illustrate an example of automatic parking by the parking assistance ECU 100 according to the embodiment. In the initial state (step S000) of the flowchart in Figure 16, vehicle 1A is automatically driving along the aisle of the parking lot under the control of the driving control unit 140, and the spatial recognition unit 130 is performing parking space detection processing. In other words, the parking assistance function of the parking assistance ECU 100 of vehicle 1A is operating in an automatic driving mode in which the vehicle autonomously drives for the purpose of searching for a parking space.

[0140] Specifically, in the initial state, vehicle 1 is at the position of vehicle 1A in Figure 17, and the spatial recognition unit 130 detects an empty parking space while detecting the angle and distance of the vehicle body relative to the aisle (white line), and the driving control unit 140 automatically drives vehicle 1 to maintain the distance from the white line (step S000). At this point, the result of the parking space detection is evaluated (step S001), and if no empty parking space is detected (step S001: No), the process returns to step S001.

[0141] When an empty parking space is detected (Step S001: Yes), the status management unit 110 sets a parking route as shown by the arrow in Figure 17 (a route that passes through OP, TP, and TE in order and stops at PS), and also sets the confirmation point OP and the stopping point PP2 (shown in Figure 20), which will be described later (Step S002). Although not shown in the diagram, in Step S002 the notification unit 160 displays the parking route and requests approval for automatic parking. Next, the status management unit 110 determines whether the driver has given approval (Step S003). If the driver has not given approval (Step S003: No), the process returns to Step S001.

[0142] When the driver approves (Step S003: Yes), the state management unit 110 starts automatic parking of vehicle 1A. In other words, it switches from an automatic driving mode that follows a passage to an automatic parking mode that follows a parking route (Step S004). Although not shown in the diagram, the spatial recognition unit 130 stops detecting white lines and parking spaces and changes the obstacle detection range from the range along the passage to the range along the parking route in conjunction with the mode change.

[0143] Figure 17 shows parking spaces (PS1, PS2, PS3, PS5) where other vehicles (1B-1E) are parked, but these are unrelated to vehicle 1A, so parking space PS4 where vehicle 1A is parked will also be simply referred to as the parking space. Assuming that there are abandoned carts CT1 and CT2 inside and outside the parking space, the carts are not lying on their sides but standing upright at the position of their wheels. In other words, carts CT1 and CT2 are abandoned in a position adjacent to the white line. Vehicle 1A has only one occupant, the driver (occupant C), and it is assumed that there are no other people around.

[0144] In this example, the parking assist ECU 100, from the position of vehicle 1A as shown in Figure 17, captures an image of the area near the opening of the parking space PS4 with its front camera, determines that an empty parking space has been detected, and starts automatic parking. In other words, it starts automatic parking at a position where it cannot confirm whether there are any obstacles inside or around the parking space PS4. Therefore, the state management unit 110 sets a confirmation point OP on the parking path to check for obstacles. As shown in Figure 18, the confirmation point OP is a point where the front camera of vehicle 1A is positioned on the extension of the longer side of the parking space on the front side, capturing the entire parking space. Also, obstacles CT2 around the parking space can be located more accurately if detected from closer up.

[0145] Returning to the explanation of Figure 16, the status management unit 110 determines whether vehicle 1A has reached confirmation point OP (step S005). If vehicle 1A has not reached confirmation point OP (step S005: No), the process returns to step S005.

[0146] As shown in Figure 18, when vehicle 1A reaches confirmation point OP (step S005: Yes), the state management unit 110 determines whether there are any obstructing obstacles based on the image captured by the front camera (step S006). If there are no obstructing obstacles (step S006: No), the process proceeds to step S020.

[0147] If an obstruction is detected (Step S006: Yes), the notification unit 160 will notify, "There is an obstacle, so we will stop" (Step S007).

[0148] Next, the status management unit 110 determines whether vehicle 1A has reached the stopping point PP2 (step S008). As shown in Figure 20, the stopping position PP2 is a position where the head of occupant C is located on the extension of the longer side on the front side of the parking space, and the entire parking space is within the field of view. If vehicle 1A has not reached the stopping point PP2 (step S008: No), the process returns to step S008.

[0149] When vehicle 1A reaches stopping point PP2 (step S008: Yes), the driving control unit 140 causes vehicle 1A to stop at stopping point PP2 (step S009). In other words, the vehicle does not stop when it detects an obstacle, but drives to a position where occupant C can see the entire parking space before stopping. The notification unit 160 then notifies occupant C about the movement of the obstacle.

[0150] If we were to set the vehicle 1 to stop as soon as it detects an obstructing obstacle, it would stop when it detects obstacle CT2 on the passageway. For example, as shown in Figure 19, it might stop at position PP1 where it cannot detect obstacle CT1 within the parking space.

[0151] In that case, the parking assist ECU 100 only detects obstacle CT2 on the aisle, so occupant C does not receive assistance based on detection for obstacle CT1 within the parking space. As a result, occupant C may overlook obstacle CT1 within the parking space, return to vehicle 1A, and then detect obstacle CT1 again after resuming automatic parking. In other words, since obstacles are not limited to those detected initially, the point at which vehicle 1 is stopped should be after it has reached a position where all obstacles can be detected.

[0152] Furthermore, as shown in position OP in Figure 18, there is a problem with positions where the front camera can detect the parking space, but the occupant C cannot see the obstacle CT1 within the parking space. In this position, even if the notification unit 160 notifies about obstacles CT2 and CT1, occupant C may only hear half of the notification, get out of the vehicle, clear only the obstacle CT2 that was visible from the driver's seat, and return to vehicle 1, resulting in a notification that obstacle CT2 has not been cleared. Rather than such a position, as shown in Figure 20, it is more efficient and time-consuming to stop vehicle 1A in a position where occupant C can see the entire parking path, notify about all obstacles, and have occupant C clear all obstacles in a single exit.

[0153] Therefore, in this embodiment, the state management unit 110 sets the parking position PP2 to a position where it can visually inspect the parking space, as shown in Figure 20.

[0154] In normal parking, not automated parking, the driver visually checks the parking space before starting to turn the vehicle and reverses into the space. In other words, if the stopping position PP2 is set under the above conditions, the stopping position PP2 will be before the turning start point TS.

[0155] Furthermore, the status management unit 110 may set the stopping position PP2 to any position between the position where the head of occupant C is on the extension of the longer side of the parking space on the front side, and the turning start point TS. In this case, when the vehicle 1A stops, it has not started turning and will not obstruct the passage of other vehicles, so occupant C can start packing up without hesitation.

[0156] In short, there are three conditions for a suitable parking position for clearing obstructions: the vehicle has not started turning, its longest side is roughly parallel to the direction of the road or passage, the detection means can detect the parking position, and the occupants can see the parking position. The best position is one that satisfies all of these conditions, but a position that satisfies at least one of them is more suitable for clearing than a position that does not. For example, even if the occupants cannot see the parking position, if the detection means can detect it, there is a high probability that the occupants can be guided to the obstacle. Even if the obstacle cannot be detected from the parking position, it is sufficient to remember the location of the obstacle, and if the occupants can see the obstacle at the parking position, there is a high probability that the obstacle can be cleared. Even if the vehicle is blocking the road diagonally, if the occupants can be guided to the obstacle and cleared in a short time, there is a high probability that traffic will not be obstructed. Thus, none of the conditions are essential, and it is better to satisfy as many conditions as possible.

[0157] Returning to the point when the obstacle is detected, let's explain the notification. When the spatial recognition unit 130 detects an obstacle, the state management unit 110 determines whether the obstacle is within an obstruction range that would hinder parking or passenger disembarkation. If the determination is Yes (step S006: Yes), the notification unit 160 notifies, for example, "There is an obstacle, so we will stop." However, as mentioned above, the vehicle does not stop immediately after the notification, but proceeds to the stopping position PP2 and stops.

[0158] When vehicle 1A stops, the notification unit 160 may display the detected obstacles (carts CT1 and CT2) surrounded by a frame, as shown in Figure 21, and notify the user that they are obstructing obstacles. Alternatively, the notification unit 160 may display the boundary line BL, as shown in Figure 22, to indicate the obstructing area. In this case, the notification unit 160 may use different display methods for the parking obstruction area and the disembarking obstruction area to make them distinguishable.

[0159] Figure 22 illustrates the area that would obstruct the driver's exit, corresponding to the presence of occupant C in the driver's seat. When the status management unit 110 asks the driver whether to switch to remote parking, the notification unit 160 may blink the exit obstruction area DR, and if the answer is Yes, it may turn off the display of the exit obstruction area DR.

[0160] The parking obstruction area TR may be represented as a surface, as shown by the hatching in Figure 22, or as a boundary line BL between the obstruction area and the non-obstruction area, as shown by the dotted line in Figure 22, or both may be used in combination.

[0161] Furthermore, when the notification unit 160 displays the boundary line BL, it may choose to display only the boundary line that overlaps with the obstacle, or the boundary line that is close to the obstacle. The notification unit 160 may also notify the destination and distance of the obstacle's movement. In this case, since the spatial recognition unit 130 has detected the parking space, the notification unit 160 may notify based on the parking space, for example, by saying, "Please move the obstacle outside the parking space."

[0162] Returning to Figure 16, the explanation continues. After step S009, the status management unit 110 determines whether crew member C has disembarked (step S010). If crew member C has not disembarked (step S010: No), the process returns to step S010.

[0163] When occupant C exits vehicle 1A (Step S010: Yes), assistance with tidying up begins. The spatial recognition unit 130 tracks occupant C and obstacles (carts CT1, CT2) (Step S011).

[0164] Next, the status management unit 110 determines whether both the occupant C and the obstacle have left the parking obstruction area PR (step S012). If at least one of the occupant C and the obstacle has not left the parking obstruction area PR (step S012: No), the process returns to step S011.

[0165] Although not shown in the diagram, steps to determine the direction of occupant C's face and to notify whether the occupant has left the parking obstruction area PR may be inserted into the loop that repeats step S012. For example, if the occupant is in the parking obstruction area when their face is turned towards the vehicle, a negative response is notified; if they are outside the area, an affirmative response is notified. In this way, the occupant can know whether they are inside or outside the parking obstruction area simply by turning their head towards the vehicle.

[0166] When the parking assist ECU 100 assists occupant C who has exited the vehicle, the state management unit 110 may provide assistance through communication using lights or sounds. For example, the state management unit 110 may control the notification unit 160 and pre-inform occupant C of a rule (code) that if both left and right lamps (LF, RF, LB, RB) of vehicle 1A light up, it means Yes (affirmative response), and if only one of the left or right lamps lights up, it means No (negative response), and then respond to occupant C's actions with the lights. Alternatively, the state management unit 110 may pre-inform occupant C of a code for voice notification, and respond with voice such as Yes (affirmative response) if the machine sound is "fan fan" twice, and No (negative response) if the machine sound is "fan" once.

[0167] When occupant C leaves the parking obstruction area PR with the obstacle (cart CT2) (step S012: Yes), the notification unit 160 signals affirmation by lighting both lamps (LF, RF, LB, RB) as shown in Figure 23, indicating that the obstacle has left the parking obstruction area PR (step S013). At this time, it is desirable to simultaneously provide an audible notification (for example, outputting two mechanical sounds of "fan fan") so that occupant C can understand the notification even if their back is turned to vehicle 1A.

[0168] Next, the status management unit 110 determines whether occupant C is approaching vehicle 1A (step S014). If occupant C is not approaching vehicle 1A (step S014: No), the process returns to step S014.

[0169] If occupant C is approaching vehicle 1A (step S014: Yes), the state management unit 110 determines whether there is an obstruction (step S015). As shown in Figure 26, if there is no obstruction (step S015: No), the process proceeds to step S017.

[0170] The reason why the judgment conditions differ in Step 12 and Step 15 is that the purpose of the notification is different. The detection means detects the direction of the occupant's face or head, or the direction of the occupant's movement, and if it is moving away from the vehicle, it only needs to inform whether the movement is sufficient, so the notification means informs whether or not the occupant has left the interference range. After leaving the interference range, if the occupant turns towards the vehicle, it only needs to inform whether automatic parking can be resumed, so the notification means informs whether or not there is an obstacle within the interfering range.

[0171] Let's continue the explanation of step S015. As shown in Figure 24, if there is an obstacle in the way when occupant C approaches vehicle 1A, proceed to step S016 (step S015: Yes). For example, as shown in Figure 24, if the obstructing obstacle (cart CT1) is to the right of vehicle 1A, the notification unit 160 will notify the denial by lighting only the right lamps (RF, RB) and convey that automatic parking cannot be resumed (step S016). At this time, the notification unit 160 will convey that the obstructing obstacle (cart CT1) is to the right of vehicle 1A by lighting the right lamps (RF, RB). After that, return to step S011. In this denial response, a single mechanical sound "fan" may be used in conjunction with the denial response, or an announcement "There is an obstacle in the parking space" may be made by voice.

[0172] After returning to step S011, as shown in Figure 25, the notification unit 160 notifies affirmation (step S013) when the obstacle (cart CT1 that was in the parking lot) leaves the parking obstruction area PR (step S012: Yes). Subsequently, as shown in Figure 26, when occupant C approaches vehicle 1A with no obstructing obstacles (step S014: Yes, step S015: No), the notification unit 160 notifies affirmation by lighting both lamps, indicating that automatic parking can be resumed (step S017). Then, proceed to step S018. This affirmation response may also be given using two "fan fan" sounds, or by announcing "Automatic parking can be resumed" by voice.

[0173] Here, we have shown an example of notification using lights and speakers in vehicle 1, but notification can also be given by vibrating or emitting a sound from a device carried by occupant C (e.g., a smart key or mobile phone). In that case, regardless of the device, a common code rule (for example, two vibrations for Yes, one vibration for No) will help avoid misunderstandings.

[0174] When assisting an occupant who has left the vehicle, the parking assist ECU 100 may have its detection means detect the direction of the occupant's face or head, or the occupant's movement, and change the notification means according to the direction. For example, the notification means may differ depending on whether the occupant's face or head is facing away from the vehicle, or whether the occupant is facing towards the vehicle or has stopped moving. If the occupant is facing away from the vehicle, the notification may be given by means including at least one of voice or vibration of a terminal carried by the occupant. If the occupant is facing towards the vehicle or has stopped moving, the notification may be given by means including the vehicle's lights.

[0175] Returning to Figure 16, let's continue the explanation. If all obstructing objects have been removed (Step S015: No), the notification unit 160 will give an affirmative response indicating that automatic parking can be resumed (Step S017). Although not shown in the diagram, the notification unit 160 will also give a notification inquiring about the resumption of automatic parking when giving the affirmative response. For example, the notification unit 160 may display a message inquiring about the resumption of automatic parking on the HMI device 20 (e.g., the navigation screen), or it may send a signal to a terminal carried by occupant C, causing the terminal to output a message inquiring about the resumption of automatic parking. The message inquiring about the resumption may be a code such as three vibrations. When giving a notification via a mobile terminal, the affirmative response may be omitted, and only the message inquiring about the resumption may be output.

[0176] After the inquiry, the status management unit 110 determines whether the occupant C has approved the resumption (step S018). For example, occupant C may return to the driver's seat and instruct the automatic parking to resume by entering "Yes" in response to "Do you want to resume automatic parking?" displayed on the navigation screen. Alternatively, occupant C may instruct the automatic parking to resume without returning to the driver's seat by operating a mobile device. If the resumption is not approved (step S018: No), the process returns to step S018.

[0177] When instructed to resume automatic parking (S018: Yes), the state management unit 110 resumes automatic parking as shown in Figure 27 (step S019). Figure 27 shows the case where occupant C approves the resumption of automatic parking without returning to the driver's seat. The driving control unit 140 drives vehicle 1A toward parking position PS, and the state management unit 110 determines whether vehicle 1A has reached parking position PS (step S020).

[0178] If vehicle 1A has not reached the parking position PS (step S020: No), the system returns to step S020. If vehicle 1A has reached the parking position PS (step S020: Yes), the driving control unit 140 parks vehicle 1A at the parking position PS, the notification unit 160 notifies that parking is complete, and then the status management unit 110 terminates all functions of the parking assist device (step S021).

[0179] Although embodiments of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms. Furthermore, various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. Moreover, this embodiment is included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of symbols]

[0180] 1 vehicle 2 cameras 10 Operating device 20 HMI device 30 Vehicle control device 40 Navigation System 50 Sonar ECU 100 Parking Assist ECU 101 CPU 102 ROM 103 RAM 104 I / O 105 IMP 106 Communication I / F 110 Status Management Department 120 Image Processing Unit 130 Spatial recognition section 140 Driving Control Unit 150 Storage section 160 Hochi Department

Claims

1. A parking assist device that automatically parks a vehicle in a designated parking position, A storage means for storing the parking route to the aforementioned parking position, A detection means that detects at least the area around the vehicle and obtains detection information, A driving control means for controlling the driving of the aforementioned automatic parking vehicle, A notification means for notifying the occupants of the vehicle, A support control means for suspending the automatic parking drive based on at least one of the detection information and the actions of the occupant, Equipped with, If the detection means detects an obstacle that would interfere with the automatic parking, or if the occupant stops the vehicle, at least one of these, the support control means suspends the automatic parking operation. The notification means provides the occupant with notification regarding the suspension of the automatic parking drive. When the occupant stops the vehicle, If the detection means detects the obstructing object, or When the aforementioned occupant performs an action to prepare to disembark, The automatic parking system will be suspended from driving. Parking assist system.

2. The aforementioned support control means is If the detection means detects an obstruction, the automatic parking vehicle will be suspended at a predetermined position. The aforementioned predetermined position is, A position on the road or on the passageway, in front of the entrance to the parking space that faces perpendicular to the road or the passageway of the parking lot on which the vehicle travels, The detection means is in a position where it can detect the parking position, From a position in which the occupant can visually confirm the parking position, It is a position that corresponds to at least one of the following: The parking assistance device according to claim 1.

3. The aforementioned support control means is The orientation of the vehicle is approximately parallel to the road. and, When the occupant can see the parking position, The notification means is instructed to issue a notification requesting confirmation of the parking position, and the driving control means is instructed to decelerate the vehicle, at least one of the above. The parking assistance device according to claim 2.

4. The aforementioned obstructing obstacles are those located within the obstruction range. The aforementioned obstruction range is a range that falls within at least one of the following categories: a parking obstruction range in which the closest approach distance to the vehicle body during automatic parking falls below a safety threshold, or a disembarking obstruction range in which the distance to the door falls below a disembarking clearance threshold when the occupant disembarks. The support control means determines whether an obstacle is obstructive based on the positional relationship between the position of the obstacle detected by the detection means and the obstruction range. The parking assistance device according to claim 1.

5. The system further includes a means for detecting when the aforementioned occupant is seated, The support control means sets the disembarkation obstruction range corresponding to the seat where the occupant is seated, and does not set the disembarkation obstruction range for seats where the occupant is not seated. The parking assistance device according to claim 4.

6. It further includes a smart key detection means for detecting the location of the smart key, The aforementioned support control means sets an exit obstruction range corresponding to the driver's seat when the smart key is inside the vehicle, and does not set an exit obstruction range corresponding to the driver's seat when the smart key is outside the vehicle. The parking assistance device according to claim 4.

7. The aforementioned support control means is At least one of the aforementioned margin threshold and the aforementioned disembarking space threshold is set according to at least one of the following: opening and closing of the side mirror, or environmental conditions. The aforementioned environmental conditions include at least one of the following: vehicle tilt, vehicle sway, outside temperature, and outside wind speed. The parking assistance device according to claim 4.

8. The notification means, when there is an obstacle in the area that obstructs vehicle disembarking and there is no obstacle in the area that obstructs parking, Notification regarding occupants whose disembarkation is prevented by the aforementioned obstacle, Notification that the vehicle can begin automatic parking once the aforementioned occupant disembarks. A notification proposing that the aforementioned crew member disembark, Notification of a means for initiating automatic parking from outside the vehicle, To provide at least one of the following notifications: The parking assistance device according to claim 4.

9. The aforementioned notification means is, In the aforementioned automatic parking and when passengers exit the vehicle, a notification is provided indicating the area where an obstacle would be in the way. Notification indicating the aforementioned obstructing obstacles, A notification indicating the location to remove the aforementioned obstructing object. A notification indicating the distance to be moved until the aforementioned obstructing object is no longer an obstruction. Notification of the presence or absence of the aforementioned obstructive obstacles, A notification indicating the direction of the aforementioned obstructing obstacle, A notification indicating that the aforementioned obstacle is no longer in the way. Perform at least one of the following: The parking assistance device according to claim 1.

10. The aforementioned support control means is If the detection means does not detect the obstructing object, When the detection means detects the obstructing object, To make it difficult to provide the aforementioned notification, or to make it easier to select a more restrained notification, The parking assistance device according to claim 1.

11. The aforementioned support control means is If the detection means does not detect the obstructing object, The detection means is made to detect an occupant who has left the vehicle. The position of the occupant is considered to be the position of the obstructing object, and the notification means is instructed to make the notification. The parking assistance device according to claim 1.

12. The aforementioned notification means is, If the detection means does not detect the obstructing obstacle, it will provide notification indicating the area where the obstacle would be an obstruction during automatic parking and / or passenger disembarkation. If the detection means detects the obstructing obstacle, it will provide notification based on the location of the obstacle. The parking assistance device according to claim 1.

13. The support control means evaluates the probability that the notification is required, Based on the probability, the support control means performs at least one of the following: selecting whether or not to provide notification, and selecting the manner of notification. The choice of whether or not to provide notification is to provide notification if the probability is high, and not to provide notification if the probability is low. The selection of the notification method is to provide proactive notification if the probability is high, and to provide restrained notification if the probability is low. The parking assistance device according to claim 1.

14. The support control means evaluates the probability based on at least one of the following: the occupant's movements, the detection result of the detection means, location information, and past parking history. The aforementioned support control means is If the time from the vehicle stopping to the action of preparing to disembark is short, the evaluation will be more likely than if the time is long. If the location of the obstacle is close to the parking path, the evaluation will be more likely than if the location is far from the parking path. If the aforementioned obstacle is detected, the evaluation should be made to increase the probability compared to the case where the aforementioned obstacle is not detected. If the obstacle is a person, the evaluation should be such that the probability is lower than if the obstacle is not a person. If the parking location is a home garage, the evaluation should be based on a lower probability than if the parking location is not a home garage. Determine whether the location where the occupant stopped outside the vehicle substantially matches the location where the occupant stopped recorded in the history, and if they substantially match, evaluate the probability to be lower than if they did not substantially match. Perform at least one of the following evaluations: The parking assistance device according to claim 13.

15. The detection means detects the direction of the occupant's face or the occupant's position when they have left the vehicle. When the position of the aforementioned occupant changes from within the range where an obstacle would be an obstruction to outside that range, Alternatively, when the speed at which the occupant's position moves away from the vehicle decreases, Or, when the occupant's face is turned towards the vehicle, In any of the following situations, a notification based on the crew's position will be issued. The parking assistance device according to claim 1.

16. The detection means detects at least one of the following: the direction of the occupant's face when they have left the vehicle, or the direction of the occupant's movement. The notification means provides different notification methods depending on whether the direction of the face or the direction of movement is away from the vehicle or toward the vehicle. The notification method when the orientation of the face or the direction of movement is away from the vehicle includes at least one of voice or vibration of a terminal carried by the occupant. The notification method when the orientation of the face or the direction of movement is toward the vehicle is a notification that includes control of the vehicle's lights. The parking assistance device according to claim 1.

17. The detection means detects at least one of the following: the direction of the occupant's face when he or she is outside the vehicle, or the direction of the occupant's movement. The aforementioned notification means is, The content of the notification is made different depending on whether the orientation of the face or the direction of movement is away from the vehicle or toward the vehicle. The parking assistance device according to claim 1.

18. A parking assistance method using a parking assistance device that includes a storage means for storing a parking route to a predetermined parking position and automatically parks a vehicle in the said parking position, A detection step of detecting at least the area around the vehicle and obtaining detection information, A driving control step that controls the driving of the automatic parking system, A notification step for providing information to the occupants of the vehicle, A support control step that suspends the automatic parking drive based on at least one of the detection information and the occupant's actions, Includes, If the detection step detects an obstacle that would interfere with the automatic parking, or if the occupant stops the vehicle, at least one of these, the support control step suspends the automatic parking drive. The notification step involves notifying the occupant about the suspension of the automatic parking drive. When the occupant stops the vehicle, If the obstructing object is detected in the detection step, or When the aforementioned occupant performs an action to prepare to disembark, The automatic parking system will be suspended from driving. Parking assistance methods.

19. The aforementioned support control step is: If an obstruction is detected in the aforementioned detection step, the automatic parking process is suspended at a predetermined position. The aforementioned predetermined position is, A position on the road or passageway that the vehicle travels on, in front of the entrance to the parking space that faces perpendicular to the road or passageway of the parking lot where the vehicle travels, A position in which the parking position can be detected in the above detection step, From a position in which the occupant can visually confirm the parking position, It is a position that corresponds to at least one of the following: The parking assistance method according to claim 18.

Images