Parking assistance method and driver assistance system

Abstract

To make it possible to perform parking support of an own vehicle to a target parking position based on an object around the target parking position even when the own vehicle never parks at the target parking position.SOLUTION: A parking support method comprises the steps of: detecting the position of an object around a vehicle 1a (S11); acquiring object positional data from a second storage device when no object positional data indicating relative positional relationship between the object around a target parking position and the target parking position exists in a first storage device inside of the vehicle 1a and when the object positional data exists in the second storage device outside of the vehicle 1a (S22 - S27); calculating relative positional relationship between the target parking position 30 and current position of the vehicle 1a based on the object positional data and the position of the object therearound (S14); calculating a traveling path from the current position of the vehicle 1a to the target parking position based on the relative positional relationship between the target parking position and the current position of the vehicle 1a (S15); and supporting parking of the vehicle 1a to the target parking position based on the traveling path (S16).SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] This invention relates to a parking assistance method and a driver assistance system. [Background technology]

[0002] Patent Document 1 describes a driving control device that extracts and stores landmarks from images taken of the area around the target parking position when the vehicle has been parked in the target parking position in the past, calculates the relative position of the target parking position to the vehicle based on the positions of the stored landmarks and the positions of landmarks extracted from images taken of the area around the vehicle during automatic parking, and automatically moves the vehicle to the target parking position based on the calculated relative position. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2017-138664 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, the driving control device described in Patent Document 1 has a problem in that the parking assistance function cannot be used when parking in a parking space where the vehicle has never been parked before. For example, the parking assistance function cannot be used when visiting a place for the first time because the target parking position is not stored in the memory. The present invention aims to enable assistance in parking a vehicle at a target parking location, even if the vehicle has never been parked there before, by using landmarks present around the target parking location. [Means for solving the problem]

[0005] In a parking assistance method according to one aspect of the present invention, the positions of surrounding objects, which are objects present around the vehicle, are detected. If the first storage device located inside the vehicle does not contain object position data representing the relative positional relationship between the objects present around the target parking position and the target parking position, but object position data exists in the second storage device located outside the vehicle, the object position data is obtained from the second storage device. Based on the object position data obtained from the second storage device and the positions of the surrounding objects, the relative positional relationship between the target parking position and the vehicle's current position is calculated. Based on the relative positional relationship between the target parking position and the vehicle's current position, a driving trajectory from the vehicle's current position to the target parking position is calculated, and parking of the vehicle to the target parking position is assisted based on the driving trajectory. [Effects of the Invention]

[0006] According to the present invention, even if the target parking location is one in which the vehicle has never been parked before, the vehicle can be assisted in parking the vehicle at the target parking location based on landmarks present around the target parking location. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram showing the general configuration of the driver assistance system according to the first embodiment. [Figure 2] (a) is an explanatory diagram of an example of the process for storing learned target data, and (b) is an explanatory diagram of an example of the process when parking assistance is implemented. [Figure 3] Figure 1 is a block diagram showing an example of the controller's functional configuration. [Figure 4] This is a flowchart illustrating an example of the process for storing learned target data. [Figure 5] This is a flowchart illustrating an example of the process when implementing parking assistance. [Figure 6] This is a sequence diagram of an example of the processing of the driver assistance system according to the first embodiment. [Figure 7] This is a sequence diagram of an example of the processing of the driver assistance system according to the second embodiment. [Figure 8] This is a schematic diagram showing the general configuration of the driver assistance system according to the third embodiment. [Figure 9] This is a sequence diagram of an example of the processing of the driver assistance system according to the third embodiment. [Figure 10] This is a sequence diagram of an example of the processing of the driver assistance system according to the fourth embodiment. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. Note that the drawings are schematic and may differ from actual ones. Furthermore, the embodiments of the present invention described below are illustrative examples of devices and methods for realizing the technical concept of the present invention, and the technical concept of the present invention is not limited to the structure, arrangement, etc., of the components described below. The technical concept of the present invention can be modified in various ways within the technical scope defined by the claims described in the patent claims.

[0009] (First Embodiment) (composition) Figure 1 is a schematic diagram showing the general configuration of the driver assistance system of the first embodiment. The driver assistance system comprises a parking assistance device 10 installed in the first vehicle 1a and the second vehicle 1b, a first mobile terminal device 2a used by the user of the first vehicle 1a (e.g., the driver or other occupant), and a second mobile terminal device 2b used by the user of the second vehicle 1b. The parking assistance device installed in the second vehicle 1b has the same configuration as the parking assistance device 10 installed in the first vehicle 1a, and is therefore omitted from the description in Figure 1. Furthermore, in the following explanation, the first vehicle 1a and the second vehicle 1b may be collectively referred to as "vehicle 1." The first mobile terminal device 2a and the second mobile terminal device 2b may be collectively referred to as "mobile terminal device 2."

[0010] The parking support device 10 supports the vehicle 1 to travel along a target travel trajectory from the current position of the vehicle 1 to the target parking position. For example, an automatic driving may be performed to control the vehicle 1 to travel to the target parking position along the target travel trajectory of the vehicle 1. The automatic driving to control the vehicle 1 to travel to the target parking position along the target travel trajectory of the vehicle 1 means controlling all or part of the steering angle, driving force, and braking force of the host vehicle to automatically perform all or part of the travel along the target travel trajectory of the vehicle 1. Further, the parking of the vehicle 1 may be supported by displaying the target travel trajectory and the current position of the vehicle 1 on a display device visible to the occupant of the vehicle 1.

[0011] The positioning device 11 measures the current position of the vehicle 1. The positioning device 11 includes, for example, a global navigation satellite system (GNSS) receiver. [[ID=⑥]]The human machine interface (HMI) 12 is an interface device that exchanges information between the parking support device 10 and the occupant. The HMI 12 includes a display device visible to the occupant of the vehicle 1, a speaker, a buzzer, and an operator. The shift switch (shift SW) 13 is a switch for the driver or the parking support device 10 to switch the shift position of the vehicle 1.

[0012] The external sensor 14 detects an object within a predetermined distance range from the vehicle 1. The external sensor 14 detects the surrounding environment of the vehicle 1, such as the relative position between the object existing around the vehicle 1 and the vehicle 1, the distance between the vehicle 1 and the object, and the direction in which the object exists. The external sensor 14 may include, for example, a camera that photographs the surrounding environment of the vehicle 1. In the following description, the camera included in the external sensor 14 is simply referred to as a "camera". The external sensor 14 may include a distance measuring device such as a laser range finder, a radar, or a LiDAR (Light Detection and Ranging).

[0013] It should be noted that there is a misrepresentation in the original text where "ヒューマンマシンインタフェース(HMI)12" is translated as "⑥" in the English version. It should be translated as "The human machine interface (HMI) 12". This correction has been made in the provided translation.The vehicle sensor 15 detects various information (vehicle information) of the vehicle 1. The vehicle sensor 15 may include, for example, a vehicle speed sensor that detects the traveling speed of the vehicle 1, a wheel speed sensor that detects the rotational speed of each tire of the vehicle 1, a three-axis acceleration sensor (G sensor) that detects the acceleration (including deceleration) of the vehicle 1 in three axial directions, a steering angle sensor that detects the steering angle, a steering angle sensor that detects the steering angle of the steering wheel, a gyro sensor, and a yaw rate sensor. The communication device 16 can communicate with the portable terminal device 2 used by the user of the vehicle 1. Specifically, the communication device 16 of the first vehicle 1a can communicate with the first portable terminal device 2a, and the communication device 16 of the second vehicle 1b can communicate with the second portable terminal device 2b. The communication method between the communication device 16 and the portable terminal device 2 may be, for example, wifi (registered trademark), bluetooth (registered trademark), USB (registered trademark), etc. The communication device 16 may also be able to communicate with an external communication device of the vehicle 1. The communication method by the communication device 16 may be, for example, wireless communication via a public mobile phone network, vehicle-to-vehicle communication, etc. The communication device 16 and the portable terminal device 2 are an example of the "interface" described in the claims.

[0014] The controller 17 is an electronic control unit that performs parking support control of the vehicle 1. The controller 17 includes a processor 20 and peripheral components such as a storage device 21. The processor 20 may be, for example, a CPU or an MPU. The storage device 21 may include a semiconductor storage device, a magnetic storage device, an optical storage device, etc. The functions of the controller 17 described below are realized, for example, when the processor 20 executes a computer program stored in the storage device 21. Note that the controller 17 may be formed by dedicated hardware for executing each information processing described below. The steering actuator 19a controls the steering direction and the steering amount of the steering mechanism of the vehicle 1 according to the control signal of the controller 17. The accelerator actuator 19b controls the accelerator opening of the drive device, which is an engine or a drive motor, according to the control signal of the controller 17. The brake actuator 19c operates the braking device according to the control signal of the controller 17.

[0015] Next, the parking assistance control by the parking assistance device 10 will be explained. Refer to Figure 2(a). When using the parking assistance by the parking assistance device 10, targets present around the target parking position 30 are extracted and stored in the storage device 21 in advance. In the following explanation, targets around the target parking position 30 stored in the storage device 21 will be referred to as "learned targets". In Figure 2(a), the circle plots represent learned targets. The data of learned targets stored in the storage device 21 is an example of the "target position data" described in the claims. For example, when the vehicle 1 is located near the target parking position 30 (for example, when the driver manually parks the vehicle 1 at the target parking position 30), the parking assist device 10 detects objects around the target parking position 30 from the surrounding image obtained by taking a picture of the area around the vehicle 1 with a camera. For example, it detects objects on the captured image obtained by the camera, such as the edges and corners of road markings, road boundaries, and obstacles, as edge points where the brightness of adjacent pixels changes by a predetermined amount or more, or points with distinctive shapes (feature points).

[0016] When storing learned targets in the memory device 21, for example, the driver operates the "parking position learning switch" provided as an operator for the HMI 12. The parking assist device 10 stores the relative positional relationship between the learned targets and the target parking position 30. For example, the driver may input that the current position of vehicle 1 is the target parking position 30 by operating the HMI 12. The parking assist device 10 may determine the relative positional relationship between the learned targets and the target parking position 30 based on the position of the learned targets detected when vehicle 1 is located at the target parking position 30. When storing the relative positional relationship between the learned targets and the target parking position 30, for example, the coordinates of the learned targets and the target parking position 30 on a coordinate system with a fixed point as the reference point (hereinafter referred to as the "map coordinate system") may be stored. In this case, the current position on the map coordinate system measured by the positioning device 11 when vehicle 1 is located at the target parking position 30 may be stored as the target parking position 30. Alternatively, instead of a map coordinate system, the relative positions of each target parking location 30 to each learned landmark may be stored.

[0017] Figure 2(b) is an explanatory diagram of an example of the process when parking assistance is performed. The parking assistance device 10 performs parking assistance for vehicle 1 when vehicle 1 is located near the target parking position 30. For example, the parking assist device 10 determines whether the driver has performed a shift operation to switch the vehicle 1 forward or backward when the vehicle 1 is located near the target parking position 30. Figure 2(b) shows an example of the vehicle 1 performing a maneuver at the maneuver position 31. The parking assist device 10 may determine that a shift operation for maneuvering has been performed when the shift position switches from the drive range (D range) to the reverse range (R range), or from the R range to the D range, and may start parking assistance. Parking assistance may also be started when the driver operates the "parking assistance activation switch" provided on the HMI 12 when the vehicle 1 is at position 32 near the target parking position 30.

[0018] The parking assist device 10 extracts objects around the vehicle 1 from the surrounding image obtained by taking pictures of the area around the vehicle 1 with a camera. In the following description, objects around the vehicle 1 that are extracted when parking assist is performed will be referred to as "surrounding objects". In Figure 2(b), the triangular plots represent surrounding objects. The parking assist device 10 matches learned targets stored in the memory device 21 with surrounding targets and associates identical feature points. Then, based on the relative positional relationship between the surrounding targets detected during parking assist and the vehicle 1, and the relative positional relationship between the learned targets associated with the surrounding targets and the target parking position 30, it calculates the relative position of the vehicle 1 with respect to the target parking position 30.

[0019] For example, the parking assistance device 10 may calculate the position of the target parking position 30 on a coordinate system (hereinafter referred to as the "vehicle coordinate system") that is based on the current position of the vehicle 1. For example, if the coordinates of learned targets in the map coordinate system and the target parking position 30 are stored in the storage device 21, the coordinates of the target parking position 30 on the map coordinate system may be converted to coordinates on the vehicle coordinate system based on the positions of surrounding targets detected when parking assistance is performed and the positions of learned targets in the map coordinate system. Alternatively, the current position of vehicle 1 in the map coordinate system may be determined based on the positions of surrounding objects detected during parking assistance and the positions of learned objects in the map coordinate system. The relative position of vehicle 1 with respect to the target parking position 30 may then be calculated from the difference between the coordinates of vehicle 1 in the map coordinate system and the coordinates of the target parking position 30. The parking assist device 10 calculates a target driving trajectory 33 from the vehicle's current position 32 to the target parking position 30, based on the vehicle's relative position to the target parking position 30. The parking assist device 10 performs parking assist control of the vehicle 1 based on the calculated target driving trajectory 33.

[0020] The functional configuration of controller 17 will be explained in more detail. Refer to Figure 3. When the parking position learning switch is operated by the occupant, the HMI control unit 40 outputs a map generation command to the map generation unit 45 to store the learned targets in the memory device 21. The HMI control unit 40 determines whether or not the occupant has performed a shift operation for reversing and outputs the determination result to the parking assistance control unit 41. In addition, when it detects that the parking assistance activation switch of HMI 12 has been operated by the occupant, it outputs the detection result to the parking assistance control unit 41. The parking assistance control unit 41 determines whether the vehicle 1 is located near the target parking position 30. For example, it determines whether the distance between the vehicle 1 and the target parking position 30 is less than or equal to a predetermined distance. For example, the positioning device 11 may be used to determine whether the distance between the vehicle 1 and the target parking position 30 is less than or equal to a predetermined distance. Alternatively, the feature quantities of targets near the target parking position 30 may be stored in advance, and the external sensor 14 may be used to determine whether or not it detects a target with similar feature quantities. When vehicle 1 is positioned near the target parking position 30 and the HMI control unit 40 detects operation of the parking assist activation switch, the parking assist control unit 41 starts parking assist control. Alternatively, when vehicle 1 is positioned near the target parking position 30 and a shift operation for maneuvering is detected, parking assist control is started.

[0021] When parking assistance control is started, the parking assistance control unit 41 outputs a parking position calculation command to the matching unit 47, which calculates the position of the target parking position 30 in the vehicle coordinate system. It also outputs a driving trajectory calculation command to the target trajectory generation unit 48, which calculates the target driving trajectory from the current position of the vehicle 1 to the target parking position 30, and the target vehicle speed profile for the vehicle 1 to travel along the target driving trajectory. The target trajectory generation unit 48 calculates the target driving trajectory and target vehicle speed profile from the vehicle 1's current position to the target parking position 30 and outputs them to the parking assistance control unit 41. Known methods used in automatic parking systems can be applied to calculate the target driving trajectory and target vehicle speed profile. The parking assistance control unit 41 outputs information on the target driving trajectory and the current position of vehicle 1 to the HMI control unit 40. If the target driving trajectory includes a U-turn, it outputs information on the U-turn position to the HMI control unit 40. The HMI control unit 40 displays the target driving trajectory, the current position of vehicle 1, and the U-turn position on the HMI 12. Furthermore, the parking assistance control unit 41 outputs a steering control command to the steering control unit 49 to control the steering so that the vehicle 1 travels along the calculated target driving trajectory, and outputs a vehicle speed control command to the vehicle speed control unit 50 to control the vehicle speed of the vehicle 1 according to the calculated target vehicle speed profile.

[0022] The image conversion unit 42 converts the image captured by the camera into an overhead view image (around view monitor image) as seen from a virtual viewpoint directly above the vehicle 1. The image conversion unit 42 converts the captured image into an overhead view image at predetermined intervals (for example, every time the vehicle 1 travels a predetermined distance (e.g., 50 cm) or a predetermined time (e.g., 1 second)), and generates an surrounding image, which is an image of the area around the vehicle 1, by accumulating the converted overhead view images along the travel path of the vehicle 1. The self-position calculation unit 43 calculates the current position of vehicle 1 on the map coordinate system by dead reckoning based on vehicle information output from vehicle sensor 15.

[0023] When storing learned targets in the memory device 21, the target detection unit 44 detects learned targets from the surrounding image output from the image conversion unit 42. Also, when performing parking assistance control, it detects surrounding targets from the surrounding image output from the image conversion unit 42. For example, the target detection unit 44 may detect feature points of targets and their image features as information about learned targets and surrounding targets. For feature point detection and image feature calculation, methods such as SIFT, SURF, ORB, BRIAK, KAZE, and AKAZE can be used. When storing learned targets in the storage device 21, the driver operates the parking position learning switch to manually park the vehicle 1 at the target parking position. At this time, the map generation unit 45 receives a map generation command from the HMI control unit 40. The map generation unit 45 generates map data 46 by storing the location information and feature quantities of the targets output from the target detection unit 44, along with the current position of the vehicle 1 synchronized with these, in the storage device 21 as learned targets. Alternatively, the position of the target in the map coordinate system may be calculated based on the current position of the vehicle 1 and stored as the target's location information. Furthermore, when the driver inputs to the parking assist device 10 that the current position of vehicle 1 is the target parking position 30, the map generation unit 45 receives the current position of vehicle 1 on the map coordinate system from the positioning device 11 or the self-position calculation unit 43 and stores it as the target parking position 30 in the map data 46. In other words, it stores the relative positional relationship between the target parking position 30 and multiple feature points as map data 46.

[0024] Subsequently, when the parking assistance control unit 41 starts parking assistance control, the matching unit 47 receives a parking position calculation command from the parking assistance control unit 41. The matching unit 47 matches the learned targets stored in the map data 46 with the surrounding targets output from the target detection unit 44 when parking assistance is performed, and associates targets with the same feature points. Based on the relative positional relationship between the surrounding targets and the vehicle 1 and the relative positional relationship between the learned targets associated with the surrounding targets and the target parking position 30, the matching unit 47 calculates the current relative position of the vehicle 1 with respect to the target parking position 30. For example, if the surrounding targets are (x i ,y i ) is written, and surrounding targets (x i ,yi ) are respectively associated with the learned object labels as (x mi , y mi ) (i = 1 to N). The matching unit 47 calculates an affine transformation matrix M affine based on the least squares method according to the following formula.

[0025]

Equation

[0026] The matching unit 47 converts the position (targetx m , targety m ) of the target parking position 30 on the map coordinate system stored in the map data 46 to the position (targetx, targety) in the vehicle coordinate system according to the following formula.

Equation

[0027] When the target trajectory generation unit 48 receives a travel trajectory calculation command from the parking support control unit 41, it calculates a target travel trajectory from the current position of the vehicle 1 on the vehicle coordinate system to the target parking position 30 and a target vehicle speed profile for the vehicle 1 to travel on the target travel trajectory. When the steering control unit 49 receives a steering control command from the parking support control unit 41, it controls the steering actuator 19a so that the vehicle 1 travels along the target travel trajectory. When the vehicle speed control unit 50 receives a vehicle speed control command from the parking support control unit 41, it controls the accelerator actuator 19b and the brake actuator 19c so that the vehicle speed of the vehicle 1 changes according to the target vehicle speed profile. Thereby, the vehicle 1 is controlled to travel along the target travel trajectory. When the vehicle 1 reaches the target parking position 30 and the parking support control is completed, the parking support control unit 41 activates the parking brake 18 and switches the shift position to the parking range (P range).

[0028] Figure 4 is a flowchart of an example of the process for storing learned target data. In step S1, the image conversion unit 42 converts the image captured by the camera into an overhead view image from a virtual viewpoint directly above the vehicle 1 to obtain an image of the surroundings. In step S2, the target detection unit 44 detects targets around the target parking position 30. In step S3, the map generation unit 45 stores the detected targets as learned targets in the storage device 21. The process then ends. Figure 5 is a flowchart of an example of the process when parking assistance is performed. In step S10, the image conversion unit 42 converts the image captured by the camera into an overhead view image from a virtual viewpoint directly above the vehicle 1 to obtain an image of the surroundings. In step S11, the object detection unit 44 detects surrounding objects present around the vehicle 1.

[0029] In step S12, the matching unit 47 reads a learned target from the storage device 21. In step S13, the matching unit 47 matches the surrounding target with the learned target. In step S14, the matching unit 47 calculates the target parking position 30 based on the matched surrounding target and the learned target. In step S15, the target trajectory generation unit 48 calculates the target driving trajectory and the target vehicle speed profile. In step S16, the steering control unit 49 and the vehicle speed control unit 50 control the steering actuator 19a, accelerator actuator 19b, and brake actuator 19c based on the target driving trajectory and the target vehicle speed profile. In step S17, when the parking assistance control is complete, the parking assistance control unit 41 activates the parking brake 18 and switches the shift position to the P range. The process then ends.

[0030] Refer to Figure 3. The data exchange unit 51 exchanges learned target data (hereinafter sometimes referred to as "learned target data") stored in the storage device 21 between the parking assistance devices 10 provided in the first vehicle 1a and the second vehicle 1b, respectively. In the driving assistance system of the first embodiment, learned target data is transmitted from the second vehicle 1b to the first vehicle 1a via the communication device 16 and the mobile terminal device 2. That is, learned target data generated by the map generation unit 45 of the second vehicle 1b and stored in the storage device 21 of the second vehicle 1b is transmitted to the first vehicle 1a. The storage device 21 of the second vehicle 1b is an example of a "storage device" as described in the claims (i.e., a storage device provided outside the first vehicle 1a).

[0031] The mobile terminal device 2 is a portable information processing device with communication capabilities, such as a smartphone, tablet, or mobile personal computer. To use the mobile terminal device 2 for transmitting learned target data from the second vehicle 1b to the first vehicle 1a, a dedicated application program or software program (sometimes referred to as "parking assistance application" in the following description) is installed on the mobile terminal device 2. The parking assistance application is stored in the storage device (not shown) of the mobile terminal device 2. The processor (not shown) of the mobile terminal device 2 executes the parking assistance application, thereby realizing the functions of the parking assistance application described below.

[0032] In the first embodiment, the parking assistance application works in conjunction with the data exchange unit 51 and the communication device 16 to provide a data transmission and reception function between the mobile terminal device 2 and the parking assistance device 10. When a user of the second vehicle 1b operates the HMI 12 of the second vehicle 1b to instruct the transfer of learned target data from the parking assistance device 10 to the mobile terminal device 2, the data exchange unit 51 of the second vehicle 1b reads the learned target data from the storage device 21 of the second vehicle 1b. The data exchange unit 51 uses the data transmission and reception function of the parking assistance application installed on the second mobile terminal device 2b to transfer the learned target data from the communication device 16 to the second mobile terminal device 2b.

[0033] Users of the second vehicle 1b transmit learned target data from the second mobile terminal device 2b to the first mobile terminal device 2a using existing data communication functions (e.g., email, short message service, social networking service) installed on the second mobile terminal device 2b. In other words, users of the second vehicle 1b select the destination (first mobile terminal device 2a) for transmitting the learned target data by operating the second mobile terminal device 2b. The data exchange unit 51 of the first vehicle 1a uses the data transmission and reception function between the parking assistance application installed on the first mobile terminal device 2a and the parking assistance device 10 to download the learned target data received by the first mobile terminal device 2a to the storage device 21 of the first vehicle 1a. As a result, the parking assistance device 10 of the first vehicle 1a can perform parking assistance control using learned targets generated by the map generation unit 45 of the second vehicle 1b. In other words, it becomes possible to provide parking assistance to park the first vehicle 1a at the target parking position where the second vehicle 1b is parked.

[0034] Figure 6 is a sequence diagram of an example of the processing of the driver assistance system of the first embodiment. In step S20, the map generation unit 45 of the second vehicle 1b stores the learned targets in the storage device 21 when the second vehicle 1b is parked at the target parking position. In step S21, the user of the second vehicle 1b performs a transfer operation by operating the HMI 12 of the second vehicle 1b, instructing the transfer of learned target data from the second vehicle 1b to the second mobile terminal device 2b. When the HMI 12 of the second vehicle 1b receives a transfer operation, in step S22, the data exchange unit 51 of the second vehicle 1b transfers the learned target data stored in the storage device 21 of the second vehicle 1b to the second mobile terminal device 2b.

[0035] In step S23, the user of the second vehicle 1b operates the second mobile terminal device 2b to specify the first mobile terminal device 2a as the destination for transmitting learned target data using the existing data communication function. In step S24, the second mobile terminal device 2b transmits the learned target data to the first mobile terminal device 2a. In step S25, the first mobile terminal device 2a, having received the learned target data, outputs a visual message (text message, message image, etc.) and / or an audio message to inform the user of the first vehicle 1a that the learned target data has been received. In step S26, the user of the first vehicle 1a performs a registration operation by operating the first mobile terminal device 2a to instruct the parking assist device 10 of the first vehicle 1a to register the received learned target data.

[0036] In step S27, the data exchange unit 51 of the first vehicle 1a downloads the learned target data received by the first mobile terminal device 2a. In step S28, the data exchange unit 51 of the first vehicle 1a stores the learned target data in the storage device 21 of the first vehicle 1a. In step S29, the parking assistance device 10 of the first vehicle 1a performs parking assistance control using the learned targets stored in step S28 (i.e., learned targets acquired from the second vehicle 1b).

[0037] (Modification of the first embodiment) In the first embodiment described above, the learned target data is transmitted from the second vehicle 1b to the first vehicle 1a via the mobile terminal device 2. Alternatively, the learned target data may be transmitted via vehicle-to-vehicle communication between the respective communication devices 16 of the first vehicle 1a and the second vehicle 1b. Alternatively, the learned target data stored in the storage device 21 of the second vehicle 1b may be recorded on a portable recording medium (such as an SD memory card, CF card, or USB memory), and the learned target data may be copied from the second vehicle 1b to the first vehicle 1a by reading it from the portable recording medium to the storage device 21 of the first vehicle 1a. The parking assist device 10 may also be equipped with a drive device capable of reading and writing data to the portable recording medium. The portable recording medium and the drive device are examples of the "interface" described in the claims.

[0038] (Second Embodiment) The schematic configuration of the driver assistance system in the second embodiment is the same as the configuration shown in Figure 1. The parking assistance application in the second embodiment works in conjunction with the data exchange unit 51 and the communication device 16 of the parking assistance device 10 to provide a data transmission and reception function between the data exchange unit 51 of the first vehicle 1a and the data exchange unit 51 of the second vehicle 1b. When transmitting learned target data stored in the storage device 21 of the second vehicle 1b to the parking assistance device 10 of another vehicle, the data exchange unit 51 displays a list of candidate destinations for the learned target data (i.e., shared destinations for sharing the learned target data) on the display device of the HMI 12. For example, the data exchange unit 51 of the second vehicle 1b may, in cooperation with the parking assistance application installed on the second mobile terminal device 2b, refer to the address book data of the second mobile terminal device 2b to generate a list of candidate destinations for the learned target data. The destination for the learned target data is specified when the user of the second vehicle 1b selects one of the destination candidates displayed on the HMI 12's display device. Here, we assume that the first vehicle 1a is selected.

[0039] The data exchange unit 51 of the second vehicle 1b, in cooperation with the parking assistance application installed on the second mobile terminal device 2b, transmits the learned target data stored in the storage device 21 of the second vehicle 1b to the first mobile terminal device 2a. The parking assistance application installed on the first mobile terminal device 2a transfers learned target data to the data exchange unit 51 of the first vehicle 1a when the parking assistance device 10 of the first vehicle 1a is activated. The data exchange unit 51 of the first vehicle 1a stores the learned target data received from the first mobile terminal device 2a in the storage device 21 of the first vehicle 1a. As a result, the parking assistance device 10 of the first vehicle 1a can perform parking assistance control using learned targets generated by the map generation unit 45 of the second vehicle 1b. In other words, it becomes possible to provide parking assistance to park the first vehicle 1a at the target parking position where the second vehicle 1b is parked.

[0040] Figure 7 is a sequence diagram of an example of the processing of the driver assistance system according to the second embodiment. In step S30, the user of the second vehicle 1b designates the first vehicle 1a as the destination for the learned target data by operating the HMI 12. In step S31, the map generation unit 45 of the second vehicle 1b stores the learned targets in the storage device 21 when the second vehicle 1b is parked at the target parking position. In steps S32 and S33, the data exchange unit 51 of the second vehicle 1b, in cooperation with the parking assistance application installed on the second mobile terminal device 2b, transmits the learned target data stored in the storage device 21 of the second vehicle 1b to the first mobile terminal device 2a.

[0041] In step S34, when the parking assist device 10 of the first vehicle 1a is activated, in step S35, the parking assist application installed on the first mobile terminal device 2a transfers the learned target data to the data exchange unit 51 of the first vehicle 1a. In step S36, the data exchange unit 51 outputs a visual message and / or an audio message to the HMI 12 to inform the user of the first vehicle 1a that it has received the learned target data. In step S37, the user of the first vehicle 1a performs a registration operation by operating the HMI 12 to instruct the parking assist device 10 of the first vehicle 1a to register the received learned target data. In step S38, the data exchange unit 51 of the first vehicle 1a stores the learned target data in the storage device 21 of the first vehicle 1a. In step S39, the parking assistance device 10 of the first vehicle 1a performs parking assistance control using the learned targets stored in step S28 (i.e., learned targets acquired from the second vehicle 1b).

[0042] (Modified version of the second embodiment) In the first embodiment described above, the first mobile terminal device 2a of the user of the first vehicle 1a was specified as the destination for transmitting the learned target data, but a specific server device may be specified instead. The data exchange unit 51 of the first vehicle 1a may download the learned target data from a specific server device when the parking assist device 10 of the first vehicle 1a is activated and store it in the storage device 21 of the first vehicle 1a. Furthermore, in the first embodiment described above, the learned target data is transmitted from the second vehicle 1b to the first vehicle 1a via the mobile terminal device 2. Alternatively, the learned target data may be transmitted via vehicle-to-vehicle communication between the respective communication devices 16 of the first vehicle 1a and the second vehicle 1b. Furthermore, in the first embodiment described above, the learned target data was stored in the storage device 21 of the second vehicle 1b after specifying the destination for transmission of the learned target data. However, the destination for transmission of the learned target data may be specified after storing the learned target data in the storage device 21 of the second vehicle 1b.

[0043] (Third embodiment) Figure 8 is a schematic diagram showing the general configuration of the driver assistance system of the third embodiment. The driver assistance system of the third embodiment comprises a parking assistance device 10 provided in the first vehicle 1a and the second vehicle 1b, a first mobile terminal device 2a, a second mobile terminal device 2b, and a public server device 3. The public server device 3 is a server device that stores learned target data generated in either vehicle 1 and mediates the learned target data between different vehicles 1a and 1b. The public server device 3 is an example of a storage device "located outside the vehicle" as described in the claims. In the third embodiment, the parking assistance application works in conjunction with the data exchange unit 51 and communication device 16 of the parking assistance device 10 and the public server device 3 to provide a data transmission and reception function between the data exchange unit 51 of the vehicle 1 and the public server device 3. The same applies to the fourth embodiment described later.

[0044] In the third embodiment, a public server device 3 can be designated as the sharing destination for the learned target data stored in the storage device 21 of the second vehicle 1b. For example, the learned target data registered in the public server device 3 may be made public so that the parking assistance device 10 of an unspecified vehicle 1 can use it. For example, learned target data for providing parking assistance in parking lots of hospitals, stores, public facilities, etc., may be registered in the public server device 3 and made public so that an unspecified vehicle 1 can use it.

[0045] Figure 9 is a sequence diagram of an example of the processing of the driver assistance system according to the third embodiment. In step S40, the data exchange unit 51 of the second vehicle 1b displays a list of sharing destinations, including the public server device 3 and the first mobile terminal device 2a, on the display device of the HMI 12 as a candidate destination for sharing the learned target data. The user of the second vehicle 1b specifies the public server device 3 as the destination for sharing the learned target data by operating the HMI 12. In step S41, the map generation unit 45 of the second vehicle 1b stores the learned targets in the storage device 21 when the second vehicle 1b is parked at the target parking position. In steps S42 and S43, the data exchange unit 51 of the second vehicle 1b, in cooperation with the parking assistance application installed on the second mobile terminal device 2b, transmits the learned target data stored in the storage device 21 of the second vehicle 1b to the public server device 3. The public server device 3 stores the received learned target data in a storage device (not shown) within the public server device 3 or in a storage device (not shown) accessible to the public server device 3.

[0046] In step S44, when the parking assist device 10 of the first vehicle 1a is activated, in steps S45 and S46, the data exchange unit 51 of the first vehicle 1a works in conjunction with the parking assist application installed on the first mobile terminal device 2a to download the learned target data stored in the public server device 3. In this case, if learned target data for multiple different target parking positions is stored in the public server device 3 (for example, if learned target data transmitted from vehicles other than the second vehicle 1b is stored), then the learned target data for these multiple target parking positions may be downloaded from the public server device 3.

[0047] In step S47, the data exchange unit 51 presents the HMI 12 with target parking positions that can be supported by the learned target data received from the public server device 3 (i.e., target parking positions that have learned the learned target data). If learned target data for multiple target parking positions has been downloaded from the public server device 3, a list of these multiple target parking positions is presented to the HMI 12. In step S48, the user of the first vehicle 1a performs a registration operation by operating the HMI 12 to instruct the parking assist device 10 of the first vehicle 1a to register the learned target data of the displayed target parking position. In step S49, the data exchange unit 51 of the first vehicle 1a stores the learned target data in the storage device 21 of the first vehicle 1a. If a list of multiple target parking locations is displayed on the HMI 12, in step S48 the data exchange unit 51 receives a selection input from the user of the first vehicle 1a to operate the HMI 12 and select one of these multiple target parking locations. In step S49, the data exchange unit 51 of the first vehicle 1a stores the learned target data for the selected target parking position in the storage device 21 of the first vehicle 1a. In step S50, the parking assistance device 10 of the first vehicle 1a performs parking assistance control using the learned targets stored in step S28 (i.e., learned targets acquired from the second vehicle 1b).

[0048] (Modified version of the third embodiment) In the third embodiment, learned target data was transmitted and received between the vehicle 1 and the public server device 3 via the mobile terminal device 2, but learned target data may also be transmitted and received directly between the communication device 16 and the public server device 3 via the public mobile phone network.

[0049] (Fourth Embodiment) The schematic configuration of the driver assistance system in the fourth embodiment is the same as the configuration shown in Figure 8. In the driver assistance system of the fourth embodiment, in response to a request from the user of the first vehicle 1a to transmit learned target data, the learned target data stored in the public server device 3 is downloaded to the storage device 21 of the first vehicle 1a and used by the parking assistance device 10 of the first vehicle 1a. Figure 10 is a sequence diagram of an example of the processing of the driver assistance system according to the fourth embodiment. The processing in steps S60 to S62 is the same as in steps S41 to S43 in Figure 9.

[0050] In step S63, the user of the first vehicle 1a operates the HMI 12 of the first vehicle 1a to perform a learned target data request operation, which requests the use of learned target data stored in the public server device 3. The data exchange unit 51 of the first vehicle 1a transmits a parking location list request signal to the public server device 3 via the first mobile terminal device 2a. The parking location list request signal is a request signal for a list of target parking locations for which learned target data is stored in the public server device 3. In step S64, the public server device 3, in response to the parking location list request signal, transmits a list of target parking locations for which learned target data is stored in the public server device 3 to the first vehicle 1a via the first mobile terminal device 2a. The list of target parking locations may include multiple different target parking locations.

[0051] In step S65, the data exchange unit 51 presents the received list of target parking locations to the HMI 12. In step S66, the data exchange unit 51 receives a selection input from the user of the first vehicle 1a, who operates the HMI 12 to select a desired target parking position from a list of target parking positions. Here, we assume that the user selects a target parking position from the learned target data transmitted from the second vehicle 1b. In step S67, the data exchange unit 51 sends a download request signal to the public server device 3 via the first mobile terminal device 2a, requesting the transmission of learned target data for the selected target parking position. In step S68, the public server device 3 sends an approval request signal to the second vehicle 1b via the second mobile terminal device 2b in response to the download request signal. The approval request signal is a notification signal that prompts the user of the second vehicle 1b to grant permission to transmit the learned target data in response to the download request signal.

[0052] In step S69, when a user of the second vehicle 1b operates the HMI 12 of the second vehicle 1b to perform an authorization operation that permits the transmission of learned target data, in step S70, the data exchange unit 51 of the second vehicle 1b transmits an authorization notification signal that permits the transmission of learned target data to the public server device 3 via the second mobile terminal device 2b. In steps S71 and S72, the public server device 3, in response to the approval notification signal, transmits the learned target data received from the second vehicle 1b to the first vehicle 1a via the first mobile terminal device 2a. In step S73, the data exchange unit 51 of the first vehicle 1a stores the received learned target data in the storage device 21 of the first vehicle 1a. In step S74, the parking assistance device 10 of the first vehicle 1a performs parking assistance control using the learned targets stored in step S28 (i.e., learned targets acquired from the second vehicle 1b).

[0053] (Modification of the fourth embodiment) In the fourth embodiment, learned target data, parking location list request signals, parking location lists, download request signals, approval request signals, and approval notification signals were transmitted and received between the vehicle 1 and the public server device 3 via the mobile terminal device 2. However, learned target data and these signals may also be transmitted and received directly between the communication device 16 and the public server device 3 via the public mobile phone network.

[0054] (Effects of the embodiment) (1) In the parking assistance method, the positions of surrounding targets, which are targets present around the first vehicle 1a, are detected. If there is no target position data in the first storage device located inside the first vehicle 1a that represents the relative positional relationship between the targets present around the target parking position and the target parking position, but there is target position data in the second storage device located outside the first vehicle 1a, the target position data is obtained from the second storage device. Based on the target position data obtained from the second storage device and the positions of the surrounding targets, the relative positional relationship between the target parking position and the current position of the first vehicle 1a is calculated. Based on the relative positional relationship between the target parking position and the current position of the first vehicle 1a, a driving trajectory from the current position of the first vehicle 1a to the target parking position is calculated, and parking of the first vehicle 1a to the target parking position is supported based on the driving trajectory. This makes it possible to assist in parking the first vehicle 1a at a target parking location based on landmarks present around the target parking location, even if the target parking location has never been used before.

[0055] (2) Object position data for multiple target parking positions may be stored in a second storage device. Multiple target parking positions may be displayed on a display device, an input operation to select one of the displayed target parking positions may be accepted, object position data for the target parking position selected by the input operation may be obtained from the second storage device, and the relative positional relationship between the selected target parking position and the current position of the first vehicle 1a may be calculated based on the object position data obtained from the second storage device and the positions of surrounding objects. This allows the user of the first vehicle 1a to select any target parking position from multiple target parking positions.

[0056] (3) The first vehicle 1a may receive, as target position data, data that is generated when the second vehicle 1b is parked at the target parking position and stored in the second memory device, which represents the relative positional relationship between the targets surrounding the second vehicle and the target parking position. This allows the system to use the data of the target object generated when the second vehicle 1b is parked at the target parking position to assist in parking the first vehicle 1a at the target parking position. (4) The first vehicle 1a may receive target position data stored in the second storage device, which has been transmitted in response to a transmission instruction from the user of the second vehicle 1b, and calculate the relative positional relationship between the target parking position and the current position of the first vehicle 1a based on the received target position data and the positions of surrounding targets around the first vehicle 1a. This makes the target data generated when the second vehicle 1b is parked at the target parking position available to the first vehicle 1a according to the instructions of the user of the second vehicle 1b.

[0057] (5) The first vehicle 1a may receive target position data stored in the second storage device, which has been transmitted in response to a request from the user of the first vehicle 1a, and calculate the relative positional relationship between the target parking position and the current position of the first vehicle 1a based on the received target position data and the positions of surrounding targets around the first vehicle 1a. This makes the data of the target object generated when the second vehicle 1b is parked at the target parking position available to the first vehicle 1a, as needed by the user of the first vehicle 1a. (6) A notification may be generated prompting the user of the second vehicle 1b to authorize the transmission of target location data in response to a request from the user of the first vehicle 1a, the system may accept the user of the second vehicle 1b's authorization operation to transmit target location data, and transmit the target location data stored in the second storage device to the first vehicle 1a in response to the authorization operation. This prevents the data of the target object generated when the second vehicle 1b is parked at the target parking position from being used for the first vehicle 1a against the will of the user of the second vehicle 1b.

[0058] (7) The second storage device may be located outside the first vehicle 1a and the second vehicle 1b. This makes it easier to share target data among unspecified vehicles. (8) The relative positional relationship between the driving trajectory, the target parking position, and the current position of the first vehicle 1a may be displayed on a display device visible to the user of the first vehicle 1a. This allows the user of the first vehicle 1a to visually confirm and verify the target parking position and the driving path to the target parking position. (9) The first vehicle 1a may be controlled to travel along the track. This reduces the effort required of users to drive to their target parking spot. [Explanation of symbols]

[0059] 1a...First vehicle, 1b...Second vehicle, 2a...First mobile terminal device, 2b...Second mobile terminal device, 3...Public server device, 10...Parking assist device, 11...Positioning device, 12...HMI, 13...Shift switch, 14...External environment sensor, 15...Vehicle sensor, 16...Communication device, 17...Controller, 18...Parking brake, 19a...Steering actuator, 19b...Accelerator actuator, 19c...Brake actuator, 20...Processor, 21...Storage device, 40...HMI control unit, 41...Parking assist control unit, 42...Image conversion unit, 43...Self-position calculation unit, 44...Target detection unit, 45...Map generation unit, 46...Map data, 47...Matching unit, 48...Target trajectory generation unit, 49...Steering control unit, 50...Vehicle speed control unit, 51...Data exchange unit

Claims

1. The system detects the positions of surrounding targets, which are objects present around the vehicle. If the first storage device located inside the vehicle contains object position data representing the relative positional relationship between objects surrounding the target parking position and the target parking position, the object position data is acquired from the first storage device. If the first storage device does not contain the object position data, but the second storage device located outside the vehicle contains the object position data, the object position data is acquired from the second storage device. Based on the acquired target position data and the positions of the surrounding targets, the relative positional relationship between the target parking position and the current position of the vehicle is calculated. Based on the relative positional relationship between the target parking position and the vehicle's current position, the vehicle's travel trajectory from its current position to the target parking position is calculated. Based on the aforementioned driving trajectory, the system assists in parking the vehicle at the target parking position. A parking assistance method characterized by the following features.

2. The object position data for multiple target parking positions is stored in the second storage device. The aforementioned multiple target parking positions are displayed on the display device, The system accepts an input operation to select one of the displayed multiple target parking positions. The target parking position data of the target parking position selected by the input operation is obtained from the second storage device. Based on the target position data obtained from the second storage device and the positions of the surrounding targets, the relative positional relationship between the selected target parking position and the current position of the vehicle is calculated. The parking assistance method according to feature 1.

3. The parking assistance method according to claim 1, characterized in that the first vehicle receives, as the target position data, data representing the relative positional relationship between a target present around the second vehicle and the target parking position, which is generated when a second vehicle other than the first vehicle is parked at the target parking position and stored in the second storage device.

4. The first vehicle receives the target position data stored in the second storage device, which has been transmitted in response to a transmission instruction from the user of the second vehicle. Based on the received target position data and the positions of the surrounding targets around the first vehicle, the relative positional relationship between the target parking position and the current position of the first vehicle is calculated. The parking assistance method according to feature 3.

5. The first vehicle receives the target position data stored in the second storage device, which has been transmitted in response to a request or instruction from the user of the first vehicle. Based on the received target position data and the positions of the surrounding targets around the first vehicle, the relative positional relationship between the target parking position and the current position of the first vehicle is calculated. The parking assistance method according to feature 3.

6. A notification is generated prompting the user of the second vehicle to authorize the transmission of the target location data in response to the user's request for the first vehicle. The user of the second vehicle grants permission to transmit the target location data. The parking assistance method according to claim 5, characterized in that the target position data stored in the second storage device is transmitted to the first vehicle in response to the transmission permission operation.

7. The parking assistance method according to claim 3, characterized in that the second storage device is provided outside the first vehicle and the second vehicle.

8. The parking assistance method according to any one of claims 1 to 7, characterized in that the relative positional relationship between the driving trajectory, the target parking position, and the current position of the vehicle is displayed on a display device visible to the user of the vehicle.

9. The parking assistance method according to any one of claims 1 to 7, characterized in that the vehicle is controlled to travel along the aforementioned travel trajectory.

10. A sensor installed on the vehicle to detect surrounding objects, A first storage device provided inside the vehicle, A second storage device that stores object position data representing the relative positional relationship between an object located outside the vehicle and the target parking position, and the target parking position, An interface for acquiring the target position data from the second storage device, A controller provided in the vehicle, which performs the following processes: detecting the position of surrounding targets, which are targets present around the vehicle, based on the detection signal of the sensor; acquiring the target position data from the first storage device if the target position data exists in the first storage device, and acquiring the target position data from the second storage device via the interface if the target position data does not exist in the first storage device but exists in the second storage device; calculating the relative positional relationship between the target parking position and the current position of the vehicle based on the target position data and the positions of the surrounding targets; calculating the driving trajectory from the current position of the vehicle to the target parking position based on the relative positional relationship between the target parking position and the current position of the vehicle; and assisting in parking the vehicle at the target parking position based on the driving trajectory. A parking assistance system characterized by having the following features.

Images