Parking assistance method and parking assistance device

Abstract

In a parking assist method for assisting parking of a host vehicle to a target parking position, it is determined whether the host vehicle is located in a first type parking lot having a plurality of parking spaces or whether the host vehicle is located near a second type parking lot capable of parking a single vehicle (S1), in the case where it is determined that the host vehicle is located in the first type parking lot, data indicating a relative positional relationship of an object to the target parking position, the object being a common object to the plurality of parking spaces when the host vehicle is parked in an arbitrary one of the plurality of parking spaces, i.e., a target parking space, and indicating that the target parking space is a parking space, is stored in a storage device (S4, S5), and in the case where it is determined that the host vehicle is located near the second type parking lot, data indicating a relative positional relationship of an object to the target parking position, the object being detected near the second type parking lot when the host vehicle is parked in the second type parking lot, is stored in the storage device (S6, S7).

Description

TECHNICAL FIELD

[0001] The present application relates to a parking assistance method and a parking assistance device. BACKGROUND

[0002] In Patent Literature 1, a driving control device has been disclosed which stores an object extracted from an image taken around a target parking position in the past, calculates a relative position of the target parking position with respect to the host vehicle based on the position of the stored object and the position of an object extracted from an image taken around the host vehicle at the time of automatic parking, and causes the host vehicle to automatically move to the target parking position based on the calculated relative position.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2017-138664 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] However, in the case where there are a plurality of parking spaces (parking frame lines) in a parking lot provided in facilities such as companies, hospitals, stores, concentrated residences, public facilities, and the like, when an arbitrary one of the parking spaces is stored as a target parking position, there is a problem that it is only possible to park in the same parking frame line all the time.

[0008] An object of the present application is to enable the host vehicle to park in an arbitrary parking space in the case where there are a plurality of parking spaces in a parking lot, in a parking assistance in which objects around a target parking position are stored in advance to assist parking of the host vehicle in the target parking position.

[0009] TECHNICAL SOLUTION FOR SOLVING THE PROBLEMS

[0010] According to one embodiment of the present invention, a parking assistance method for assisting parking of a host vehicle in a target parking position is provided. In the parking assistance method, data indicating a relative positional relationship between a target parking position and an object existing around the target parking position is stored in advance as learned object data in a storage device, a position of an object existing around the host vehicle, i.e., a surrounding object, is detected, a relative positional relationship between the target parking position and a current position of the host vehicle is calculated based on the learned object data and the position of the surrounding object, a travel trajectory of the host vehicle from the current position to the target parking position is calculated based on the relative positional relationship between the target parking position and the current position of the host vehicle, and parking of the host vehicle in the target parking position is assisted based on the travel trajectory. In the parking assistance method, it is determined whether the host vehicle is located in a first type of parking lot having a plurality of parking spaces or whether the host vehicle is located near a second type of parking lot capable of parking a single vehicle, learned object data indicating a relative positional relationship between an object indicating a parking space common to the plurality of parking spaces and a parking position in an object parking space, i.e., an arbitrary one of the plurality of parking spaces, is stored in the storage device when it is determined that the host vehicle is located in the first type of parking lot, and learned object data indicating a relative positional relationship between an object detected near the second type of parking lot and the target parking position is stored in the storage device when it is determined that the host vehicle is located near the second type of parking lot.

[0011] Effects of the Invention

[0012] In the parking assistance method of the present invention, in which learned object data is stored in advance to assist parking of a host vehicle in a target parking position, the host vehicle can be parked in an arbitrary parking space when a plurality of parking spaces exist in a parking lot. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 FIG. 1 is a diagram showing an outline of a structure of a parking assistance device.

[0014] Figure 2A FIG. 2 is a diagram showing a first example of a process of storing learned object data.

[0015] Figure 2B FIG. 3 is a diagram showing a first example of a process when the parking assistance is performed.

[0016] Figure 2C FIG. 4 is a diagram showing one example of a method of detecting a parking space.

[0017] Figure 3A FIG. 5 is a diagram showing a second example of a process of storing learned object data.

[0018] Figure 3Bis a diagram for explaining a second example of the process when the parking assist is implemented.

[0019] Figure 4 is a block diagram of one example of the functional structure of the controller. Figure 1

[0020] Figure 5 is a diagram of one example of a partial region including a pattern representing a parking space.

[0021] Figure 6 is a flowchart of one example of the storage process of the object data that has been learned.

[0022] Figure 7 is a flowchart of a first example of the process when the parking assist is implemented.

[0023] Figure 8 is a flowchart of a second example of the process when the parking assist is implemented. DETAILED DESCRIPTION

[0024] (Configuration)

[0025] Referring to Figure 1 The host vehicle 1 has a parking assist device 10 that assists parking of the host vehicle 1 to a target parking position. The parking assist device 10 assists travel of the host vehicle 1 along a target travel trajectory from a current position to the target parking position. For example, automatic driving of the host vehicle 1 can be controlled so that the host vehicle 1 travels along the target travel trajectory to the target parking position. The automatic driving that controls the host vehicle 1 to travel along the target travel trajectory to the target parking position means that all or a part of the steering angle, the driving force, and the braking force of the host vehicle are controlled, and all or a part of the control of the host vehicle 1 to travel along the target travel trajectory is automatically implemented. In addition, the host vehicle 1 can be assisted to park by displaying the target travel trajectory and the current position of the host vehicle 1 on a display device that can be seen by an occupant of the host vehicle 1.

[0026] The positioning device 11 measures the current position of the host vehicle 1. The positioning device 11 has, for example, a global navigation satellite system (GNSS) receiver.

[0027] Map data is stored in a map database (map DB) 12. The map data stored in the map database 12 can be, for example, navigation map data or high-precision map data suitable as an automatic driving map. The map data includes information on the positions and ranges of parking lots provided in facilities such as companies, hospitals, stores, concentrated residences, and public facilities.

[0028] ​A human-machine interface (HMI) 13 is an interface device that receives and transmits information between the parking assist device 10 and an occupant, and has a display device, a speaker or a buzzer, and an operation member, which the occupant of the host vehicle 1 can see.

[0029] A shift switch (shift SW) 14 is a switch used by the driver or the parking assist device 10 to switch the gear of the host vehicle 1.

[0030] An outside sensor 15 detects an object within a prescribed distance range from the host vehicle 1. The outside sensor 15 detects the relative position of an object existing around the host vehicle 1 to the host vehicle 1, the distance between the host vehicle 1 and the object, the direction in which the object exists, and the like, that is, the environment around the host vehicle 1. The outside sensor 15 can include, for example, a camera that takes a picture of the environment around the host vehicle 1. In the following description, the camera possessed by the outside sensor 15 will be simply referred to as "camera". The outside sensor 15 can also include a range finder such as a laser range finder, a radar, a LiDAR (Light Detection and Ranging), or the like.

[0031] A vehicle sensor 16 detects various information (vehicle information) of the host vehicle 1. The vehicle sensor 16 can include, for example, a vehicle speed sensor that detects the running speed of the host vehicle 1, a wheel speed sensor that detects the rotational speed of each tire possessed by the host vehicle 1, a three-axis acceleration sensor (G sensor) that detects the acceleration (including deceleration) of three axes of the host vehicle 1, a steering angle sensor that detects the steering angle, a steering angle sensor that detects the steering angle of the steering wheel, a gyro sensor, and a yaw rate sensor.

[0032] A controller 17 is an electronic control unit that performs parking assist control of the host vehicle 1. The controller 17 includes a processor 20 and a storage device 21, and the like, as peripheral accessories. The processor 20 can be, for example, a CPU or an MPU. The storage device 21 can have a semiconductor storage device, a magnetic storage device, an optical storage device, and the like. The functions of the controller 17 described below are realized, for example, by the processor 20 executing a computer program stored in the storage device 21. Note that the controller 17 can also be formed by a dedicated hardware for performing each information processing described below.

[0033] A steering actuator 19a controls the steering direction and the steering amount of the steering mechanism of the host vehicle 1 according to a control signal of the controller 17. An accelerator actuator 19b controls the accelerator opening degree of the engine or the drive motor, that is, the drive device, according to a control signal of the controller 17. A brake actuator 19c operates the brake device according to a control signal of the controller 17.

[0034] Next, the parking assist control by the parking assist device 10 will be described. When parking assist by the parking assist device 10 is utilized, data indicating the relative positional relationship between the object existing around the position at which the host vehicle 1 is parked, i.e., the target parking position, and the target parking position is stored in advance in the storage device 21. In the following description, there are cases in which the data indicating the relative positional relationship between the object stored in the storage device 21 and the target parking position is referred to as "learned object data".

[0035] For example, the object around the target parking position when the host vehicle 1 is located in the vicinity of the target parking space can be detected by the outside sensor 15. For example, the object around the host vehicle 1 can be detected by the outside sensor 15 when the occupant (e.g., the driver) of the host vehicle 1 parks the host vehicle 1 at the target parking position by manual driving. For example, the object around the host vehicle 1 can be detected by the outside sensor 15 when the host vehicle 1 is parked at the target parking position. At this time, for example, the object can be detected from an image of the surroundings of the host vehicle 1 taken by a camera, or the object around the host vehicle 1 can be detected by a range-finding device.

[0036] The learned object data stored in the storage device 21 can include data indicating the characteristics of the detected object (in the following description, there are cases in which the data is referred to as "object characteristic data") and data indicating the relative positional relationship between the object and the target parking position (in the following description, there are cases in which the data is referred to as "relative position data").

[0037] Figure 2A and Figure 3A is a diagram for explaining a processing example of storing learned object data. When the learned object data is stored in the storage device 21, for example, the occupant operates a "parking position learning switch" prepared as an operating member of the HMI 13.

[0038] When the learned object data is stored, the controller 17 determines whether the host vehicle 1 is located within a parking lot having a plurality of parking spaces (in the following description, there are cases in which the parking lot is referred to as a "first-type parking lot") or whether the host vehicle is located in the vicinity of a parking lot capable of parking a single vehicle (in the following description, there are cases in which the parking lot is referred to as a "second-type parking lot").

[0039] Examples of the first-type parking lot are parking lots provided in facilities such as companies, hospitals, stores, concentrated residences, public facilities, and the like. Examples of the second-type parking lot are parking lots provided in private residences.

[0040] Figure 2AThe illustrated parking lot is a first-type parking lot having a plurality of parking spaces 30a to 30d. For example, assume a case where the host vehicle 1 moves along the route 31 by manual driving and parks in the parking space 30c. The parking space 30c is an example of the "target parking space" described in the claims.

[0041] The controller 17 detects, as the landmark indicating that the parking space 30c is a parking space, a stereoscopic object, i.e., an object landmark feature, such as a road marking (e.g., a white line indicating a parking space frame) or a wheel stop, painted on the road surface of the parking lot, based on the image of the surroundings of the host vehicle 1 captured by the camera. The controller 17 stores, as the learned landmark data, the detected object landmark feature data indicating the road marking feature, and relative position data between the detected road marking and the target parking position within the parking space, in the storage device 21.

[0042] For example, the controller 17 can store, as the object landmark feature data, the pattern 32 of the detected road marking in the storage device 21. In addition, the relative position data between the parking position Pr at which the host vehicle 1 parks in the parking space 30c and the pattern 32 can be stored in the storage device 21. For example, the coordinates of the road marking in the vehicle coordinate system (i.e., a coordinate system with the current position of the host vehicle 1 as the reference) at the time when the host vehicle 1 parks in the parking space 30c can be stored as the relative position data.

[0043] In the following description, there are cases where the pattern 32 of the road marking indicating a parking space is referred to as a "road surface pattern". For example, the controller 17 can extract, as the road surface pattern 32, a partial image of a local area including the entire or a part of the road marking indicating a single parking space from the image captured by the camera of the surroundings of the host vehicle 1, and store the partial image in the storage device 21.

[0044] In this way, the controller 17 stores, as the object landmark feature data, partial information of the feature of the road marking indicating a parking space unit among the road markings of the entire parking lot in the first-type parking lot having a plurality of parking spaces 30a to 30d in the storage device 21. For example, the local road surface pattern 32 of the road marking indicating a parking space unit is stored in the storage device 21. At this time, the controller 17 determines whether the road surface pattern 32 is a feature of a parking space common to the plurality of parking spaces 30a to 30d, and stores the road surface pattern 32 in the case where it is a feature of a parking space common to the plurality of parking spaces 30a to 30d. For example, the controller 17 can determine whether it is a feature of a parking space common to the plurality of parking spaces 30a to 30d by determining whether a plurality of road surface patterns 32 are continuously present in the image captured by the camera of the surroundings of the host vehicle 1.

[0045] Figure 3AThe illustrated parking lot is a second-type parking lot having a parking space 40 in which a single vehicle can be parked. Assume, for example, a case where the parking space 40 is stored in the storage device 21 as a parking target position when the host vehicle 1 moves along the route 41 by manual driving and parks in the parking space 40.

[0046] The controller 17 detects the object around the parking space 40 from an image captured by the camera of the surroundings of the host vehicle 1 when the host vehicle 1 is located near the parking space 40, for example, when the occupant parks the host vehicle 1 in the parking space 40 by manual driving. The controller 17 can detect, for example, a feature point on the image as an object around the parking space 40. The controller 17 can detect, for example, a point having a feature on an edge or a shape of an object such as an edge point or a corner of an edge or a corner of an object such as a road surface marking or a road boundary, an obstacle, or the like on a captured image obtained by capturing with the camera, as a feature point.

[0047] The controller 17 stores object feature data indicating a feature of an object detected within a prescribed region 42 around the parking space 40 as learned object data in the storage device 21. The controller 17 can store, for example, a feature point of a detected object and a feature amount as the object feature data in the storage device 21. In addition, the controller 17 stores relative position data between the parking target position, that is, the parking space 40 and an object as learned object data in the storage device 21.

[0048] In Figure 3A The objects stored as learned object data are schematically represented by circular icons.

[0049] When storing the relative position data of the object and the parking space 40, the coordinates of the object and the parking space 40 in a coordinate system (hereinafter referred to as "map coordinate system") having a fixed point as a reference point can be stored, for example. In this case, the current position in the map coordinate system measured when the host vehicle 1 is located in the parking space 40 can be stored as the position of the parking space 40. Alternatively, the relative positions of the parking space 40 with respect to the respective objects can be stored instead of the map coordinate system.

[0050] Note that the prescribed region 42 in which the learned object data is stored can be a region having a length of one side or a diameter of about 20 m to 30 m, for example. In this case, where the learned object data around the parking space 40 in the second-type parking lot is stored, the entire configuration pattern of the objects (feature points) in a relatively large range including the parking space 40 is stored in the storage device 21.

[0051] On the other hand, in a case where learned object data of the parking space 30c in the first-type parking lot is stored, a partial pattern representing an object (a road surface mark) in a relatively small range of one parking space is stored in the storage device 21. That is, the first range of the object in the first-type parking lot is stored smaller than the second range of the object in the second-type parking lot.

[0052] Figure 2B is a diagram for explaining a process example when the parking assist is implemented in the first-type parking lot. The controller 17 implements the parking assist of the host vehicle 1 in a case where the host vehicle 1 is located in the first-type parking lot and a prescribed parking assist start condition has been established. For example, the controller 17 can determine whether or not the host vehicle 1 is located in the first-type parking lot on the basis of a current position of the host vehicle 1 measured by the positioning device 11 and information of the parking lot stored in the map DB 12.

[0053] As the parking assist start condition, the controller 17 determines whether or not a shift operation for reversing for a U-turn has been performed by an occupant when the host vehicle 1 is located in the first-type parking lot. The controller 17 determines that the shift operation for reversing for a U-turn has been performed in a case where the gear position is switched from the forward gear (D range) to the reverse gear (R range) or from the R range to the D range, and can start the parking assist.

[0054] The controller 17 can also determine whether or not an operation has been performed on a "parking assist start switch" prepared in the HMI 13 by the occupant as the parking assist start condition, and start the parking assist when the operation has been performed on the parking assist start switch.

[0055] The controller 17 generates an image (a surrounding image) of a surrounding area 35 of the host vehicle 1 by traveling along a trajectory 34 on a drive aisle 33 in the first-type parking lot and accumulating images captured by a camera of the surroundings of the host vehicle 1.

[0056] The controller 17 detects one or more parking spaces 36a to 36c existing in the surroundings of the host vehicle 1 by detecting a region including an object having features similar to object feature data (i.e., object feature data representing an object of a parking space) stored as learned object data from the surrounding image. For example, the parking spaces 36a to 36c are detected by detecting a region including a road surface mark similar to a road surface pattern 32 of a parking space common to the plurality of parking spaces 36a to 36c from the surrounding image. The parking spaces 36a to 36c are one example of the "surrounding object" described in the claims.

[0057] The controller 17 can detect the parking spaces 36a to 36c, for example, by template matching the surrounding image with the road surface pattern 32 as a template. For example, the controller 17 scans the road surface pattern 32 on the surrounding image, calculates the similarity of the road surface pattern 32 at each position on the surrounding image to the surrounding image, and detects the region where the similarity is above a threshold value.

[0058] Figure 2C is an explanatory diagram of one example of a detection method of a parking space. For example, the extension direction of the travel lane 33 is set as a main scanning direction, and each time one main scanning is completed, the sub scanning direction position is changed, and the road surface pattern 32 is scanned on the surrounding image of the surrounding area 35 of the host vehicle 1. Then, the similarity of the road surface pattern 32 at each position on the surrounding image to the surrounding image is calculated, and the positions Pfa, Pfb, and Pfc where the similarity is above a threshold value Th are detected as the positions of the parking spaces 36a to 36c.

[0059] Referring to Figure 2B The controller 17 selects the parking space in which the host vehicle 1 is to be parked, from among the detected parking spaces 36a to 36c. For example, the controller 17 can display the detected parking spaces 36a to 36c on the HMI 13, prompt the occupant with the parking space candidates, and accept a selection input from the occupant to select any of the above parking space candidates, thereby selecting the parking space in which the host vehicle 1 is to be parked. Alternatively, for example, the controller 17 can select the parking space in which the host vehicle 1 is to be parked, based on the shortest movement route. Here, it is assumed that the parking space 36c has been selected.

[0060] When the parking space 36c in which the host vehicle 1 is to be parked is selected, the controller 17 calculates the relative position of the host vehicle 1 with respect to the target parking position, based on the detected position of the parking space 36c and the relative position data between the target parking position and the road surface marker of the parking space 30c stored as the learned landmark data (i.e., the relative position data between the road surface marker of the parking space and the target parking position).

[0061] For example, the relative position of the host vehicle 1 with respect to the target parking position is calculated based on the detected position of the parking space 36c and the relative position data between the target parking position Pr in which the host vehicle 1 is to be parked in the parking space 30c and the road surface marker of the parking space 30c stored as the learned landmark data.

[0062] The controller 17 calculates the target travel trajectory 37 of the host vehicle 1 from the current position to the target parking position, based on the relative position of the host vehicle 1 with respect to the target parking position. Then, the controller 17 implements the parking assist control of the host vehicle 1 based on the calculated target travel trajectory.

[0063] Figure 3Bis a diagram for explaining a processing example when the parking assist is implemented in the second type of parking lot. The controller 17 implements the parking assist of the host vehicle 1 in a case where the above-mentioned parking assist start condition is established in the vicinity of the parking space (i.e., the target parking position) 40.

[0064] The controller 17 detects the object in the vicinity of the host vehicle 1 from the surrounding image obtained by capturing the surroundings of the host vehicle 1 with the camera pair. In the following description, the object in the vicinity of the host vehicle 1 extracted when the parking assist is implemented is another example of the "surrounding object" described in the claims. In the following description, the object in the vicinity of the host vehicle 1 extracted when the parking assist is implemented is another example of the "surrounding object" described in the claims. In the following description, the object in the vicinity of the host vehicle 1 extracted when the parking assist is implemented is another example of the "surrounding object" described in the claims. Figure 3B In the map, the triangular icon represents the surrounding object.

[0065] The controller 17 matches the learned object data stored in the storage device 21 with the surrounding object, and associates the same objects with each other.

[0066] The controller 17 calculates the relative position of the host vehicle 1 with respect to the target parking position 40 (i.e., the relative position of the host vehicle 1 with respect to the target parking position) based on the relative positional relationship between the surrounding object detected when the parking assist is implemented and the host vehicle 1, and the relative positional relationship between the object of the learned object data associated with the surrounding object and the target parking position 40.

[0067] For example, the controller 17 can also calculate the position of the target parking position 40 in the vehicle coordinate system. For example, in a case where the coordinates of the object and the coordinates of the target parking position 40 are expressed as the coordinates of the map coordinate system in the learned object data, the coordinates of the target parking position 40 in the map coordinate system can be converted into the coordinates in the vehicle coordinate system based on the coordinates of the surrounding object detected when the parking assist is implemented in the vehicle coordinate system, and the coordinates of the object of the learned object data in the map coordinate system.

[0068] Alternatively, the current position of the host vehicle 1 in the map coordinate system can be found based on the coordinates of the surrounding object detected when the parking assist is implemented in the vehicle coordinate system, and the coordinates of the object of the learned object data in the map coordinate system, and the relative position of the host vehicle 1 with respect to the target parking position 40 can be calculated from the difference between the coordinates of the host vehicle 1 and the coordinates of the target parking position 40 in the map coordinate system.

[0069] The controller 17 calculates the target travel trajectory 44 of the host vehicle 1 from the current position 43 to the target parking position 40 based on the relative position of the host vehicle 1 with respect to the target parking position 40. Then, the parking assist control of the host vehicle 1 is implemented based on the calculated target travel trajectory 44.

[0070] Next, the functional structure of the controller 17 will be described in detail. Referring to Figure 4The HMI control section 50 outputs a map generation instruction to the map generation section 56 to cause the learned landmark data stored in the storage device 21 to be generated into a map when the parking position learning switch is operated by the occupant. The HMI control section 50 determines whether or not the occupant has performed a shift operation for a U-turn, and outputs the determination result to the parking assist control section 51. In addition, when it is detected that the parking assist start switch of the HMI 13 has been operated by the occupant, the detection result is output to the parking assist control section 51.

[0071] The parking assist control section 51 determines whether or not to activate the parking assist control. For example, when it is determined by the parking lot type determination section 54 described later that the vehicle 1 is located in a first-type parking lot and the above parking assist activation condition has been satisfied, the parking assist control is activated.

[0072] Alternatively, the parking assist control section 51 determines whether or not the vehicle 1 is located in the vicinity of a second-type parking lot stored as a target parking position. For example, it is determined whether or not the distance between the vehicle 1 and the second-type parking lot is equal to or less than a predetermined distance. For example, it can be determined by the positioning device 11 whether or not the distance between the vehicle 1 and the second-type parking lot is equal to or less than a predetermined distance. Alternatively, the feature amount of a landmark in the vicinity of the second-type parking lot can be stored in advance, and it can be determined whether or not the outside sensor 15 detects a landmark having a similar feature amount. The parking assist control section 51 activates the parking assist control when it is determined that the vehicle 1 is located in the vicinity of the second-type parking lot stored as a target parking position and the above parking assist activation condition has been satisfied.

[0073] When the parking assist control is activated, the parking assist control section 51 outputs a parking position calculation instruction to the comparison section 58 to calculate the position of the target parking position in the vehicle coordinate system. A travel trajectory calculation instruction to calculate a target travel trajectory of the vehicle 1 from the current position to the target parking position and a target speed profile of the vehicle 1 traveling along the target travel trajectory is output to the target trajectory generation section 59.

[0074] The target trajectory generation section 59 calculates the target travel trajectory of the vehicle 1 from the current position to the target parking position and the target speed profile, and outputs them to the parking assist control section 51. The calculation of the target travel trajectory and the target speed profile can be performed by using a known method employed in an automatic parking device.

[0075] The parking assist control section 51 outputs the target travel trajectory and the information of the current position of the vehicle 1 to the HMI control section 50. In the case where the target travel trajectory includes a U-turn, the information of the U-turn position is output to the HMI control section 50. The HMI control section 50 displays the target travel trajectory, the current position of the vehicle 1, and the U-turn position on the HMI 13.

[0076] In addition, the parking assist control unit 51 outputs a steering control command to the steering control unit 60 to control the vehicle 1 to travel along the calculated target driving trajectory. Furthermore, it outputs a vehicle speed control command to the vehicle speed control unit 61 to control the vehicle speed according to the calculated target vehicle speed change diagram.

[0077] The image conversion unit 52 converts the captured images from the camera into a bird's-eye view (panoramic monitoring image) observed from a virtual perspective directly above the vehicle 1. The image conversion unit 52 converts the captured images into bird's-eye views at predetermined intervals (e.g., every predetermined distance (e.g., 50 cm) or predetermined time (e.g., one second) of the vehicle 1, and accumulates the converted bird's-eye views along the driving path of the vehicle 1, thereby generating an image of the area surrounding the vehicle 1, i.e., a surrounding image.

[0078] The self-position calculation unit 53 uses a dead reckoning algorithm based on vehicle information output from vehicle sensor 16 to calculate the current position of the vehicle 1 in the map coordinate system.

[0079] Parking type determination unit 54 determines whether vehicle 1 is located in a Class I parking lot.

[0080] For example, such as Figure 2A As shown, in the first type of parking lot, similar patterns exist consecutively to represent multiple parking spaces 30a to 30d. Therefore, the parking lot type determination unit 54 can determine that the vehicle 1 is located in the first type of parking lot if similar patterns exist consecutively in the surrounding image generated by the image conversion unit 52. Alternatively, for example, it can also determine whether the vehicle 1 is located in the first type of parking lot based on the current position of the vehicle 1 measured by the positioning device 11 and the parking lot information stored in the map DB12.

[0081] When storing the learned target data in the storage device 21, the occupant manually drives the vehicle 1 to the target parking position and operates the parking position learning switch.

[0082] If it is determined that the vehicle 1 is located in a first-class parking lot, the object detection unit 55 detects the object indicating the parking space where the vehicle 1 is parked based on the surrounding image output from the image conversion unit 52, and generates object feature data by generating information indicating the features of the detected object.

[0083] For example, the object detection unit 55 can generate object feature data representing the parking space from a partial image (road pattern) of the whole or a part of the road markings that indicate a parked parking space.

[0084] Figure 5is an example of a schematic view of a partial region including a pattern of a road surface mark indicating a parking space. For example, the object detection unit 55 detects, as an object, a road surface mark 71 of an end portion of a road surface mark indicating the parking space 70 on a side of a travel lane adjacent to the parking space 70 among both end portions in the front-rear direction, and generates a partial image of a partial region 72 including the road surface mark 71 as object feature data of an object indicating a parking space.

[0085] In addition, for example, the object detection unit 55 can detect, as an object, a road surface mark 73 of an end portion of a road surface mark indicating the parking space 70 on a side opposite to the travel lane among both end portions in the front-rear direction adjacent to the parking space 70, and generate a partial image of a partial region 74 including the road surface mark 73 as object feature data of an object indicating a parking space.

[0086] In addition, for example, the object detection unit 55 can detect, as an object, a road surface mark 75 provided within the parking space 70, and generate a partial image of a partial region 76 including the road surface mark 75 as object feature data of an object indicating a parking space.

[0087] With reference to Figure 4 The object detection unit 55 receives the current position of the host vehicle 1 when the host vehicle 1 is parked in a parking space from the self-position calculating unit 53, and acquires a parking position Pr.

[0088] The map generating unit 56 stores, in the storage device 21, object feature data of an object detected by the object detection unit 55 and relative position data indicating a relative positional relationship between the parking position Pr and the object as learned object data, and generates map data 57.

[0089] On the other hand, in a case where it is determined that the host vehicle 1 is located in the vicinity of the second-type parking area, the object detection unit 55 detects an object around a parking space of the second-type parking area on the basis of the surrounding image output from the image converting unit 52. For example, the object detection unit 55 detects a feature point of an object and an image feature amount thereof. The detection of the feature point and the calculation of the image feature amount can be performed, for example, by a method such as SIFT, SURF, ORB, BRIAK, KAZE, AKAZE, or the like.

[0090] Further, the object detection unit 55 receives the current position of the host vehicle 1 from the self-position calculating unit 53. The map generating unit 56 calculates a position of an object on a map coordinate system on the basis of the current position of the host vehicle 1 when the object is detected, and calculates a parking position of the map coordinate system on the basis of the current position of the host vehicle 1 when the host vehicle 1 is parked in a parking space.

[0091] The map generation section 56 stores the feature points of the object detected by the object detection section 55 and the image feature amounts as object feature data in the storage device 21, and stores the position of the object in the map coordinate system and the parking position as relative position data between the object and the parking space in the storage device 21, thereby generating map data 57.

[0092] After that, when the parking assist control section 51 starts the parking assist control, the matching section 58 receives the parking position calculation instruction from the parking assist control section 51.

[0093] In the case where it is determined that the host vehicle 1 is located in the first type of parking lot, the matching section 58 reads the object feature data of the object indicating the parking space stored in the storage device 21 as the learned object data, and the relative position data between the object and the target parking position. The matching section 58 performs template matching on the surrounding image generated by the image conversion section 52 using the object feature data, thereby detecting the object indicating the parking space from the surrounding image. The matching section 58 calculates the current relative position of the host vehicle 1 with respect to the target parking position, i.e., the parking space, based on the detected position of the object on the surrounding image and the relative position data read from the storage device 21.

[0094] On the other hand, in the case where it is determined that the host vehicle 1 is located in the vicinity of the second type of parking lot, the object detection section 55 detects the surrounding objects around the host vehicle 1 based on the surrounding image output from the image conversion section 52. The matching section 58 matches the objects of the learned object data stored as the learned object data with the surrounding objects detected by the object detection section 55 at the time of implementing the parking assist, and associates the same objects with each other.

[0095] The matching section 58 calculates the current relative position of the host vehicle with respect to the target parking position based on the relative positional relationship between the objects of the learned object data associated with the surrounding objects and the target parking position, and the relative positional relationship between the surrounding objects and the host vehicle 1.

[0096] For example, let the position of the surrounding object be (x i , y i ), and the position of the object of the learned object data associated with the surrounding object be (x mi , y mi ) (i = 1 ~ N). The matching section 58 calculates the affine transformation matrix M affine based on the least square method using the following equation.

[0097] [Num 1]

[0098]

[0099] wherein,

[0100]

[0101] The collating section 58 converts the position of the target parking position in the map coordinate system stored in the map data 57 to the position of the target parking position in the vehicle coordinate system (targetx, targety) using the following equation. m m

[0102] [Num 2]

[0103]

[0104] When the target trajectory generating section 59 receives the travel trajectory calculation instruction from the parking assist control section 51, the target travel trajectory of the host vehicle 1 from the current position to the target parking position in the vehicle coordinate system and the target vehicle speed change map for the host vehicle 1 to travel along the target travel trajectory are calculated. When the steering control section 60 receives the steering control instruction from the parking assist control section 51, the steering actuator 19a is controlled so that the host vehicle 1 travels along the target travel trajectory. When the vehicle speed control section 61 receives the vehicle speed control instruction from the parking assist control section 51, the accelerator actuator 19b and the brake actuator 19c are controlled so that the vehicle speed of the host vehicle 1 is changed in accordance with the target vehicle speed change map. Thus, the host vehicle 1 is controlled to travel along the target travel trajectory.

[0105] When the parking assist control section 51 completes the parking assist control as the host vehicle 1 reaches the target parking position, the parking assist control section 51 operates the parking brake 18 and switches the gear to the parking gear (P range).

[0106] (Action)

[0107] Figure 6 is a flowchart of one example of a process of storing learned landmark data.

[0108] In step S1, the image conversion section 52 converts the captured image of the camera to an overhead image viewed from a virtual viewpoint directly above the host vehicle 1, and acquires a surrounding image. In step S2, the landmark detection section 55 detects a landmark from the acquired surrounding image. In the case where the host vehicle 1 parks in the first type of parking lot, the entire or a part of the road surface marking indicating the parked parking space is detected as a landmark. In the case where the host vehicle 1 parks in the second type of parking lot, a landmark around the parking space is detected.

[0109] ​​In step S3, the parking lot type determination section 54 determines whether the host vehicle 1 is located in a first type parking lot. In the case where the host vehicle 1 is located in the first type parking lot (step S4: Y), the process proceeds to step S5. In the case where the host vehicle 1 is located in the vicinity of a second type parking lot in which a single vehicle can be parked (step S4: N), the process proceeds to step S6. In step S5, the map generation section 56 stores, in the storage device 21, as learned landmark data, landmark feature data representing features of landmarks represented as parking spaces of the first type parking lot, and relative position data of the landmarks to the parking position. Thereafter, the process ends. In step S6, the map generation section 56 stores, in the storage device 21, as learned landmark data, landmark feature data representing features of landmarks in the vicinity of the parking space of the second type parking lot, and relative position data of the landmarks to the parking space. Thereafter, the process ends.

[0110] Figure 7 is a flowchart of a first example of the process when the parking assist is implemented. In step S10, the image conversion section 52 converts a captured image of the camera into an overhead image, and acquires a surrounding image. In step Sll, the landmark detection section 55 detects surrounding landmarks from the surrounding image. In step S12, the collation section 58 reads learned landmark data from the storage device 21. In step S13, the collation section 58 matches the learned landmark data and the surrounding landmarks. In step S14, the collation section 58 determines whether a plurality of parking spaces have been detected. In the case where a plurality of parking spaces have been detected (step S14: Y), the process proceeds to step S15. In the case where a plurality of parking spaces have not been detected (step S14: N), the process proceeds to step S16. In step S15, the collation section 58 selects an arbitrary one of the plurality of parking spaces as a parking space in which the host vehicle 1 is parked, by accepting a selection input of a passenger selecting an arbitrary one of the plurality of parking spaces. Thereafter, the process proceeds to step S16.

[0111] In step S16, the collation section 58 calculates a current relative position of the host vehicle 1 with respect to the target parking position, i.e., the parking space. In step S17, the target trajectory generation section 59 calculates a target travel trajectory and a target vehicle speed change pattern. In step S18, the steering control section 60 and the vehicle speed control section 61 control the steering actuator 19a, the accelerator actuator 19b, and the brake actuator 19c, based on the target travel trajectory and the target vehicle speed change pattern. In step S19, the parking assist control section 51 operates the parking brake 18 and switches the gear to the P range when the parking assist control is completed.

[0112] Figure 8is a flowchart of a second example of the process of implementing the parking assist. In step S20, the parking lot type determination section 54 determines whether the host vehicle 1 is located in the first type parking lot. In step S21, the image conversion section 52 converts the captured image of the camera into an overhead image observed from a virtual viewpoint directly above the host vehicle 1, and acquires the surrounding image. In the case where the host vehicle 1 is located in the first type parking lot (step S22: Y), the process proceeds to step S23. In the case where the host vehicle 1 is located in the vicinity of the second type parking lot capable of parking a single vehicle (step S22: N), the process proceeds to step S25. In step S23, the collation section 58 reads the object feature data of the object indicating the parking space of the first type parking lot, and the relative position data between the object and the target parking position, which are stored as the learned object data. In step S24, the collation section 58 detects the object indicating the parking space from the surrounding image acquired in step S21 by template matching of the surrounding image using the object feature data, and calculates the current relative position of the host vehicle 1 with respect to the target parking position, i.e., the parking space. Thereafter, the process proceeds to step S29.

[0113] In step S25, the object detection section 55 detects the surrounding object around the host vehicle 1. In step S26, the collation section 58 reads the object feature data of the object around the parking space of the second type parking lot, and the relative position data between the object and the target parking position, which are stored as the learned object data. In step S27, the collation section 58 matches the object of the learned object data with the surrounding object. In step S28, the target parking position is calculated based on the object matched by the collation section 58. Thereafter, the process proceeds to step S29. The process of steps S29 to S31 is the same as that of steps S18 to S19 of Figure 7

[0114] (EFFECTS OF THE EMBODIMENTS)

[0115] (1) In the parking assist method of the embodiments, it is determined whether the host vehicle 1 is located in the first type parking lot having a plurality of parking spaces, or whether the host vehicle 1 is located in the vicinity of the second type parking lot capable of parking a single vehicle, in the case where it is determined that the host vehicle 1 is located in the first type parking lot, when the host vehicle 1 is parked in an arbitrary one of the plurality of parking spaces, i.e., the target parking position, data indicating the object of the parking space common to the plurality of parking spaces and the relative positional relationship of the parking position in the target parking position is stored in the storage device as the learned object data, and in the case where it is determined that the host vehicle 1 is located in the vicinity of the second type parking lot, data indicating the relative positional relationship of the object detected in the vicinity of the second type parking lot when the host vehicle 1 is parked in the second type parking lot and the target parking position is stored in the storage device as the learned object data.

[0116] ​Thus, in the first type of parking lot having a plurality of parking spaces, by detecting a local pattern indicating a parking space from the surroundings of the host vehicle 1, a plurality of parking spaces indicated by similar patterns can be detected as each target parking position. As a result, the parking space in which the host vehicle 1 is parked can be arbitrarily selected. On the other hand, in the second type of parking lot in which a single vehicle can be parked, by storing the overall configuration pattern of the object detected in the vicinity of the parking lot, stable detection of the target parking position can be performed regardless of changes in the environment or changes in the vehicle position.

[0117] (2) In the case where it is determined that the host vehicle 1 is located in the first type of parking lot, the parking space in which the host vehicle is parked can be selected by accepting the selection input of the occupant selecting an arbitrary one of the plurality of parking spaces. Thus, the occupant can arbitrarily select the parking space in which the host vehicle 1 is parked among the plurality of parking spaces.

[0118] (3) Whether or not similar patterns are continuously present in the image obtained by capturing the surroundings of the host vehicle 1, i.e., the surrounding image, can be determined, and in the case where similar patterns are continuously present in the surrounding image, it can be determined that the host vehicle 1 is located in the first type of parking lot.

[0119] Thus, whether or not the host vehicle 1 is located in the first type of parking lot can be easily determined on the basis of the surrounding image.

[0120] (4) Whether or not the host vehicle 1 is located in the first type of parking lot can also be determined on the basis of the positioning result of the positioning device possessed by the host vehicle 1 and map information. Thus, for example, in the case where a navigation device is provided, whether or not the host vehicle is located in the first type of parking lot can be easily determined in the vehicle.

[0121] (5) The data indicating the relative positional relationship between the road surface marking of the end portion of the driveway adjacent to the object parking space among the both end portions in the front-rear direction of the object parking space and the parking position in the object parking space can be stored in the storage device as the learned object data. Thus, the learned object data indicating the object as a parking space can be stored.

[0122] (6) The data indicating the relative positional relationship between the road surface marking of the end portion of the driveway on the opposite side to the object parking space among the both end portions in the front-rear direction of the object parking space and the parking position in the object parking space can also be stored in the storage device as the learned object data. Thus, the learned object data indicating the object as a parking space can be stored.

[0123] (7) The data indicating the relative positional relationship between the road surface marking provided in the object parking space and the parking position in the object parking space can also be stored in the storage device as the learned object data. Thus, the learned object data indicating the object as a parking space can be stored.

[0124] Reference numerals

[0125] 1 host vehicle; 10 parking assistance device; 17 controller.

Claims

1. A parking assistance method that assists parking of a host vehicle to a target parking position, characterized by comprising: storing, in advance, in a storage device, data indicating a relative positional relationship between a target parking position and a landmark existing around the target parking position as learned landmark data; detecting a position of a landmark existing around the host vehicle; calculating a relative positional relationship between the target parking position and a current position of the host vehicle, based on the learned landmark data and the position of the landmark around the host vehicle; calculating a travel trajectory of the host vehicle from the current position to the target parking position, based on the relative positional relationship between the target parking position and the current position of the host vehicle; assisting parking of the host vehicle to the target parking position, based on the travel trajectory; further, determining whether the host vehicle is located in a first type of parking lot having a plurality of parking spaces and from which any one of the plurality of parking spaces can be selected, or whether the host vehicle is located in the vicinity of a second type of parking lot in which a single vehicle can be parked, in a case where it is determined that the host vehicle is located in the first type of parking lot, determining whether a landmark around an arbitrary one of the plurality of parking spaces, i.e., an object parking space, in which the host vehicle is parked is a landmark common to the plurality of parking spaces and indicating a parking space, in a case where the landmark around the object parking space is a landmark common to the plurality of parking spaces and indicating a parking space, storing the landmark around the object parking space, and storing, in the storage device, data indicating a relative positional relationship between the stored landmark and a parking position in the object parking space as the learned landmark data, in a case where it is determined that the host vehicle is located in the vicinity of the second type of parking lot, storing, in the storage device, data indicating a relative positional relationship between a landmark detected in the vicinity of the second type of parking lot and the target parking position as the learned landmark data, in a case where the host vehicle is parked in the second type of parking lot.

2. The parking assistance method according to claim 1, characterized in that, in a case where it is determined that the host vehicle is located in the first type of parking lot, a parking space in which the host vehicle is parked is selected by accepting a selection input of a passenger selecting an arbitrary one of the plurality of parking spaces.

3. The parking assistance method according to claim 1, characterized in that, it is determined whether a similar pattern is continuously present in an image around the host vehicle, i.e., a surrounding image, in a case where it is determined that the host vehicle is located in the first type of parking lot.

4. The parking assistance method according to claim 1, characterized in that, it is determined whether the host vehicle is located in the first type of parking lot, based on a positioning result of a positioning device possessed by the host vehicle and map information.

5. The parking assistance method according to any one of claims 1 to 4, characterized in that, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ Data indicating a relative positional relationship between a road surface mark of an end portion of a driveway adjacent to an end portion of the target parking space on the opposite side among both end portions of the target parking space in the front-rear direction and a parking position within the target parking space is stored in the storage device as the learned landmark data.

6. The parking assist method according to any one of claims 1 to 4, characterized in that, Data indicating a relative positional relationship between a road surface mark of an end portion of a driveway adjacent to an end portion of the target parking space on the opposite side among both end portions of the target parking space in the front-rear direction and a parking position within the target parking space is stored in the storage device as the learned landmark data.

7. The parking assist method according to any one of claims 1 to 4, characterized in that, Data indicating a relative positional relationship between a road surface mark of an end portion of a driveway adjacent to an end portion of the target parking space on the opposite side among both end portions of the target parking space in the front-rear direction and a parking position within the target parking space is stored in the storage device as the learned landmark data.

8. A parking assist device characterized by comprising: having: a sensor that detects a landmark around a host vehicle; a storage device; a controller that stores, in the storage device, in advance, data indicating a relative positional relationship between a landmark existing around a target parking position and the target parking position as learned landmark data, detects a position of a landmark existing around the host vehicle, i.e., a surrounding landmark, using the sensor, calculates a relative positional relationship between the target parking position and a current position of the host vehicle based on the learned landmark data and the position of the surrounding landmark, calculates a travel trajectory of the host vehicle from the current position to the target parking position based on the relative positional relationship between the target parking position and the current position of the host vehicle, and assists parking of the host vehicle into the target parking position based on the travel trajectory; the controller, determines whether the host vehicle is located in a first type of parking lot having a plurality of parking spaces and from which any one of the plurality of parking spaces can be selected or whether the host vehicle is located in the vicinity of a second type of parking lot in which a single vehicle can be parked; in a case where it is determined that the host vehicle is located in the first type of parking lot, determines whether a landmark of an arbitrary one of the plurality of parking spaces, i.e., a target parking space, is a landmark common to the plurality of parking spaces and indicating a parking space when the host vehicle parks in the target parking space, stores a landmark around the target parking space in a case where the landmark around the target parking space is a landmark common to the plurality of parking spaces and indicating a parking space, and stores, in the storage device, data indicating a relative positional relationship between the stored landmark and a parking position within the target parking space as the learned landmark data; in a case where it is determined that the host vehicle is located in the vicinity of the second type of parking lot, stores, in the storage device, data indicating a relative positional relationship between a landmark detected in the vicinity of the second type of parking lot and the target parking position as the learned landmark data when the host vehicle parks in the second type of parking lot.

Images