Parking assistance method and parking assistance device

Abstract

In a parking assistance method for assisting a host vehicle in parking to a target parking position, it is determined whether the host vehicle is located in a first-type parking lot having a plurality of parking spaces, or in the vicinity of a second-type parking lot in which a single vehicle can be parked (S1), and if it is determined that the host vehicle is located in the first-type parking lot, the host vehicle is parked in the second-type parking lot (S2). When a host vehicle is parked in a target parking space, which is any one of a plurality of parking spaces, data indicating a relative positional relationship between a target object indicating that the target parking space is a parking space, and a parking position in the target parking space, is stored in a storage device (S4, S5). When it is determined that the host vehicle is in the vicinity of the second-type parking lot, data indicating a relative positional relationship between a target parking position and a target object detected in the vicinity of the second-type parking lot when the host vehicle is parked in the second-type parking lot is stored in a storage device (S6, S7).

Description

Technical Field

[0001] The invention relates to a parking assistance method and a parking assistance device. Background Art

[0002] Patent document 1 discloses a driving control device that extracts and stores landmarks from an image previously taken of the surroundings of a target parking position, calculates the relative position of the target parking position with respect to the vehicle based on the positions of the stored landmarks and the positions of the landmarks extracted from an image previously taken of the surroundings of the vehicle during automatic parking, and automatically moves the vehicle to the target parking position based on the calculated relative position.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: (Japanese) Patent Application Publication No. 2017-138664 Summary of the invention

[0006] Technical problem to be solved by the invention

[0007] However, in the case of a parking lot with multiple parking spaces (parking frame lines) such as those installed in companies, hospitals, stores, residential complexes, public facilities, etc., when any parking space is stored as a target parking position, there is a problem that the vehicle can only park on the same parking frame line.

[0008] An object of the present invention is to enable a host vehicle to be parked in any parking space when there are a plurality of parking spaces in a parking lot, in parking assistance for assisting the parking of a host vehicle at the target parking space by pre-storing landmarks near the target parking space.

[0009] Technical solutions for solving technical problems

[0010] According to one embodiment of the present invention, a parking assistance method for assisting a vehicle to park at a target parking position is provided. In the parking assistance method, data indicating the relative positional relationship between the target parking position and the object markers existing around the target parking position is stored in a storage device as learned object marker data in advance, the positions of the object markers existing around the vehicle, i.e., the positions of the surrounding object markers, are detected, and the relative positional relationship between the target parking position and the current position of the vehicle is calculated based on the learned object marker data and the positions of the surrounding object markers, and the driving trajectory of the vehicle from the current position to the target parking position is calculated based on the relative positional relationship between the target parking position and the current position of the vehicle, and the parking of the vehicle at the target parking position is assisted based on the driving trajectory. In the parking assistance method, it is determined whether the vehicle is located in a first-class parking lot having multiple parking spaces, or whether the vehicle is located near a second-class parking lot that can accommodate a single vehicle. When it is determined that the vehicle is located in the first-class parking lot, data indicating the relative positional relationship between an object mark common to the multiple parking spaces and indicating the parking spaces and the parking position in the target parking space when the vehicle is parked in any one of the multiple parking spaces, i.e., the target parking space, is stored in a storage device as learned object mark data. When it is determined that the vehicle is located near a second-class parking lot, data indicating the relative positional relationship between an object mark detected near the second-class parking lot when the vehicle is parked in the second-class parking lot and the target parking position is stored in the storage device as learned object mark data.

[0011] Effects of the Invention

[0012] In the present invention, in the parking assistance for assisting the parking of the host vehicle at the target parking position by pre-storing landmarks near the target parking position, the host vehicle can be parked in any parking space when there are a plurality of parking spaces in the parking lot. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a diagram showing an example of the schematic configuration of a parking assist device.

[0014] Figure 2A This is an explanatory diagram of a first example of the process of storing learned object label data.

[0015] Figure 2B It is an explanatory diagram of a first example of the process when performing parking assistance.

[0016] Figure 2C This is an explanatory diagram of an example of a parking space detection method.

[0017] Figure 3A This is an explanatory diagram of a first example of the process of storing learned object label data.

[0018] Figure 3BIt is an explanatory diagram of a second example of the process when performing parking assistance.

[0019] Figure 4 yes Figure 1 A block diagram of an example of the functional structure of a controller.

[0020] Figure 5 It is a schematic diagram of an example of a partial area including a pattern representing a parking space.

[0021] Figure 6 This is a flowchart of an example of storage processing of learned object label data.

[0022] Figure 7 This is a flowchart of a first example of processing when performing parking assistance.

[0023] Figure 8 This is a flowchart of a second example of the process when performing parking assistance. DETAILED DESCRIPTION

[0024] (constitute)

[0025] Reference Figure 1 . The vehicle 1 has a parking assistance device 10 for assisting the vehicle 1 in parking at a target parking position. The parking assistance device 10 assists the vehicle 1 in traveling along a target driving trajectory from the current position to the target parking position. For example, the automatic driving of the vehicle 1 may be controlled so that the vehicle 1 travels to the target parking position along the target driving trajectory. The automatic driving that controls the vehicle 1 to travel to the target parking position along the target driving trajectory refers to controlling all or part of the steering angle, driving force, and braking force of the vehicle to automatically implement all or part of the control of the vehicle 1 traveling along the target driving trajectory. In addition, the parking of the vehicle 1 may be assisted by displaying the target driving trajectory and the current position of the vehicle 1 on a display device that can be seen by the passengers of the vehicle 1.

[0026] The positioning device 11 measures the current position of the host vehicle 1. The positioning device 11 includes, for example, a global navigation satellite system (GNSS) receiver.

[0027] Map data is stored in the map database (map DB) 12. The map data stored in the map database 12 may be, for example, navigation map data or high-precision map data suitable as an autonomous driving map. The map data includes information on the location and range of parking lots set up in facilities such as companies, hospitals, stores, residential complexes, and public facilities.

[0028] The human machine interface (HMI) 13 is an interface device for transmitting and receiving information between the parking assistance device 10 and the occupant, and includes a display device, a speaker or a buzzer, and an operating element visible to the occupant of the host vehicle 1 .

[0029] The shift switch (shift SW) 14 is a switch used by the driver or the parking assist device 10 to switch the shift position of the host vehicle 1 .

[0030] The external sensor 15 detects objects within a specified distance range from the vehicle 1. The external sensor 15 detects the surrounding environment of the vehicle 1, such as the relative position of the objects existing around the vehicle 1 and the vehicle 1, the distance between the vehicle 1 and the objects, and the direction in which the objects exist. The external sensor 15 may include, for example, a camera that takes pictures of the surrounding environment of the vehicle 1. In the following description, the camera of the external sensor 15 is simply referred to as a "camera". The external sensor 15 may also include a distance measuring device such as a laser rangefinder, a radar, or LiDAR (Light Detection and Ranging).

[0031] The vehicle sensor 16 detects various information (vehicle information) of the vehicle 1. The vehicle sensor 16 may include, for example, a vehicle speed sensor for detecting the running speed of the vehicle 1, a wheel speed sensor for detecting the rotation speed of each tire of the vehicle 1, a three-axis acceleration sensor (G sensor) for detecting acceleration (including deceleration) in three axial directions of the vehicle 1, a steering angle sensor for detecting a steering angle, a steering angle sensor for detecting a steering angle of a steering wheel, a gyro sensor, and a yaw rate sensor.

[0032] The controller 17 is an electronic control unit that performs parking assist control of the vehicle 1. The controller 17 includes a processor 20, and peripheral components such as a storage device 21. The processor 20 may be, for example, a CPU or an MPU. The storage device 21 may include a semiconductor storage device, a magnetic storage device, an optical storage device, and the like. The functions of the controller 17 described below are implemented, for example, by the processor 20 executing a computer program stored in the storage device 21. It should be noted that the controller 17 may also be formed by dedicated hardware for performing each information processing described below.

[0033] The steering actuator 19a controls the steering direction and steering amount of the steering mechanism of the vehicle 1 according to the control signal of the controller 17. The accelerator actuator 19b controls the accelerator opening of the engine or the drive motor, i.e., the drive device according to the control signal of the controller 17. The brake actuator 19c operates the brake device according to the control signal of the controller 17.

[0034] Next, the parking assistance control performed by the parking assistance device 10 is described. When the parking assistance is performed by the parking assistance device 10, data indicating the relative positional relationship between the object markers existing around the target parking position, which is the position where the host vehicle 1 is to be parked, and the target parking position are pre-stored in the storage device 21. In the following description, the data indicating the relative positional relationship between the object markers stored in the storage device 21 and the target parking position may be recorded as "learned object marker data".

[0035] For example, the external sensor 15 can be used to detect the object mark around the target parking position when the host vehicle 1 is located near the target parking lot. For example, the external sensor 15 can be used to detect the object mark around the host vehicle 1 when the occupant of the host vehicle 1 (such as the driver) parks the host vehicle 1 at the target parking position by manual driving. For example, the external sensor 15 can be used to detect the object mark around the host vehicle 1 when the host vehicle 1 is parked at the target parking position. At this time, for example, the object mark can be detected from the image around the host vehicle 1 captured by the camera, or the object mark around the host vehicle 1 can be detected by using a distance measuring device.

[0036] The learned object mark data stored in the storage device 21 may include: data representing the characteristics of the detected object mark (sometimes recorded as "object mark feature data" in the following description), and data on the relative position relationship between the object mark and the target parking position (sometimes recorded as "relative position data" in the following description).

[0037] Figure 2A and Figure 3A 2 is an explanatory diagram of an example of a process of storing learned object marker data. When the learned object marker data is stored in the storage device 21 , for example, the occupant operates a “parking position learning switch” provided as an operating element of the HMI 13 .

[0038] When storing the learned object marker data, the controller 17 determines whether the vehicle 1 is located in a parking lot with multiple parking spaces (sometimes referred to as a "first-class parking lot" in the following description), or whether the vehicle is located near a parking lot that can accommodate a single vehicle (sometimes referred to as a "second-class parking lot" in the following description).

[0039] Examples of the first type of parking lot include parking lots installed in companies, hospitals, stores, residential complexes, public facilities, etc. Examples of the second type of parking lot include parking lots installed in private homes.

[0040] Figure 2AThe parking lot shown is a first type parking lot having a plurality of parking spaces 30a to 30d. For example, it is assumed that the vehicle 1 moves along the route 31 by manual driving and parks in the parking space 30c. The parking space 30c is an example of a "target parking space" described in the claims.

[0041] The controller 17 detects the road marking indicating the parking space 30c (e.g., white lines indicating the parking space frame) or three-dimensional objects such as wheel stoppers painted on the road surface of the parking lot, i.e., object marker features, as object markers indicating that the parking space 30c is a parking space, based on the image of the surroundings of the host vehicle 1 captured by the camera. The controller 17 stores the object marker feature data indicating the detected road marking features and the relative position data between the detected road marking and the target parking position in the parking space as learned object marker data in the storage device 21.

[0042] For example, the controller 17 may store the detected road marking pattern 32 as object feature data in the storage device 21. In addition, the relative position data between the parking position Pr when the host vehicle 1 is parked in the parking space 30c and the pattern 32 may be stored in the storage device 21. For example, the coordinates of the road marking detected when the host vehicle 1 is parked in the parking space 30c on the vehicle coordinate system (i.e., the coordinate system based on the current position of the host vehicle 1) may be stored as relative position data.

[0043] In the following description, the pattern 32 of the road surface marking indicating the parking space may be recorded as a “road surface pattern.” For example, the controller 17 may extract a partial image of a local area including the entirety or a portion of the road surface marking indicating a single parking space from an image captured by a camera of the surroundings of the vehicle 1, and store the partial image in the storage device 21 as the road surface pattern 32.

[0044] In this way, in the first type parking lot having a plurality of parking spaces 30a to 30d, the controller 17 stores local information of the features of the road surface markings of the parking space unit among the road surface markings of the entire parking lot as object feature data in the storage device 21. For example, a local road surface pattern 32 of the road surface markings of the parking space unit is stored in the storage device 21. At this time, the controller 17 determines whether the road surface pattern 32 is a feature of the parking space common to the plurality of parking spaces 30a to 30d, and stores the road surface pattern 32 if it is a parking space common to the plurality of parking spaces 30a to 30d. For example, the controller 17 may determine whether the road surface pattern 32 is a feature of the parking space common to the plurality of parking spaces 30a to 30d by determining whether a plurality of road surface patterns 32 are continuously present in the image captured by the camera of the surroundings of the host vehicle 1.

[0045] Figure 3AThe parking lot shown is a second type parking lot having a parking space 40 capable of parking a single vehicle. For example, it is assumed that when the vehicle 1 moves along the route 41 by manual driving and parks in the parking space 40, the parking space 40 is stored in the storage device 21 as the parking target position.

[0046] The controller 17 detects the objects around the parking space 40 from the image captured by the camera of the surroundings of the host vehicle 1 when the host vehicle 1 is located near the parking space 40 (for example, when the occupant parks the host vehicle 1 in the parking space 40 by manual driving). For example, the controller 17 can detect the feature points on the image as the objects around the parking space 40. For example, the edge points or points with features in the shape of the adjacent pixels such as the edge or corner of the object such as the road surface marking, road boundary, obstacle, etc. on the photographed image obtained by shooting with the camera, where the brightness changes by more than a predetermined amount, are detected as feature points.

[0047] The controller 17 stores the object feature data indicating the features of the object detected in the predetermined area 42 around the parking space 40 as the learned object feature data in the storage device 21. For example, the controller 17 may store the feature points and feature quantities of the detected object as the object feature data in the storage device 21. In addition, the controller 17 stores the relative position data between the parking target position, i.e., the parking space 40 and the object as the learned object feature data in the storage device 21.

[0048] exist Figure 3A The circular icons in the middle schematically represent objects stored as learned object data.

[0049] When storing the relative position data of the object and the parking space 40, for example, the coordinates of the object and the parking space 40 on a coordinate system (hereinafter referred to as a "map coordinate system") with a fixed point as a reference point may be stored. In this case, the current position of the host vehicle 1 on the map coordinate system measured when it is located in the parking space 40 may be stored as the position of the parking space 40. In addition, instead of the map coordinate system, the relative positions of the parking space 40 with respect to each object may be stored.

[0050] It should be noted that the predetermined area 42 for storing the learned object marker data may be, for example, an area with a length of about 20m to 30m on one side or in diameter. In this way, when storing the learned object marker data around the parking space 40 in the second type parking lot, the overall arrangement diagram pattern of the object markers (feature points) in a relatively large range including the parking space 40 is stored in the storage device 21.

[0051] On the other hand, when the learned object marker data of the parking space 30c in the first type parking lot is stored, a partial pattern representing the object marker (road surface marking) within a relatively small range of one parking space is stored in the storage device 21. That is, the first range for storing the object marker in the first type parking lot is smaller than the second range for storing the object marker in the second type parking lot.

[0052] Figure 2B 1 is an explanatory diagram of a processing example when parking assistance is performed in a first-class parking lot. When the host vehicle 1 is located in a first-class parking lot and a predetermined parking assistance activation condition is satisfied, the controller 17 performs parking assistance for the host vehicle 1. For example, the controller 17 can determine whether the host vehicle 1 is located in a first-class parking lot based on the current position of the host vehicle 1 measured by the positioning device 11 and the parking lot information stored in the map DB 12.

[0053] As a parking assist activation condition, the controller 17 determines whether the occupant has performed a shift operation for switching the vehicle 1 forward and backward when the vehicle 1 is in the first type parking lot. When the gear position is switched from the forward gear (D gear) to the reverse gear (R gear), or from the R gear to the D gear, the controller 17 determines that the shift operation for switching back has been performed, and the parking assist can be activated.

[0054] The controller 17 may determine whether or not the occupant has operated a “parking assist start switch” provided in the HMI 13 as the parking assist start condition, and start parking assist when the parking assist start switch is operated.

[0055] The controller 17 generates an image of a surrounding area 35 of the host vehicle 1 (surrounding image) by driving along a track 34 in a first-type parking lot and accumulating images of the surroundings of the host vehicle 1 captured by a camera.

[0056] The controller 17 detects one or more parking spaces 36a to 36c existing around the vehicle 1 by detecting an area including an object having features similar to the object feature data stored as the learned object feature data (i.e., the object feature data indicating that the object is a parking space) from the surrounding image. For example, the parking spaces 36a to 36c are detected by detecting an area including a road surface marking similar to the road surface pattern 32 indicating a parking space common to the plurality of parking spaces 36a to 36c from the surrounding image. The parking spaces 36a to 36c are an example of the "surrounding object" described in the claims.

[0057] For example, the controller 17 can detect the parking spaces 36a to 36c by performing template matching on the surrounding image using the road surface pattern 32 as a template. For example, the controller 17 scans the road surface pattern 32 on the surrounding image, calculates the similarity between the road surface pattern 32 at each position on the surrounding image and the surrounding image, and detects the area where the similarity is above a threshold.

[0058] Figure 2C 3 is an explanatory diagram of an example of a method for detecting a parking space. For example, the extension direction of the driving passage 33 is set as the main scanning direction, and each time the main scanning is completed, the position in the sub-scanning direction is changed, and the road surface pattern 32 is scanned on the surrounding image of the surrounding area 35 of the vehicle 1. Then, the similarity between the road surface pattern 32 at each position on the surrounding image and the surrounding image is calculated, and the positions Pfa, Pfb, and Pfc where the similarity is greater than the threshold value Th are detected as the positions of the parking spaces 36a to 36c.

[0059] Reference Figure 2B . The controller 17 selects a parking space for the vehicle 1 to be parked from the detected parking spaces 36a to 36c. For example, the controller 17 may display the detected parking spaces 36a to 36c on the HMI 13, present the parking space candidates to the occupant, and accept the occupant's selection input to select any parking space from the above-mentioned parking space candidates, thereby selecting a parking space for the vehicle 1 to be parked. In addition, for example, the controller 17 may also select a parking space that can be parked based on the shortest movement route. Here, it is assumed that the parking space 36c has been selected.

[0060] When a parking space 36c is selected for parking the vehicle 1, the controller 17 calculates the relative position of the vehicle 1 with respect to the target parking position based on the detected position of the parking space 36c and the relative position data between the object marker stored as learned object marker data and the target parking position (i.e., the relative position data between the road surface marking representing the parking space and the target parking position).

[0061] For example, based on the detected position of the parking space 36c and the relative position data between the parking position Pr when the vehicle 1 is parked in the parking space 30c and the road surface marking of the parking space 30c stored as learned object marker data, the relative position of the vehicle 1 relative to the target parking position is calculated.

[0062] The controller 17 calculates a target travel trajectory 37 of the host vehicle 1 from the current position to the target parking position based on the relative position of the host vehicle 1 to the target parking position. Then, the controller 17 performs parking assist control of the host vehicle 1 based on the calculated target travel trajectory.

[0063] Figure 3B1 is an explanatory diagram of a processing example when parking assistance is performed in a second type parking lot. The controller 17 performs parking assistance for the host vehicle 1 when the host vehicle 1 is located near a parking space (ie, a target parking position) 40 and the parking assistance start condition is satisfied.

[0064] The controller 17 detects the object landmarks around the host vehicle 1 from the surrounding image obtained by photographing the surroundings of the host vehicle 1 with a camera. In the following description, the object landmarks around the host vehicle 1 extracted when performing parking assistance are other examples of the "surrounding object landmarks" described in the claims. Figure 3B In the figure, triangle icons indicate surrounding objects.

[0065] The controller 17 matches the learned object data stored in the storage device 21 with the surrounding objects to associate the same objects with each other.

[0066] The controller 17 calculates the relative position of the vehicle 1 relative to the target parking position 40 (i.e., the relative position of the vehicle 1 relative to the target parking position) based on the relative position relationship between the surrounding objects detected when implementing parking assistance and the vehicle 1, and the relative position relationship between the objects of the learned object data associated with the surrounding objects and the target parking position 40.

[0067] For example, the controller 17 may also calculate the position of the target parking position 40 on the vehicle coordinate system. For example, when the coordinates of the object and the coordinates of the target parking position 40 are expressed as coordinates of the map coordinate system in the learned object data, the coordinates of the target parking position 40 on the map coordinate system may be converted into coordinates on the vehicle coordinate system based on the coordinates of the surrounding objects detected when the parking assistance is implemented and the coordinates of the map coordinate system of the objects in the learned object data.

[0068] Alternatively, the current position of the vehicle 1 on the map coordinate system can be determined based on the coordinates of the surrounding objects detected when implementing parking assistance and the coordinates of the objects whose object data has been learned on the map coordinate system. The relative position of the vehicle 1 with respect to the target parking position 40 can be calculated based on the difference between the coordinates of the vehicle 1 on the map coordinate system and the coordinates of the target parking position 40.

[0069] The controller 17 calculates a target travel trajectory 44 of the host vehicle 1 from the current position 43 to the target parking position 40 based on the relative position of the host vehicle 1 to the target parking position 40. Then, based on the calculated target travel trajectory 44, the parking assist control of the host vehicle 1 is performed.

[0070] Next, the functional structure of the controller 17 will be described in detail. Figure 4When the parking position learning switch is operated by the passenger, the HMI control unit 50 outputs a map generation instruction to store the learned object marker data in the storage device 21 to the map generation unit 56. The HMI control unit 50 determines whether the passenger has performed a shift operation for turning back, and outputs the determination result to the parking assist control unit 51. In addition, when it is detected that the parking assist start switch of the HMI 13 has been operated by the passenger, the detection result is output to the parking assist control unit 51.

[0071] The parking assist control unit 51 determines whether to activate the parking assist control. For example, the parking assist control is activated when the parking lot type determination unit 54 described later determines that the vehicle is located in a first-type parking lot and the parking assist activation condition is satisfied.

[0072] Alternatively, the parking assist control unit 51 determines whether the vehicle 1 is located near the second type parking lot stored as the target parking position. For example, it is determined whether the distance between the vehicle 1 and the second type parking lot is less than a specified distance. For example, the positioning device 11 can be used to determine whether the distance between the vehicle 1 and the second type parking lot is less than a specified distance. Alternatively, feature quantities of object markers near the second type parking lot can be pre-stored to determine whether the external sensor 15 detects object markers with similar feature quantities. The parking assist control unit 51 starts the parking assist control when it is determined that the vehicle 1 is located near the second type parking lot stored as the target parking position and the above-mentioned parking assist start condition has been met.

[0073] When the parking assist control is activated, the parking assist control unit 51 outputs a parking position calculation instruction for calculating the position of the target parking position in the vehicle coordinate system to the comparison unit 58. A driving trajectory calculation instruction for calculating a target driving trajectory of the host vehicle 1 from the current position to the target parking position and a target vehicle speed change diagram of the host vehicle 1 traveling along the target driving trajectory is output to the target trajectory generation unit 59.

[0074] The target trajectory generation unit 59 calculates a target travel trajectory of the host vehicle 1 from the current position to the target parking position and a target vehicle speed variation map, and outputs them to the parking assist control unit 51. The target travel trajectory and the target vehicle speed variation map can be calculated using a known method used in an automatic parking device.

[0075] The parking assist control unit 51 outputs the target driving trajectory and the information of the current position of the host vehicle 1 to the HMI control unit 50. When the target driving trajectory includes a turnaround, the information of the turnaround position is output to the HMI control unit 50. The HMI control unit 50 displays the target driving trajectory, the current position of the host vehicle 1, and the turnaround position on the HMI 13.

[0076] The parking assist control unit 51 outputs a steering control command to the steering control unit 60 for steering the host vehicle 1 along the calculated target driving trajectory. The parking assist control unit 51 outputs a vehicle speed control command to the vehicle speed control unit 61 for controlling the vehicle speed according to the calculated target vehicle speed map.

[0077] The image conversion unit 52 converts the captured image of the camera into a bird's-eye view image (panoramic monitoring image) observed from a virtual perspective directly above the host vehicle 1. The image conversion unit 52 converts the captured image into a bird's-eye view image at a predetermined interval (for example, every time the host vehicle 1 travels a predetermined distance (for example, 50 cm) or a predetermined time (for example, one second)), and accumulates the converted bird's-eye view images along the travel route of the host vehicle 1, thereby generating an image of the surrounding area of ​​the host vehicle 1, that is, a surrounding image.

[0078] The own position calculation unit 53 calculates the current position of the own vehicle 1 on the map coordinate system using the dead reckoning method based on the vehicle information output from the vehicle sensor 16 .

[0079] The parking lot type determination unit 54 determines whether the host vehicle 1 is located in a first-type parking lot.

[0080] For example, Figure 2A As shown in FIG. 1 , similar patterns are continuously present in order to represent the plurality of parking spaces 30a to 30d in the first type parking lot. Therefore, the parking lot type determination unit 54 can determine that the vehicle 1 is located in the first type parking lot when similar patterns are continuously present in the surrounding image generated by the image conversion unit 52. Alternatively, for example, it is also possible to determine whether the vehicle 1 is located in the first type parking lot based on the current position of the vehicle 1 measured by the positioning device 11 and the parking lot information stored in the map DB 12.

[0081] When the learned object marker data is stored in the storage device 21 , the occupant parks the host vehicle 1 at the target parking position by manual driving and operates the parking position learning switch.

[0082] When it is determined that the vehicle 1 is located in the first type parking lot, the object marker detection unit 55 detects an object marker indicating the parking space where the vehicle 1 is parked based on the surrounding image output from the image conversion unit 52, and generates information indicating the features of the detected object marker as object marker feature data.

[0083] For example, the object marker detection unit 55 may generate a partial image (road surface pattern) including the entirety or a portion of a local area of ​​a road surface marking indicating a parked parking space as object marker feature data indicating the parking space.

[0084] Figure 51 is a schematic diagram of an example of a partial area of ​​a pattern including a road surface marking indicating a parking space. For example, the object marker detecting unit 55 detects a road surface marking 71 at the end of the road surface marking indicating a parking space 70 at the end of the driving lane adjacent to the parking space 70 in the front-rear direction as an object marker, and generates a partial image of a partial area 72 including the road surface marking 71 as object marker feature data of the object marker indicating the parking space.

[0085] In addition, for example, the object marker detection unit 55 can detect a road surface marking 73 at the end of the road surface marking representing the parking space 70 in the front-rear direction on the opposite side of the driving lane adjacent to the parking space 70 as an object marker, and generate a partial image of a local area 74 including the road surface marking 73 as object marker feature data of the object marker representing the parking space.

[0086] For example, the object marker detection unit 55 may detect a road surface marking 75 provided in the parking space 70 as an object marker, and generate a partial image of a local area 76 including the road surface marking 75 as object marker feature data indicating the object marker of the parking space.

[0087] Reference Figure 4 The object marker detection unit 55 receives the current position of the host vehicle 1 when the host vehicle 1 is parked in the parking space from the own position calculation unit 53, and acquires the parking position Pr.

[0088] The map generation unit 56 stores the object feature data of the object detected by the object detection unit 55 and the relative position data indicating the relative positional relationship between the parking position Pr and the object as learned object data in the storage device 21 , and generates the map data 57 .

[0089] On the other hand, when it is determined that the vehicle 1 is located near the second type parking lot, the object marker detection unit 55 detects the object markers around the parking space of the second type parking lot based on the surrounding image output from the image conversion unit 52. For example, the object marker detection unit 55 detects the feature points of the object markers and the image feature quantities thereof. The detection of the feature points and the calculation of the image feature quantities can be performed using methods such as SIFT, SURF, ORB, BRIAK, KAZE, and AKAZE.

[0090] In addition, the object detection unit 55 receives the current position of the host vehicle 1 from the own position calculation unit 53. The map generation unit 56 calculates the position of the object on the map coordinate system based on the current position of the host vehicle 1 when the object is detected, and calculates the parking position on the map coordinate system based on the current position of the host vehicle 1 when the host vehicle 1 is parked in the parking space.

[0091] The map generation unit 56 stores the object points and image feature quantities of the object detected by the object detection unit 55 as object feature data in the storage device 21, and stores the position of the object and the parking position in the map coordinate system as relative position data between the object and the parking space in the storage device 21, thereby generating map data 57.

[0092] Thereafter, when the parking assist control unit 51 starts the parking assist control, the comparison unit 58 receives a parking position calculation instruction from the parking assist control unit 51 .

[0093] When it is determined that the host vehicle 1 is located in the first type parking lot, the comparison unit 58 reads the object feature data of the object representing the parking space and the relative position data between the object and the target parking position stored in the storage device 21 as the learned object feature data. The comparison unit 58 performs template matching on the surrounding image generated by the image conversion unit 52 using the object feature data, thereby detecting the object representing the parking space from the surrounding image. The comparison unit 58 calculates the current relative position of the host vehicle 1 relative to the target parking position, i.e., the parking space, based on the detected position of the object on the surrounding image and the relative position data read from the storage device 21.

[0094] On the other hand, when it is determined that the host vehicle 1 is located near the second type parking lot, the object marker detection unit 55 detects surrounding object markers around the host vehicle 1 based on the surrounding image output from the image conversion unit 52. The comparison unit 58 matches the object markers stored as the learned object marker data with the surrounding object markers detected by the object marker detection unit 55 when the parking assistance is performed, so that the same object markers are associated with each other.

[0095] The comparison unit 58 calculates the current relative position of the host vehicle with respect to the target parking position based on the relative positional relationship between the object and the target parking position in the learned object data associated with the surrounding object and the relative positional relationship between the surrounding object and the host vehicle 1 .

[0096] For example, the positions of surrounding objects are recorded as (x i ,y i ), the position of the object in the learned object data associated with the surrounding object is recorded as (x mi ,y mi )(i=1 to N). The comparison unit 58 calculates the affine transformation matrix M using the following equation based on the least squares method: affine .

[0097] [Number 1]

[0098]

[0099] The comparison unit 58 calculates the position of the target parking position (targetx m , targety m ) is converted to the position (targetx, targety) in the vehicle coordinate system.

[0100] [Number 2]

[0101]

[0102] When the target trajectory generating unit 59 receives a driving trajectory calculation instruction from the parking assist control unit 51, it calculates the target driving trajectory of the vehicle 1 from the current position to the target parking position on the vehicle coordinate system, and the target vehicle speed change diagram of the vehicle 1 traveling along the target driving trajectory. When the steering control unit 60 receives a steering control instruction from the parking assist control unit 51, it controls the steering actuator 19a so that the vehicle 1 travels along the target driving trajectory. When the vehicle speed control unit 61 receives a vehicle speed control instruction from the parking assist control unit 51, it controls the accelerator actuator 19b and the brake actuator 19c so that the vehicle speed of the vehicle 1 changes according to the target vehicle speed change diagram. In this way, the vehicle 1 is controlled to travel along the target driving trajectory.

[0103] When the host vehicle 1 reaches the target parking position and the parking assist control is completed, the parking assist control unit 51 operates the parking brake 18 and switches the shift position to the parking position (P position).

[0104] (action)

[0105] Figure 6 This is a flowchart of an example of a process of storing learned object label data.

[0106] In step S1, the image conversion unit 52 converts the camera image into a bird's-eye view image observed from a virtual perspective directly above the vehicle 1, and obtains a surrounding image. In step S2, the object marker detection unit 55 detects object markers from the acquired surrounding image. When the vehicle 1 is parked in the first type parking lot, the entire or a part of the road surface marking indicating the parked parking space is detected as an object marker. When the vehicle 1 is parked in the second type parking lot, the object markers around the parking space are detected.

[0107] In step S3, the parking lot type determination unit 54 determines whether the vehicle 1 is located in the first type parking lot. When the vehicle 1 is located in the first type parking lot (step S4: Y), the process proceeds to step S5. When the vehicle 1 is located near the second type parking lot where a single vehicle can be parked (step S4: N), the process proceeds to step S6. In step S5, the map generation unit 56 stores the object feature data representing the features of the object represented as the parking space of the first type parking lot and the relative position data between the object and the parking position in the storage device 21 as learned object feature data. Thereafter, the process ends. In step S6, the map generation unit 56 stores the object feature data representing the features of the object around the parking space of the second type parking lot and the relative position data between the object and the parking space as learned object feature data in the storage device 21. Thereafter, the process ends.

[0108] Figure 7 This is a flowchart of a first example of processing when implementing parking assistance. In step S10, the image conversion unit 52 converts the camera's photographed image into a bird's-eye view image and acquires a surrounding image. In step S11, the object detection unit 55 detects surrounding object markers from the surrounding image. In step S12, the comparison unit 58 reads the learned object marker data from the storage device 21. In step S13, the comparison unit 58 matches the object markers of the learned object marker data with the surrounding object markers. In step S14, the comparison unit 58 determines whether a plurality of parking spaces have been detected. In the case where a plurality of parking spaces have been detected (step S14: Y), the processing proceeds to step S15. In the case where a plurality of parking spaces have not been detected (step S14: N), the processing proceeds to step S16. In step S15, the comparison unit 58 selects any one of the plurality of parking spaces as the parking space for the vehicle 1 to park by accepting the selection input of the occupant selecting any one of the plurality of parking spaces. Thereafter, the processing proceeds to step S16.

[0109] In step S16, the comparison unit 58 calculates the current relative position of the vehicle 1 relative to the target parking position, i.e., the parking space. In step S17, the target trajectory generation unit 59 calculates the target driving trajectory and the target vehicle speed change map. In step S18, the steering control unit 60 and the vehicle speed control unit 61 control the steering actuator 19a, the accelerator actuator 19b, and the brake actuator 19c based on the target driving trajectory and the target vehicle speed change map. In step S19, when the parking assist control is completed, the parking assist control unit 51 operates the parking brake 18 and switches the gear to the P gear.

[0110] Figure 81 is a flowchart of a second example of the process when parking assistance is implemented. In step S20, the parking lot type determination unit 54 determines whether the vehicle 1 is located in the first type parking lot. In step S21, the image conversion unit 52 converts the camera image into a bird's-eye view image observed from a virtual perspective directly above the vehicle 1, and acquires a surrounding image. In the case where the vehicle 1 is located in the first type parking lot (step S22: Y), the process proceeds to step S23. In the case where the vehicle 1 is located near the second type parking lot where a single vehicle can be parked (step S22: N), the process proceeds to step S25. In step S23, the comparison unit 58 reads the object feature data of the object representing the parking space of the first type parking lot stored as the learned object feature data, and the relative position data between the object and the target parking position. In step S24, the comparison unit 58 performs template matching on the surrounding image acquired in step S21 using the object feature data, detects the object representing the parking space from the surrounding image, and calculates the current relative position of the vehicle 1 relative to the target parking position, i.e., the parking space. After that, the processing enters step S29.

[0111] In step S25, the object detection unit 55 detects surrounding object markers around the vehicle 1. In step S26, the comparison unit 58 reads the object feature data of the object markers around the parking space of the second type parking lot stored as the learned object marker data, and the relative position data between the object markers and the target parking position. In step S27, the comparison unit 58 matches the object markers of the learned object marker data with the surrounding object markers. In step S28, the target parking position is calculated based on the object markers matched by the comparison unit 58. Thereafter, the processing enters step S29. The processing of steps S29 to S31 is the same as that of step S29. Figure 7 The processing of steps S18 to S19 is the same.

[0112] (Effects of implementation methods)

[0113] (1) In the parking assistance method of the embodiment, it is determined whether the vehicle 1 is located in a first-class parking lot having multiple parking spaces, or whether the vehicle 1 is located near a second-class parking lot that can accommodate a single vehicle. If it is determined that the vehicle 1 is located in the first-class parking lot, when the vehicle 1 is parked in any one of the multiple parking spaces, i.e., the target parking space, data on the relative positional relationship between an object mark common to the multiple parking spaces and the parking position in the target parking space is stored in a storage device as learned object mark data. If it is determined that the vehicle 1 is located near a second-class parking lot, data on the relative positional relationship between an object mark detected near the second-class parking lot when the vehicle 1 is parked in the second-class parking lot and the target parking position is stored in the storage device as learned object mark data.

[0114] Thus, in the first type parking lot having a plurality of parking spaces, by detecting a partial pattern representing a parking space from the vicinity of the host vehicle 1, a plurality of parking spaces represented by similar patterns can be detected as respective target parking positions. As a result, a parking space for parking the host vehicle 1 can be arbitrarily selected. On the other hand, in the second type parking lot where a single vehicle can be parked, by storing the overall arrangement pattern of the object landmarks detected near the parking lot, a stable target parking position can be detected without depending on changes in the environment or the position of the vehicle.

[0115] (2) When it is determined that the host vehicle 1 is located in the first type parking lot, a parking space for parking the host vehicle 1 may be selected by accepting a selection input from the occupant to select any one of the plurality of parking spaces. Thus, the occupant can arbitrarily select a parking space for parking the host vehicle 1 from among the plurality of parking spaces.

[0116] (3) It is determined whether similar patterns continue in the surrounding image, which is an image of the surroundings of the host vehicle 1 . If similar patterns continue in the surrounding image, it can be determined that the host vehicle 1 is located in the first type parking lot.

[0117] This makes it possible to easily determine whether the host vehicle 1 is located in the first type parking lot based on the surrounding image.

[0118] (4) Whether the vehicle 1 is located in the first type parking lot may be determined based on the positioning result of the positioning device of the vehicle 1 and the map information. In this way, for example, if a navigation device is provided, it is possible to easily determine whether the vehicle is located in the first type parking lot.

[0119] (5) Data indicating the relative positional relationship between the road marking at the end of the target parking space on the driving lane side adjacent to the target parking space and the parking position in the target parking space can be stored in the storage device as learned object marker data. In this way, learned object marker data indicating the object marker of the parking space can be stored.

[0120] (6) Data indicating the relative positional relationship between a road marking at an end of the target parking space in the front-rear direction opposite to the driving lane adjacent to the target parking space and a parking position in the target parking space may be stored in the storage device as learned object marker data. In this way, learned object marker data indicating an object marker of a parking space can be stored.

[0121] (7) Data indicating the relative positional relationship between the road surface marking provided in the target parking space and the parking position in the target parking space may be stored in the storage device as learned object marker data. In this way, learned object marker data indicating the object of the parking space can be stored.

[0122] Description of Reference Numerals

[0123] 1 Vehicle; 10 Parking aid; 17 Controller.

Claims

1. A parking assistance method for assisting a vehicle in parking at a target parking position, the parking assistance method comprising: Preliminarily storing data indicating a relative positional relationship between an object existing around a target parking position and the target parking position as learned object data in a storage device; detecting positions of objects existing around the vehicle, namely, surrounding objects; Calculating a relative position relationship between the target parking position and the current position of the host vehicle based on the learned object marker data and the positions of the surrounding object markers; Calculating a driving trajectory of the host vehicle from the current position to the target parking position based on a relative position relationship between the target parking position and the current position of the host vehicle; Based on the driving trajectory, assisting the host vehicle in parking at the target parking position; Further, it is determined whether the host vehicle is located in a first type parking lot having a plurality of parking spaces, or whether the host vehicle is located near a second type parking lot capable of parking a single vehicle. In the case where it is determined that the host vehicle is located in the first type parking lot, when the host vehicle is parked in any one of the plurality of parking spaces, namely, the target parking space, data indicating a relative positional relationship between an object mark common to the plurality of parking spaces and indicating a parking space and a parking position in the target parking space is stored in the storage device as the learned object mark data, When it is determined that the vehicle is located near the second type of parking lot, when the vehicle is parked in the second type of parking lot, data representing the relative positional relationship between the object mark detected near the second type of parking lot and the target parking position is stored in the storage device as the learned object mark data.

2. The parking assistance method according to claim 1, characterized in that: When it is determined that the host vehicle is located in the first-type parking lot, a parking space in which the host vehicle is to be parked is selected by accepting a selection input from an occupant selecting any one of the plurality of parking spaces.

3. The parking assistance method according to claim 1, characterized in that: determining whether similar patterns continue in an image obtained by photographing the surroundings of the host vehicle, that is, a surrounding image, When similar patterns are continuously present in the surrounding image, it is determined that the host vehicle is located in the first type parking lot.

4. The parking assistance method according to claim 1, characterized in that: Based on the positioning result of the positioning device of the host vehicle and the map information, it is determined whether the host vehicle is located in the first type parking lot.

5. The parking assistance method according to any one of claims 1 to 4, characterized in that: Data indicating the relative positional relationship between the road marking at the end portion of the target parking space on the driving lane side adjacent to the target parking space and the parking position in the target parking space is stored in the storage device as the learned object landmark data.

6. The parking assistance method according to any one of claims 1 to 4, characterized in that: Data indicating the relative positional relationship between a road marking at the end opposite to a driving lane adjacent to the target parking space and a parking position in the target parking space, at both ends in the front-rear direction of the target parking space, and the parking position in the target parking space are stored in the storage device as the learned object marker data.

7. The parking assistance method according to any one of claims 1 to 4, characterized in that: Data indicating a relative positional relationship between a road surface marking provided in the target parking space and a parking position in the target parking space is stored in the storage device as the learned object marker data.

8. A parking assist device, characterized in that: have: A sensor that detects objects around the vehicle; Storage device; a controller that stores data indicating a relative positional relationship between an object existing around a target parking position and the target parking position in advance as learned object marker data in the storage device, detects positions of objects existing around the host vehicle, i.e., surrounding object markers, using the sensor, calculates a relative positional relationship between the target parking position and a current position of the host vehicle based on the learned object marker data and the positions of the surrounding object markers, calculates a driving trajectory of the host vehicle from the current position to the target parking position based on the relative positional relationship between the target parking position and the current position of the host vehicle, and assists the host vehicle in parking at the target parking position based on the driving trajectory; the controller, determining whether the host vehicle is located in a first type parking lot having a plurality of parking spaces, or whether the host vehicle is located near a second type parking lot capable of parking a single vehicle; When it is determined that the host vehicle is located in the first type parking lot, when the host vehicle is parked in any one of the plurality of parking spaces, namely, a target parking space, data indicating a relative positional relationship between an object mark common to the plurality of parking spaces and indicating a parking space and a parking position in the target parking space are stored in the storage device as the learned object mark data; When it is determined that the vehicle is located near the second type of parking lot, when the vehicle is parked in the second type of parking lot, data representing the relative positional relationship between the object mark detected near the second type of parking lot and the target parking position is stored in the storage device as the learned object mark data.

Images