Parking assistance method and parking assistance device

Abstract

This parking assistance method sets a target parking position for parking an own vehicle, generates an overhead image (62) of surroundings of the own vehicle from a captured image obtained by photographing the surroundings of the own vehicle, and displays the overhead image (62) and a guide figure (64), which indicates a predetermined size range enclosing the target parking position, in a predetermined display area (61) of a display screen (60) which is visually recognizable by an occupant of the own vehicle (1). When a predetermined range or more of the guide figure (64) can be displayed within the display area (61) when the overhead image (62) is displayed at a first display position (Pb1) on the display screen (60), the overhead image (62) is displayed at the first display position (Pb1) on the display screen (60). When the predetermined range or more of the guide figure (64) cannot be displayed within the display area (61) when the overhead image (62) is displayed at the first display position (Pb1) on the display screen (60), the overhead image (62) is displayed at a second display position (Pb2) different from the first display position (Pb1).

Description

Parking assistance method and parking assistance device

[0001] The present invention relates to a parking assistance method and a parking assistance device.

[0002] The parking assistance device described in Patent Document 1 below draws an extension line of the target parking space and a predicted trajectory line of the vehicle on the overhead image when the target parking space is located outside the overhead image generated from images taken around the vehicle.

[0003] Japanese Patent Application Laid-Open No. 2008-80959

[0004] However, if the vehicle is far from the target parking position and the display position of the target parking frame is set outside the range of the overhead image, it is not possible to correctly grasp the relative position of the target parking frame with respect to the vehicle. The present invention aims to provide an image that makes it easy to grasp the relative position of the target parking frame with respect to the vehicle, even if the vehicle is far from the target parking position.

[0005] According to one aspect of the present invention, there is provided a parking assistance method which sets a target parking position where a vehicle is to be parked, generates an overhead image of the surroundings of the vehicle from captured images obtained by photographing the surroundings of the vehicle, and displays the overhead image and a guide figure representing an area of ​​a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen visible to an occupant of the vehicle. In the parking assistance method, when the overhead image is displayed at a first display position on the display screen, if a predetermined area or more of the guide figure can be displayed within the display area, the overhead image is displayed at the first display position on the display screen, and when the overhead image is displayed at the first display position on the display screen, if a predetermined area or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen different from the first display position.

[0006] According to the present invention, even if the vehicle is far from the target parking position, an image can be provided that makes it easy to understand the relative position of the target parking space with respect to the vehicle. The objects and advantages of the present invention are realized and achieved by using the elements and combinations thereof set forth in the claims. It should be understood that both the foregoing general description and the following detailed description are merely exemplary and explanatory and are not intended to limit the invention as defined by the claims.

[0007] 1 is a diagram showing an example of the schematic configuration of a parking assistance device. (a) is an explanatory diagram of an example of a process for registering a target parking position, and (b) is an explanatory diagram of an example of a process when parking assistance is performed. It is a block diagram of an example of the functional configuration of the controller of FIG. 1. (a) to (c) are explanatory diagrams of an example of a parking assistance method of the first embodiment. (a) to (d) are schematic diagrams of an example of superimposition of a target parking frame. It is a flowchart of an example of a parking assistance method of the first embodiment. (a) and (b) are explanatory diagrams of an example of a parking assistance method of the second embodiment. It is a flowchart of an example of a parking assistance method of the second embodiment.

[0008] (First embodiment) (Configuration) See FIG. 1 . A host vehicle 1 is equipped with a parking assistance device 10 that assists in parking the host vehicle 1 at a target parking position. The parking assistance device 10 assists the host vehicle 1 in traveling along a target traveling trajectory from the current position of the host vehicle 1 to the target parking position. For example, an automatic driving system may be implemented in which the host vehicle 1 is controlled to travel along the target traveling trajectory of the host vehicle 1 to the target parking position (i.e., control is performed to automatically perform all or part of traveling along the target traveling trajectory of the host vehicle 1 by controlling all or part of the steering angle, driving force, and braking force of the host vehicle). Parking of the host vehicle 1 may be assisted by displaying the target parking position, the target traveling trajectory, and the current position of the host vehicle 1 on a display device visible to an occupant (e.g., the driver) of the host vehicle 1.

[0009] The positioning device 11 measures the current position and attitude of the vehicle 1 (for example, in the longitudinal direction of the vehicle body). The positioning device 11 includes a Global Navigation System (GNSS) receiver such as a Global Positioning System (GPS) receiver. The human-machine interface (HMI) 13 is an interface device that exchanges information between the parking assistance device 10 and the occupant, and includes a display device with a display screen visible to the occupant, a speaker, a buzzer, and controls. The shift switch (shift SW) 14 is a switch that allows the occupant of the vehicle 1 or the parking assistance device 10 to change the shift position of the vehicle 1.

[0010] The external sensor 15 detects objects within a predetermined distance range from the host vehicle 1. The external sensor 15 detects the surrounding environment of the host vehicle 1, such as the relative position of the host vehicle 1 and an object present around the host vehicle 1, the distance between the host vehicle 1 and the object, and the direction in which the object is present. The external sensor 15 may include, for example, a camera that captures the surrounding environment of the host vehicle 1. In the following description, the camera included in the external sensor 15 will be simply referred to as a "camera." The external sensor 15 may include a distance measuring device such as a laser range finder, radar, or LiDAR. The vehicle sensor 16 detects various information (vehicle information) about the host vehicle 1. The vehicle sensor 16 may include, for example, a vehicle speed sensor, a wheel speed sensor, a three-axis acceleration sensor that detects the acceleration of the host vehicle 1 in three axial directions, a steering angle sensor that detects the steering angle, a steering angle sensor that detects the steering angle of the steered wheels, a gyro sensor, and a yaw rate sensor.

[0011] The controller 17 is an electronic control unit that performs parking assistance control. The controller 17 includes a processor 20 and peripheral components such as a storage device 21. The processor 20 may be, for example, a CPU or an MPU. The storage device 21 may include a semiconductor storage device, a magnetic storage device, an optical storage device, or the like. The functions of the controller 17 are realized, for example, by the processor 20 executing a computer program stored in the storage device 21. The steering actuator 19a controls the steering direction and steering amount of the steering mechanism of the host vehicle 1 in response to a control signal from the controller 17. The accelerator actuator 19b controls the accelerator opening of the drive device, which is an engine or a drive motor, in response to a control signal from the controller 17. The brake actuator 19c activates a braking device in response to a control signal from the controller 17.

[0012] Next, parking assistance control by the parking assistance device 10 will be described. When using parking assistance by the parking assistance device 10, a target parking position where the vehicle 1 should be parked is registered in the parking assistance device 10. Specifically, targets present around the target parking position are extracted and stored in advance in the storage device 21. In the following description, targets around the target parking position stored in the storage device 21 will be referred to as "learned targets." FIG. 2( a) is an explanatory diagram of an example of a process for registering a target parking position, and circles represent learned targets. When registering a target parking position 30 in the parking assistance device 10, the occupant performs an operation to instruct registration of the target parking position 30 (hereinafter, sometimes referred to as a "registration operation"). The registration operation may be, for example, operation of a "parking position registration switch" provided in the HMI 13.

[0013] For example, when the vehicle 1 is located near the target parking position 30 (for example, when the occupant manually parks the vehicle 1 at the target parking position 30), the parking assistance device 10 detects the surroundings of the vehicle 1 using the external sensor 15 and stores the detected targets as learned targets. For example, targets may be detected from a surrounding image obtained by capturing an image of the surroundings of the vehicle 1 with a camera. In this case, the targets are detected as edge points or points (feature points) with characteristic shapes where the brightness of adjacent pixels changes by more than a predetermined amount, such as edges or corners of targets such as road markings, road boundaries, and obstacles in the surrounding image. Targets around the vehicle 1 may also be detected using a distance measuring device.

[0014] The parking assistance device 10 stores learned target data related to learned targets in the storage device 21. For example, the learned target data includes data representing the features of the learned targets (hereinafter referred to as "feature data"), data on the relative positional relationship between the learned targets and the target parking position (hereinafter referred to as "relative position data"), and coordinate data of the target parking position 30 (hereinafter referred to as "target parking position coordinate data"). As the relative position data, for example, the relative position of the learned target with respect to the target parking position 30 may be stored. The parking assistance device 10 can acquire the position of the learned target detected when the host vehicle 1 is located at the target parking position 30 as the relative position of the learned target with respect to the target parking position 30. For example, the occupant may operate the HMI 13 to input that the host vehicle 1 is located at the target parking position 30. Alternatively, the coordinates of the learned target and the target parking position 30 in a coordinate system with a fixed point as the reference point (hereinafter referred to as "map coordinate system") may be stored. As the target parking position coordinate data, coordinate data of the target parking position 30 in the map coordinate system and angle data indicating the fore-and-aft direction 32 of the parking space where the vehicle is to be parked at the target parking position 30 are stored. As the target parking position coordinate data, the own position of the vehicle 1 in the map coordinate system and the fore-and-aft direction of the vehicle body when the vehicle 1 is located at the target parking position 30 may be stored.

[0015] 2B is an explanatory diagram of an example of processing when parking assistance is performed. The parking assistance device 10 starts parking assistance for the host vehicle 1 when the host vehicle 1 is positioned near a registered target parking position 30 and an occupant performs an operation (hereinafter sometimes referred to as an "activation operation") to instruct activation of parking assistance control of the host vehicle 1 toward the target parking position 30. The activation operation may be, for example, an operation of a "parking assistance activation switch" provided in the HMI 13, or a shift operation for switching between forward and reverse movement of the host vehicle 1 (for example, an operation of switching from a drive range (D range) to a reverse range (R range) or an operation of switching from the R range to the D range).

[0016] The parking assistance device 10 extracts targets around the host vehicle 1 from a surrounding image obtained by capturing images of the surroundings of the host vehicle 1 with a camera. In the following description, targets around the host vehicle 1 extracted during parking assistance are referred to as "surrounding targets." In FIG. 2(b), triangular plots represent surrounding targets. The parking assistance device 10 matches the learned targets stored in the storage device 21 with the surrounding targets and associates identical feature points. Then, based on the relative positional relationship between the host vehicle 1 and the surrounding targets detected during parking assistance and the relative positional relationship between the learned targets associated with the surrounding targets and the target parking position 30, the parking assistance device 10 calculates the relative position of the host vehicle 1 with respect to the target parking position 30. For example, the parking assistance device 10 calculates the position of the target parking position 30 in a coordinate system based on the current position of the host vehicle 1 (hereinafter referred to as the "vehicle coordinate system"). In addition, when the coordinates of the learned targets and the target parking position 30 on the map coordinate system are stored in the storage device 21, the coordinates of the target parking position 30 on the map coordinate system may be converted into coordinates on the vehicle coordinate system based on the positions of the surrounding targets detected when parking assistance is performed and the positions of the learned targets in the map coordinate system. The own position of the vehicle 1 on the map coordinate system may be obtained based on the positions of the surrounding targets detected when parking assistance is performed and the positions of the learned targets in the map coordinate system, and the relative position of the vehicle 1 with respect to the target parking position 30 may be calculated from the difference between the coordinates of the vehicle 1 and the coordinates of the target parking position 30 in the map coordinate system.

[0017] The parking assistance device 10 calculates a target driving trajectory 34 from the current position 33 of the vehicle 1 to the target parking position 30 based on the relative position of the vehicle 1 with respect to the target parking position 30. The parking assistance device 10 performs parking assistance control of the vehicle 1 based on the calculated target driving trajectory 34. Furthermore, the parking assistance device 10 generates an overhead image (around view monitor image) of the surroundings of the vehicle 1 from the captured image generated by the camera, and displays a target parking frame representing a parking space where the vehicle will be parked at the target parking position 30 on the display device of the HMI 13 by superimposing it on the overhead image. The target parking frame is an example of a "guide figure representing an area of ​​a predetermined size surrounding the target parking position" as set forth in the claims.

[0018] The functional configuration of the controller 17 will be described in more detail below. FIG. 3 is a block diagram of an example of the functional configuration of the controller 17. The HMI control unit 40 detects an operation by the occupant to register a target parking position 30 and an operation to activate parking assist control for the registered target parking position 30. When the registration operation is detected, the HMI control unit 40 outputs a map generation command to the map generation unit 45 to store learned target data in the storage device 21. When the activation operation is detected, the HMI control unit 40 outputs a control start command to the parking assist control unit 41 to start parking assist control of the host vehicle 1 toward the target parking position 30. The image conversion unit 42 converts the captured image from the camera into a bird's-eye view image viewed from a virtual viewpoint directly above the host vehicle 1. The image conversion unit 42 converts the captured image into a bird's-eye view image at predetermined intervals (e.g., every time the host vehicle 1 travels a predetermined distance (e.g., 50 cm) or a predetermined time (e.g., 1 second)), and generates a surrounding image, which is an image of the area surrounding the host vehicle 1, by accumulating the converted bird's-eye view images along the travel path of the host vehicle 1.

[0019] The vehicle position calculation unit 43 calculates the vehicle's current position and attitude (e.g., the longitudinal direction of the vehicle body) on a map coordinate system by odometry (e.g., dead reckoning) based on vehicle information output from the vehicle sensor 16. The vehicle position calculation unit 43 corrects the calculation results of the vehicle's position and attitude based on the detection results of the vehicle's position and attitude by the positioning device 11. The target detection unit 44 detects targets from the surrounding image output from the image conversion unit 42. The target detection unit 44 may detect the positions of feature points of the targets and their image feature amounts. Methods such as SIFT, SURF, ORB, BRIAK, KAZE, and AKAZE can be used to detect the positions of the feature points and calculate the image feature amounts. The target detection unit 44 outputs the detected positions of the feature points and the image feature amounts to the map generation unit 45 and the matching unit 47 as target data. In addition, the self-position acquired from the self-position calculation unit 43 in synchronization with the detection of the target object is output to the map generation unit 45 and the matching unit 47 .

[0020] When the map generation unit 45 receives a map generation command from the HMI control unit 40 (i.e., when the target parking position 30 is registered), the map generation unit 45 generates learned target data and stores it as map data 46 in the storage device 21. For example, the map generation unit 45 receives target data and the host vehicle's own position on the map coordinate system synchronized with the target data from the target detection unit 44. The map generation unit 45 acquires coordinate data of the target parking position 30 in the map coordinate system and angle data in the fore-and-aft direction 32 of the parking space where the vehicle is to be parked at the target parking position 30. For example, the host vehicle's own position calculated by the host position calculation unit 43 when the host vehicle 1 is located at the target parking position 30 may be acquired as the coordinate data of the target parking position 30, and the angle of the vehicle body in the fore-and-aft direction 32 may be acquired as angle data in the fore-and-aft direction 32. The map generation unit 45 generates relative position data based on the positions of the feature points included in the target data, the position information of the host vehicle 1 synchronized therewith, and the position data of the target parking position 30. The map generator 45 also acquires feature data from the target object data. The position data of the target parking position 30 and the angle data of the parking space in the longitudinal direction 32 are used as target parking position coordinate data. The learned target object data including the relative position data, feature data, and target parking position coordinate data is stored in the storage device 21 as map data 46.

[0021] When the parking assist control unit 41 receives a control start command from the HMI control unit 40 (i.e., when the parking assist control is activated), it outputs a parking position calculation command to the matching unit 47. The matching unit 47 receives the target data output from the target detection unit 44 as target data of surrounding targets, and receives the host vehicle 1's own position in the map coordinate system in synchronization with the target data. The matching unit 47 reads the learned target data stored in the storage device 21 as map data 46, and determines whether the host vehicle 1 is located near a registered target parking position 30 based on the target parking position coordinate data. When the host vehicle 1 is located near a registered target parking position 30, the matching unit 47 matches the learned targets stored in the map data 46 with the surrounding targets to associate targets with the same feature points. The matching unit 47 calculates the current relative position of the vehicle 1 with respect to the target parking position 30 based on the relative positional relationship between the surrounding targets and the vehicle 1 and the relative positional relationship between the learned targets associated with the surrounding targets and the target parking position 30. For example, if the surrounding targets are (x ｉ , y ｉ ) and the surrounding targets (x ｉ , y ｉ ) are the learned targets associated with each of the ｍｉ , y ｍｉ ) (i=1 to N). The matching unit 47 calculates the affine transformation matrix M ａｆｆｉｎｅ１ Calculate.

[0022]

[0023] The abutting portion 47 is the position (targetx) of the target parking position 30 on the map coordinate system stored in the map data 46. ｍ , target ｍ ) is converted into a position (targetx, targety) in the vehicle coordinate system based on the following equation:

[0024]

[0025] Furthermore, the abutting portion 47 is an angle targetyaw indicating the front-rear direction 32 of the parking space of the target parking position 30 on the map coordinate system. ｍis converted into an angle target aw indicating a direction on the vehicle coordinate system based on the following equation:

[0026]

[0027] When the target trajectory generation unit 48 receives a driving trajectory calculation command from the parking assist control unit 41, it calculates a target driving trajectory from the current position of the host vehicle 1 on the vehicle coordinate system to the target parking position 30, and a target vehicle speed profile. When the steering control unit 49a receives a steering control command from the parking assist control unit 41, it controls the steering actuator 19a so that the host vehicle 1 drives along the target driving trajectory. When the vehicle speed control unit 49b receives a vehicle speed control command from the parking assist control unit 41, it controls the accelerator actuator 19b and the brake actuator 19c so that the vehicle speed of the host vehicle 1 changes in accordance with the target vehicle speed profile. When the host vehicle 1 reaches the target parking position 30 and the parking assist control is completed, the parking assist control unit 41 activates the parking brake 18 and switches the shift position to the parking range (P range).

[0028] While parking assist control is being performed, the HMI control unit 40 receives an overhead image of the surroundings of the vehicle 1 from the image conversion unit 42 and displays it on the display device of the HMI 13. The HMI control unit 40 also receives information on the coordinates (targetx, targety) of the target parking position 30 and the angle targetyaw in the fore-and-aft direction 32 from the butting unit 47, and displays a target parking frame representing a parking space in which the vehicle will be parked at the target parking position 30 by superimposing it on the overhead image. FIG. 4A is a schematic diagram of an example of a display area in which the overhead image is displayed on the display device of the HMI 13. Reference numeral 60 indicates a display screen of the display device of the HMI 13. The overhead image 62 is displayed within a predetermined display area 61 on the display screen 60. For convenience in the following explanation, we will use the reference point Pr at the bottom left corner of the display area 61 as the coordinate origin (0,0), and the rightward and upward directions as the positive x-axis and positive y-axis directions, respectively, as an example; however, the setting of the coordinate system is not limited to this example and may be changed as appropriate.

[0029] The display area 61 is a pre-defined area within the display screen 60. The number of pixels in the display area 61 in the x-axis direction and the y-axis direction are Wavm and Havm, respectively. For example, the number of pixels in the x-axis direction and the y-axis direction of the display area 61 may be set equal to the number of pixels in the x-axis direction and the y-axis direction of the overhead image 62 output from the image conversion unit 42 to the HMI control unit 40, respectively. In FIG. 4A , the overhead image 62 is rendered slightly smaller than the display area 61 to make the boundary of the overhead image 62 easier to see. However, the overhead image 62 may be displayed filling the display area 61 (i.e., the number of pixels in the x-axis direction and the y-axis direction of the overhead image 62 displayed in the display area 61 may be Wavm and Havm, respectively). The overhead image 62 is an image in which an icon 63 representing the host vehicle 1 is superimposed at the center of the overhead image representing the range of the longitudinal length DL and the vehicle width direction DT centered on the current position of the host vehicle 1. The longitudinal length DL and the transverse length DT are appropriately set so that the occupant can grasp the sense of distance from the features (road markings such as white lines and road boundaries such as road shoulders) around the vehicle 1 that are shown in the overhead image 62. In the following description, the position where the overhead image 62 is displayed in the display area 61 is specified as the position where the lower left corner of the overhead image 62 is displayed (position Pb1 in the example of FIG. 4A).

[0030] The HMI control unit 40 calculates a position where the target parking space 64 is superimposed and displayed on the overhead image 62. Specifically, the target parking position Pt in the coordinate system on the display screen 60 is calculated based on the coordinates (targetx, targety) of the target parking position 30 received from the matching unit 47 and the scale at which the overhead image 62 is displayed on the display screen 60. In addition, the HMI control unit 40 sets the display position and direction of the target parking space 64 in the coordinate system on the display screen 60 based on the angle targetyaw and the target parking position Pt received from the matching unit 47. For example, the target parking position Pt is set to the center position of the target parking space 64, and the direction indicated by the angle targetyaw is set to the front-rear direction of the target parking space 64. The size (front-rear length and lateral length) of the target parking space 64 to be displayed on the display screen 60 may be set according to, for example, the scale of the overhead image 62 and the size of the host vehicle 1. For example, the size of the target parking space 64 may be set to be approximately equal to the size of the area occupied by the host vehicle 1 in the overhead image 62. Also, the size of the target parking space 64 may be set to be slightly larger than the area occupied by the host vehicle 1 to provide a margin, or may be set to be slightly smaller than the area occupied by the host vehicle 1. Also, the size of the target parking space 64 when the overhead image 62 is displayed to fill the display area 61 (i.e., when the overhead image 62 is displayed with pixel numbers Wavm and Havm in the x-axis and y-axis directions, respectively) may be set in advance as an initial value.

[0031] Here, when parking assist control is started in a state where the host vehicle 1 is away from the target parking position 30, the display position of the target parking frame 64 may be set outside the range of the overhead image 62 as shown in FIG. 4A. In this case, it is not possible to correctly grasp the relative position of the target parking frame 64 with respect to the icon 63 representing the host vehicle 1. Therefore, when the overhead image 62 is displayed at the first display position Pb1 of the display screen 60, if a predetermined range or more of the target parking frame 64 can be displayed within the display area 61, the HMI control unit 40 displays the overhead image 62 at the first display position Pb1 of the display screen 60. For example, when the display position for displaying the overhead image 62 is set to the first display position Pb1, it is determined whether a predetermined range or more of the target parking frame 64 can be displayed within the display area 61. 4(a), when the overhead image 62 is displayed to fill the display area 61 (that is, when the overhead image 62 has pixel counts Wavm and Havm in the x-axis direction and the y-axis direction, respectively, and the first display position Pb1 of the overhead image 62 is set to the reference point Pr in the lower left corner of the display area 61), it is determined whether or not a predetermined range of the target parking space 64 or more can be displayed within the display area 61. In the following description, the predetermined range of the target parking space 64 may be referred to as a "required display range."

[0032] On the other hand, when the bird's-eye view image 62 is displayed at the first display position Pb1 of the display screen 60, if a predetermined range or more of the target parking space 64 cannot be displayed within the display area 61, the bird's-eye view image 62 is displayed at a second display position Pb2 of the display screen 60 different from the first display position Pb1 as shown in Fig. 4(b). This makes it possible to display a necessary display range or more of the target parking space 64 within the display area 61. In this case, for example, the part of the bird's-eye view image 62 that is outside the display area 61 due to a change in the display position and the part of the target parking space 64 that does not fit within the display area 61 may be cut out, and the remaining part may be displayed within the display area 61.

[0033] The HMI control unit 40 determines the offset direction (movement direction) and offset amount (movement distance) of the display position of the overhead image 62 relative to the reference position (i.e., first display position Pb1) so that more than the required display range of the target parking space 64 can be displayed within the display area 61, and calculates the second display position Pb2 by offsetting the first display position Pb1 in the offset direction by the offset amount. See FIG. 4( c). The offset direction of the display position of the overhead image 62 is determined by the position of the target parking position Pt when the overhead image 62 is displayed to fill the display area 61. Region R1 is a region whose x coordinate is less than 0 and whose y coordinate is greater than Havm, region R2 is a region whose x coordinate is between 0 and Wavm but less than Wavm and whose y coordinate is greater than Havm, and region R3 is a region whose x coordinate and y coordinate are greater than Wavm and Havm, respectively. Region R4 is a region where the x coordinate is less than 0 and the y coordinate is between 0 and Havm, region R5 is a region where the x coordinate is greater than Wavm and the y coordinate is between 0 and Havm, region R6 is a region where both the x coordinate and the y coordinate are less than 0, region R7 is a region where the x coordinate is between 0 and Wavm and the y coordinate is less than 0, and region R8 is a region where the x coordinate is greater than Wavm and the y coordinate is less than 0.

[0034] For example, when the target parking position Pt is in region R1, the offset direction of the display position of the overhead image 62 may be a downward and right direction (increasing the x coordinate and decreasing the y coordinate), when the target parking position Pt is in region R2, the offset direction of the display position of the overhead image 62 may be a downward direction (decreasing the y coordinate without changing the x coordinate), and when the target parking position Pt is in region R3, the offset direction of the display position of the overhead image 62 may be a downward and left direction (decreasing the x coordinate and y coordinate).When the target parking position Pt is in region R4, the offset direction of the display position of the overhead image 62 may be a right direction (increasing the x coordinate without changing the y coordinate), and when the target parking position Pt is in region R5, the offset direction of the display position of the overhead image 62 may be a left direction (decreasing the x coordinate without changing the y coordinate). When the target parking position Pt is in region R6, the offset direction of the display position of the overhead image 62 may be set to the upper right direction (increasing the x-coordinate and y-coordinate), when the target parking position Pt is in region R7, the offset direction of the display position of the overhead image 62 may be set to the upper right direction (increasing the y-coordinate without changing the x-coordinate), and when the target parking position Pt is in region R8, the offset direction of the display position of the overhead image 62 may be set to the upper left direction (decreasing the x-coordinate and increasing the y-coordinate).

[0035] 4(b) shows an example of a case where the offset direction and offset amount of the display position of the overhead image 62 are set so that the required display range of the target parking frame 64 can be displayed within the display area 61 in an area including the center of the target parking frame 64 (for example, the target parking position Pt). In this example, the x-direction offset amount ofx is set to Px, and the y-direction offset amount ofy is set to Py based on the coordinates (-Px, -Py) of the target parking position Pt when the overhead image 62 is displayed to fill the display area 61 in FIG. 4(a). The required display range of the target parking frame 64 is not limited to the above example, and may be set appropriately so that the target parking frame 64 displayed on the display screen 60 is easily understood by the occupant. For example, the required display range may be a range including a portion of each of at least two sides 64a, 64b of the rectangular target parking space 64 (FIG. 5(a)), a range including the entire length of each of at least two sides 64a, 64b of the rectangular target parking space 64 (FIG. 5(b)), a range including the entire length of at least one short side 64a of the rectangular target parking space 64 (FIG. 5(c)), or a range including all four sides of the rectangle (FIG. 5(d)). For example, the HMI control unit 40 may calculate the coordinates of each of the four corners of the rectangular target parking space 64 based on the target parking position Pt and the angle targetyaw, and determine the offset direction and offset amount of the display position of the overhead image 62 so that more than the required display range can be displayed in the display area 61.

[0036] (Operation) FIG. 6 is a flowchart of an example of the parking assistance method of the first embodiment. In step S1, the HMI control unit 40 calculates the target parking position Pt in the coordinate system on the display screen 60. Also, it calculates the display position and direction of the target parking space 64. In step S2, the HMI control unit 40 determines whether or not a predetermined range or more of the target parking space 64 can be displayed within the display area 61 when the overhead image 62 is displayed at the first display position Pb1. If a predetermined range or more of the target parking space 64 cannot be displayed within the display area 61 (step S2: N), the process proceeds to step S3. If a predetermined range or more of the target parking space 64 can be displayed within the display area 61 (step S2: Y), the process proceeds to step S7. In step S3, the HMI control unit 40 calculates an offset amount (ofx, ofy) of the display position of the overhead image 62. In step S4, the HMI control unit 40 calculates the display position of the target parking frame 64 after the offset by the offset amount (ofx, ofy).

[0037] In step S5, the HMI control unit 40 displays the overhead image 62 at the second display position Pb2, which is a display position offset by the offset amount (ofx, ofy). In step S6, the HMI control unit 40 displays the target parking space 64 superimposed on the display position offset by the offset amount (ofx, ofy). Thereafter, the processing ends. In step S7, the HMI control unit 40 displays the overhead image 62 at the first display position Pb1. In step S8, the HMI control unit 40 displays the target parking space 64 superimposed on the position calculated in step S1. Thereafter, the processing ends.

[0038] Second Embodiment If the display position of the overhead-view image 62 is offset while maintaining its size (i.e., the scale of the overhead-view image 62), a portion of the overhead-view image 62 may extend outside the display area 61 and become unable to be displayed on the display screen 60. Therefore, in the second embodiment, as shown in Fig. 7A, the overhead-view image 62 displayed in the display area 61 is reduced. This allows the overhead-view image 62 to be displayed at the second display position Pb2 so as to include the entire range that can be displayed in the display area 61 when the overhead-view image 62 is displayed at the first display position Pb1 (i.e., the range of the longitudinal length DL and the transverse length DT centered on the current position of the vehicle 1) as shown in Fig. 4A.

[0039] For example, when the x-direction offset amount ofx is smaller than the y-direction offset amount ofy, the HMI control unit 40 calculates the reduction rate scale by the following formula: scale=(Havm-ofy) / Havm When the x-direction offset amount ofx is larger than the y-direction offset amount ofy, the HMI control unit 40 calculates the reduction rate scale by the following formula: scale=(Wavm-ofx) / Wavm The HMI control unit 40 calculates an affine transformation matrix M that offsets the display positions of the overhead image 62 and the target parking space 64 by the x-direction offset amount ofx and the y-direction offset amount ofy and reduces the overhead image 62 and the target parking space 64 by the reduction rate scale. ａｆｆｉｎｅ２ is set by the following formula:

[0040]

[0041] The HMI control unit 40 converts the coordinates of the overhead image 62 and the target parking space 64 into an affine transformation matrix M ａｆｆｉｎｅ２ Specifically, if the coordinates of each pixel of the overhead image 62 before conversion and the coordinates of the boundary of the target parking space 64 are (Px1, Py1), the coordinates of each pixel of the overhead image 62 after offset and reduction and the coordinates of the boundary of the target parking space 64 (Px2, Py2) are calculated by the following equations.

[0042]

[0043] 7A, the HMI control unit 40 may draw grid lines 65 in the background portion of the display area 61 (the area where the overhead image 62 is not displayed). For example, the HMI control unit 40 may display in the display area 61 an image obtained by adding a background image 66 on which a grid is drawn to the overhead image 62, as shown in FIG. 7B.

[0044] FIG. 8 is a flowchart of an example of a parking assistance method of the second embodiment. The processes of steps S10 and S11 are the same as the processes of steps S1 and S2 in FIG. 6. If a predetermined range or more of the target parking space 64 cannot be displayed within the display area 61 (step S11: N), the process proceeds to step S12. If a predetermined range or more of the target parking space 64 can be displayed within the display area 61 (step S11: Y), the process proceeds to step S17. The process of step S12 is the same as step S3 in FIG. 6. In step S13, the HMI control unit 40 calculates the reduction rate "scale". In step S14, the HMI control unit 40 calculates the display position of the target parking space 64 when the overhead image 62 is offset by the offset amount (ofx, ofy) and reduced by the reduction rate "scale".

[0045] In step S15, the HMI control unit 40 displays the overhead image 62 offset by the offset amount (ofx, ofy) and reduced by the reduction rate scale. In step S16, the HMI control unit 40 displays the target parking space 64 offset by the offset amount (ofx, ofy) and reduced by the reduction rate scale in a superimposed manner. Thereafter, the processing ends. The processing in steps S17 and S18 is the same as the processing in steps S7 and S8 in FIG. 6.

[0046] (Modification) In the first and second embodiments, examples have been described in which targets around the target parking position 30 stored in advance are matched with targets detected around the vehicle 1 during execution of parking assist control to detect the relative position of the target parking position 30 with respect to the current position of the vehicle 1, but the parking assist method of the present invention is not limited to such a method of determining the target parking position. For example, during execution of parking assist control, targets indicating a parking space (for example, road markings such as parking frame lines or three-dimensional objects such as wheel chocks) may be detected to detect the relative position of the target parking position 30 with respect to the current position of the vehicle 1.

[0047] (Effects of the embodiment) (1) In a parking assistance method that sets a target parking position where the host vehicle 1 will park, generates an overhead image of the surroundings of the host vehicle 1 from captured images obtained by photographing the surroundings of the host vehicle 1, and displays the overhead image and a guide figure representing a predetermined size range surrounding the target parking position in a predetermined display area of ​​a display screen visible to an occupant of the host vehicle 1, when the overhead image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the overhead image is displayed at the first display position on the display screen, and when the overhead image is displayed at the first display position on the display screen, if a predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen different from the first display position. This makes it possible to simultaneously display the overhead image of the surroundings of the host vehicle 1 and a predetermined range or more of the guide figure even if the host vehicle is far from the target parking position, thereby providing an image that makes it easy to grasp the relative position of the target parking space with respect to the host vehicle.

[0048] (2) When the overhead-view image is displayed at the second display position, the overhead-view image displayed in the display area may be reduced. This makes it possible to prevent a reduction in the display range of the overhead-view image after changing the display position of the overhead-view image. (3) By reducing the overhead-view image displayed in the display area, an overhead-view image including the entire range that can be displayed in the display area when the display position of the overhead-view image is set to the first display position may be displayed at the second display position. This makes it possible to display the entire range of the overhead-view image that was displayable before the change, even when the display position of the overhead-view image is changed.

[0049] (4) A background image with a drawn grid may be added to the overhead image and displayed in the display area. This makes it easier to grasp the sense of distance to the guide figure displayed outside the overhead image. (5) The predetermined range of the guide figure may be a range that includes a portion of each of at least two sides of the rectangular guide figure. Furthermore, the predetermined range of the guide figure may be a range that includes the entire length of each of at least two sides of the rectangular guide figure. Furthermore, the predetermined range of the guide figure may be a range that includes the entire length of at least one short side of the rectangular guide figure. Furthermore, the predetermined range of the guide figure may be a range that includes all four sides of the rectangular guide figure. Furthermore, the predetermined range of the guide figure may be a range that includes the center point of the guide figure. This makes it possible to display the guide figure in a way that is easy for the occupant to understand.

[0050] (6) A target parking position may be detected based on captured images of the surroundings of the vehicle 1, a target driving trajectory from the current position of the vehicle 1 to the target parking position may be calculated, and the vehicle 1 may be controlled to drive along the target driving trajectory. This may assist parking of the vehicle 1.

[0051] All examples and conditional terms described herein are intended for educational purposes to aid the reader in understanding the present invention and the concepts provided by the inventor for the advancement of technology, and should be construed without limitation to the specifically described examples and conditions above, and the configuration of examples herein for illustrating the advantages and disadvantages of the present invention. Although the embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the present invention.

[0052] 1...Own vehicle, 10...Parking assistance device, 11...Positioning device, 13...Human machine interface (HMI), 14...Shift switch (shift SW), 15...External sensor, 16...Vehicle sensor, 17...Controller, 18...Parking brake, 19a...Steering actuator, 19b...Accelerator actuator, 19c...Brake actuator, 20...Processor, 21...Storage device, 40...HMI control unit, 41...Parking assistance control unit, 42...Image conversion unit, 43...Own position calculation unit, 44...Target detection unit, 45...Map generation unit, 46...Map data, 47...Matching unit, 48...Target trajectory generation unit, 49a...Steering control unit, 49b...Vehicle speed control unit

Claims

1. (delete)

2. A parking assistance method comprising: setting a target parking position where a vehicle is to be parked; generating an overhead image of the surroundings of the vehicle from an image captured by photographing the surroundings of the vehicle; and displaying the overhead image and a guide figure representing a range of a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen that is visible to an occupant of the vehicle; When an overhead image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the overhead image is displayed at the first display position on the display screen; When an overhead image is displayed at the first display position of the display screen, if the predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position of the display screen that is different from the first display position; A parking assistance method comprising the steps of: when displaying an overhead image in the second display position, reducing the overhead image displayed in the display area.

3. 3. The parking assistance method according to claim 2, wherein the overhead image displayed in the display area is reduced, so that the overhead image including the entire range that can be displayed in the display area when the display position of the overhead image is set to the first display position is displayed in the second display position.

4. A parking assistance method comprising: setting a target parking position where a vehicle is to be parked; generating an overhead image of the surroundings of the vehicle from an image captured by photographing the surroundings of the vehicle; and displaying the overhead image and a guide figure representing a range of a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen that is visible to an occupant of the vehicle; When an overhead image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the overhead image is displayed at the first display position on the display screen; When an overhead image is displayed at the first display position of the display screen, if the predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position of the display screen that is different from the first display position; A parking assistance method comprising the steps of: adding a background image on which a grid is drawn to the overhead image and displaying the background image in the display area.

5. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range including a part of each of at least two sides of the guide figure that is rectangular.

6. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range including the total lengths of at least two sides of the guide figure that is rectangular.

7. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range including a total length of at least one short side of the guide figure that is rectangular.

8. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range including all four sides of the guide figure which is rectangular in shape.

9. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range including a center point of the guide figure.

10. The parking assistance method according to any one of claims 2 to 9, further comprising the steps of: detecting the target parking position based on an image obtained by photographing the surroundings of the vehicle; calculating a target driving trajectory from a current position of the vehicle to the target parking position; and controlling the vehicle to travel along the target driving trajectory.

11. A parking assistance device comprising: a camera that captures images of the surroundings of a vehicle; and a display device having a display screen that can be viewed by an occupant of the vehicle; the parking assistance device sets a target parking position where the vehicle is to be parked; generates an overhead image of the surroundings of the vehicle from an image captured by the camera; and displays the overhead image and a guide figure that represents a range of a predetermined size surrounding the target parking position in a predetermined display area of ​​the display screen, a controller that displays an overhead image at a first display position on the display screen when a predetermined range or more of the guide figure can be displayed within the display area, displays an overhead image at the first display position on the display screen, and when a predetermined range or more of the guide figure cannot be displayed within the display area when an overhead image is displayed at the first display position on the display screen, displays an overhead image at a second display position on the display screen that is different from the first display position, and reduces the overhead image displayed in the display area when the overhead image is displayed at the second display position.

12. A parking assistance device comprising: a camera that captures images of the surroundings of a vehicle; and a display device having a display screen that can be viewed by an occupant of the vehicle; the parking assistance device sets a target parking position where the vehicle is to be parked; generates an overhead image of the surroundings of the vehicle from an image captured by the camera; and displays the overhead image and a guide figure that represents a range of a predetermined size surrounding the target parking position in a predetermined display area of ​​the display screen, a controller that displays an overhead image at a first display position on the display screen when a predetermined range or more of the guide figure can be displayed within the display area, and displays an overhead image at a second display position on the display screen that is different from the first display position when a predetermined range or more of the guide figure cannot be displayed within the display area when an overhead image is displayed at the first display position on the display screen, and adds a background image on which a grid is drawn to the overhead image and displays it in the display area.