Parking assistance method and parking assistance device

Abstract

In this parking assistance method, a target parking position at which a host vehicle is parked is set, and a bird's-eye view image (62) of the surroundings of the host vehicle is generated on the basis of a captured image obtained by capturing the surroundings of the host vehicle. The overhead image (62) and a guide pattern (64) indicating a range of a predetermined size surrounding the target parking position are displayed in a predetermined display region (61) of a display screen (60) that can be viewed by an occupant of the host vehicle (1). When the bird's-eye view image (62) is displayed at a first display position (Pb1) of the display screen (60), the bird's-eye view image (62) is displayed at the first display position (Pb1) of the display screen (60) when the guide pattern (64) can be displayed within the display region (61) or more than a predetermined range. When the bird's-eye-view image (62) is displayed at a first display position (Pb1) of the display screen (60), the bird's-eye-view image (62) is displayed at a second display position (Pb2) of the display screen, which is different from the first display position (Pb1), if the guide pattern (64) cannot be displayed within the display region (61).

Description

Technical Field

[0001] The invention relates to a parking assistance method and a parking assistance device. Background Art

[0002] The parking assist device described in Patent Document 1 below draws an extension line of the target parking frame and an expected trajectory line of the vehicle on the bird's-eye view image when the target parking frame is outside the bird's-eye view image generated from an image of the surroundings of the vehicle.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2008-80959

[0006] Problems to be Solved by the Invention

[0007] However, when the host vehicle is far away from the target parking position and the display position of the target parking frame is set outside the range of the bird's-eye view image, the relative position of the target parking frame with respect to the host vehicle cannot be accurately grasped. Summary of the invention

[0008] An object of the present invention is to provide an image that makes it easy to grasp the relative position of a target parking frame with respect to the vehicle even if the vehicle is far away from the target parking position.

[0009] According to one aspect of the present invention, there is provided a parking assistance method, which sets a target parking position for a vehicle to park, generates an overhead image of the surroundings of the vehicle based on a captured image obtained by capturing the surroundings of the vehicle, and displays the overhead image and a guide graphic indicating a range of a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen that can be visually recognized by an occupant of the vehicle. In the parking assistance method, when the overhead image can be displayed at a first display position of the display screen, the overhead image is displayed at the first display position of the display screen, and when the overhead image cannot be displayed at the first display position of the display screen, the overhead image is displayed at a second display position of the display screen that is different from the first display position.

[0010] Effects of the Invention

[0011] According to the present invention, it is possible to provide an image that allows the relative position of the target parking frame with respect to the host vehicle to be easily grasped even if the host vehicle is far away from the target parking position.

[0012] The objects and advantages of the present invention are embodied and realized by the elements and combinations thereof shown in the scope of the claims. The above general description and the following detailed description are only examples and illustrations and should not be understood as limiting the present invention as the scope of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a diagram showing a schematic configuration example of a parking assist device.

[0014] Figure 2 (a) is an explanatory diagram of an example of a process for registering a target parking position, and (b) is an explanatory diagram of an example of a process for performing parking assistance.

[0015] Figure 3 yes Figure 1 A block diagram showing an example of the functional structure of a controller in FIG.

[0016] Figure 4 (a) to (c) are explanatory diagrams of an example of the parking assistance method according to the first embodiment.

[0017] Figure 5 (a) to (d) are schematic diagrams of overlapping examples of target parking frames.

[0018] Figure 6 This is a flowchart of an example of the parking assistance method according to the first embodiment.

[0019] Figure 7 (a) and (b) are explanatory diagrams of an example of a parking assistance method according to the second embodiment.

[0020] Figure 8 This is a flowchart of an example of the parking assistance method according to the second embodiment. DETAILED DESCRIPTION

[0021] (First Embodiment)

[0022] (structure)

[0023] Reference Figure 1 . The vehicle 1 is provided with a parking assistance device 10 for assisting the vehicle 1 in parking at a target parking position. The parking assistance device 10 assists the vehicle 1 in traveling from the current position to the target parking position along the target driving trajectory. For example, automatic driving may be performed to control the vehicle 1 so that the vehicle 1 travels to the target parking position along the target driving trajectory (i.e., controlling all or part of the steering angle, driving force, and braking force of the vehicle to automatically implement all or part of the control of the vehicle 1 traveling along the target driving trajectory). The parking of the vehicle 1 may also be assisted by displaying the target parking position, the target driving trajectory, and the current position of the vehicle 1 on a display device that can be visually recognized by the occupants of the vehicle 1 (e.g., the driver).

[0024] The positioning device 11 measures the current position of the host vehicle 1, that is, its own position and posture (for example, the front-back direction of the vehicle body). The positioning device 11 has a global navigation satellite system (GNSS) receiver such as a global positioning system (GPS) receiver. The human-machine interface (HMI) 13 is an interface device for transmitting and receiving information between the parking assistance device 10 and the occupant, and has: a display device with a display screen that can be visually recognized by the occupant, a speaker or a buzzer, and an operating element. The shift switch (shift SW) 14 is a switch for the occupant of the host vehicle 1 or the parking assistance device 10 to switch the gear position of the host vehicle 1.

[0025] The external sensor 15 detects objects within a specified distance range from the vehicle 1. The external sensor 15 detects the surrounding environment of the vehicle 1, such as the relative position between the object existing around the vehicle 1 and the vehicle 1, the distance between the vehicle 1 and the object, and the direction in which the object exists. The external sensor 15 may include, for example, a camera that photographs the surrounding environment of the vehicle 1. In the following description, the camera possessed by the external sensor 15 is referred to as a "camera". The external sensor 15 may include a distance measuring device such as a laser viewfinder, a radar, and a LiDAR. The vehicle sensor 16 detects various information (vehicle information) of the vehicle 1. The vehicle sensor 16 may include, for example, a vehicle speed sensor, a wheel speed sensor, a three-axis acceleration sensor that detects the acceleration of the three axes of the vehicle 1, a steering angle sensor that detects the steering angle, a steering angle sensor that detects the steering angle of the steering wheel, a gyro sensor, and a yaw rate sensor.

[0026] The controller 17 is an electronic control unit that performs parking assist control. The controller 17 includes peripheral components such as a processor 20 and a storage device 21. The processor 20 may be, for example, a CPU or an MPU. The storage device 21 may include a semiconductor memory, a magnetic memory, an optical memory, etc. The functions of the controller 17 are realized, for example, by causing the processor 20 to execute a computer program stored in the storage device 21. The steering actuator 19a controls the steering direction and steering amount of the steering mechanism of the vehicle 1 according to the control signal of the controller 17. The accelerator actuator 19b controls the accelerator opening of the engine or the drive motor, i.e., the drive device, according to the control signal of the controller 17. The brake actuator 19c activates the brake device according to the control signal of the controller 17.

[0027] Next, the parking assistance control of the parking assistance device 10 will be described. When the parking assistance of the parking assistance device 10 is used, the target parking position where the host vehicle 1 is to be parked is registered in the parking assistance device 10. Specifically, the landmark objects existing around the target parking position are extracted and stored in advance in the storage device 21. In the following description, the landmark objects around the target parking position stored in the storage device 21 are expressed as "learning completed landmark objects". Figure 2 (a) is an explanatory diagram of an example of a target parking position registration process, wherein a circle icon indicates a learning completion mark object. When registering the target parking position 30 in the parking assist device 10, the occupant performs an operation (hereinafter sometimes referred to as a "registration operation") to instruct registration of the target parking position 30. The registration operation may be, for example, an operation of a "parking position registration switch" prepared on the HMI 13.

[0028] For example, when the vehicle 1 is located near the target parking position 30 (for example, when the occupant parks the vehicle 1 at the target parking position 30 by manual driving), the parking assistance device 10 detects the landmark objects around the vehicle 1 through the external sensor 15 and stores them as learned landmark objects. For example, the landmark objects can be detected from the surrounding image obtained by photographing the surroundings of the vehicle 1 by a camera. In this case, the edges or corners of landmark objects such as road markings, road boundaries, obstacles, etc. on the surrounding image, edge points where the brightness of adjacent pixels changes by more than a specified value, or points with characteristics in the shape (feature points) are detected as landmark objects. The landmark objects around the vehicle 1 can also be detected by a distance measuring device.

[0029] The parking assist device 10 stores the learning completion mark object data related to the learning completion mark object in the storage device 21. For example, the learning completion mark object data includes: data indicating the feature quantity of the learning completion mark object (hereinafter referred to as "feature quantity data"), data on the relative positional relationship between the learning completion mark object and the target parking position (hereinafter referred to as "relative position data"), and coordinate data of the target parking position 30 (hereinafter referred to as "target parking position coordinate data").

[0030] As the relative position data, for example, the relative position of the learning completion mark object relative to the target parking position 30 may be stored. The parking assist device 10 can obtain the position of the learning completion mark object detected when the host vehicle 1 is located at the target parking position 30 as the relative position of the learning completion mark object relative to the target parking position 30. For example, the occupant can operate the HMI 13 to input that the host vehicle 1 is located at the target parking position 30. Instead of this, the coordinates of the learning completion mark object and the target parking position 30 in a coordinate system (hereinafter referred to as a "map coordinate system") with a fixed location as a reference point may be stored.

[0031] As the target parking position coordinate data, the coordinate data of the target parking position 30 in the map coordinate system and the angle data indicating the front-rear direction 32 of the parking space for parking the vehicle at the target parking position 30 are stored. As the target parking position coordinate data, the own position of the vehicle 1 in the map coordinate system when the vehicle 1 is located at the target parking position 30 and the front-rear direction of the vehicle body may also be stored.

[0032] Figure 2 (b) is an explanatory diagram of an example of processing when performing parking assistance. The parking assistance device 10 starts parking assistance for the vehicle 1 when the vehicle 1 is located near the registered target parking position 30 and an operation (hereinafter sometimes referred to as "start operation") of the occupant is performed to instruct the start of parking assistance control for the vehicle 1 to move to the target parking position 30. The start operation may be, for example, an operation of a "parking assistance start switch" prepared on the HMI 13, or a shift operation for switching the vehicle 1 between forward and reverse movement (for example, a switching operation from a drive range (D range) to a reverse range (R range) or a switching operation from an R range to a D range).

[0033] The parking assist device 10 extracts landmark objects around the host vehicle 1 from the surrounding image obtained by capturing the surroundings of the host vehicle 1 by a camera. In the following description, the landmark objects around the host vehicle 1 extracted when performing parking assist are referred to as "surrounding landmark objects". Figure 2In (b), the triangular icon represents a surrounding marker object. The parking assist device 10 matches the learning completion marker object and the surrounding marker object stored in the storage device 21, and makes the same feature points correspond to each other. Then, based on the relative positional relationship between the surrounding marker objects detected when the parking assist is implemented and the host vehicle 1, and the relative positional relationship between the learning completion marker object corresponding to the surrounding marker object and the target parking position 30, the relative position of the host vehicle 1 with respect to the target parking position 30 is calculated. For example, the parking assist device 10 calculates the position of the target parking position 30 on a coordinate system (hereinafter referred to as "vehicle coordinate system") based on the current position of the host vehicle 1. In addition, when the coordinates of the learning completion marker object and the target parking position 30 on the map coordinate system are stored in the storage device 21, the coordinates of the target parking position 30 on the map coordinate system may be converted into coordinates on the vehicle coordinate system based on the positions of the surrounding marker objects detected when the parking assist is implemented and the positions of the learning completion marker objects on the map coordinate system. The position of the vehicle 1 on the map coordinate system can also be determined based on the positions of surrounding marker objects detected when implementing parking assistance and the positions of learning completion marker objects in the map coordinate system. The relative position of the vehicle 1 relative to the target parking position 30 can be calculated based on the difference between the coordinates of the vehicle 1 in the map coordinate system and the coordinates of the target parking position 30.

[0034] The parking assist device 10 calculates a target driving trajectory 34 from the current position 33 of the host vehicle 1 to the target parking position 30 based on the relative position of the host vehicle 1 to the target parking position 30. The parking assist device 10 performs parking assist control of the host vehicle 1 based on the calculated target driving trajectory 34.

[0035] Furthermore, the parking assist device 10 generates a bird's-eye view image (bird's-eye view monitor image) of the surroundings of the vehicle 1 based on the camera image generated by the camera, superimposes a target parking frame indicating a parking space for parking the vehicle at the target parking position 30 on the bird's-eye view image, and displays the target parking frame on the display device of the HMI 13. The target parking frame is an example of "a guide graphic indicating a range of a predetermined size surrounding the target parking position" described in the scope of the claim.

[0036] The functional structure of the controller 17 will be described in more detail below. Figure 3 This is a block diagram showing an example of the functional configuration of the controller 17 .

[0037] The HMI control unit 40 detects the registration operation of the occupant for the target parking position 30 and the start operation of the parking assist control for the registered target parking position 30. When the registration operation is detected, the HMI control unit 40 outputs a map generation command to the map generation unit 45 to store the learning completion marker object data in the storage device 21. When the start operation is detected, the HMI control unit 40 outputs a control start command to start the parking assist control of the host vehicle 1 to the target parking position 30 to the parking assist control unit 41. The image conversion unit 42 converts the captured image of the camera into a bird's-eye view image observed from a virtual viewpoint directly above the host vehicle 1. The image conversion unit 42 converts the captured image into a bird's-eye view image at a predetermined interval (for example, every time the host vehicle 1 travels a predetermined distance (for example, 50 cm) or a predetermined time (for example, 1 second)), and accumulates the converted bird's-eye view images along the travel path of the host vehicle 1, thereby generating an image of the surrounding area of ​​the host vehicle 1, that is, a surrounding image.

[0038] The own position calculation unit 43 calculates the current position of the own vehicle 1 on the map coordinate system, that is, the own position and posture (for example, the front and rear direction of the vehicle body) by the odometry method (for example, the dead reckoning method) based on the vehicle information output from the vehicle sensor 16. The own position calculation unit 43 corrects the calculation result of the own position and posture based on the detection result of the own position and posture of the positioning device 11. The landmark object detection unit 44 detects the landmark object based on the surrounding image output from the image conversion unit 42. The landmark object detection unit 44 can detect the position of the feature point of the landmark object and its image feature value as the landmark object. In the detection of the position of the feature point and the calculation of the image feature value, for example, SIFT, SURF, ORB, BRIAK, KAZE, AKAZE and other methods can be used. The landmark object detection unit 44 outputs the position of the detected feature point and the image feature value as the landmark object data to the map generation unit 45 and the docking unit (corresponding unit) 47. In addition, the own position obtained from the own position calculation unit 43 is output to the map generation unit 45 and the docking unit 47 in synchronization with the detection of the landmark object.

[0039] When receiving a map generation instruction from the HMI control unit 40 (i.e., when a registration operation of the target parking position 30 is performed), the map generation unit 45 generates learning-completed landmark object data and stores it in the storage device 21 as map data 46. For example, the map generation unit 45 receives landmark object data and the own position of the host vehicle 1 on the map coordinate system synchronized with the landmark object data from the landmark object detection unit 44.

[0040] The map generation unit 45 obtains the coordinate data of the target parking position 30 in the map coordinate system and the angle data of the front-rear direction 32 of the parking space for parking the vehicle at the target parking position 30. For example, the own position calculated by the own position calculation unit 43 when the host vehicle 1 is located at the target parking position 30 may be obtained as the coordinate data of the target parking position 30, and the angle of the front-rear direction of the vehicle body may be obtained as the angle data of the front-rear direction 32.

[0041] The map generation unit 45 generates relative position data based on the positions of the feature points included in the landmark object data, the position information of the host vehicle 1 synchronized therewith, and the position data of the target parking position 30. In addition, the map generation unit 45 obtains feature quantity data from the landmark object data. The position data of the target parking position 30 and the angle data of the front-rear direction 32 of the parking space are used as the target parking position coordinate data. The learning completed landmark object data including these relative position data, feature quantity data, and target parking position coordinate data are stored in the storage device 21 as map data 46.

[0042] When receiving a control start command from the HMI control unit 40 (i.e., when a parking assist control start operation is performed), the parking assist control unit 41 outputs a parking position calculation command to the docking unit 47. The docking unit 47 receives the landmark object data output from the landmark object detection unit 44 as landmark object data of the surrounding landmark objects, and synchronously receives the own position of the host vehicle 1 in the map coordinate system.

[0043] The docking unit 47 reads the learning completion mark object data stored in the storage device 21 as the map data 46, and determines whether the vehicle 1 is located near the registered target parking position 30 based on the target parking position coordinate data. When the vehicle 1 is located near the registered target parking position 30, the docking unit 47 matches the learning completion mark object stored as the map data 46 with the surrounding mark objects and makes the mark objects with the same feature points correspond to each other. The docking unit 47 calculates the current relative position of the vehicle 1 relative to the target parking position 30 based on the relative position relationship between the surrounding mark objects and the vehicle 1, and the relative position relationship between the learning completion mark objects corresponding to the surrounding mark objects and the target parking position 30. For example, the surrounding mark objects are expressed as (x i ,y i ), and will be respectively compared with the surrounding landmark objects (x i ,y i ) The corresponding learning completion mark object is expressed as (x mi ,y mi )(i=1 to N). The docking unit 47 calculates the affine transformation matrix M according to the following equation based on the least squares method: affine1 .

[0044] (Equation 1)

[0045]

[0046] in,

[0047]

[0048] The docking unit 47 converts the position (target x) of the target parking position 30 on the map coordinate system stored in the map data 46 into the position (target x m , targety m ) is transformed into the position (targetx, targety) of the vehicle coordinate system.

[0049] (Formula 2)

[0050]

[0051] Furthermore, the docking unit 47 converts the angle targetyaw of the front-rear direction 32 of the parking space representing the target parking position 30 on the map coordinate system into: m Transformed into the angle targetyaw representing the direction in the vehicle coordinate system.

[0052] (Formula 3)

[0053]

[0054] The target trajectory generation unit 48 calculates the target driving trajectory and the target vehicle speed curve from the current position of the host vehicle 1 on the vehicle coordinate system to the target parking position 30 when receiving the driving trajectory calculation instruction from the parking assist control unit 41. The steering control unit 49 controls the steering actuator 19a to make the host vehicle 1 travel along the target driving trajectory when receiving the steering control instruction from the parking assist control unit 41. The vehicle speed control unit 50 controls the accelerator actuator 19b and the brake actuator 19c to change the vehicle speed of the host vehicle 1 according to the target vehicle speed curve when receiving the vehicle speed control instruction from the parking assist control unit 41. When the host vehicle 1 reaches the target parking position 30 and the parking assist control is completed, the parking assist control unit 41 activates the parking brake 18 to switch the gear position to the parking gear (P gear position).

[0055] During the parking assist control, the HMI control unit 40 receives the bird's-eye view image of the surroundings of the host vehicle 1 from the image conversion unit 42 and displays it on the display device of the HMI 13. In addition, the HMI control unit 40 receives information on the coordinates (targetx, targety) of the target parking position 30 and the angle targetyaw in the front-rear direction 32 from the docking unit 47, and displays a target parking frame indicating a parking space for parking the vehicle at the target parking position 30 in an overlapping manner on the bird's-eye view image.

[0056] Figure 4 (a) is a schematic diagram of an example of a display area in which a bird's-eye view image is displayed on a display device of HMI13. Reference symbol 60 indicates a display screen of the display device of HMI13. A bird's-eye view image 62 is displayed in a predetermined display area 61 of the display screen 60. In the following description, for convenience, the reference point Pr at the lower left corner of the display area 61 is used as the coordinate origin (0, 0), and the right direction and the upward direction are respectively set as the positive direction of the x-axis and the positive direction of the y-axis, but the setting of the coordinate system is not limited to this example and can also be appropriately changed.

[0057] The display area 61 is an area pre-set in the display screen 60. The number of pixels of the display area 61 in the x-axis direction and the y-axis direction are Wavm and Havm, respectively. For example, the number of pixels in the x-axis direction and the y-axis direction of the display area 61 can be set to be equal to the number of pixels in the x-axis direction and the y-axis direction of the bird's-eye view image 62 output from the image conversion unit 42 to the HMI control unit 40. Figure 4 In (a), the bird's-eye view image 62 is depicted slightly smaller than the display area 61 in order to make the boundary of the bird's-eye view image 62 easier to see. However, the bird's-eye view image 62 may be displayed in the entire display area 61 (i.e., the number of pixels in the x-axis direction and the y-axis direction of the bird's-eye view image 62 displayed in the display area 61 may be set to Wavm and Havm, respectively).

[0058] The bird's-eye view image 62 is an image in which an icon 63 representing the host vehicle 1 is superimposed at the center of the bird's-eye view image representing a range of a length DL in the front-rear direction and a length DT in the vehicle width direction centered on the current position of the host vehicle 1. The length DL in the front-rear direction and the length DT in the vehicle width direction are appropriately set so that the occupant can understand the sense of distance from the ground features (road markings such as white lines or road boundaries such as shoulders) around the host vehicle 1 mapped in the bird's-eye view image 62.

[0059] In the following description, the position of the lower left corner of the bird's-eye view image 62 (in Figure 4 In the example of (a), the position Pb1) is designated as the display position where the bird's-eye view image 62 is displayed in the display area 61 .

[0060] The HMI control unit 40 calculates the position at which the target parking frame 64 is to be displayed superimposed on the bird's-eye view image 62. Specifically, the target parking position Pt in the coordinate system on the display screen 60 is calculated based on the coordinates (targetx, targety) of the target parking position 30 received from the docking unit 47 and the scale at which the bird's-eye view image 62 is displayed on the display screen 60. In addition, based on the angle targetyaw received from the docking unit 47 and the target parking position Pt, the display position and direction of the target parking frame 64 in the coordinate system on the display screen 60 are set. For example, the target parking position Pt is set as the center position of the target parking frame 64, and the direction indicated by the angle targetyaw is set as the front-rear direction of the target parking frame 64.

[0061] The size (length in the front-to-back direction and lateral length) of the target parking frame 64 displayed on the display screen 60 can be set, for example, based on the scale of the bird's-eye view image 62 and the size of the vehicle 1. For example, the size of the target parking frame 64 can be set to be approximately equal to the size of the occupied area of ​​the vehicle 1 in the bird's-eye view image 62. In addition, the size of the target parking frame 64 can be set to be slightly larger than the occupied area of ​​the vehicle 1 to set a margin, or can be set to be slightly smaller than the occupied area of ​​the vehicle 1. In addition, the size of the target parking frame 64 can be pre-set to an initial value when the bird's-eye view image 62 is displayed on the entire area of ​​the display area 61 (i.e., when the number of pixels in the x-axis direction and the y-axis direction display the bird's-eye view image 62 of Wavm and Havm, respectively).

[0062] Here, if the parking assist control is started when the vehicle 1 is away from the target parking position 30, Figure 4 As shown in (a), the display position of the target parking frame 64 may be set outside the range of the bird's-eye view image 62. In this case, the relative position of the target parking frame 64 with respect to the icon 63 representing the host vehicle 1 cannot be accurately grasped.

[0063] Therefore, when the HMI control unit 40 displays the bird's-eye view image 62 at the first display position Pb1 of the display screen 60, if the target parking frame 64 can be displayed in the display area 61 within a specified range, the bird's-eye view image 62 is displayed at the first display position Pb1 of the display screen 60. For example, when the display position for displaying the bird's-eye view image 62 is set to the first display position Pb1, it is determined whether the target parking frame 64 can be displayed in the display area 61 within a specified range. Figure 4In the example of (a), when the bird's-eye view image 62 is displayed in the entire area of ​​the display area 61 (i.e., when the bird's-eye view image 62 is displayed with the number of pixels in the x-axis direction and the y-axis direction being Wavm and Havm, respectively, and the first display position Pb1 of the bird's-eye view image 62 is set to the reference point Pr at the lower left corner of the display area 61), it is determined whether or not the target parking frame 64 can be displayed in the display area 61 to a greater extent than the prescribed range. In the following description, the prescribed range of the target parking frame 64 is sometimes expressed as "necessary display range".

[0064] On the other hand, when the bird's-eye view image 62 is displayed at the first display position Pb1 of the display screen 60, if the target parking frame 64 cannot be displayed within the display area 61, the target parking frame 64 may be displayed within the specified range. Figure 4 As shown in FIG. 5( b), the bird's-eye view image 62 is displayed at a second display position Pb2 of the display screen 60 that is different from the first display position Pb1. As a result, it is possible to display more than the necessary display range of the target parking frame 64 in the display area 61. In this case, for example, the portion of the bird's-eye view image 62 that is exposed outside the display area 61 and the portion of the target parking frame 64 that is not within the display area 61 can be clipped by changing the display position, and the remaining portion can be displayed in the display area 61.

[0065] The HMI control unit 40 obtains the offset direction (movement direction) and the offset amount (movement amount) of the display position of the bird's-eye view image 62 relative to the reference position (i.e., the first display position Pb1), and calculates the second display position Pb2 by shifting the first display position Pb1 in the offset direction by the offset amount, so that the target parking frame 64 can be displayed in the display area 61 in an area larger than the required display range. Figure 4 (c). The offset direction of the display position of the bird's-eye view image 62 is determined based on the position of the target parking position Pt when the bird's-eye view image 62 is displayed in the entire area of ​​the display area 61. Region R1 is a region where the x coordinate is less than 0 and the y coordinate is greater than Havm, region R2 is a region where the x coordinate is greater than 0 and less than Wavm and the y coordinate is greater than Havm, and region R3 is a region where the x coordinate and the y coordinate are greater than Wavm and Havm. Region R4 is a region where the x coordinate is less than 0 and the y coordinate is greater than 0 and less than Havm, region R5 is a region where the x coordinate is greater than Wavm and the y coordinate is greater than 0 and less than Havm. Region R6 is a region where the x coordinate and the y coordinate are less than 0, region R7 is a region where the x coordinate is greater than 0 and less than Wavm and the y coordinate is less than 0, and region R8 is a region where the x coordinate is greater than Wavm and the y coordinate is less than 0.

[0066] For example, when the target parking position Pt is located in the region R1, the offset direction of the display position of the overhead image 62 may be set to the lower right direction (increasing the x coordinate and decreasing the y coordinate), when the target parking position Pt is located in the region R2, the offset direction of the display position of the overhead image 62 may be set to the lower direction (decreasing the y coordinate without changing the x coordinate), and when the target parking position Pt is located in the region R3, the offset direction of the display position of the overhead image 62 may be set to the lower left direction (decreasing the x coordinate and the y coordinate). When the target parking position Pt is located in the region R4, the offset direction of the display position of the overhead image 62 may be set to the right (increasing the x coordinate without changing the y coordinate), and when the target parking position Pt is located in the region R5, the offset direction of the display position of the overhead image 62 may be set to the left (decreasing the x coordinate without changing the y coordinate). When the target parking position Pt is located in area R6, the offset direction of the display position of the overhead image 62 can be set to the upper right direction (increasing the x coordinate and the y coordinate); when the target parking position Pt is located in area R7, the offset direction of the display position of the overhead image 62 can be set to the upper direction (increasing the y coordinate without changing the x coordinate); and when the target parking position Pt is located in area R8, the offset direction of the display position of the overhead image 62 can be set to the upper left direction (reducing the x coordinate and increasing the y coordinate).

[0067] For example, Figure 4 (b) shows an example of setting the offset direction and offset amount of the display position of the bird's-eye view image 62 so that the range including the center of the target parking frame 64 (e.g., the target parking position Pt) can be displayed as the necessary display range of the target parking frame 64 in the display area 61. Figure 4 In (a), the coordinates of the target parking position Pt when the bird's-eye view image 62 is displayed on the entire display area 61 are (-Px, -Py), the x-direction offset ofx is set to Px, and the y-direction offset ofy is set to Py.

[0068] The required display range of the target parking frame 64 is not limited to the above example, and may be appropriately set so that the occupant can easily understand the target parking frame 64 displayed on the display screen 60. For example, the required display range may be a range including a portion of each of at least two sides 64a and 64b of the rectangular target parking frame 64 ( Figure 5 (a)), or may be a range including the full length of at least two sides 64a, 64b of the rectangular target parking frame 64 ( Figure 5 (b)), or may be a range including the entire length of at least one short side 64a of the rectangular target parking frame 64 ( Figure 5 (c)), or a range that includes any of the four sides of the rectangular shape ( Figure 5 (d)).

[0069] For example, the HMI control unit 40 can calculate the coordinates of the four corners of the rectangular target parking frame 64 based on the target parking position Pt and the angle targetyaw, and determine the offset direction and offset amount of the display position of the overhead image 62 so as to be able to display above the above-mentioned necessary display range within the display area 61.

[0070] (action)

[0071] Figure 6 Flowchart of an example of the parking assistance method of the first embodiment. In step S1, the HMI control unit 40 calculates the target parking position Pt in the coordinate system on the display screen 60. In addition, the display position and direction of the target parking frame 64 are calculated. In step S2, the HMI control unit 40 determines whether the target parking frame 64 can be displayed above the prescribed range in the display area 61 when the overhead image 62 is displayed at the first display position Pb1. When the target parking frame 64 cannot be displayed above the prescribed range in the display area 61 (step S2: No), the process proceeds to step S3. When the target parking frame 64 can be displayed above the prescribed range in the display area 61 (step S2: Yes), the process proceeds to step S7.

[0072] In step S3, the HMI control unit 40 calculates the offset amount (ofx, ofy) of the display position of the bird's-eye view image 62. In step S4, the HMI control unit 40 calculates the display position of the target parking frame 64 after the offset based on the offset amount (ofx, ofy).

[0073] In step S5, the HMI control unit 40 displays the bird's-eye view image 62 at the second display position Pb2 after being shifted by the offset amount (ofx, ofy). In step S6, the HMI control unit 40 overlays and displays the target parking frame 64 at the (moved) display position after being shifted by the offset amount (ofx, ofy). Then, the process ends.

[0074] In step S7, the HMI control unit 40 displays the bird's-eye view image 62 at the first display position Pb1. In step S8, the HMI control unit 40 displays the target parking frame 64 in an overlapping manner at the position calculated in step S1. Then, the process ends.

[0075] (Second Embodiment)

[0076] If the display position is shifted while maintaining the size of the bird's-eye view image 62 (i.e., the scale of the bird's-eye view image 62), a portion of the bird's-eye view image 62 may be exposed outside the display area 61 and cannot be displayed on the display screen 60. Figure 7 As shown in (a), the bird's-eye view image 62 displayed in the display area 61 is reduced in size.

[0077] Therefore, if Figure 4 As shown in (a), the overhead image 62 can be displayed at the second display position Pb2 in a manner that includes the entire range that can be displayed in the display area 61 when the overhead image 62 is displayed at the first display position Pb1 (i.e., the range of the length DL in the front-to-back direction and the length DT in the vehicle width direction centered on the current position of the vehicle 1).

[0078] For example, when the x-direction offset ofx is smaller than the y-direction offset ofy, the HMI control unit 40 calculates the reduction ratio scale by the following equation.

[0079] scale=(Havm-ofy) / Havm

[0080] When the x-direction offset ofx is greater than the y-direction offset ofy, the HMI controller 40 calculates the reduction ratio scale by the following equation.

[0081] scale = (Wavm-ofx) / Wavm

[0082] The HMI control unit 40 offsets the display positions of the bird's-eye view image 62 and the target parking frame 64 by the x-direction offset ofx and the y-direction offset ofy, and sets the affine transformation matrix M for reducing the bird's-eye view image 62 and the target parking frame 64 by the reduction ratio scale according to the following formula: affine2 .

[0083] (Formula 4)

[0084]

[0085] The HMI control unit 40 uses the affine transformation matrix M affine2 The coordinates of the bird's-eye view image 62 and the target parking frame 64 are transformed. Specifically, if the coordinates of each pixel of the bird's-eye view image 62 before transformation and the coordinates of the boundary of the target parking frame 64 are set to (Px1, Py1), the coordinates of each pixel of the bird's-eye view image 62 after offset and reduction and the coordinates of the boundary of the target parking frame 64 are calculated by the following formula (Px2, Py2).

[0086] (Equation 5)

[0087]

[0088] In addition, if Figure 7 As shown in (a), the HMI control unit 40 may draw grid lines 65 in the background portion of the display area 61 (an area where the bird's-eye view image 62 is not displayed). Figure 7 As shown in (b), the HMI control unit 40 may display an image in which a background image 66 in which grid lines are drawn is added to the bird's-eye view image 62 in the display area 61 .

[0089] Figure 8 is a flowchart of an example of the parking assistance method according to the second embodiment. The processing of steps S10 and S11 is similar to Figure 6 The processing of steps S1 and S2 is the same. When the target parking frame 64 cannot be displayed in the display area 61 above the specified range (step S11: No), the processing proceeds to step S12. When the target parking frame 64 can be displayed in the display area 61 above the specified range (step S11: Yes), the processing proceeds to step S17.

[0090] The processing of step S12 is Figure 6 In step S13, the HMI control unit 40 calculates the reduction ratio scale. In step S14, the HMI control unit 40 calculates the display position of the target parking frame 64 when the overhead image 62 is offset by the offset amount (ofx, ofy) and reduced by the reduction ratio scale.

[0091] In step S15, the HMI control unit 40 displays the bird's-eye view image 62 offset by the offset amount (ofx, ofy) and reduced by the reduction ratio scale. In step S16, the HMI control unit 40 displays the target parking frame 64 offset by the offset amount (ofx, ofy) and reduced by the reduction ratio scale in an overlapping manner. Then, the process ends.

[0092] The processing of steps S17 and S18 is Figure 6 The processing of steps S7 and S8 is the same.

[0093] (Variation Example)

[0094] In the first and second embodiments, an example is described in which the relative position of the target parking position 30 with respect to the current position of the host vehicle 1 is detected by matching the pre-stored marker objects around the target parking position 30 with the marker objects detected around the host vehicle 1 when the parking assist control is executed, but the parking assist method of the present invention is not limited to such a method for determining the target parking position. For example, when the parking assist control is executed, a marker object indicating that it is a parking area (for example, a road surface display such as a parking frame line or a three-dimensional object such as a wheel stopper) may be detected to detect the relative position of the target parking position 30 with respect to the current position of the host vehicle 1.

[0095] (Effects of Embodiments)

[0096] (1) The parking assistance method of the present invention sets a target parking position for a host vehicle 1 to park, generates a bird's-eye view image of the surroundings of the host vehicle 1 based on a camera image obtained by photographing the surroundings of the host vehicle 1, and displays the bird's-eye view image and a guide graphic indicating a range of a specified size surrounding the target parking position in a specified display area of ​​a display screen that can be visually recognized by an occupant of the host vehicle 1, wherein when the bird's-eye view image can be displayed at a first display position on the display screen and a range larger than a specified range of the guide graphic can be displayed within the display area, the bird's-eye view image is displayed at the first display position on the display screen, and when the bird's-eye view image cannot be displayed at the first display position on the display screen and a range larger than a specified range of the guide graphic can be displayed within the display area, the bird's-eye view image is displayed at a second display position on the display screen that is different from the first display position.

[0097] Thus, even if the vehicle is far away from the target parking position, an overhead image of the surroundings of the vehicle 1 and a predetermined range of guide graphics can be simultaneously displayed, thereby providing an image that makes it easy to understand the relative position of the target parking frame with respect to the vehicle.

[0098] (2) When displaying the bird's-eye view image at the second display position, the bird's-eye view image displayed in the display area can be reduced. This can prevent the display range of the bird's-eye view image from being reduced after the display position of the bird's-eye view image is changed.

[0099] (3) By reducing the overhead image displayed in the display area, the overhead image including the entire range that can be displayed in the display area when the display position of the overhead image is set to the first display position can be displayed at the second display position. Thus, even if the display position of the overhead image is changed, the entire range of the overhead image that can be displayed before the change can be displayed.

[0100] (4) A background image with grid lines drawn on it may be added to the bird's-eye view image and displayed in the display area. This makes it easier to understand the distance to the guide graphic displayed outside the bird's-eye view image.

[0101] (5) The prescribed range of the guide graphic may be a range that includes a portion of at least two sides of the rectangular guide graphic. In addition, the prescribed range of the guide graphic may be a range that includes the full length of at least two sides of the rectangular guide graphic. In addition, the prescribed range of the guide graphic may be a range that includes the full length of at least one short side of the rectangular guide graphic. In addition, the prescribed range of the guide graphic may be a range that includes any one of the four sides of the rectangular guide graphic. In addition, the prescribed range of the guide graphic may be a range that includes the center point of the guide graphic. Thus, the guide graphic can be displayed in a manner that is easy for the occupant to understand.

[0102] (6) The target parking position can be detected based on the captured image of the surroundings of the host vehicle 1, the target driving trajectory from the current position of the vehicle 1 to the target parking position can be calculated, and the host vehicle 1 can be controlled to travel along the target driving trajectory. In this way, the parking of the host vehicle 1 can be assisted.

[0103] All examples and conditional terms described herein are intended for the purpose of teaching in order to help readers understand the present invention and the concepts provided by the inventors for technological advancement, and should be interpreted as not being limited to the above examples and conditions specifically described, as well as the structure of the examples of this specification related to the superiority and deterioration of the present invention. Although the embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be applied thereto without departing from the spirit and scope of the present invention.

[0104] Explanation of symbols

[0105] 1: own vehicle, 10: parking assistance device, 11: positioning device, 13: human-machine interface (HMI), 14: shift switch (shift SW), 15: external sensor, 16: vehicle sensor, 17: controller, 18: parking brake, 19a: steering gear actuator, 19b: accelerator actuator, 19c: brake actuator, 20: processor, 21: storage device, 40: HMI control unit, 41: parking assistance control unit, 42: image conversion unit, 43: own position calculation unit, 44: landmark object detection unit, 45: map generation unit, 46: map data, 47: docking unit, 48: target trajectory generation unit, 49: steering control unit, 50: vehicle speed control unit.

Claims

1. A parking assistance method, comprising: setting a target parking position for a host vehicle, generating an overhead image of the surroundings of the host vehicle based on a camera image obtained by photographing the surroundings of the host vehicle, and displaying the overhead image and a guide graphic indicating a range of a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen that can be visually recognized by an occupant of the host vehicle, characterized in that: If a predetermined range or more of the guide graphic can be displayed in the display area when the bird's-eye view image is displayed at the first display position on the display screen, displaying the bird's-eye view image at the first display position on the display screen, If the guide graphic cannot be displayed within the display area beyond the predetermined range when the bird's-eye view image is displayed at the first display position of the display screen, the bird's-eye view image is displayed at a second display position of the display screen different from the first display position.

2. The parking assistance method according to claim 1, characterized in that: When the bird's-eye view image is displayed at the second display position, the bird's-eye view image displayed in the display area is reduced in size.

3. The parking assistance method according to claim 2, characterized in that: By reducing the overhead image displayed in the display area, the overhead image including the entire range that can be displayed in the display area when the display position of the overhead image is set to the first display position is displayed at the second display position.

4. The parking assistance method according to claim 1, characterized in that: A background image in which grid lines are drawn is added to the bird's-eye view image and displayed in the display area.

5. The parking assistance method according to claim 1, characterized in that: The predetermined range of the guide figure is a range including a portion of each of at least two sides of the rectangular guide figure.

6. The parking assistance method according to claim 1, characterized in that: The predetermined range of the guide figure is a range including the full length of each of at least two sides of the rectangular guide figure.

7. The parking assistance method according to claim 1, characterized in that: The predetermined range of the guide figure is a range including the entire length of at least one short side of the rectangular guide figure.

8. The parking assistance method according to claim 1, characterized in that: The predetermined range of the guide figure is a range including any one of four sides of the rectangular guide figure.

9. The parking assistance method according to claim 1, characterized in that: The predetermined range of the guide figure is a range including a center point of the guide figure.

10. The parking assistance method according to any one of claims 1 to 9, characterized in that: The target parking position is detected based on a camera image obtained by photographing the surroundings of the host vehicle, a target driving trajectory from the current position of the host vehicle to the target parking position is calculated, and the host vehicle is controlled to travel along the target driving trajectory.

11. A parking assistance device comprising: a camera for photographing the surroundings of a host vehicle, and a display device having a display screen that can be visually recognized by an occupant of the host vehicle, wherein a target parking position for the host vehicle is set, an overhead image of the surroundings of the host vehicle is generated based on a camera image generated by the camera, and the overhead image and a guide graphic indicating a range of a predetermined size surrounding the target parking position are displayed in a predetermined display area of ​​the display screen, characterized in that: A controller is provided, and the controller performs the following processing: When the bird's-eye view image is displayed at the first display position on the display screen, a process of displaying the bird's-eye view image at the first display position on the display screen if a predetermined range or more of the guide graphic can be displayed in the display area; and displaying the bird's-eye view image at a second display position of the display screen different from the first display position when the guide graphic cannot be displayed within the display area beyond the predetermined range when the bird's-eye view image is displayed at the first display position of the display screen.

Images