Parking assistance method and parking assistance device

Abstract

This parking assistance method sets a target parking position for parking an own vehicle, generates an overhead image (62) of surroundings of the own vehicle from a captured image obtained by photographing the surroundings of the own vehicle, and displays the overhead image (62) and a guide figure (64), which indicates a predetermined size range enclosing the target parking position, in a predetermined display area (61) of a display screen (60) which is visually recognizable by an occupant of the own vehicle (1). When a predetermined range or more of the guide figure (64) can be displayed within the display area (61) when the overhead image (62) is displayed at a first display position (Pb1) on the display screen (60), the overhead image (62) is displayed at the first display position (Pb1) on the display screen (60). When the predetermined range or more of the guide figure (64) cannot be displayed within the display area (61) when the overhead image (62) is displayed at the first display position (Pb1) on the display screen (60), the overhead image (62) is displayed at a second display position (Pb2) different from the first display position (Pb1).

Description

[Technical Field]

[0001] The present invention relates to a parking assistance method and a parking assistance device. [Background technology]

[0002] The parking assistance device described in Patent Document 1 below draws an extension line of the target parking space and a predicted trajectory line of the vehicle on the overhead image when the target parking space is located outside the overhead image generated from images taken around the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-80959 Summary of the Invention [Problem to be solved by the invention]

[0004] However, if the vehicle is far from the target parking position and the display position of the target parking frame is set outside the range of the overhead image, the relative position of the target parking frame with respect to the vehicle cannot be correctly grasped. An object of the present invention is to provide an image that makes it easy to grasp the relative position of a target parking frame with respect to one's own vehicle even when the own vehicle is far from the target parking position. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided a parking assistance method which sets a target parking position where a host vehicle is to be parked, generates an overhead image of the surroundings of the host vehicle from captured images obtained by photographing the surroundings of the host vehicle, and displays the overhead image and a guide figure representing an area of ​​a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen visible to an occupant of the host vehicle. In the parking assistance method, when the overhead image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the overhead image is displayed at the first display position on the display screen, and when the overhead image is displayed at the first display position on the display screen, if a predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen different from the first display position. [Effects of the Invention]

[0006] According to the present invention, an image can be provided that makes it easy to grasp the relative position of the target parking frame with respect to the vehicle, even if the vehicle is far from the target parking position. The objects and advantages of the invention will be realized and attained by means of the elements and combinations set forth in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a parking assistance device. [Figure 2] 1A is an explanatory diagram of an example of a process for registering a target parking position, and FIG. 1B is an explanatory diagram of an example of a process when parking assistance is performed. [Figure 3] 2 is a block diagram illustrating an example of a functional configuration of a controller in FIG. 1. FIG. [Figure 4] 5(a) to 5(c) are explanatory diagrams of an example of a parking assistance method according to the first embodiment. [Figure 5] 10(a) to 10(d) are schematic diagrams of examples of overlapping target parking frames. [Figure 6] 3 is a flowchart of an example of a parking assistance method according to the first embodiment. [Figure 7] 10(a) and 10(b) are explanatory diagrams of an example of a parking assistance method according to a second embodiment. [Figure 8] 10 is a flowchart illustrating an example of a parking assistance method according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] (First embodiment) (composition) Referring to FIG. 1, the host vehicle 1 is equipped with a parking assistance device 10 that assists in parking the host vehicle 1 at a target parking position. The parking assistance device 10 assists the host vehicle 1 in traveling along a target traveling trajectory from the current position of the host vehicle 1 to the target parking position. For example, the host vehicle 1 may be automatically driven to travel along the target traveling trajectory of the host vehicle 1 to the target parking position (i.e., control to automatically perform all or part of traveling along the target traveling trajectory of the host vehicle 1 by controlling all or part of the steering angle, driving force, and braking force of the host vehicle). Parking of the host vehicle 1 may be assisted by displaying the target parking position, the target traveling trajectory, and the current position of the host vehicle 1 on a display device that is visible to an occupant of the host vehicle 1 (e.g., the driver).

[0009] The positioning device 11 measures the current position and attitude of the host vehicle 1 (for example, in the longitudinal direction of the vehicle body). The positioning device 11 includes a Global Navigation System (GNSS) receiver such as a Global Positioning System (GPS) receiver. The human-machine interface (HMI) 13 is an interface device that exchanges information between the parking assistance device 10 and the occupant, and includes a display device with a display screen visible to the occupant, a speaker, a buzzer, and an operator. The shift switch (shift SW) 14 is a switch that allows the occupant of the host vehicle 1 or the parking assistance device 10 to change the shift position of the host vehicle 1.

[0010] The external sensor 15 detects objects within a predetermined distance range from the host vehicle 1. The external sensor 15 detects the surrounding environment of the host vehicle 1, such as the relative position of the host vehicle 1 and an object present around the host vehicle 1, the distance between the host vehicle 1 and the object, and the direction in which the object is present. The external sensor 15 may include, for example, a camera that captures the surrounding environment of the host vehicle 1. In the following description, the camera included in the external sensor 15 will be simply referred to as a "camera." The external sensor 15 may include a distance measuring device such as a laser range finder, radar, or LiDAR. The vehicle sensor 16 detects various information (vehicle information) about the host vehicle 1. The vehicle sensor 16 may include, for example, a vehicle speed sensor, a wheel speed sensor, a three-axis acceleration sensor that detects the acceleration of the host vehicle 1 in three axial directions, a steering angle sensor that detects the steering angle, a steering angle sensor that detects the steering angle of the steered wheels, a gyro sensor, and a yaw rate sensor.

[0011] The controller 17 is an electronic control unit that performs parking assistance control. The controller 17 includes a processor 20 and peripheral components such as a storage device 21. The processor 20 may be, for example, a CPU or an MPU. The storage device 21 may include a semiconductor storage device, a magnetic storage device, an optical storage device, or the like. The functions of the controller 17 are realized, for example, by the processor 20 executing a computer program stored in the storage device 21. The steering actuator 19a controls the steering direction and steering amount of the steering mechanism of the host vehicle 1 in response to a control signal from the controller 17. The accelerator actuator 19b controls the accelerator opening of the drive device, which is an engine or a drive motor, in response to a control signal from the controller 17. The brake actuator 19c activates a braking device in response to a control signal from the controller 17.

[0012] Next, parking assistance control by the parking assistance device 10 will be described. When using parking assistance by the parking assistance device 10, a target parking position where the vehicle 1 should be parked is registered in the parking assistance device 10. Specifically, targets existing around the target parking position are extracted and stored in advance in the storage device 21. In the following description, targets around the target parking position stored in the storage device 21 will be referred to as "learned targets." FIG. 2(a) is an explanatory diagram of an example of the process of registering a target parking position, with circles representing learned targets. When registering a target parking position 30 in the parking assistance device 10, the occupant performs an operation to instruct the registration of the target parking position 30 (hereinafter, this may be referred to as a "registration operation"). The registration operation may be, for example, the operation of a "parking position registration switch" provided in the HMI 13.

[0013] For example, when the vehicle 1 is located near the target parking position 30 (for example, when the occupant manually parks the vehicle 1 at the target parking position 30), the parking assistance device 10 detects the surroundings of the vehicle 1 using the external sensor 15 and stores the detected targets as learned targets. For example, targets may be detected from a surrounding image obtained by capturing an image of the surroundings of the vehicle 1 with a camera. In this case, targets are detected as edge points or points (feature points) with characteristic shapes where the brightness of adjacent pixels changes by more than a predetermined amount, such as edges or corners of targets such as road markings, road boundaries, and obstacles in the surrounding image. Targets around the vehicle 1 may also be detected by a distance measuring device.

[0014] The parking assistance device 10 stores learned target data related to the learned target in the storage device 21. For example, the learned target data includes data representing the feature amounts of the learned target (hereinafter referred to as "feature amount data"), data on the relative positional relationship between the learned target and the target parking position (hereinafter referred to as "relative position data"), and coordinate data of the target parking position 30 (hereinafter referred to as "target parking position coordinate data"). As the relative position data, for example, the relative position of the learned target with respect to the target parking position 30 may be stored. The parking assistance device 10 can acquire the position of the learned target detected when the host vehicle 1 is located at the target parking position 30 as the relative position of the learned target with respect to the target parking position 30. For example, the occupant may operate the HMI 13 to input that the host vehicle 1 is located at the target parking position 30. Alternatively, the coordinates of the learned target and the target parking position 30 in a coordinate system with a fixed point as the reference point (hereinafter referred to as the "map coordinate system") may be stored. As the target parking position coordinate data, coordinate data of the target parking position 30 in the map coordinate system and angle data indicating the fore-and-aft direction 32 of the parking space where the vehicle is to be parked at the target parking position 30 are stored. As the target parking position coordinate data, the own position of the vehicle 1 in the map coordinate system and the fore-and-aft direction of the vehicle body when the vehicle 1 is located at the target parking position 30 may be stored.

[0015] 2(b) is an explanatory diagram of an example of processing when parking assistance is performed. The parking assistance device 10 starts parking assistance for the host vehicle 1 when the host vehicle 1 is positioned near a registered target parking position 30 and an occupant performs an operation (hereinafter sometimes referred to as an "activation operation") to instruct activation of parking assistance control of the host vehicle 1 to the target parking position 30. The activation operation may be, for example, an operation of a "parking assistance activation switch" provided in the HMI 13, or a shift operation for switching between forward and reverse movement of the host vehicle 1 (for example, an operation of switching from drive range (D range) to reverse range (R range) or an operation of switching from R range to D range).

[0016] The parking assistance device 10 extracts targets around the host vehicle 1 from a surrounding image obtained by capturing images of the surroundings of the host vehicle 1 with a camera. In the following description, targets around the host vehicle 1 extracted when parking assistance is performed are referred to as "surrounding targets." In FIG. 2(b), triangular plots represent surrounding targets. The parking assistance device 10 matches the learned targets stored in the storage device 21 with the surrounding targets and associates identical feature points. Then, based on the relative positional relationship between the host vehicle 1 and the surrounding targets detected when parking assistance is performed, and the relative positional relationship between the learned targets associated with the surrounding targets and the target parking position 30, the parking assistance device 10 calculates the relative position of the host vehicle 1 with respect to the target parking position 30. For example, the parking assistance device 10 calculates the position of the target parking position 30 in a coordinate system based on the current position of the host vehicle 1 (hereinafter referred to as the "vehicle coordinate system"). In addition, when the coordinates of the learned targets and the target parking position 30 on the map coordinate system are stored in the storage device 21, the coordinates of the target parking position 30 on the map coordinate system may be converted into coordinates on the vehicle coordinate system based on the positions of the surrounding targets detected when parking assistance is performed and the positions of the learned targets in the map coordinate system. The own position of the vehicle 1 on the map coordinate system may be obtained based on the positions of the surrounding targets detected when parking assistance is performed and the positions of the learned targets in the map coordinate system, and the relative position of the vehicle 1 with respect to the target parking position 30 may be calculated from the difference between the coordinates of the vehicle 1 and the coordinates of the target parking position 30 in the map coordinate system.

[0017] The parking assistance device 10 calculates a target driving trajectory 34 from the current position 33 of the vehicle 1 to the target parking position 30 based on the relative position of the vehicle 1 with respect to the target parking position 30. The parking assistance device 10 performs parking assistance control of the vehicle 1 based on the calculated target driving trajectory 34. Furthermore, the parking assistance device 10 generates an overhead image (around view monitor image) of the surroundings of the vehicle 1 from the captured image generated by the camera, and displays a target parking frame representing a parking space for parking the vehicle at the target parking position 30 on the overhead image on the display device of the HMI 13. The target parking frame is an example of a "guide figure representing an area of ​​a predetermined size surrounding the target parking position" as set forth in the claims.

[0018] The following describes in more detail the functional configuration of the controller 17. Figure 3 is a block diagram showing an example of the functional configuration of the controller 17. The HMI control unit 40 detects an operation by the occupant to register a target parking position 30 and an operation to activate parking assist control for the registered target parking position 30. When the registration operation is detected, the HMI control unit 40 outputs a map generation command to the map generation unit 45 to store learned target data in the storage device 21. When the activation operation is detected, the HMI control unit 40 outputs a control start command to the parking assist control unit 41 to start parking assist control of the host vehicle 1 to the target parking position 30. The image conversion unit 42 converts the captured image from the camera into a bird's-eye view image seen from a virtual viewpoint directly above the host vehicle 1. The image conversion unit 42 converts the captured image into a bird's-eye view image at predetermined intervals (for example, every time the host vehicle 1 travels a predetermined distance (for example, 50 cm) or a predetermined time (for example, 1 second)), and generates a surrounding image, which is an image of the area around the host vehicle 1, by accumulating the converted bird's-eye view images along the travel route of the host vehicle 1.

[0019] The vehicle position calculation unit 43 calculates the vehicle's current position and attitude (e.g., the longitudinal direction of the vehicle body) on a map coordinate system by odometry (e.g., dead reckoning) based on vehicle information output from the vehicle sensor 16. The vehicle position calculation unit 43 corrects the calculation results of the vehicle's position and attitude based on the detection results of the vehicle's position and attitude by the positioning device 11. The target detection unit 44 detects targets from the surrounding image output from the image conversion unit 42. The target detection unit 44 may detect the positions of feature points of the targets and their image feature amounts. Methods such as SIFT, SURF, ORB, BRIAK, KAZE, and AKAZE can be used to detect the positions of the feature points and calculate the image feature amounts. The target detection unit 44 outputs the detected positions of the feature points and the image feature amounts as target data to the map generation unit 45 and the matching unit 47. The target detection unit 44 also outputs the vehicle's position acquired from the vehicle position calculation unit 43 to the map generation unit 45 and the matching unit 47 in synchronization with the detection of the targets.

[0020] When the map generation unit 45 receives a map generation command from the HMI control unit 40 (i.e., when the target parking position 30 is registered), the map generation unit 45 generates learned target data and stores it in the storage device 21 as map data 46. For example, the map generation unit 45 receives, from the target detection unit 44, the target data and the vehicle's own position on the map coordinate system synchronized with the target data. The map generation unit 45 acquires coordinate data of the target parking position 30 in the map coordinate system and angle data in the fore-and-aft direction 32 of the parking space where the vehicle is to be parked at the target parking position 30. For example, when the host vehicle 1 is located at the target parking position 30, the host position calculated by the host position calculation unit 43 may be acquired as the coordinate data of the target parking position 30, and the angle of the vehicle body in the fore-and-aft direction may be acquired as angle data in the fore-and-aft direction 32. The map generation unit 45 generates relative position data based on the positions of the feature points included in the target object data, the position information of the host vehicle 1 synchronized therewith, and the position data of the target parking position 30. The map generation unit 45 also acquires feature amount data from the target object data. The position data of the target parking position 30 and the angle data of the parking space in the fore-aft direction 32 are used as target parking position coordinate data. The learned target object data including the relative position data, feature amount data, and target parking position coordinate data is stored in the storage device 21 as map data 46.

[0021] When the parking assist control unit 41 receives a control start command from the HMI control unit 40 (i.e., when an operation to start parking assist control is performed), it outputs a parking position calculation command to the matching unit 47. The matching unit 47 receives the target data output from the target detection unit 44 as target data of surrounding targets, and also receives the own position of the vehicle 1 in the map coordinate system in synchronization therewith. The matching unit 47 reads out the learned target data stored in the storage device 21 as map data 46, and determines whether the vehicle 1 is located near the registered target parking position 30 based on the target parking position coordinate data. When the vehicle 1 is located near the registered target parking position 30, the matching unit 47 matches the learned targets stored in the map data 46 with the surrounding targets to associate targets with each other that have the same characteristic points. The matching unit 47 calculates the current relative position of the vehicle 1 with respect to the target parking position 30 based on the relative positional relationship between the surrounding targets and the vehicle 1 and the relative positional relationship between the learned targets associated with the surrounding targets and the target parking position 30. For example, if the surrounding targets are (x i ,y i ) and the surrounding targets (x i ,y i ) are the learned targets associated with each of the mi ,y mi ) (i=1 to N). The matching unit 47 calculates the affine transformation matrix M affine1 Calculate.

[0022]

number

[0023] The abutment 47 is the position (targetx) of the target parking position 30 on the map coordinate system stored in the map data 46. m ,targety m ) is converted into a position (targetx, targety) in the vehicle coordinate system based on the following equation:

[0024]

number

[0025] Furthermore, the abutting portion 47 is formed by an angle targetyaw which indicates the front-rear direction 32 of the parking space of the target parking position 30 on the map coordinate system. m is converted into an angle targetyaw indicating the direction on the vehicle coordinate system based on the following equation:

[0026]

number

[0027] When the target trajectory generating unit 48 receives a driving trajectory calculation command from the parking assist control unit 41, it calculates a target driving trajectory from the current position of the vehicle 1 on the vehicle coordinate system to the target parking position 30, and a target vehicle speed profile. 9 is When the vehicle speed control unit 41 receives a steering control command from the parking assist control unit 41, the vehicle speed control unit 41 controls the steering actuator 19a so that the vehicle 1 travels along the target travel path. 50 When the vehicle speed control command is received from the parking assist control unit 41, the accelerator actuator 19b and the brake actuator 19c are controlled so that the vehicle speed of the host vehicle 1 changes in accordance with the target vehicle speed profile. When the host vehicle 1 reaches the target parking position 30 and the parking assist control is completed, the parking assist control unit 41 activates the parking brake 18 and switches the shift position to the parking range (P range).

[0028] While parking assist control is being performed, the HMI control unit 40 receives an overhead image of the surroundings of the vehicle 1 from the image conversion unit 42 and displays it on the display device of the HMI 13. The HMI control unit 40 also receives information on the coordinates (targetx, targety) of the target parking position 30 and the angle targetyaw in the fore-and-aft direction 32 from the matching unit 47, and displays a target parking frame representing a parking space for parking the vehicle at the target parking position 30 by superimposing it on the overhead image. 4(a) is a schematic diagram of an example of a display area in which an overhead image is displayed on the display device of HMI 13. Reference numeral 60 denotes the display screen of the display device of HMI 13. An overhead image 62 is displayed within a predetermined display area 61 on the display screen 60. For convenience in the following explanation, a reference point Pr in the lower left corner of the display area 61 is taken as the coordinate origin (0,0), and the rightward and upward directions are taken as the positive x-axis and positive y-axis directions, respectively. However, the setting of the coordinate system is not limited to this example and may be changed as appropriate.

[0029] Display area 61 is a pre-defined area within display screen 60. The number of pixels in display area 61 in the x-axis and y-axis directions is Wavm and Havm, respectively. For example, the number of pixels in display area 61 in the x-axis and y-axis directions may be set equal to the number of pixels in the x-axis and y-axis directions of overhead image 62 output from image conversion unit 42 to HMI control unit 40, respectively. In FIG. 4(a), overhead image 62 is drawn slightly smaller than display area 61 to make the boundary of overhead image 62 easier to see, but overhead image 62 may also be displayed filling display area 61 (i.e., the number of pixels in the x-axis and y-axis directions of overhead image 62 displayed in display area 61 may be Wavm and Havm, respectively). The overhead image 62 is an image in which an icon 63 representing the vehicle 1 is superimposed at the center of the overhead image, which represents the range of the longitudinal length DL and the vehicle width length DT centered on the current position of the vehicle 1. The longitudinal length DL and the vehicle width length DT are set appropriately so that the occupant can grasp the sense of distance from the features (road markings such as white lines and road boundaries such as road shoulders) around the vehicle 1 shown in the overhead image 62. In the following description, the display position for displaying the overhead image 62 in the display area 61 is specified as the position where the bottom left corner of the overhead image 62 is displayed (position Pb1 in the example of FIG. 4(a)).

[0030] The HMI control unit 40 calculates the position where the target parking space 64 is superimposed and displayed on the overhead image 62. Specifically, the target parking position Pt in the coordinate system on the display screen 60 is calculated based on the coordinates (targetx, targety) of the target parking position 30 received from the matching unit 47 and the scale at which the overhead image 62 is displayed on the display screen 60. In addition, based on the angle targetyaw and the target parking position Pt received from the matching unit 47, the display position and direction of the target parking space 64 in the coordinate system on the display screen 60 are set. For example, the target parking position Pt is set to the center position of the target parking space 64, and the direction indicated by the angle targetyaw is set to the front-rear direction of the target parking space 64. The size (front-rear length and lateral length) of the target parking space 64 to be displayed on the display screen 60 may be set, for example, according to the scale of the overhead image 62 and the size of the host vehicle 1. For example, the size of the target parking space 64 may be set to be approximately equal to the size of the occupied area of ​​the host vehicle 1 in the overhead image 62. Also, the size of the target parking space 64 may be set slightly larger than the occupied area of ​​the host vehicle 1 to provide a margin, or may be set slightly smaller than the occupied area of ​​the host vehicle 1. Also, the size of the target parking space 64 when the overhead image 62 is displayed to fill the display area 61 (i.e., when the overhead image 62 is displayed with pixel counts Wavm and Havm in the x-axis and y-axis directions, respectively) may be set in advance as an initial value.

[0031] Here, when parking assist control is started in a state where the host vehicle 1 is away from the target parking position 30, the display position of the target parking frame 64 may be set outside the range of the overhead image 62 as shown in Fig. 4(a). In this case, the relative position of the target parking frame 64 with respect to the icon 63 representing the host vehicle 1 cannot be correctly grasped. Therefore, when the bird's-eye view image 62 is displayed at the first display position Pb1 of the display screen 60, if a predetermined range or more of the target parking space 64 can be displayed within the display area 61, the HMI control unit 40 displays the bird's-eye view image 62 at the first display position Pb1 of the display screen 60. For example, when the display position for displaying the bird's-eye view image 62 is set to the first display position Pb1, it is determined whether or not a predetermined range or more of the target parking space 64 can be displayed within the display area 61. In the example of FIG. 4(a), when the bird's-eye view image 62 is displayed to fill the display area 61 (i.e., when the bird's-eye view image 62 having pixel counts Wavm and Havm in the x-axis and y-axis directions, respectively, is displayed, and the first display position Pb1 of the bird's-eye view image 62 is set to the reference point Pr in the lower left corner of the display area 61), it is determined whether or not a predetermined range or more of the target parking space 64 can be displayed within the display area 61. In the following description, the predetermined range of the target parking space 64 may be referred to as a "required display range."

[0032] On the other hand, when the overhead image 62 is displayed at the first display position Pb1 of the display screen 60, if a predetermined range or more of the target parking space 64 cannot be displayed within the display area 61, the overhead image 62 is displayed at a second display position Pb2 of the display screen 60 different from the first display position Pb1 as shown in FIG. 4(b). This makes it possible to display a necessary display range or more of the target parking space 64 within the display area 61. In this case, for example, the portion of the overhead image 62 that is outside the display area 61 due to the change in display position and the portion of the target parking space 64 that does not fit within the display area 61 may be cut out, and the remaining portion may be displayed within the display area 61.

[0033] The HMI control unit 40 determines the offset direction (movement direction) and offset amount (movement distance) of the display position of the overhead image 62 relative to the reference position (i.e., first display position Pb1) so that more than the required display range of the target parking space 64 can be displayed within the display area 61, and calculates the second display position Pb2 by offsetting the first display position Pb1 in the offset direction by the offset amount. See FIG. 4(c). The offset direction of the display position of the overhead image 62 is determined by the position of the target parking position Pt when the overhead image 62 is displayed to fill the entire display area 61. Region R1 is a region where the x coordinate is less than 0 and the y coordinate is greater than Havm, region R2 is a region where the x coordinate is between 0 and Wavm and is greater than Havm, and region R3 is a region where the x coordinate and the y coordinate are greater than Wavm and Havm, respectively. Region R4 is a region where the x coordinate is less than 0 and the y coordinate is between 0 and Havm, region R5 is a region where the x coordinate is greater than Wavm and the y coordinate is between 0 and Havm, region R6 is a region where both the x and y coordinates are less than 0, region R7 is a region where the x coordinate is between 0 and Wavm and the y coordinate is less than 0, and region R8 is a region where the x coordinate is greater than Wavm and the y coordinate is less than 0.

[0034] For example, when the target parking position Pt is in region R1, the offset direction of the display position of the overhead image 62 may be set to a downward and right direction (increasing the x coordinate and decreasing the y coordinate), when the target parking position Pt is in region R2, the offset direction of the display position of the overhead image 62 may be set to a downward direction (decreasing the y coordinate without changing the x coordinate), and when the target parking position Pt is in region R3, the offset direction of the display position of the overhead image 62 may be set to a downward and left direction (decreasing the x coordinate and y coordinate).When the target parking position Pt is in region R4, the offset direction of the display position of the overhead image 62 may be set to a right direction (increasing the x coordinate without changing the y coordinate), and when the target parking position Pt is in region R5, the offset direction of the display position of the overhead image 62 may be set to a left direction (decreasing the x coordinate without changing the y coordinate). When the target parking position Pt is in region R6, the offset direction of the display position of the overhead image 62 may be set to the upper right direction (increasing the x-coordinate and y-coordinate), when the target parking position Pt is in region R7, the offset direction of the display position of the overhead image 62 may be set to the upper right direction (increasing the y-coordinate without changing the x-coordinate), and when the target parking position Pt is in region R8, the offset direction of the display position of the overhead image 62 may be set to the upper left direction (decreasing the x-coordinate and increasing the y-coordinate).

[0035] 4(b) shows an example of setting the offset direction and offset amount of the display position of the overhead image 62 so that an area including the center of the target parking frame 64 (for example, the target parking position Pt) or more can be displayed within the display area 61 as the required display range of the target parking frame 64. In this example, the x-direction offset amount ofx is set to Px, and the y-direction offset amount ofy is set to Py based on the coordinates (-Px, -Py) of the target parking position Pt when the overhead image 62 is displayed to fill the display area 61 in FIG. The required display range of the target parking frame 64 is not limited to the above example, and may be appropriately set so that the target parking frame 64 displayed on the display screen 60 is easily understood by the occupant. For example, the required display range may be a range including a part of at least two sides 64a, 64b of the rectangular target parking frame 64 (FIG. 5(a)), a range including the entire length of at least two sides 64a, 64b of the rectangular target parking frame 64 (FIG. 5(b)), a range including the entire length of at least one short side 64a of the rectangular target parking frame 64 (FIG. 5(c)), or a range including all four sides of the rectangular shape (FIG. 5(d)). For example, the HMI control unit 40 may calculate the coordinates of the four corners of the rectangular target parking space 64 based on the target parking position Pt and the angle targetyaw, and determine the offset direction and offset amount of the display position of the overhead image 62 so that more than the required display range can be displayed within the display area 61.

[0036] (operation) 6 is a flowchart of an example of the parking assistance method of the first embodiment. In step S1, the HMI control unit 40 calculates the target parking position Pt in the coordinate system on the display screen 60. Also, it calculates the display position and direction of the target parking space 64. In step S2, the HMI control unit 40 determines whether or not a predetermined range or more of the target parking space 64 can be displayed within the display area 61 when the overhead image 62 is displayed at the first display position Pb1. If a predetermined range or more of the target parking space 64 cannot be displayed within the display area 61 (step S2: N), the process proceeds to step S3. If a predetermined range or more of the target parking space 64 can be displayed within the display area 61 (step S2: Y), the process proceeds to step S7. In step S3, the HMI control unit 40 calculates the offset amount (ofx, ofy) of the display position of the overhead image 62. In step S4, the HMI control unit 40 calculates the display position of the target parking frame 64 after offset by the offset amount (ofx, ofy).

[0037] In step S5, the HMI control unit 40 displays the overhead image 62 at the second display position Pb2, which is the display position offset by the offset amount (ofx, ofy). In step S6, the HMI control unit 40 displays the target parking space 64 superimposed on the display position offset by the offset amount (ofx, ofy). Then, the processing ends. In step S7, the HMI control unit 40 displays the overhead image 62 at the first display position Pb1. In step S8, the HMI control unit 40 displays the target parking frame 64 superimposed on the position calculated in step S1. Thereafter, the processing ends.

[0038] (Second embodiment) If the display position is offset while maintaining the size of the overhead image 62 (i.e., the scale of the overhead image 62), a part of the overhead image 62 may go outside the display area 61 and become unable to be displayed on the display screen 60. Therefore, in the second embodiment, the overhead image 62 displayed in the display area 61 is reduced in size, as shown in FIG. 7(a). This allows the overhead image 62 to be displayed at the second display position Pb2 so as to include the entire range that can be displayed in the display area 61 when the overhead image 62 is displayed at the first display position Pb1 (i.e., the range of the longitudinal length DL and the transverse length DT centered on the current position of the vehicle 1) as shown in Figure 4(a).

[0039] For example, when the x-direction offset amount ofx is smaller than the y-direction offset amount ofy, the HMI control unit 40 calculates the reduction ratio scale using the following formula. scale=(Havm-ofy) / Havm When the x-direction offset amount ofx is greater than the y-direction offset amount ofy, the HMI control unit 40 calculates the reduction ratio scale using the following formula. scale=(Wavm-ofx) / Wavm The HMI control unit 40 offsets the display positions of the overhead image 62 and the target parking space 64 by an x-direction offset amount ofx and a y-direction offset amount ofy, and calculates an affine transformation matrix M affine2is set by the following formula:

[0040]

number

[0041] The HMI control unit 40 converts the coordinates of the overhead image 62 and the target parking space 64 into an affine transformation matrix M affine2 Specifically, if the coordinates of each pixel of the overhead image 62 before conversion and the coordinates of the boundary of the target parking space 64 are (Px1, Py1), the coordinates of each pixel of the overhead image 62 after offset and reduction and the coordinates of the boundary of the target parking space 64 (Px2, Py2) are calculated by the following equation.

[0042]

number

[0043] 7(a), the HMI control unit 40 may draw grid lines 65 in the background portion of the display area 61 (the area where the overhead image 62 is not displayed). For example, the HMI control unit 40 may display in the display area 61 an image obtained by adding a background image 66 on which a grid is drawn to the overhead image 62, as shown in FIG. 7(b).

[0044] Fig. 8 is a flowchart of an example of the parking assistance method of the second embodiment. The processes of steps S10 and S11 are the same as the processes of steps S1 and S2 in Fig. 6. If the predetermined range or more of the target parking frame 64 cannot be displayed within the display area 61 (step S11: N), the process proceeds to step S12. If the predetermined range or more of the target parking frame 64 can be displayed within the display area 61 (step S11: Y), the process proceeds to step S17. The processing of step S12 is the same as step S3 in Fig. 6. In step S13, the HMI control unit 40 calculates the reduction ratio "scale". In step S14, the HMI control unit 40 calculates the display position of the target parking space 64 when the overhead image 62 is offset by the offset amount (ofx, ofy) and reduced by the reduction ratio "scale".

[0045] In step S15, the HMI control unit 40 displays the overhead image 62 offset by the offset amount (ofx, ofy) and reduced by the reduction rate scale. In step S16, the HMI control unit 40 displays the target parking frame 64 offset by the offset amount (ofx, ofy) and reduced by the reduction rate scale in a superimposed manner. Thereafter, the processing ends. The processing in steps S17 and S18 is the same as the processing in steps S7 and S8 in FIG.

[0046] (Variation) In the first and second embodiments, examples have been described in which targets around the target parking position 30 stored in advance are matched with targets detected around the host vehicle 1 when parking assist control is being executed, thereby detecting the relative position of the target parking position 30 with respect to the current position of the host vehicle 1, but the parking assist method of the present invention is not limited to such a method of determining the target parking position. For example, when parking assist control is being executed, targets indicating a parking space (for example, road markings such as parking frame lines or three-dimensional objects such as wheel chocks) may be detected to detect the relative position of the target parking position 30 with respect to the current position of the host vehicle 1.

[0047] (Effects of the embodiment) (1) A parking assistance method that sets a target parking position where vehicle 1 will park, generates an overhead image of the surroundings of vehicle 1 from captured images obtained by photographing the surroundings of vehicle 1, and displays the overhead image and a guide figure representing an area of ​​a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen that is visible to an occupant of vehicle 1.When the overhead image is displayed at a first display position on the display screen, if more than a predetermined range of the guide figure can be displayed within the display area, the overhead image is displayed at the first display position on the display screen, and when the overhead image is displayed at the first display position on the display screen, if more than the predetermined range of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen that is different from the first display position. This allows the bird's-eye view image of the surroundings of the vehicle 1 and the guide figure beyond a predetermined range to be displayed simultaneously even if the vehicle is far from the target parking position, so that an image that makes it easy to grasp the relative position of the target parking frame with respect to the vehicle can be provided.

[0048] (2) When the overhead image is displayed at the second display position, the overhead image displayed in the display area may be reduced in size, thereby preventing a reduction in the display range of the overhead image after changing the display position of the overhead image. (3) By reducing the size of the overhead image displayed in the display area, an overhead image including the entire range that can be displayed in the display area when the display position of the overhead image is set to the first display position may be displayed in the second display position. This allows the entire range of the overhead image that could be displayed before the change to be displayed even when the display position of the overhead image is changed.

[0049] (4) A background image with a grid drawn on it may be added to the overhead image and displayed in the display area, which makes it easier to grasp the sense of distance to the guide figure displayed outside the overhead image. (5) The predetermined range of the guide figure may be a range that includes a portion of each of at least two sides of the rectangular guide figure. The predetermined range of the guide figure may also be a range that includes the entire length of each of at least two sides of the rectangular guide figure. The predetermined range of the guide figure may also be a range that includes the entire length of at least one short side of the rectangular guide figure. The predetermined range of the guide figure may also be a range that includes all four sides of the rectangular guide figure. The predetermined range of the guide figure may also be a range that includes the center point of the guide figure. This allows the guide figure to be displayed in a way that is easy for the occupant to understand.

[0050] (6) The target parking position may be detected based on captured images obtained by photographing the surroundings of the vehicle 1, a target driving trajectory from the current position of the vehicle 1 to the target parking position may be calculated, and the vehicle 1 may be controlled to drive along the target driving trajectory. This may assist the vehicle 1 in parking.

[0051] All examples and conditional terms described herein are intended for educational purposes to aid the reader in understanding the present invention and the concepts provided by the inventor for the advancement of technology, and should be construed without limitation to the specifically described examples and conditions above, and the configuration of examples herein for illustrating the advantages and disadvantages of the present invention. Although the embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0052] 1...Vehicle, 10...Parking assistance device, 11...Positioning device, 13...Human-machine interface (HMI), 14...Shift switch (shift SW), 15...External sensor, 16...Vehicle sensor, 17...Controller, 18...Parking brake, 19a...Steering actuator, 19b...Accelerator actuator, 19c...Brake actuator, 20...Processor, 21...Storage device, 40...HMI control unit, 41...Parking assistance control unit, 42...Image conversion unit, 43...Self-position calculation unit, 44...Target detection unit, 45...Map generation unit, 46...Map data, 47...Matching unit, 48...Target trajectory generation unit, 4 9… Steering control unit, 50 ...Vehicle speed control unit

Claims

2. A parking assistance method comprising: setting a target parking position where a vehicle is to be parked; generating an overhead image of the surroundings of the vehicle from captured images obtained by capturing images of the surroundings of the vehicle; and displaying the overhead image and a guide figure representing an area of ​​a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen visible to an occupant of the vehicle; When a bird's-eye view image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the bird's-eye view image is displayed at the first display position on the display screen; When an overhead image is displayed at the first display position on the display screen, if the predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen that is different from the first display position; A parking assistance method, comprising: when displaying an overhead image at the second display position, reducing the overhead image displayed in the display area.

3. 3. The parking assistance method according to claim 2, wherein the overhead image displayed in the display area is reduced, so that the overhead image including the entire range that can be displayed in the display area when the display position of the overhead image is set to the first display position is displayed in the second display position.

4. A parking assistance method comprising: setting a target parking position where a vehicle is to be parked; generating an overhead image of the surroundings of the vehicle from captured images obtained by capturing images of the surroundings of the vehicle; and displaying the overhead image and a guide figure representing an area of ​​a predetermined size surrounding the target parking position in a predetermined display area of ​​a display screen visible to an occupant of the vehicle; When a bird's-eye view image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the bird's-eye view image is displayed at the first display position on the display screen; When an overhead image is displayed at the first display position on the display screen, if the predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen that is different from the first display position; A parking assistance method characterized in that a background image on which a grid is drawn is added to the overhead image and displayed in the display area.

5. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range that includes a part of each of at least two sides of the guide figure that is rectangular.

6. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range that includes the total lengths of at least two sides of the rectangular guide figure.

7. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range that includes the entire length of at least one short side of the rectangular guide figure.

8. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range that includes all four sides of the guide figure that is rectangular.

9. 5. The parking assistance method according to claim 2, wherein the predetermined range of the guide figure is a range that includes a center point of the guide figure.

10. 10. The parking assistance method according to claim 2, wherein the target parking position is detected based on an image obtained by photographing the surroundings of the host vehicle, a target driving trajectory from the current position of the host vehicle to the target parking position is calculated, and the host vehicle is controlled to travel along the target driving trajectory.

11. A parking assistance device comprising: a camera that captures images of the surroundings of a vehicle; and a display device having a display screen that can be viewed by an occupant of the vehicle; the parking assistance device sets a target parking position where the vehicle will be parked; generates an overhead image of the surroundings of the vehicle from an image captured by the camera; and displays the overhead image and a guide figure that represents a range of a predetermined size surrounding the target parking position in a predetermined display area of ​​the display screen, a controller that displays an overhead image at a first display position on the display screen when a predetermined range or more of the guide figure can be displayed within the display area, displays an overhead image at the first display position on the display screen, and when a predetermined range or more of the guide figure cannot be displayed within the display area when an overhead image is displayed at the first display position on the display screen, displays an overhead image at a second display position on the display screen that is different from the first display position, and reduces the overhead image displayed in the display area when the overhead image is displayed at the second display position.

12. A parking assistance device comprising: a camera that captures images of the surroundings of a vehicle; and a display device having a display screen that can be viewed by an occupant of the vehicle; the parking assistance device sets a target parking position where the vehicle will be parked; generates an overhead image of the surroundings of the vehicle from an image captured by the camera; and displays the overhead image and a guide figure that represents a range of a predetermined size surrounding the target parking position in a predetermined display area of ​​the display screen, a controller that displays an overhead image at a first display position on the display screen when a predetermined range or more of the guide figure can be displayed within the display area when the overhead image is displayed at a first display position on the display screen, and displays an overhead image at a second display position on the display screen that is different from the first display position when a predetermined range or more of the guide figure cannot be displayed within the display area when the overhead image is displayed at the first display position on the display screen, and adds a background image on which a grid is drawn to the overhead image and displays it in the display area.

Images