Parking assistance method and parking assistance device

Abstract

In this parking assistance method: a target position, which is the position of a target in the surroundings of a host vehicle, is detected from an image obtained by photographing the surroundings of the host vehicle (S2); a first photograph situation, which is the photograph situation when the image was photographed, is detected (S3); a combination of one or more relative positional relationships, which are relationships between a goal parking position and a target detected from a past image obtained by photographing the surroundings of the goal parking position in the past, and a second photograph situation, which is the photograph situation when the past image was photographed, is read from a predetermined storage device in which there are stored one or more combinations of relative positional relationships and second photograph situations (S4); a relative positional relationship stored in combination with a second photograph situation for which the difference with the first photograph situation is not greater than a predetermined difference is selected from among the read relative positional relationships (S5); and the relative positions between the current position of the host vehicle and the goal parking position is calculated on the basis of the selected relative positional relationship and the target position of the stored target in the surroundings of the host vehicle, and parking assistance is carried out to guide the host vehicle to the goal parking position (S7−S9).

Description

[Technical Field]

[0001] The present invention relates to a parking assistance method and a parking assistance device. [Background technology]

[0002] The following Patent Document 1 is known as a parking assistance technology that assists in parking the vehicle at a target parking position. In Patent Document 1, targets are detected from images taken in the past around the target parking position and stored, the relative position of the target parking position with respect to the vehicle is calculated based on the stored target positions and the target positions detected from images taken around the vehicle during automatic parking, and the vehicle is automatically moved to the target parking position based on the calculated relative position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-138664 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the target detected from the image changes depending on the image capturing conditions, so if the image capturing conditions at the time of parking assistance differ from the past image capturing conditions under which the stored target was detected, the accuracy of the parking assistance may be reduced. The present invention aims to suppress a decrease in the accuracy of parking assistance caused by changes in the shooting environment of the captured image in a parking assistance system that assists in parking a vehicle into a target parking position based on a target position detected from an captured image. [Means for solving the problem]

[0005] In one aspect of the parking assistance method of the present invention, a first target position, which is the position of a target around the host vehicle, is detected from an image obtained by photographing the surroundings of the host vehicle, a first photographing situation, which is the photographing situation when the image was photographed, is detected, and for a second target position, which is the target position of a target detected from a past image, which is an image obtained by photographing the surroundings of the target parking position in the past, one or more combinations of the relative position relationship between the second target position and the target parking position and the second photographing situation, which is the photographing situation when the past image was photographed, are stored from a predetermined storage device in which one or more combinations of the relative position relationship between the second target position and the target parking position and the second photographing situation, which is the photographing situation when the past image was photographed, are read out, from the read relative position relationships, a relative position relationship stored in combination with the second photographing situation, whose difference from the first photographing situation is equal to or less than a predetermined difference, and a relative position between the current position of the host vehicle and the target parking position is calculated based on the selected relative position relationship and the target position of the target stored around the host vehicle, and parking assistance is performed for the host vehicle to the target parking position. [Effects of the Invention]

[0006] According to the present invention, in a parking assistance system that assists in parking a vehicle into a target parking position based on a target object position detected from an image, it is possible to suppress a decrease in the accuracy of the parking assistance system due to changes in the shooting environment of the image. The objects and advantages of the invention will be realized and attained by means of the elements and combinations set forth in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a parking assistance device according to an embodiment; [Figure 2] FIG. 2 is a schematic diagram of target positions around a target parking position. [Figure 3] FIG. 2 is an explanatory diagram of an example of a parking assistance method. [Figure 4] FIG. 3 is an explanatory diagram of a first example of parking assistance data. [Figure 5] FIG. 10 is an explanatory diagram of a second example of parking assistance data. [Figure 6] 2 is a block diagram illustrating an example of a functional configuration of a controller in FIG. 1. FIG. [Figure 7] 1 is a flowchart illustrating an example of a parking assistance method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] (composition) 1 is a diagram showing an example of a schematic configuration of a parking assistance device according to an embodiment. A host vehicle 1 is equipped with a parking assistance device 10 that assists the host vehicle 1 in parking at a target parking position. In parking assistance by the parking assistance device 10, a target driving trajectory from the current position of the host vehicle 1 to the target parking position is calculated, and assistance is provided to the host vehicle 1 to drive along the target driving trajectory. Parking assistance by the parking assistance device 10 includes various forms of assistance in driving the host vehicle 1 along a target driving trajectory. For example, parking of the host vehicle 1 may be assisted by performing automatic driving that controls the host vehicle 1 to drive to a target parking position along the target driving trajectory of the host vehicle 1. Note that automatic driving that controls the host vehicle 1 to drive to a target parking position along the target driving trajectory of the host vehicle 1 refers to control that automatically controls all or part of the steering angle, driving force, and braking force of the host vehicle to drive all or part of the host vehicle 1 along the target driving trajectory, thereby assisting the occupant in parking. For example, parking of the vehicle 1 may be assisted by displaying the target driving trajectory and the current position of the vehicle 1 on a display device that is visible to the occupants of the vehicle 1, making it easier for the occupants of the vehicle 1 to drive the vehicle so that it travels along the target driving trajectory.

[0009] The parking assistance device 10 includes a positioning device 11, an object sensor 12, a vehicle sensor 13, a communication device 14, a human-machine interface 15, an actuator 16, and a controller 17. In the drawings, the human-machine interface is abbreviated as "HMI." The positioning device 11 measures the current position of the vehicle 1. The positioning device 11 may include, for example, a Global Positioning System (GNSS) receiver. The GNSS receiver is, for example, a Global Positioning System (GPS) receiver, and receives radio waves from multiple navigation satellites to measure the current position of the vehicle 1. The object sensor 12 detects objects within a predetermined distance range from the host vehicle 1 (for example, the detection area of the object sensor 12). The object sensor 12 detects the surrounding environment of the host vehicle 1, such as the relative position of the host vehicle 1 and the object present around the host vehicle 1, the distance between the host vehicle 1 and the object, and the direction in which the object is present. The object sensor 12 may include, for example, a camera that captures the surrounding environment of the host vehicle 1. The camera may be, for example, an around-view monitor camera that captures the surroundings of the host vehicle 1 and generates captured images that are converted into a bird's-eye view (around-view monitor image). The object sensor 12 may include a ranging device such as a laser range finder (LRF), radar, or a laser radar of LiDAR (Light Detection and Ranging).

[0010] Vehicle sensor 13 detects various information (vehicle information) obtained from host vehicle 1. Vehicle sensor 13 may include, for example, a vehicle speed sensor that detects the traveling speed (vehicle speed) of host vehicle 1, a wheel speed sensor that detects the rotational speed of each tire equipped on host vehicle 1, a three-axis acceleration sensor (G sensor) that detects the acceleration (including deceleration) in three axial directions of host vehicle 1, a steering angle sensor that detects the steering angle of the steering wheel, a turning angle sensor that detects the turning angle of the steered wheels, a gyro sensor that detects the angular velocity generated in host vehicle 1, and a yaw rate sensor that detects the yaw rate. The communication device 14 performs wireless communication with a communication device outside the vehicle 1. The communication method used by the communication device 14 may be, for example, wireless communication via a public mobile communication network, vehicle-to-vehicle communication, road-to-vehicle communication, or satellite communication.

[0011] The human-machine interface 15 is an interface device that exchanges information between the parking assistance device 10 and the occupant. The human-machine interface 15 includes a display device that can be seen by the occupant of the vehicle 1 (for example, a display screen of a navigation system or a display device provided near the meter in front of the driver's seat), and a speaker or buzzer that outputs warning sounds, notification sounds, and audio information. The human-machine interface 15 also includes an operator that accepts operational inputs from the occupant to the parking assistance device 10. For example, the operator may be a button, a switch, a lever, a dial, a keyboard, a touch panel, or the like. The actuator 16 includes a steering actuator, an accelerator actuator, and a brake control actuator. The steering actuator controls the steering angle of the steering mechanism of the host vehicle 1 in response to a control signal from the controller 17. The accelerator actuator controls the accelerator opening of the drive device, which is the engine or drive motor, in response to a control signal from the controller 17. The brake actuator operates the braking device in response to a control signal from the controller 17.

[0012] The controller 17 is an electronic control unit (ECU) that performs parking assistance control of the host vehicle 1. The controller 17 includes a processor 18 and a storage device 19. and other peripheral components. The processor 18 may be, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). The storage device 19 may include a semiconductor storage device, a magnetic storage device, an optical storage device, etc. The storage device 19 may include a register, a cache memory, a ROM (Read Only Memory) used as a main storage device, etc. The functions of the controller 17, which will be described below, are realized by, for example, the processor 18 executing a computer program stored in the storage device 19. The controller 17 may be formed of dedicated hardware for executing each information processing described below. For example, the controller 17 may include a functional logic circuit set in a general-purpose semiconductor integrated circuit. For example, the controller 17 may be formed of a processor such as a field-programmable gate array (FPGA). It has a programmable logic device (PLD), etc. That's fine.

[0013] Next, an example of parking assist control by the parking assist device 10 will be described. See FIG. 2. When starting to use parking assistance by the parking assist device 10, first, the relative positional relationship between the target parking position 2 and the target object position of a target object existing near the target parking position 2 is stored in the storage device 19. Here, the target parking position 2 is the target position at which the host vehicle 1 is to be parked. The target object is a feature that serves as a landmark for identifying the current position of the host vehicle 1. The target object may be, for example, a road marking (lane boundary line 3a, stop line 3b, road sign, etc.), a road boundary (curbstones 3c to 3e, guardrail, etc.), or an obstacle (house 3f, fence 3g, object 3h, etc.).

[0014] When the relative positional relationship between the target object position and the target parking position 2 is stored in the storage device 19 for the first time, the operation mode of the parking assistance device 10 is set to the "target object learning mode." Then, the host vehicle 1 is parked at the target parking position 2 by manual driving. When the host vehicle 1 is parked at the target parking position 2 by manual driving, the parking assistance device 10 may automatically set the operation mode to the "target object learning mode." While parking the vehicle 1 at the target parking position 2 by manual driving, the object sensor 12 detects the target positions of the objects around the vehicle 1. The object sensor 12 is a sensor that detects the target positions present in a detection area within a predetermined detection distance range from the object sensor 12. For example, the parking assistance device 10 may detect the edge and corner portions of targets such as road markings (lane boundary lines 3a and stop lines 3b in the example of Figure 2), road boundaries (grounding points of curbs 3c to 3e in the example of Figure 2), and obstacles (grounding points of house 3f, fence 3g, and object 3h in the example of Figure 2) on the captured image obtained by the camera of the object sensor 12 as feature points, and may regard the positions of the feature points as the target positions. Furthermore, the parking assistance device 10 calculates the feature amounts (e.g., the shading and attributes of the feature points) of the detected feature points. For detecting feature points and calculating feature amounts, methods such as SIFT, SURF, ORB, BRIAK, KAZE, and AKAZE can be used. Note that detection of the feature amounts of the feature points is not essential; it is sufficient to at least detect the positions of the feature points.

[0015] Next, the parking assistance device 10 determines the relative positional relationship between the target position (position of the feature point) detected by the object sensor 12 and the target parking position 2. For example, the parking assistance device 10 calculates the relative position of the feature point with respect to the host vehicle 1 based on the position of the feature point on the image and camera information related to the mounting state of the camera on the host vehicle 1 (mounting position, optical axis angle, and angle of view). Next, the parking assistance device 10 estimates the current position of the host vehicle 1 in a fixed coordinate system at the time the target position is detected by the object sensor 12, and calculates the target position in the fixed coordinate system based on the estimated current position and the relative position of the target with respect to the host vehicle 1. Here, the fixed coordinate system is a coordinate system (e.g., a map coordinate system) with a specific point as the coordinate origin. The current position of the vehicle 1 in the fixed coordinate system may be estimated by, for example, the positioning device 11, odometry, or dead reckoning. The current position of the vehicle 1 in the fixed coordinate system may also be estimated by map mapping of the target position detected by the object sensor 12 with known target positions or high-precision map information. Next, the parking assistance device 10 identifies the target parking position 2 in a fixed coordinate system. For example, the position of the host vehicle 1 when the host vehicle 1 is located at the target parking position 2 may be detected as the target parking position 2. By determining the positions of the target position and the target parking position 2 in the fixed coordinate system, the relative positional relationship between the target position and the target parking position 2 is determined.

[0016] The parking assistance device 10 stores the relative positional relationship between the target position and the target parking position 2 in the storage device 19. For example, the target position in a fixed coordinate system and the position of the target parking position 2 may be stored in the storage device 19. Alternatively, the target position in a relative coordinate system with the target parking position 2 as the coordinate origin may be calculated and stored in the storage device 19. In the following explanation, an example will be described in which the target position in the fixed coordinate system and the position of the target parking position 2 are stored in the storage device 19. In the following description, the target stored in the storage device 19 may be referred to as a “learned target.” The circle plots in FIG.

[0017] Next, a parking assistance method using the parking assistance device 10 will be described with reference to Fig. 3. To use parking assistance, the operation mode of the parking assistance device 10 is set to "parking assistance mode." In the parking assistance mode, the parking assistance device 10 reads out the target position and the position of the target parking position 2 stored in the memory device 19. The circle plots in Fig. 3 represent the target positions of the learned targets read out from the memory device 19 in the parking assistance mode. The parking assistance device 10 uses the object sensor 12 to detect the relative positions of the targets around the vehicle 1 relative to the vehicle 1 as the target positions of the targets around the vehicle 1. The method for detecting the target positions is the same as the detection method in the target learning mode. In the parking assist mode, the target positions detected by the object sensor 12 are shown as triangular plots. In the example of Figure 3, the lane boundary line 3a, the curbs 3c and 3e, and the corners of the fence 3g are detected as target positions. In the parking assistance mode, the parking assistance device 10 matches each target position (triangular plot) detected by the object sensor 12 with the target position (circle plot) of the learned target read from the storage device 19, thereby associating identical target positions. For example, the parking assistance device 10 may determine that target positions having the same or similar feature amounts are the same target position. Alternatively, regardless of the feature amount, the parking assistance device 10 may match the target positions (triangular plot) detected by the object sensor 12 with the target positions (circle plot) of the learned target read from the storage device 19 by comparing the relative positional relationship between the target positions (triangular plot) detected by the object sensor 12 with the relative positional relationship between the target positions (circle plot) of the learned target read from the storage device 19. Alternatively, the target positions (triangular plot) detected by the object sensor 12 may be associated with the target positions (circle plot) of the learned target read from the storage device 19 using both the feature amount and the relative positional relationship of the feature points described above. In the example of FIG. 3, lane boundary lines 3a, curbs 3c and 3e, and corners of a fence 3g are associated with each other.

[0018] The parking assistance device 10 calculates the relative position between the target parking position 2 and the current position of the vehicle 1 based on the relative positional relationship between the target position (triangular plot) detected during the parking assistance mode and the vehicle 1, and the relative positional relationship between the target position (circular plot) of the learned target associated with these feature points (triangular plot) and the target parking position 2. For example, the parking assistance device 10 may calculate the position of the target parking position 2 on a relative coordinate system (hereinafter referred to as the "vehicle coordinate system") based on the current position of the host vehicle 1. Alternatively, the current position of the host vehicle 1 on the fixed coordinate system may be calculated based on the relative positional relationship between each target position (triangle plot) detected in the parking assistance mode and the host vehicle 1, and the target position (circle plot) of the learned target on the fixed coordinate system. By determining the positions of the host vehicle 1 and the target parking position 2 in the fixed coordinate system, the relative position of the target parking position 2 with respect to the current position of the host vehicle 1 is determined. The parking assistance device 10 calculates a target driving trajectory from the current position of the vehicle 1 to the target parking position 2 based on the relative position of the target parking position 2 with respect to the current position of the vehicle 1. The parking assistance device 10 performs automatic driving to control the vehicle 1 so that the vehicle 1 travels along the calculated target driving trajectory to the target parking position.

[0019] However, when detecting feature points of a target from an image in this way, the feature points detected from the image change depending on the conditions under which the image was taken. For example, the feature points detected from the image change depending on the illuminance (amount of light) and the direction of the light when the image was taken. Also, feature points detected from the image (for example, feature points detected from hedges or lawns) change depending on the difference in vegetation between seasons. If the shooting conditions at the time when the target position of a learned target was detected in the past differ significantly from the shooting conditions at the time when parking assistance is used, the number of target positions (circle plots) of learned targets that can be matched to the target positions (triangular plots) detected at the time when parking assistance is used will decrease, which may reduce the detection accuracy of the relative position of the target parking position 2 to the current position of the vehicle 1 and result in a decrease in the accuracy of parking assistance.

[0020] Therefore, when assisting the vehicle 1 in parking at a target parking position, the parking assistance device 10 of the embodiment detects the target position around the vehicle 1 from an image obtained by photographing the surroundings of the vehicle 1, and also detects the photographing conditions when this image was photographed (sometimes referred to as the "first photographing conditions" in the following description). In addition, one or more combinations of the shooting conditions (hereinafter referred to as "second shooting conditions") when an image in which a learned target was detected in the past (hereinafter referred to as "past image") was taken, and the relative positional relationship between the learned target detected from the past image and the target parking position 2 are stored in the memory device 19 as parking assistance data 20.

[0021] Fig. 4 is an explanatory diagram of a first example of the parking assistance data 20. The parking assistance data 20 in Fig. 4 includes j sets of parking assistance data 20a1 to 20aj. The parking assistance data 20a1 to 20aj are combinations of a plurality of different second photographing situations and target positions of learned targets detected from images photographed in these second photographing situations. For example, each of the parking assistance data 20a1 to 20aj includes information on the date, time, and weather at the time of shooting as the second shooting situation, and also includes image feature information of the feature points of the learned targets (positions and feature amounts of the feature points in a fixed coordinate system) as the target positions of the learned targets. The parking assistance data 20 also includes position information of the target parking position 2 on a fixed coordinate system.

[0022] The parking assistance device 10 selects, from among these parking assistance data 20a1 to 20aj, the target position of the learned target combined with a second shooting situation whose difference from the first shooting situation is less than a predetermined difference, and calculates the relative position of the target parking position 2 with respect to the current position of the vehicle 1 based on the relative positional relationship between the selected learned target and the target parking position 2 and the target positions of the targets detected around the vehicle 1. In this way, by selecting learned targets detected from images taken in a second shooting situation whose difference from the first shooting situation is less than a predetermined difference, it becomes possible to select and use learned targets detected from images taken in a shooting situation that is close (similar) to the first shooting situation at the time of providing parking assistance to the target parking position of the vehicle 1. As a result, it is possible to prevent a decrease in the number of learned target positions (circle plots) that can be associated with the detected target positions (triangular plots) when performing parking assistance, thereby preventing a decrease in the accuracy of parking assistance.

[0023] The parking assistance data 20 may be stored in the storage device 19 for a plurality of different target parking positions. FIG. 5 shows the first parking assistance data 20. 25 includes j sets of parking assistance data 20a1-20aj, k sets of parking assistance data 20b1-20bk, m sets of parking assistance data 20c1-20cn, and n sets of parking assistance data 20d1-20dn for different target parking positions, points A to D. Parking assistance data 20a1-20aj are combinations of multiple different second photographing situations and the target positions of learned objects detected from images captured around the target parking position at point A in these second photographing situations. Parking assistance data 20b1-20bk are combinations of multiple different second photographing situations and the target positions of learned objects detected from images captured around the target parking position at point B in these second photographing situations. Parking assistance data 20c1-20cm are combinations of multiple different second photographing situations and the target positions of learned objects detected from images captured around the target parking position at point C in these second photographing situations. Parking assistance data 20d1-20dn are combinations of multiple different second photographing situations and the target positions of learned objects detected from images captured around the target parking position at point D in these second photographing situations.

[0024] In the parking assistance mode, the parking assistance device 10 selects and reads out the parking assistance data 20 stored for the target parking position where the vehicle 1 is to be parked from among the plurality of points A to D. For example, the parking assistance device 10 may read out parking assistance data 20 corresponding to the current position of the vehicle 1 measured by the positioning device 11. For example, the parking assistance device 10 may read out parking assistance data stored for a target parking position that is closest to the current position of the vehicle 1. Furthermore, attribute data of the parking assistance data 20 (e.g., "parking lot at home" or "parking lot at work") may be stored in the storage device 19, and a passenger (e.g., the driver) may select parking assistance data based on the attribute data. Instead of storing the parking assistance data 20 in the storage device 19, the parking assistance data 20 may be stored in an external server device, and the parking assistance data 20 may be transmitted and received via the communication device .

[0025] The functional configuration of the controller 17 will be described in more detail below with reference to Fig. 6. The controller 17 functions as an image conversion unit 40, a self-position calculation unit 41, a shooting situation detection unit 42, a feature point detection unit 43, an assistance data generation unit 44, an assistance data selection unit 45, a relative position estimation unit 46, a target trajectory generation unit 47, a steering control unit 48, a vehicle speed control unit 49, and an assistance image generation unit 50. The image conversion unit 40 converts the image captured by the camera of the object sensor 12 into a bird's-eye view image (around view monitor image) seen from a virtual viewpoint directly above the vehicle 1 as shown in Figures 2 and 3. Hereinafter, the bird's-eye view image converted by the image conversion unit 40 may be referred to as a "surrounding image."

[0026] The self-position calculation unit 41 calculates the current position of the vehicle 1 on a fixed coordinate system by dead reckoning or the like based on the vehicle information output from the vehicle sensor 13. The self-position calculation unit 41 may correct the calculated current position by map mapping or the like between the target position detected by the object sensor 12 and a known target position or high-precision map information. The feature point detection unit 43 detects feature points of targets around the vehicle 1 from the surrounding image output from the image conversion unit 40, and calculates feature amounts of the feature points. The feature point detection unit 43 outputs the detected feature points and their feature amounts, together with the current position of the vehicle 1 received from the self-position calculation unit 41, to the assistance data generation unit 44 and the relative position estimation unit 46.

[0027] The shooting condition detection unit 42 detects the shooting condition of the camera of the object sensor 12 at the time of shooting the image in which the feature point detection unit 43 detected the feature points. The shooting condition detection unit 42 may include a clock that detects the time or date and time at the time of shooting as the shooting condition. The shooting condition detection unit 42 may also obtain weather information at the current position of the vehicle 1 as the shooting condition. The shooting condition detection unit 42 may also obtain weather information from an external device via the communication device 14. Furthermore, the photographing situation detection unit 42 may detect, as the photographing situation, the brightness (illuminance) at the current position of the vehicle 1 using the camera of the object sensor 12 or other sensors. Furthermore, the photographing situation detection unit 42 may calculate the position of the sun at the photographing position based on the date and time at the time of photographing, and use this as the photographing situation. The shooting situation detection unit 42 outputs information about the detected shooting situation to the assistance data selection unit 45 and the assistance data generation unit 44. The shooting situation output to the assistance data selection unit 45 is used as the first shooting situation. The shooting situation output to the assistance data generation unit 44 is used as the second shooting situation.

[0028] The assistance data generation unit 44 calculates the positions of the feature points on a fixed coordinate system based on the feature points output from the feature point detection unit 43 and the current position of the vehicle 1. The assistance data generation unit 44 combines image feature information including the calculated positions of the feature points and feature amount information with the second shooting situation output from the shooting situation detection unit 42, and stores the combined data in the storage device 19 as parking assistance data 20. When the current position of the vehicle 1 reaches the target parking position 2, the assistance data generation unit 44 receives the current position of the vehicle 1 on the fixed coordinate system from the positioning device 11 or the self-position calculation unit 41 as the target parking position 2 and stores it in the storage device 19 as parking assistance data 20. The assistance data generating unit 44 may generate the parking assistance data 20 from the feature points detected while manually parking the vehicle 1 at the target parking position 2 in the target object learning mode, and may also generate the parking assistance data 20 from the feature points detected while subsequently parking the vehicle 1 at the target parking position 2 in the parking assistance mode. This makes it easier to generate the parking assistance data 20 under different shooting conditions.

[0029] Next, when providing parking assistance to the target parking position of the vehicle 1 in the parking assistance mode, the assistance data selection unit 45 acquires the current position of the vehicle 1 from the vehicle position calculation unit 41. Then, of the parking assistance data 20 stored for the target parking positions of a plurality of points (for example, points A to D in FIG. 5), the parking assistance data 20 corresponding to the current position is read from the storage device 19. Here, a case where the parking assistance data 20a1 to 20aj for point A are read will be described.

[0030] The assistance data selection unit 45 compares the second shooting situation included in the parking assistance data 20a1 to 20aj for point A with the first shooting situation output from the shooting situation detection unit 42, and determines the difference between the first shooting situation and the second shooting situation. For example, the assistance data selection unit 45 may calculate the difference between variables (for example, shooting time, brightness, sun position) indicating the first and second shooting situations as the difference between the first and second shooting situations. Furthermore, when using weather (for example, sunny, cloudy, rainy, snowy) as variables indicating the first and second shooting situations, it may assign a corresponding value corresponding to each weather (for example, a value of "0" for sunny, a value of "0.4" for cloudy, a value of "1" for rainy, and a value of "1.2" for snowy) and calculate the difference between the corresponding values as the difference between the first and second shooting situations.

[0031] Alternatively, a function, a map, or a lookup table may be used to derive corresponding values for the variables indicating the first and second shooting situations, and the difference between the corresponding values may be calculated as the difference between the first and second shooting situations. For example, a corresponding value corresponding to the shooting date and time in the first shooting situation and a corresponding value corresponding to the shooting date and time in the second shooting situation may be derived, and the difference between the corresponding values may be calculated as the difference between the first and second shooting situations. As the corresponding value corresponding to the shooting date and time, for example, binary information that distinguishes whether the shooting time was daytime or nighttime for each shooting date may be derived, or an estimated value of brightness at the shooting date and time may be derived. A corresponding value for a combination of shooting date and time and weather may be derived using a function, map, or lookup table, and the difference between the corresponding values may be calculated as the difference between the first and second shooting conditions. Alternatively, for example, a corresponding value for the shooting date and time and a corresponding value for the weather may be calculated separately, and the difference between the weighted sums of these values may be calculated as the difference between the first and second shooting conditions.

[0032] The assistance data selection unit 45 selects, from the parking assistance data 20a1 to 20aj for point A, image feature information of the feature points of the learned target combined with a second shooting situation whose difference from the first shooting situation is equal to or less than a predetermined difference. The difference between the first shooting situation and the second shooting situation being less than a predetermined difference may mean, for example, that the difference between the first shooting situation and the second shooting situation is less than a predetermined value, or that the difference between the first shooting situation and the second shooting situation is the smallest. In addition, when the differences between the first and second shooting conditions are both greater than a predetermined value, the assistance data selection unit 45 may select image feature information combined with the second shooting condition that has the smallest difference from the first shooting condition. Furthermore, if there are multiple parking assistance data among the parking assistance data 20a1 to 20aj for point A, in which the difference between the first shooting situation and the second shooting situation is less than a predetermined difference, the assistance data selection unit 45 may select the parking assistance data that includes the image feature information of the most feature points from among these parking assistance data. The assistance data selection unit 45 outputs the selected image feature information and information on the target parking position 2 corresponding to the current position of the vehicle 1 to the relative position estimation unit 46.

[0033] The relative position estimation unit 46 matches the target position of the learned target indicated by the image feature information output from the assistance data selection unit 45 (circle plot in Figure 3) with the target position indicated by the feature points output from the feature point detection unit 43 during the parking assistance mode (triangle plot in Figure 3), and associates the feature point information detected for the same target with each other. The relative position estimation unit 46 estimates the relative position of the target parking position 2 with respect to the current position of the vehicle 1 based on the relative position relationship between the target position (triangular plot) detected in the parking assistance mode and the vehicle 1, and the relative position relationship between the feature point information (circular plot) of the learned target associated with these feature points (triangular plot) and the target parking position 2.

[0034] For example, in the parking assistance mode, the target position detected is (x i ,y i ) and the target position (x i ,y i ) are the target positions of the learned targets associated with each of the mi ,y mi ) (i=1 to N). For example, the relative position estimation unit 46 calculates the affine transformation matrix M affine may be calculated.

number

[0035] Using the weighted least squares method, the column vector (a1, a2, a3, a4) is calculated as follows: T may be calculated.

number

[0036] The relative position estimation unit 46 calculates the position of the target parking position 2 on the fixed coordinate system (targetx m ,targety m ) is converted into a position (targetx, targety) in the vehicle coordinate system.

number

[0037] The target trajectory generating unit 47 generates a target driving trajectory from the current position of the vehicle 1 on the vehicle coordinate system (ie, the coordinate origin) to the position (targetx, targety) of the target parking position 2 on the vehicle coordinate system. A well-known method already used in commonly known automatic parking devices can be applied to calculate the target driving trajectory from the current position of the vehicle 1 to the target parking position 2, but as an example, the target driving trajectory can be calculated by connecting a clothoid curve from the current position of the vehicle 1 to the target parking position 2. If the target driving trajectory includes a turning point, the target driving trajectory can be calculated by connecting a clothoid curve from the current position of the vehicle to the target parking position 2 via the turning point.

[0038] Furthermore, the target trajectory generation unit 47 calculates a target vehicle speed profile that sets the travel speed at each position on the target driving trajectory from the current position of the host vehicle to the target parking position 2. For example, the target vehicle speed profile can be calculated based on a predetermined set speed, such that the host vehicle accelerates from the current position of the host vehicle 1 to the set speed and then stops at the target parking position 2. If the target driving trajectory includes a turning point, the vehicle speed profile may be calculated such that the host vehicle decelerates before the turning point and stops at the turning point, accelerates from the turning point to the set speed, decelerates before the target parking position 2, and stops at the target parking position 2. The set speed when calculating the speed profile may be set based on the curvature of the calculated target driving trajectory so that the larger the curvature, the lower the speed.

[0039] The steering control unit 48 controls the steering actuator of the actuator 16 so that the host vehicle 1 travels along the target travel path. Furthermore, the vehicle speed control unit 49 controls the accelerator actuator and the brake actuator of the actuator 16 so that the vehicle speed of the host vehicle 1 changes in accordance with the travel speed plan calculated by the target trajectory generation unit 47. In this way, the host vehicle 1 is controlled so as to travel along the target travel trajectory. The support image generation unit 50 generates a parking support image that shows the target driving trajectory calculated by the target trajectory generation unit 47 and the current position of the vehicle 1. For example, the parking support image may be an image in which the target driving trajectory and the current position of the vehicle 1 are superimposed on a bird's-eye view or an overhead view of the area around the vehicle 1. The support image generation unit 50 displays the parking support image on the display device of the human-machine interface 15.

[0040] (operation) FIG. 7 is a flowchart of an example of the operation of the parking assistance device 10 in the parking assistance mode. In step S1, the positioning device 11 measures the current position of the vehicle 1 on a fixed coordinate system. In step S2, the feature point detection unit 43 of the controller 17 detects the target positions (feature points) of the targets around the host vehicle 1. In step S3, the photographing situation detection unit 42 detects the photographing situation around the vehicle 1 photographed by the camera of the object sensor 12 as a first photographing situation.

[0041] In step S 4 , the assistance data selection unit 45 reads out the parking assistance data 20 corresponding to the current position of the host vehicle 1 from the storage device 19 . In step S5, the assistance data selection unit 45 selects, from the parking assistance data 20 read out in step S4, parking assistance data 20 for which the difference between the first and second photographing conditions detected in step S3 is equal to or less than a predetermined difference. In step S6, the assistance data selection unit 45 selects the data including the most feature points from the parking assistance data 20 selected in step S5.

[0042] In step S7, the relative position estimation unit 46 estimates the relative position of the target parking position 2 with respect to the vehicle 1 based on the target position detected in step S2 and the parking assistance data 20 selected in step S6. In step S8, the target trajectory generation unit 47 generates a target driving trajectory for the host vehicle 1 to drive from the current position of the host vehicle 1 to the target parking position 2 and a target vehicle speed profile based on the relative position of the target parking position 2 with respect to the host vehicle 1. In step S9, the steering control unit 48 controls the steering angle so that the host vehicle 1 travels along the target travel trajectory. The vehicle speed control unit 49 controls the travel speed of the host vehicle 1 in accordance with the target vehicle speed profile. When the host vehicle 1 reaches the target parking position 2, the process ends.

[0043] (Effects of the embodiment) (1) The controller 17 detects a first target position, which is the position of a target around the vehicle 1, from an image obtained by photographing the area around the vehicle 1, and detects a first photographing situation, which is the photographing situation when the image was photographed. For a second target position, which is the target position of a target detected from a past image, which is an image obtained by photographing the area around the target parking position in the past, the controller 17 reads out one or more combinations of the relative position relationship and the second photographing situation from the storage device 19, which stores one or more combinations of the relative position relationship between the second target position and the target parking position and the second photographing situation, which is the photographing situation when the past image was photographed. From the read-out relative position relationships, the controller 17 selects a relative position relationship stored in combination with a second photographing situation whose difference from the first photographing situation is equal to or less than a predetermined difference. Based on the selected relative position relationship and the target position of the target stored around the vehicle 1, the controller 17 calculates the relative position between the current position of the vehicle 1 and the target parking position, and performs parking assistance for the vehicle 1 to the target parking position.

[0044] As a result, targets are detected in advance from images taken around the target parking position and the relative positional relationship between the target parking position and the target position is stored, and then at the time of subsequent parking assistance, the target position is detected from images taken around the vehicle 1 and the detected target position is associated with the pre-stored target position to calculate the relative position between the current position of the vehicle 1 and the target parking position, and the target position detected from the image taken in a second shooting situation close to (similar to) the first shooting situation, which is the shooting situation at the time of parking assistance, is associated with the target position detected at the time of parking assistance, thereby calculating the relative position between the current position of the vehicle 1 and the target parking position. As a result, it is possible to prevent a decrease in target positions that can be associated due to changes in the shooting environment, and therefore it is possible to prevent a decrease in the accuracy of parking assistance.

[0045] (2) The controller 17 may select, from the relative positional relationships stored in the storage device, the relative positional relationship stored in combination with the second photographing situation that has the smallest difference from the first photographing situation, as the selected relative positional relationship. This allows the target position detected from the image captured in the second shooting situation that is closest (most similar) to the first shooting situation to be matched with the target position detected at the time of parking assistance, thereby calculating the relative position between the current position of the vehicle 1 and the target parking position. (3) The first and second photographing conditions may include at least one of the brightness or weather at the photographing location, the photographing time, the photographing date and time, and the position of the sun at the time of photographing. This makes it possible to determine the difference between the first and second photographing situations based on factors that affect the detection of the target position.

[0046] (4) The controller 17 may calculate a target driving trajectory from the current position of the vehicle 1 to the target parking position based on the relative positional relationship between the current position of the vehicle 1 and the target parking position, and control the vehicle 1 to drive along the target driving trajectory from the current position of the vehicle 1 to the target parking position. This makes it possible to realize parking assistance that controls the host vehicle 1 so that the host vehicle 1 travels to the target parking position 2. (5) The controller 17 may calculate a target driving trajectory from the current position of the vehicle 1 to the target parking position based on the relative positional relationship between the current position of the vehicle 1 and the target parking position, and display the target driving trajectory and the position of the vehicle 1 on a display device that is visible to the occupants. This allows the occupant to visually recognize the target driving trajectory along which the vehicle 1 will travel to the target parking position 2.

[0047] All examples and conditional terms described herein are intended for educational purposes to aid the reader in understanding the present invention and the concepts provided by the inventor for the advancement of technology, and should be construed without limitation to the specifically described examples and conditions above, and the configuration of examples herein for illustrating the advantages and disadvantages of the present invention. Although the embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0048] 1...own vehicle, 10...parking assistance device, 11...positioning device, 12...object sensor, 13...vehicle sensor, 14...communication device, 15...human-machine interface, 16...actuator, 17...controller, 18...processor, 19...storage device, 40...image conversion unit, 41...self-position calculation unit, 42...shooting situation detection unit, 43...feature point detection unit, 44...assistance data generation unit, 45...assistance data selection unit, 46...relative position estimation unit, 47...target trajectory generation unit, 48...steering control unit, 49...vehicle speed control unit, 50...assistance image generation unit

Claims

1. A controller comprising: Detecting a first target position, which is a position of a target around the host vehicle, from an image obtained by photographing the surroundings of the host vehicle; Detecting a first photographing situation that is a photographing situation when the image was photographed; With respect to a second target position, which is the target position of a target detected from a past image, which is an image obtained by photographing the periphery of the target parking position in the past, one or more combinations of a relative positional relationship between the second target position and the target parking position and a second photographing situation, which is the photographing situation when the past image was photographed, are stored in a predetermined storage device, and one or more combinations of the relative positional relationship and the second photographing situation are read out from the predetermined storage device, Among the read relative positional relationships, a relative positional relationship stored in combination with the second photographing situation, the difference between which is equal to or smaller than a predetermined difference from the first photographing situation, is selected; a first target position and a second target position; a first target position and a second target position; a second target position and a second target position; a first target position and a second target position; a first target position and a second target position;

2. The parking assistance method described in Claim 1, characterized in that the controller selects, from the read-out relative position relationships, the relative position relationship stored in combination with the second shooting situation that has the smallest difference from the first shooting situation as the selected relative position relationship.

3. 3. The parking assistance method according to claim 1, wherein the first and second shooting conditions include at least one of brightness or weather at the shooting location, shooting time, shooting date and time, or solar position at the time of shooting.

4. The controller calculating a target driving trajectory from the current position of the vehicle to the target parking position based on a relative positional relationship between the current position of the vehicle and the target parking position; controlling the vehicle so that the vehicle travels from the current position of the vehicle to the target parking position along the target travel trajectory; 4. The parking assistance method according to claim 1, wherein the parking assistance method comprises:

5. The controller calculating a target driving trajectory from the current position of the vehicle to the target parking position based on a relative positional relationship between the current position of the vehicle and the target parking position; displaying the target travel trajectory and the position of the vehicle on a display device that is visible to a passenger; 5. The parking assistance method according to claim 1, wherein the vehicle is driven in a direction parallel to the road surface.

6. an imaging device that captures images of the surroundings of the vehicle; a controller that detects a first target position, which is the position of a target around the host vehicle, from an image generated by the imaging device, detects a first photographing situation, which is the photographing situation when the image was photographed, and for a second target position, which is the target position of a target detected from a past image, which is an image obtained by photographing the surroundings of the target parking position in the past, reads out one or more combinations of the relative positional relationship between the second target position and the target parking position and the second photographing situation, which is the photographing situation when the past image was photographed, from a predetermined storage device in which one or more combinations of the relative positional relationship between the second target position and the target parking position and the second photographing situation, are stored, selects one or more relative positional relationships stored in combination with the second photographing situation, the difference from the first photographing situation being a predetermined difference or less, and calculates a relative position between the current position of the host vehicle and the target parking position based on the selected relative positional relationship and the first target position, and performs parking assistance for the host vehicle to the target parking position; A parking assistance device comprising:

Images