Parking assistance device

Abstract

A parking assistance device of an embodiment of the present invention comprises, for example: an image acquisition unit (50) that acquires a captured image from an image capturing unit (14) that captures an image of a surrounding situation of a vehicle (10); a setting unit (58) that, at the start of parking assistance, sets a reference movement target position of the vehicle (10) in a parking area (42) contained in the captured image and defined by a pair of partition lines disposed spaced apart from each other, and that, after the start of parking assistance, sets, at each predetermined control cycle, a movement target position of the vehicle (10) in the parking area (42) contained in the captured image and defined by the pair of partition lines; a correction unit (66) that corrects the movement target position on the basis of the reference movement target position; a position acquisition unit (54) that acquires the current position of the vehicle (10); and a guidance control unit (64) that executes control for guiding the vehicle to the corrected movement target position, on the basis of the corrected movement target position and the current position.

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a parking assistance device. [Background technology]

[0002] Conventionally, parking assistance devices have been known that assist a driver in parking a vehicle in a parking lot, etc. For example, in a parking lot, etc., the parking assistance device sets a target position for the vehicle to move within a parking area defined by dividing lines (e.g., a pair of white lines) included in a captured image, and guides the vehicle to the target position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-74254 Summary of the Invention [Problem to be solved by the invention]

[0004] When recognizing lane markings using captured images, it is assumed that the camera position and angle are known. However, in reality, errors occur in the recognition results of lane markings using captured images because the camera position and angle change depending on the weight of passengers and luggage, tire air pressure, etc. The errors vary depending on the distance and direction from the vehicle to the lane markings.

[0005] If there is an error in the recognition results of the lane markings, the vehicle will have to turn around when guiding the vehicle to the target position, which is undesirable as it reduces the accuracy of vehicle guidance. Also, while it is possible to improve this by adding a vehicle height sensor, etc., this is costly.

[0006] Therefore, one of the objectives of the embodiment is to provide a parking assistance device that can guide a vehicle into a parking area with high accuracy even if there is an error in the recognition results of the lane markings using captured images, without adding any physical configuration. [Means for solving the problem]

[0007] A parking assistance device according to an embodiment of the present invention includes, for example, an image acquisition unit that acquires an image from an imaging unit that images the surroundings of the vehicle; a setting unit that, when parking assistance starts, sets a reference movement target position for the vehicle within a parking area defined by a pair of spaced-apart demarcation lines included in the image, and, after parking assistance starts, sets a movement target position for the vehicle within the parking area defined by the pair of demarcation lines included in the image for each predetermined control period; a correction unit that corrects the movement target position based on the reference movement target position; a position acquisition unit that acquires the current position of the vehicle; and a guidance control unit that executes control to guide the vehicle to the corrected movement target position based on the corrected movement target position and the current position. According to this configuration, by correcting the movement target position after the start of parking assistance using the reference movement target position set when parking assistance begins, the vehicle can be guided to the parking area with high accuracy without adding any physical configuration, even if there is an error in the recognition results of the lane markings using the captured image.

[0008] In the parking assistance device described above, for example, the correction unit may be configured to reduce the amount of correction used to correct the movement target position as the distance from the vehicle to the movement target position becomes shorter. This configuration makes it possible to avoid situations where the movement target position is over-corrected due to factors other than changes in the camera position or angle (for example, a slope, a step in the road, etc.).

[0009] Furthermore, in the parking assistance device described above, for example, when the vehicle moves in a direction approaching the movement target position on a turning trajectory during parking assistance, if the movement target position is farther from the turning center of the turning trajectory than the reference movement target position, the correction unit may perform a correction to bring the movement target position closer to the reference movement target position, and if the movement target position is closer to the turning center of the turning trajectory than the reference movement target position, the correction unit may not correct the movement target position. According to this configuration, when the movement target position is closer to the turning center of the turning trajectory than the reference movement target position, where correction is less necessary, the movement target position is not corrected, thereby avoiding instability in vehicle guidance that may occur if correction is made.

[0010] Furthermore, in the parking assistance device described above, for example, when the calculation result indicates that the correction amount for correcting the movement target position increases over time, the correction unit may maintain the correction amount without increasing it. This configuration can prevent the vehicle from swaying left and right during guidance.

[0011] In the parking assistance device described above, for example, the correction unit may reset the correction amount used to correct the movement target position when a new lane marking is recognized in the captured image. According to this configuration, when a new lane marking is recognized, an appropriate response can be realized, in which the correction amount is reset.

[0012] Furthermore, in the above-mentioned parking assistance device, for example, the correction unit may terminate the correction of the movement target position when the angle between the direction of movement of the vehicle and the direction in which the pair of demarcation lines extends becomes equal to or less than a predetermined angle. According to this configuration, it is possible to realize an appropriate response of ending the correction when the angle becomes equal to or smaller than a predetermined angle, that is, when the possibility of a turnaround occurring during vehicle guidance is eliminated.

[0013] Furthermore, in the above-described parking assistance device, for example, the setting unit may recognize the start of parking assistance based on at least one of the following: the gear shift operating unit having reached a predetermined position; and the user having performed an operation to start parking assistance. According to this configuration, the start of parking assistance can be recognized by a specific trigger, such as the shift operation unit being in a predetermined position or the user performing an operation to start parking assistance.

[0014] Furthermore, in the parking assistance device described above, for example, when correcting the movement target position based on the reference movement target position, the correction unit may correct the movement target position to the reference movement target position itself, or may correct the movement target position so as to approach the reference movement target position. According to this configuration, it is possible to perform more specific corrections, such as correcting the movement target position to the reference movement target position itself, or bringing the movement target position closer to the reference movement target position. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic plan view showing an example of a vehicle on which a parking assistance device according to an embodiment can be installed. [Figure 2] FIG. 2 is an exemplary block diagram of a configuration of a parking assistance system including the parking assistance device of the embodiment. [Figure 3] FIG. 3 is an exemplary block diagram of the functional configuration of the CPU of the parking assistance device according to the embodiment. [Figure 4] FIG. 4 is a diagram for explaining an example of a case where parking assistance for a vehicle is performed by the parking assistance device of the embodiment. [Figure 5] FIG. 5 is a diagram for explaining that the position of an object recognized in a captured image differs depending on the height position of the camera. [Figure 6] FIG. 6 is a diagram for explaining an example of the recognition result of the demarcation lines when performing parallel backward parking using the conventional technology. [Figure 7] FIG. 7 is a diagram for explaining an example of the recognition result of the demarcation line when performing parallel backward parking by the parking assistance device of the embodiment. [Figure 8] FIG. 8 is a graph showing the relationship between the distance and the gain used in the parking assistance device of the embodiment. [Figure 9] FIG. 9 is a diagram for explaining an example in which a new demarcation line is recognized in the parking assistance device of the embodiment. [Figure 10] FIG. 10 is a flowchart illustrating an example of parking assistance processing by the parking assistance device of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Exemplary embodiments of the present invention are disclosed below. The configurations of the embodiments described below, as well as the actions, results, and advantages brought about by the configurations, are examples. The present invention can be realized with configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of the various advantages and derivative advantages based on the basic configurations.

[0017] FIG. 1 is a schematic plan view showing an example of a vehicle 10 that can be equipped with a parking assistance device according to an embodiment. The vehicle 10 may be, for example, an automobile (internal combustion engine automobile) that uses an internal combustion engine (engine, not shown) as a drive source, an automobile (electric automobile, fuel cell automobile, etc.) that uses an electric motor (motor, not shown) as a drive source, or an automobile that uses both of these as drive sources (hybrid automobile). The vehicle 10 may be equipped with various transmissions and various devices (systems, parts, etc.) required to drive the internal combustion engine or electric motor. The type, number, layout, etc. of devices related to driving the wheels 12 (front wheels 12F, rear wheels 12R) of the vehicle 10 may be variously configured.

[0018] As illustrated in FIG. 1, the vehicle 10 is provided with a plurality of imaging units 14 (hereinafter also referred to as "cameras"), for example, four imaging units 14a to 14d. The imaging units 14 are digital cameras incorporating imaging elements such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) image sensor (CIS). The imaging units 14 can output video data (captured image data) at a predetermined frame rate. Each imaging unit 14 has a wide-angle lens or a fisheye lens and can capture images in a horizontal range of, for example, 140° to 220°. The optical axis of the imaging unit 14 may be set to point diagonally downward. Thus, the imaging units 14 sequentially capture images of the surrounding environment outside the vehicle 10, including road surfaces on which the vehicle 10 can travel, indicators on the road surfaces (including demarcation lines, lane dividers, arrows, etc. indicating parking areas), and objects (obstacles, such as pedestrians and vehicles), and output the captured image data.

[0019] The imaging unit 14 is provided on the outer periphery of the vehicle 10. The imaging unit 14a is provided, for example, at the rear side of the vehicle 10, i.e., at the rear end of the vehicle in the longitudinal direction and approximately in the center in the vehicle width direction, for example, above the rear bumper 10a, and is capable of capturing an image of the rear region including the rear end of the vehicle 10 (e.g., the rear bumper 10a). The imaging unit 14b is provided, for example, at the front side of the vehicle 10, i.e., at the front end of the vehicle in the longitudinal direction and approximately in the center in the vehicle width direction, for example, on the front bumper 10b or front grille, and is capable of capturing a front image including the front end of the vehicle 10 (e.g., the front bumper 10b).

[0020] Furthermore, the imaging unit 14c is provided, for example, at the right end of the vehicle 10, for example, on the right door mirror 10c, and is capable of capturing a right side image including an area centered on the right side of the vehicle 10 (for example, an area from the right front to the right rear). The imaging unit 14d is provided, for example, at the left end of the vehicle 10, for example, on the left door mirror 10d, and is capable of capturing a left side image including an area centered on the left side of the vehicle 10 (for example, an area from the left front to the left rear).

[0021] The parking assistance device of this embodiment can recognize (detect) lane lines and the like by performing arithmetic processing and image processing based on image data (images) obtained by the multiple image capture units 14. By recognizing lane lines, the parking assistance device can perform operations such as searching for a parking area for parking the vehicle 10, setting a movement target position to be used when guiding the vehicle 10 to the parking area, estimating the current position of the vehicle 10, and calculating a movement path for guiding the vehicle 10 to the movement target position. Note that by performing arithmetic processing and image processing based on the image data obtained by the multiple image capture units 14, it is possible to generate images with a wider viewing angle and generate virtual images of the vehicle 10 viewed from above, in front, to the side, etc. (bird's-eye view images (planar images), side-view images, front-view images, etc.).

[0022] FIG. 2 is an exemplary block diagram of a configuration of a parking assistance system 100 including a parking assistance device according to an embodiment. A display device 16 and an audio output device 18 are provided in the cabin of the vehicle 10. The display device 16 is, for example, a liquid crystal display (LCD) or an organic electroluminescent display (OLED). The audio output device 18 is, for example, a speaker. The display device 16 is covered with a transparent operation input unit 20 such as a touch panel. An occupant (e.g., the driver) can visually recognize an image displayed on the display screen of the display device 16 via the operation input unit 20. The occupant can perform an operation input by touching, pressing, or moving the operation input unit 20 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 16.

[0023] The display device 16, audio output device 18, operation input unit 20, etc. are provided on a monitor device 22 located, for example, in the center of the dashboard of the vehicle 10 in the vehicle width direction, i.e., the left-right direction. The monitor device 22 may have operation input units (not shown), such as switches, dials, joysticks, and push buttons. The monitor device 22 may also be used as, for example, a navigation system or an audio system.

[0024] 2, the parking assistance system 100 (parking assistance device) includes an ECU 24 (Electronic Control Unit) in addition to the imaging units 14 (14a to 14d) and the monitor device 22. In the parking assistance system 100, the ECU 24 and the monitor device 22 are electrically connected via an in-vehicle network 26 serving as an electrical communication line. The in-vehicle network 26 is configured as, for example, a CAN (Controller Area Network).

[0025] The ECU 24 can control various systems by sending control signals via the in-vehicle network 26. For example, in the parking assistance system 100, in addition to the ECU 24 and the monitor device 22, a steering system 28, a steering angle sensor 30, a brake system 32, a drive system 34, an accelerator sensor 36, a shift sensor 38, a wheel speed sensor 40, and the like are electrically connected via the in-vehicle network 26. The ECU 24 can control the steering system 28, the brake system 32, the drive system 34, and the like by sending control signals via the in-vehicle network 26. The ECU 24 can also receive detection results from the torque sensor 28a, the brake sensor 32a, the steering angle sensor 30, the accelerator sensor 36, the shift sensor 38, the wheel speed sensor 40, and the like, as well as operation signals from the operation input unit 20, and the like, via the in-vehicle network 26.

[0026] The ECU 24 has, for example, a CPU 24a (Central Processing Unit), a ROM 24b (Read Only Memory), a RAM 24c (Random Access Memory), a display control unit 24d, an audio control unit 24e, and an SSD 24f (Solid State Drive, flash memory), etc. The CPU 24a can execute various types of arithmetic processing and control, such as determining a movement target position of the vehicle 10 when executing parking assistance, estimating the current position of the vehicle 10, calculating a movement distance, calculating a target vehicle speed and a movement target position for each time, and image processing related to images displayed on the display device 16.

[0027] The CPU 24a can read out programs installed and stored in a non-volatile storage device such as the ROM 24b and execute arithmetic processing in accordance with the programs. The ROM 24b stores each program and parameters necessary for executing the programs. The RAM 24c temporarily stores various data used in the calculations performed by the CPU 24a. Furthermore, the display control unit 24d mainly executes image processing using image data obtained by the imaging unit 14 and composition of image data displayed on the display device 16, among the arithmetic processing performed by the ECU 24.

[0028] Furthermore, the audio control unit 24e mainly processes audio data to be output by the audio output device 18, among the arithmetic processing performed by the ECU 24. The SSD 24f is a rewritable nonvolatile storage unit that can store data even when the power to the ECU 24 is turned off. The CPU 24a, ROM 24b, RAM 24c, etc. may be integrated in the same package. The ECU 24 may be configured to use other logic operation processors, logic circuits, etc., such as a DSP (Digital Signal Processor), instead of the CPU 24a. A HDD (Hard Disk Drive) may be provided instead of the SSD 24f, or the SSD 24f and HDD may be provided separately from the ECU 24.

[0029] As illustrated in Fig. 1, the vehicle 10 is, for example, a four-wheeled vehicle, and has two front wheels 12F (left and right) and two rear wheels 12R (left and right). All four wheels 12 can be configured to be steerable. As illustrated in Fig. 2, a steering system 28 steers at least two wheels 12 of the vehicle 10. The steering system 28 has a torque sensor 28a and an actuator 28b. The steering system 28 is electrically controlled by the ECU 24 or the like to operate the actuator 28b. The steering system 28 is, for example, an electric power steering system or a steer-by-wire (SBW) system.

[0030] Steering system 28 uses actuator 28b to apply torque, i.e., assist torque, to a steering unit (for example, a steering wheel) to supplement steering force, and actuator 28b to steer wheels 12. In this case, actuator 28b may steer one wheel 12 or multiple wheels 12. Furthermore, torque sensor 28a detects, for example, the torque applied to the steering unit by the driver.

[0031] The steering angle sensor 30 is, for example, a sensor that detects the steering amount of the steering unit. The steering angle sensor 30 is configured using, for example, a Hall element. The ECU 24 acquires the steering amount of the steering unit by the driver, the steering amount of each wheel 12 during automatic steering, etc. from the steering angle sensor 30 and performs various controls. The steering angle sensor 30 detects the rotation angle of a rotating part included in the steering unit.

[0032] The brake system 32 is, for example, an anti-lock brake system (ABS) that prevents the brakes from locking, an electronic stability control (ESC) that prevents the vehicle 10 from skidding when cornering, an electric brake system that increases the braking force (performing brake assist), or a brake-by-wire (BBW). The brake system 32 applies braking force to the wheels 12 and thus the vehicle 10 via an actuator 32b. The brake system 32 can also detect signs of brake lock, freewheeling of the wheels 12, skidding, etc. from the rotation difference between the left and right wheels 12, and execute various controls. The brake sensor 32a is, for example, a sensor that detects the position of a movable part of a brake operating unit (e.g., a brake pedal).

[0033] The drive system 34 is an internal combustion engine (engine) system or a motor system serving as a drive source. The drive system 34 controls the engine fuel injection amount, intake air amount, and motor output value in accordance with the driver (user) operation amount (e.g., accelerator pedal depression amount) detected by an accelerator sensor 36. Furthermore, regardless of the user's operation, the drive system 34 can control the engine and motor output values in cooperation with the control of the steering system 28 and the brake system 32 in accordance with the driving state of the vehicle 10. The same applies when the vehicle 10 is driving in autonomous driving mode.

[0034] The accelerator sensor 36 is a sensor that detects the position of a movable part of an acceleration operation part (for example, an accelerator pedal), for example. The accelerator sensor 36 can detect the position of the accelerator pedal as a movable part.

[0035] The shift sensor 38 is a sensor that detects the position of a movable part of a gear change operation part (e.g., a shift lever). The shift sensor 38 can detect the position of a movable part such as a lever, an arm, or a button. The shift sensor 38 may include a displacement sensor or may be configured as a switch. Based on the detection result of the shift sensor 38, the ECU 24 can determine whether the vehicle 10 has received a request to travel forward or a request to travel backward.

[0036] The wheel speed sensors 40 are provided on each wheel 12 and detect the amount of rotation and the number of rotations per unit time of each wheel 12, and output the number of wheel speed pulses indicating the detected number of rotations as a detection value. The wheel speed sensors 40 may be configured using, for example, Hall elements. The ECU 24 calculates the vehicle speed and movement amount of the vehicle 10 based on the detection values acquired from the wheel speed sensors 40 and executes various controls. When calculating the vehicle speed of the vehicle 10 based on the detection values of the wheel speed sensors 40 of each wheel 12, the ECU 24 determines the vehicle speed of the vehicle 10 based on the speed of the wheel 12 with the smallest detected value among the four wheels, and executes various controls.

[0037] Furthermore, if there is a wheel 12 among the four wheels 12 whose detected value is larger than that of the other wheels 12, for example, if there is a wheel 12 whose number of rotations per unit period (unit time or unit distance) is greater by a predetermined number or more than that of the other wheels 12, the ECU 24 determines that the wheel 12 is in a slipping state (idling state) and executes various controls. The wheel speed sensor 40 may be provided in the brake system 32. In this case, the ECU 24 obtains the detection result of the wheel speed sensor 40 via the brake system 32.

[0038] The configurations, arrangements, electrical connection forms, etc. of the various sensors and actuators described above are merely examples, and can be set (changed) in various ways.

[0039] For example, the ECU 24 executes parking assistance processing based on the captured image data acquired from the imaging unit 14, and transmits data relating to surrounding images and sounds generated based on the captured image data to the monitor device 22.

[0040] The CPU 24a includes various modules such as those shown in Fig. 3, which will be described later, and executes processes related to parking assistance, for example. For example, the CPU 24a performs arithmetic processing, image processing, etc. on the captured image data captured by the imaging unit 14 to search for (present) candidate parking areas where the vehicle 10 can be parked, set a parking target position for guiding the vehicle 10 to the parking area, estimate the current position of the vehicle 10, and guide the vehicle 10 to the parking target position. Details of the CPU 24a will be described later.

[0041] In this embodiment, the ECU 24, through cooperation between hardware and software (control program), automatically or semi-automatically parks the vehicle 10 in a parking area while reducing the burden on the driver. Parking assistance modes include "fully automatic parking assistance," in which the parking assistance system 100 performs steering, acceleration / deceleration, braking, and other operations to guide the vehicle 10 to a destination position set within the parking area, and the driver does not need to perform any operations other than starting the assistance. Other modes include "semi-automatic parking assistance," in which at least one operation of the fully automatic parking assistance is left to the driver, and "guidance assistance," in which the driver is provided with instructions on how to operate the vehicle 10 by voice or display, and the driver drives the vehicle 10 in accordance with the instructions. In the following explanation, "fully automatic parking assistance," in which the vehicle 10 is driven backward into a parking area, will be described as an example.

[0042] First, an overview of parking assistance will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining an example of parking assistance for a vehicle 10 by a parking assistance system 100 (parking assistance device) of an embodiment, and is a schematic bird's-eye view diagram conceptually explaining the search for a parking area 42 and the guidance of the vehicle 10 along a travel route L1. Fig. 4 shows an example in which the vehicle 10 enters a parking lot P for parking, where a plurality of parking areas 42 defined by a pair of demarcation lines 44a and 44b (hereinafter referred to as demarcation lines 44 when no distinction is made) are aligned (for example, parking areas 42A to 42F are aligned side by side).

[0043] The demarcation lines 44a, 44b are, for example, white lines painted on the parking road surface and are arranged to define a parking area 42 with a width W and depth D sufficient to allow parking of a typical passenger car (e.g., vehicle 10). The parking assistance system 100 searches for a parking area 42 where parking is possible when the vehicle 10 is traveling at a speed below the parking area search speed (e.g., 30 km / h or less). In the case of FIG. 4, while the parking assistance system 100 is traveling (e.g., traveling along traveling trajectory L2) in a parking lot traveling space PS where parking areas 42 are lined up within the parking lot P, it uses the imaging unit 14 mounted on the vehicle 10 to search for parking areas 42 (only one parking area 42 is shown in FIG. 4) that may exist around the vehicle 10 (e.g., on the left and right).

[0044] The parking assistance system 100 sequentially performs image processing such as edge processing on images captured while traveling, and also performs, for example, straight line detection, to recognize demarcation lines 44, such as demarcation line 44a and demarcation line 44b, and similar features. When a pair of adjacent demarcation lines 44a and 44b, spaced apart at a distance wider than the width of the vehicle 10, is recognized, the parking assistance system 100 recognizes the area defined by the pair of demarcation lines 44a and 44b as a parking area 42. Furthermore, when no objects, such as other vehicles, are present in the recognized parking area 42, the parking assistance system 100 registers the parking area 42 as a candidate parking area 42 and presents it to the driver by, for example, displaying it on the display device 16.

[0045] When the driver selects a desired parking area 42 from the presented candidate parking areas 42, the parking assistance system 100 sets a movement target position T1 for guiding the vehicle 10 into the selected parking area 42. The parking assistance system 100 sets the movement target position T1 based on, for example, the width and length of the vehicle 10, as well as the positions of the leading ends of the demarcation lines 44a and 44b that make up the selected parking area 42 and the extending direction (angle) of the demarcation lines 44a and 44b obtained as a result of straight line detection.

[0046] The movement target position T1 is set to a position where, for example, a reference position T2 is set to the center position in the vehicle width direction of the rear wheel axle 12a supporting the rear wheels 12R of the vehicle 10 when the vehicle 10 fits into the selected parking area 42. Therefore, the parking assistance system 100 calculates a movement path L1 connecting the reference position T2 indicating the current position of the vehicle 10 and the movement target position T1 using well-known technology, and guides the vehicle 10 by driving backward along the movement path L1, thereby parking the vehicle 10 in the selected parking area 42.

[0047] As mentioned above, when recognizing lane markings using captured images, it is assumed that the camera position and angle are known. However, in reality, errors occur in the recognition results of lane markings using captured images because the camera position and angle change depending on the weight of passengers and luggage, tire air pressure, etc. The errors vary depending on the distance and direction from the vehicle to the lane markings.

[0048] 5 is a diagram illustrating that the position of an object recognized in a captured image differs depending on the height position of the camera. Camera position H1 is a height position (known height position) set for the camera. On the other hand, camera position H2 is the actual height position of the camera due to the body of the vehicle 10 sinking due to the weight of passengers and luggage, etc.

[0049] The recognition positions TR1 and TR2 are correct recognition positions based on image recognition of a predetermined object (for example, a white line), whereas the recognition positions FA1 and FA2 are recognition positions with errors based on image recognition of the predetermined object.

[0050] That is, the recognition position FA1 has an error of distance D1 compared to the correct recognition position TR1. Also, the recognition position FA2 has an error of distance D2 compared to the correct recognition position TR2. Thus, the greater the distance from the camera to the image capture target, the greater the error. Also, in this example, the error caused by the height position of the camera has been described, but the same applies to errors caused by the angle of the camera.

[0051] If there is an error in the recognition results of the lane markings, the vehicle may have to turn around when being guided to a target position, which reduces the accuracy of vehicle guidance, which is undesirable. Also, while it is possible to improve the situation by adding a vehicle height sensor or the like, this is costly.

[0052] Therefore, below we will explain a technology that can guide a vehicle into a parking area with high accuracy even if there is an error in the recognition results of the lane markings using captured images, without adding any physical configuration.

[0053] First, an overview of the conventional technology and this embodiment will be described with reference to Figures 6 and 7. Figure 6 is a diagram for explaining an example of the recognition result of lane markings when performing parallel backward parking using the conventional technology.

[0054] As shown in state C1 in Figure 6(a), parking assistance is initiated when the vehicle 10 is located next to the demarcation line TR11 (actual position of the demarcation line). At this time, the distance from the vehicle 10 to the demarcation line TR11 is short, so the recognition result FA11 of the demarcation line is close to the demarcation line TR11. In other words, the error is small.

[0055] After parking assistance begins, the vehicle 10 temporarily moves away from the demarcation line TR11, as shown in state C2 in FIG. 6(b). At this time, the distance from the vehicle 10 to the demarcation line TR11 is far, so the recognition result FA12 of the demarcation line is located far from the demarcation line TR11. In other words, there is a large error. Furthermore, because the distance from the vehicle 10 to one end of each of the two demarcation lines TR11 is different, the recognition result FA12 of the demarcation line also has an error in the extension direction compared to the demarcation line TR11.

[0056] Thereafter, as shown in state C3 in FIG. 6(c), the vehicle 10 gradually approaches the lane marking TR11. At this time, the distance from the vehicle 10 to the lane marking TR11 becomes shorter, so the lane marking recognition result FA13 gradually approaches the lane marking TR11. In other words, the error gradually decreases. Furthermore, the error in the direction in which the lane marking recognition result FA13 extends also gradually decreases.

[0057] If the lane marking recognition results are as described above, the vehicle 10 will need to move further inward than the planned turning path when entering the parking lot, which requires greater steering. However, there is also a limit to the steering angle, and following the path may be difficult. In this case, it may be necessary to turn the vehicle back. Even if the path can be followed, the driver may experience a strange behavior.

[0058] Next, FIG. 7 is a diagram for explaining an example of the recognition result of the demarcation line when performing parallel backward parking by the parking assistance device of the embodiment.

[0059] First, as shown in state C11 in FIG. 7(a), parking assistance is initiated when the vehicle 10 is positioned next to the demarcation line TR21 (actual position of the demarcation line). At this time, the distance from the vehicle 10 to the demarcation line TR21 is short, so the recognition result FA21 of the demarcation line is close to the demarcation line TR21. In other words, the error is small. The recognition result FA21 of the demarcation line at this time (or the reference movement target position T10 based on it) is stored.

[0060] After parking assistance begins, the vehicle 10 temporarily moves away from the demarcation line TR21, as shown in state C12 in FIG. 7(b). At this time, the distance from the vehicle 10 to the demarcation line TR21 is large, so the demarcation line recognition result FA22 is positioned far from the demarcation line TR21. In other words, there is a large error. Furthermore, because the distance from the vehicle 10 to each of the two demarcation lines TR21 is different, the demarcation line recognition result FA22 also has an error in the extension direction compared to the demarcation line TR21. Therefore, the demarcation line recognition result FA22 (or the movement target position T11 based thereon) is corrected using the demarcation line recognition result FA21 (or the reference movement target position T10 based thereon) to obtain the corrected demarcation line R22 (or the movement target position T12 based thereon). Parking assistance is performed using this corrected demarcation line R22 (or the movement target position T12 based thereon).

[0061] Thereafter, as shown in state C13 in FIG. 7(c), the vehicle 10 gradually approaches the demarcation line TR21. At this time, the distance from the vehicle 10 to the demarcation line TR21 becomes shorter, so the demarcation line recognition result FA23 gradually approaches the demarcation line TR21. In other words, the error gradually decreases. Furthermore, the error in the direction in which the demarcation line recognition result FA23 extends also gradually decreases. Here, as in the case of FIG. 7(b), the demarcation line recognition result FA23 is corrected using the demarcation line recognition result FA21, and a corrected demarcation line R23 is obtained. This corrected demarcation line R23 is used to perform parking assistance.

[0062] In this way, even if the error in the recognition result of the lane marking is large, it is possible to achieve highly accurate vehicle guidance by correcting the error using the recognition result of the lane marking when the error is small. This will be explained in detail below.

[0063] FIG. 3 is an exemplary block diagram of the functional configuration of a CPU 24a of the parking assistance device according to the embodiment. The CPU 24a includes various modules for executing the parking assistance process described above. The various modules are implemented by the CPU 24a reading and executing programs installed and stored in a storage device such as the ROM 24b. For example, as shown in FIG. 3, the CPU 24a includes modules such as an image acquisition unit 50, a projection transformation unit 52, a vehicle position acquisition unit 54 (position acquisition unit), a recognition unit 56, a setting unit 58, a path generation unit 60, a trigger acquisition unit 62, a guidance control unit 64, and a correction unit 66. Note that the modules shown in FIG. 3 are exemplified by function, and the modules may be modified as appropriate. Similar effects can be achieved by integrating or separating the functions described below.

[0064] The image acquisition unit 50 acquires, via the display control unit 24d, captured images from the imaging unit 14, which captures images of the surrounding environment of the vehicle 10. In this embodiment, when searching for a parking area 42, side images of the vehicle 10 captured by the imaging units 14c and 14d are primarily acquired, and when receiving parking assistance when reversing into a parking space, a rear image of the vehicle 10 captured by the imaging unit 14a is primarily acquired. Note that, when receiving parking assistance when reversing into a parking space, a front image of the vehicle 10 captured by the imaging unit 14b is primarily acquired. The image acquisition unit 50 provides the acquired images to the projection conversion unit 52. Note that, the display control unit 24d may perform image processing such as distortion correction on the images captured by the imaging unit 14 before displaying them, display them without performing image processing, or display them superimposed with other data, and then display the images on the display device 16.

[0065] The projection transformation unit 52 performs, for example, a well-known projection transformation process on the image of the surroundings of the vehicle 10 acquired by the image acquisition unit 50, and converts it into a three-dimensional image that provides a bird's-eye view of the parking area 42, as shown in Figure 4.

[0066] The host vehicle position acquisition unit 54 acquires the current position of the vehicle 10. Specifically, the host vehicle position acquisition unit 54 acquires the current position of the vehicle 10 (host vehicle position) on a coordinate system defined by the three-dimensional image converted by the projection transformation unit 52. For example, the relationship between the mounting position of the imaging unit 14a and the center position of the rear wheel axle 12a in the vehicle width direction is known as a vehicle design value of the vehicle 10. Therefore, when the projection transformation unit 52 performs projection transformation on the image captured by the imaging unit 14a, the position of the reference position T2 can be determined on the three-dimensional coordinate system. Similarly, the relationship between the mounting positions of the imaging units 14c and 14d and the center position of the rear wheel axle 12a in the vehicle width direction is known. Therefore, the host vehicle position acquisition unit 54 can sequentially acquire the current position of the vehicle 10 relative to a reference point (e.g., an origin) determined on the projection-transformed three-dimensional coordinate system acquired when searching for the parking area 42 or after parking assistance has started.

[0067] The recognition unit 56 performs, for example, edge processing on the projection-transformed image, and also performs straight line detection processing and the like to recognize the demarcation lines 44 .

[0068] At the start of parking assistance, the setting unit 58 sets a reference movement target position (reference symbol T10 in FIG. 7) of the vehicle within the parking area 42 defined by a pair of spaced-apart demarcation lines 44 included in the captured image. After the start of parking assistance, the setting unit 58 sets a movement target position T1 (reference symbol T11 in FIG. 7) of the vehicle within the parking area 42 defined by the pair of demarcation lines 44 included in the captured image for each predetermined control period.

[0069] The movement target position T1 is a position to which the reference position T2 is guided when the vehicle 10 enters the parking area 42. The widthwise position of the movement target position T1 in the parking area 42 is, for example, the midpoint position in the separation direction of a pair of linearly detected demarcation lines 44a and 44b. The depthwise position of the movement target position T1 in the parking area 42 can be, for example, a position corresponding to the distance from the front end of the vehicle 10 (front bumper 10b) to the reference position T2, with the tip of the recognized demarcation line 44 on the entrance side as the reference point.

[0070] The route generation unit 60 calculates the travel route L1 using a well-known recommended travel route calculation technique based on the set travel target position T1 and a reference position T2 at the current position of the vehicle 10. If the travel target position T1 has been corrected by the correction unit 66, the route generation unit 60 calculates the travel route L1 based on the corrected travel target position T1 and the reference position T2 at the current position of the vehicle 10.

[0071] The trigger acquisition unit 62 acquires a trigger signal for starting parking assistance processing in the parking assistance system 100. For example, when parking the vehicle 10 while driving it backward, the preparation processing for parking assistance includes searching for candidate parking areas 42, selecting the parking area 42 where the vehicle is desired to park, setting a movement target position T1 (for example, initial setting), detecting a reference position T2 (for example, initial estimation), and the like.

[0072] Then, when parking assistance actually starts, the driver moves the gearshift operating unit (e.g., a shift lever) to the "R position." In the case of parking assistance system 100, for example, the movement of the gearshift operating unit to the "R position" based on the detection result of shift sensor 38 is used as the trigger to start parking assistance. Note that the start of parking assistance may also be initiated by input via operation input unit 20 or by voice input, and trigger acquisition unit 62 may acquire these input signals. In other words, for example, the trigger acquisition unit 62 acquires a trigger signal based on at least one of the gearshift operating unit being in a predetermined position and the user performing a parking assistance start operation, and thereby setting unit 58 and the like recognize the start of parking assistance.

[0073] The correction unit 66 corrects the movement target position T1 based on the reference movement target position. For example, the correction unit 66 corrects the movement target position T1 to the reference movement target position itself. Furthermore, for example, the correction unit 66 corrects the movement target position T1 so that it approaches the reference movement target position.

[0074] Furthermore, the correction unit 66 may be configured to reduce the amount of correction for correcting the movement target position T1 as the distance from the vehicle 10 to the movement target position T1 becomes shorter. Here, Fig. 8 is a graph showing the relationship between the distance and the gain used in the parking assistance device of the embodiment.

[0075] The distance is the distance from the vehicle 10 to the lane marking. The gain is a number (0 to 1 (0 to 100%)) by which the difference between the movement target position T1 and the reference movement target position is multiplied. In the graph of FIG. 8, starting from the smallest distance, the gain increases from distance D1, becomes 100% at distance D2, and remains 100% at distances D2 and above.

[0076] Then, the difference between the movement target position T1 and the reference movement target position is multiplied by a corresponding gain and added to the movement target position T1, thereby correcting the movement target position T1. In this way, the correction amount can be reliably set to zero before the vehicle 10 reaches the movement target position T1, and the behavior of the vehicle 10 during parking assistance can be stabilized.

[0077] Returning to FIG. 3, when correcting the movement target position T1, the correction unit 66 may correct the movement target position T1 with a correction amount equal to or less than a predetermined upper limit of the correction amount.

[0078] Furthermore, during parking assistance, when the vehicle 10 moves in a direction approaching the movement target position T1 on a turning trajectory, if the movement target position T1 is farther from the turning center of the turning trajectory (the center of the circle when the turning trajectory is approximated as part of a circle) than the reference movement target position, the correction unit 66 may perform a correction to bring the movement target position T1 closer to the reference movement target position, and if the movement target position T1 is closer to the turning center of the turning trajectory than the reference movement target position, the correction unit 66 may not correct the movement target position.

[0079] Furthermore, if the calculation result indicates that the amount of correction for correcting the movement target position T1 increases over time, the correction unit 66 may maintain the amount of correction without increasing it.

[0080] Furthermore, the correction unit 66 may reset the correction amount for correcting the movement target position T1 when a new lane marking is recognized in the captured image. Here, Fig. 9 is a diagram for explaining an example in which a new lane marking is recognized in the parking assistance device of the embodiment.

[0081] In FIG. 9(a), the correction unit 66 recognizes the image of the demarcation lines 44d to 44f, but does not recognize the image of the demarcation line 44g. In this case, the parking area becomes an area AR1 closer to the demarcation line 44f. Thereafter, as shown in FIG. 9(b), the correction unit 66 recognizes the image of the demarcation line 44g. In this case, the parking area is changed to an area AR2 midway between the demarcation lines 44f and 44g. In this way, when a new demarcation line is recognized, the movement target position T1 (parking area) changes, so it is preferable to reset the correction amount used to correct the movement target position T1.

[0082] 3, for example, the correction unit 66 corrects the movement target position T1 when the angle between the movement direction of the vehicle 10 and the extension direction of the pair of lane lines 44 is equal to or greater than a predetermined angle. In this case, the correction unit 66 ends the correction of the movement target position T1 when the angle between the movement direction of the vehicle 10 and the extension direction of the pair of lane lines 44 becomes equal to or smaller than the predetermined angle.

[0083] The guidance control unit 64 executes control to guide the vehicle 10 to the movement target position T1 based on the movement target position T1 and the current position of the vehicle 10. If the movement target position T1 has been corrected, the guidance control unit 64 executes control to guide the vehicle 10 to the corrected movement target position T1 based on the corrected movement target position T1 and the current position of the vehicle 10.

[0084] The guidance control unit 64 performs guidance processing to move the vehicle 10 along the movement path L1 generated by the path generation unit 60 based on the corrected movement target position T1. When performing "fully automatic parking assistance," the guidance control unit 64 controls the steering system 28, the brake system 32, the drive system 34, etc. to reverse the vehicle 10 along the movement path L1, and moves the reference position T2 closer to the movement target position T1. In the case of "semi-automatic parking assistance" or "guidance assistance," the details of the operation are provided to the driver by voice or display, and the driver performs some or all of the operation to perform the parking operation.

[0085] Next, an example of parking assistance processing by the parking assistance device of the embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of parking assistance processing by the parking assistance device of the embodiment.

[0086] First, the imaging unit 14 of the vehicle 10 normally captures images of the surroundings of the vehicle 10 at all times, but when the vehicle is traveling at, for example, 30 km / h or less (less than the parking area search speed), the parking assistance system 100 assumes that the driver is driving while searching for a parking space. Then, in step S1, the image acquisition unit 50 acquires images of the surroundings of the vehicle 10 from the imaging unit 14 in order to search for a parking area 42 as a preparation step for parking assistance.

[0087] Next, the projection transformation unit 52 performs sequential projection transformation processing on the images captured while driving, and the recognition unit 56 performs edge processing, straight line detection processing, obstacle detection processing, etc. on the projection transformed images to search for candidates for parking areas 42, and displays the search results on the display device 16 (step S2).

[0088] Next, if the driver or the setting unit 58 does not select a specific parking area 42 from the displayed candidate parking areas 42 (No in step S3), the process returns to step S1 and the display of candidate parking areas 42 based on newly acquired images is repeated.

[0089] On the other hand, if the parking area 42 is selected by the driver or the setting unit 58 (Yes in step S3), the process proceeds to step S4.

[0090] In step S4, the trigger acquisition unit 62 determines whether or not a trigger signal for starting the parking assistance process has been received (acquired), and if Yes, the process proceeds to step S5, and if No, the process returns to step S4.

[0091] In step S5, the image acquisition unit 50 acquires an image of the surroundings of the vehicle 10 from the imaging unit 14.

[0092] Next, in step S6, the setting unit 58 sets a reference movement target position of the vehicle within the parking area 42 defined by the pair of demarcation lines 44 included in the captured image.

[0093] Next, in step S7, the image acquisition unit 50 acquires an image of the surroundings of the vehicle 10 from the imaging unit 14.

[0094] Next, in step S8, the setting unit 58 sets a target position for the vehicle to move to within the parking area 42 defined by the pair of demarcation lines 44 included in the captured image.

[0095] Next, in step S9, the correction unit 66 corrects the movement target position T1 based on the reference movement target position. Note that if the distance from the vehicle 10 to the movement target position becomes equal to or shorter than a predetermined distance (for example, about 3 meters), the correction may not be performed.

[0096] Next, in step S10, the vehicle position acquisition unit 54 acquires the current position of the vehicle 10 (the vehicle position).

[0097] Next, in step S11, the route generation unit 60 generates a travel route based on the corrected travel target position T1 and the current position of the vehicle 10 using a known recommended travel route calculation technique.

[0098] Next, in step S12, the guidance control unit 64 executes control to guide the vehicle 10 to the corrected movement target position T1 based on the current position of the vehicle 10 and the movement route.

[0099] Next, the guidance control unit 64 determines whether or not the vehicle 10 has reached the movement target position, and if Yes, ends the process, and if No, returns to step S7.

[0100] In this way, according to the parking assistance system 100 of this embodiment, by correcting the movement target position after the start of parking assistance using the reference movement target position set at the start of parking assistance, it is possible to guide the vehicle 10 into the parking area with high accuracy without adding any physical configuration, even if there is an error in the recognition result of the lane markings using the captured image. This makes it possible, for example, to reduce the number of times the vehicle 10 must turn around during parking assistance.

[0101] Furthermore, the shorter the distance from the vehicle 10 to the movement target position, the smaller the amount of correction for correcting the movement target position. This makes it possible to avoid a situation where the movement target position is over-corrected due to factors other than changes in the position or angle of the camera (imaging unit 14) (for example, a slope, a step on the road, etc.).

[0102] Furthermore, when correcting the movement target position, the correction amount may be set to a predetermined upper limit or less, which makes it possible to avoid the movement target position being over-corrected due to factors other than changes in the camera position or angle.

[0103] Furthermore, during parking assistance, when the vehicle 10 moves on a turning trajectory in a direction approaching a movement target position, the correction unit 66 may perform a correction to bring the movement target position closer to the reference movement target position if the movement target position is farther from the turning center of the turning trajectory than the reference movement target position, and may not correct the movement target position if the movement target position is closer to the turning center of the turning trajectory than the reference movement target position. This makes it possible to avoid instability in vehicle guidance that may occur if the movement target position is corrected, by not correcting the movement target position when the movement target position is closer to the turning center of the turning trajectory than the reference movement target position, for which correction is less necessary.

[0104] Furthermore, if the calculation result indicates that the amount of correction for correcting the target position increases over time, the amount of correction may be maintained without increasing, thereby preventing the vehicle from swaying left and right during guidance.

[0105] Furthermore, when a new lane marking is recognized in the captured image, the correction amount for correcting the movement target position may be reset. This allows for an appropriate response of resetting the correction amount when a new lane marking is recognized.

[0106] Furthermore, correction of the movement target position may be terminated when the angle between the direction of movement of the vehicle 10 and the direction in which the pair of demarcation lines 44 extends becomes equal to or less than a predetermined angle. This allows for an appropriate response of terminating the correction when the angle becomes equal to or less than the predetermined angle, i.e., when there is no longer a possibility of a change of direction during vehicle guidance.

[0107] Alternatively, the start of parking assistance may be recognized based on at least one of the following: the gearshift operating unit being in a predetermined position, or the user performing a parking assistance start operation. This allows the start of parking assistance to be recognized based on specific triggers, such as the gearshift operating unit being in a predetermined position or the user performing a parking assistance start operation.

[0108] Furthermore, when correcting the movement target position based on the reference movement target position, the movement target position may be corrected to the reference movement target position itself, or the movement target position may be corrected so as to approach the reference movement target position. This allows for more specific correction, such as correcting the movement target position to the reference movement target position itself or bringing the movement target position closer to the reference movement target position.

[0109] In the above embodiment, the demarcation lines 44 are painted on the parking road surface, but the present invention is not limited to this and may be any demarcation line that can be recognized by applying image processing to a captured image. For example, this embodiment can be applied to demarcation lines formed with ropes or the like, and similar effects can be obtained.

[0110] The program for the parking assistance processing executed by the CPU 24a of this embodiment may be configured to be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0111] Furthermore, the parking assistance processing program may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the parking assistance processing program executed in this embodiment may be provided or distributed via a network such as the Internet.

[0112] Although the embodiments and modifications of the present invention have been described, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims.

[0113] For example, although the above embodiment has been described with reference to backward parallel parking, the present invention is not limited to this and can also be applied to forward parallel parking, parallel parking, and the like. [Explanation of symbols]

[0114] 10...vehicle, 12a...rear wheel axle, 14, 14a, 14b, 14c, 14d...imaging unit, 16...display device, 24...ECU, 24a...CPU, 42...parking area, 44, 44a, 44b...landing line, 44c...tip portion, 50...image acquisition unit, 52...projection conversion unit, 54...vehicle position acquisition unit, 56...recognition unit, 58...setting unit, 60...route generation unit, 62...trigger acquisition unit, 64...guidance control unit, 66...correction unit, 100...parking assistance system, L1...movement path, L2...driving trajectory, T1, T11...movement target position, T2...reference position (vehicle position), T10...reference movement target position, T12...corrected movement target position.

Claims

1. an image acquisition unit that acquires an image from an imaging unit that images the surroundings of the vehicle; When starting parking assistance when the vehicle is located next to a parking area, a reference movement target position of the vehicle is set within the parking area defined by a pair of partition lines arranged at a distance and included in the captured image; and a setting unit that sets a movement target position of the vehicle within a parking area defined by the pair of demarcation lines included in the captured image for each predetermined control period after starting parking assistance; a correction unit that corrects the movement target position based on the reference movement target position; a position acquisition unit that acquires a current position of the vehicle; a guidance control unit that performs control to guide the vehicle to the corrected movement target position based on the corrected movement target position and the current position; Equipped with The correction unit reduces the amount of correction used to correct the movement target position as the distance from the vehicle to the movement target position becomes shorter, and in this case, when the distance from the vehicle to the lane marking is equal to or shorter than a predetermined distance, the correction amount is set to zero.

2. During parking assistance, when the vehicle moves along a turning trajectory in a direction approaching the movement target position, The correction unit When the movement target position is farther from the turning center of the turning trajectory than the reference movement target position, a correction is made to bring the movement target position closer to the reference movement target position; 2. The parking assistance device according to claim 1, wherein the movement target position is not corrected when the movement target position is closer to the turning center of the turning locus than the reference movement target position.

3. 2. The parking assistance device according to claim 1, wherein the correction unit maintains the correction amount without increasing it when the calculation result indicates that the correction amount for correcting the movement target position increases over time.

4. The parking assistance device according to claim 1 , wherein the correction unit resets a correction amount for correcting the movement target position when a new lane marking is recognized in the captured image.

5. 2. The parking assistance device according to claim 1, wherein the correction unit terminates correction of the movement target position when an angle formed between the movement direction of the vehicle and the extension direction of the pair of demarcation lines becomes equal to or smaller than a predetermined angle.

6. 2. The parking assistance device according to claim 1, wherein the setting unit recognizes that the parking assistance has started based on at least one of a gear shift operation unit being in a predetermined position and a user performing an operation to start parking assistance.

7. 2. The parking assistance device according to claim 1, wherein, when correcting the movement target position based on the reference movement target position, the correction unit corrects the movement target position to the reference movement target position itself, or corrects the movement target position so as to approach the reference movement target position.

Images