Parking Assistance Device

Abstract

To more appropriately detect a partition line defining a parking section for a vehicle.SOLUTION: A parking assistance device according to an embodiment includes: a captured image acquisition unit that acquires a captured image in which an image of a road surface around a vehicle is captured; an end portion detection unit that detects an end portion of a ground object attached to the road surface included in a detection area set in the captured image; a position information calculation unit that calculates position information indicating a positional relation of the detected end portion of the ground object with the vehicle; a partition line determination unit that determines whether or not the ground object having the end portion is a partition line defining a parking section for the vehicle, or not; and an exclusion unit that determines that a ground object located entirely in front of other partition lines among the ground objects determined as the partition lines is not a partition line and that excludes it from the partition lines.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] An embodiment of the present invention relates to a parking assistance device. [Background technology]

[0002] Conventionally, parking assistance devices have been used to assist the driver in parking a vehicle in order to reduce the driver's burden. Such parking assistance includes detection assistance for detecting a parking space where the vehicle can park, setting assistance for setting a target parking space where the vehicle will be parked from the detected parking space, start position guidance assistance for guiding the vehicle to a parking start position where parking begins, and parking position guidance assistance for guiding the vehicle from the parking start position to a target parking space. During this series of assistance, it is important to recognize the demarcation lines that define the parking space.

[0003] A known technology in such parking assistance devices focuses on the regularity of the placement of demarcation lines, groups the necessary demarcation lines based on the detected front end positions and angles of the demarcation lines, and groups the unnecessary demarcation lines due to erroneous detections, etc., and calculates the target parking position from the demarcation lines of the necessary demarcation line group. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-64140 Summary of the Invention [Problem to be solved by the invention]

[0005] In such parking assistance technology, it is desirable not only to prevent road markings and the like present in a parking space from being mistakenly detected as demarcation lines, but also to detect the demarcation lines more accurately.

[0006] This embodiment has been made in consideration of the above, and aims to provide a parking assistance device that can more appropriately detect the partition lines that define the parking space of a vehicle, and provide parking assistance at a more accurate parking target position. [Means for solving the problem]

[0007] The parking assistance device of the embodiment includes an image acquisition unit that acquires an image of a road surface around a vehicle, an edge detection unit that detects an edge of a feature attached to the road surface included in a detection area set in the captured image, a position information calculation unit that calculates position information indicating a positional relationship between the edge of the detected feature and the vehicle, a demarcation line determination unit that determines whether the feature having the edge is a demarcation line that defines a parking space for the vehicle, and a removal range that includes a range including a front area that is closer to the vehicle than a position including a vicinity of the tip of a demarcation line that has its tip on the farthest side from the vehicle before the start of a parking operation. by determining whether the feature determined to be the demarcation line is located closer to the vehicle than other demarcation lines as a whole before the parking assistance is started; When the rear end is included in the removal range, the feature determined to be the division line is , a feature located generally forward of the other division line, A removal unit that determines that the mark is not a demarcation line and removes the mark from the demarcation line; Equipped with.

[0008] With this configuration, for example, it is possible to prevent the mistaken detection of features such as road markings that are located generally forward of other demarcation lines as demarcation lines, and to more appropriately detect the demarcation lines that define the vehicle's parking space, thereby providing parking assistance at a more accurate target parking position.

[0010] With this configuration, as an example, it is possible to prevent the misdetection of features such as road surface marks that are included in the range extending from a position including near the tip of the demarcation line that has its tip on the side farthest from the vehicle before the parking operation begins, as a demarcation line, and to more appropriately detect the demarcation lines that define the vehicle's parking space, thereby providing parking assistance at a more accurate target parking position.

[0011] In addition, the parking assistance device of the embodiment further includes a judgment area setting unit that sets a predetermined judgment area along the road surface in the captured image based on position information of the end of the demarcation line, and an belonging / non-belonging determination unit that determines whether an end of an object other than the object having the end used to set the judgment area belongs to the judgment area, and the demarcation line determination unit determines whether the object having the end used to set the judgment area is a demarcation line that defines a parking space for the vehicle based on the determination result of the belonging / non-belonging determination unit, and the removal unit sets the judgment area as the removal range and determines whether the rear end of the object determined to be the demarcation line is included in the removal range.

[0012] With this configuration, as an example, it is possible to more appropriately detect the demarcation lines that define the parking space for the vehicle while reducing the effort required for setting the removal range, and to more appropriately detect the demarcation lines that define the parking space for the vehicle, thereby providing parking assistance at a more accurate parking target position.

[0013] In addition, the parking assistance device of the embodiment further includes a judgment area setting unit that sets a predetermined judgment area along the road surface in the captured image based on position information of the end of the demarcation line, and the judgment area setting unit sets a predetermined judgment area along the road surface in the captured image based on the end of the feature. The side closest to the vehicle Forward and The side farther from the vehicle The determination area set in a predetermined range behind is set as the removal area.

[0014] With this configuration, as an example, it is possible to prevent the mistaken detection of features such as road markings that exist not only in front of but also in a certain range rearward from near the tip of the demarcation line as demarcation lines, and to more appropriately detect the demarcation lines that define the vehicle's parking space, thereby providing parking assistance at a more accurate target parking position.

[0015] In the parking assistance device according to the embodiment, the determination area setting unit determines whether the edge of the feature is a reference. A direction perpendicular to the dividing line The determination region set in the predetermined range is set as the removal range.

[0016] With this configuration, as an example, it is possible to prevent false detection of features such as road surface marks that exist within a certain range to the right and left of the end of the demarcation line as a demarcation line, and to more appropriately detect the demarcation lines that define the vehicle's parking space, thereby providing parking assistance at a more accurate target parking position. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is an exemplary perspective view showing a state in which a part of a vehicle interior of a vehicle according to the present embodiment is seen through. [Diagram 2] FIG. 2 is an exemplary plan view of the vehicle according to the present embodiment. [Diagram 3] FIG. 3 is a view of an example of a dashboard of a vehicle according to this embodiment, viewed from the rear of the vehicle. [Figure 4] FIG. 4 is an exemplary block diagram of the configuration of the parking assistance system provided in the vehicle according to this embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of a functional configuration of an ECU included in the vehicle according to the present embodiment. [Figure 6] FIG. 6 is a schematic diagram showing an example of the determination region of this embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a road marking according to the present embodiment. [Figure 8] FIG. 8 is a diagram showing an example of the removal range and the demarcation lines in this embodiment. [Figure 9] FIG. 9 is a flowchart showing the procedure of the parking assistance process according to the embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of a procedure for the lane line management process according to this embodiment. [Figure 11] FIG. 11 is a flowchart showing an example of a procedure for lane line management processing in the modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Exemplary embodiments of the present invention are disclosed below. The configurations of the embodiments described below, as well as the actions, results, and effects brought about by the configurations, are merely examples. The present invention can be realized with configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of various effects based on the basic configuration and derivative effects.

[0019] (Embodiment) The vehicle 1 of this embodiment may be, for example, an automobile using an internal combustion engine (not shown) as a drive source, i.e., an internal combustion engine automobile, an automobile using an electric motor (not shown) as a drive source, i.e., an electric automobile or a fuel cell automobile, a hybrid automobile using both of them as drive sources, or an automobile equipped with other drive sources. The vehicle 1 may be equipped with various transmissions and various devices, such as systems and parts, required for driving the internal combustion engine or the electric motor. The type, number, layout, etc. of the devices related to driving the wheels 3 of the vehicle 1 may be set in various ways.

[0020] Fig. 1 is an exemplary perspective view showing a state in which a part of a cabin of a vehicle according to the present embodiment is seen through. Fig. 2 is an exemplary plan view of a vehicle according to the present embodiment. Fig. 3 is a view of an example of a dashboard of a vehicle according to the present embodiment as viewed from the rear of the vehicle. Fig. 4 is an exemplary block diagram of a configuration of a parking assistance system provided in a vehicle according to the present embodiment.

[0021] First, an example of the configuration of a vehicle 1 according to the present embodiment will be described with reference to FIGS.

[0022] 1, the vehicle body 2 defines a vehicle compartment 2a in which a passenger (not shown) rides. In the vehicle compartment 2a, a steering unit 4, an acceleration operation unit 5, a braking operation unit 6, a gear shift operation unit 7, etc. are provided in a state facing a driver's seat 2b as a passenger.

[0023] The steering unit 4 is, for example, a steering wheel protruding from the dashboard 24. The acceleration operation unit 5 is, for example, an accelerator pedal located under the driver's feet. The braking operation unit 6 is, for example, a brake pedal located under the driver's feet. The gear shift operation unit 7 is, for example, a shift lever protruding from a center console. Note that the steering unit 4, acceleration operation unit 5, braking operation unit 6, gear shift operation unit 7, etc. are not limited to these.

[0024] A display device 8 as a display output unit and an audio output device 9 as an audio output unit are provided in the vehicle interior 2a. The display device 8 is, for example, a liquid crystal display (LCD) or an organic electroluminescent display (OLED). The audio output device 9 is, for example, a speaker. The display device 8 is covered with a transparent operation input unit 10 such as a touch panel. The passenger can visually recognize an image displayed on the display screen of the display device 8 via the operation input unit 10. The passenger can also execute an operation input by touching, pressing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8.

[0025] The display device 8, audio output device 9, operation input unit 10, etc. are provided in a monitor device 11 located, for example, in the center of the dashboard 24 in the vehicle width direction, i.e., in the left-right direction. The monitor device 11 can have an operation input unit (not shown) such as a switch, a dial, a joystick, a push button, etc. Also, an audio output device (not shown) can be provided in another position in the vehicle compartment 2a different from the monitor device 11, and audio can be output from the audio output device 9 of the monitor device 11 and another audio output device. The monitor device 11 can also be used as, for example, a navigation system or an audio system.

[0026] In addition, a display device 12 separate from the display device 8 is provided in the vehicle interior 2a. As illustrated in FIG. 3, the display device 12 is provided, for example, in the instrument panel 25 of the dashboard 24, and is positioned approximately in the center of the instrument panel 25 between the speed display section 25a and the RPM display section 25b. The size of the screen 12a of the display device 12 is smaller than the size of the screen 8a of the display device 8. The display device 12 may display an image that mainly shows information related to parking assistance for the vehicle 1. The amount of information displayed by the display device 12 may be less than the amount of information displayed by the display device 8. The display device 12 is, for example, an LCD or an OELD. The information displayed by the display device 12 may be displayed on the display device 8.

[0027] 1 and 2, the vehicle 1 is, for example, a four-wheeled vehicle, and has two front wheels 3F, one on the left and one on the right, and two rear wheels 3R, one on the left and one on the right. All of these four wheels 3 can be configured to be steerable. As illustrated in FIG. 4, the vehicle 1 has a steering system 13 that steers at least two of the wheels 3.

[0028] The steering system 13 has an actuator 13a and a torque sensor 13b. The steering system 13 is electrically controlled by an ECU 14 (Electronic Control Unit) or the like to operate the actuator 13a. The steering system 13 is, for example, an electric power steering system or a steer-by-wire (SBW) system. The steering system 13 supplements the steering force by adding torque, i.e., assist torque, to the steering unit 4 by the actuator 13a, and steers the wheels 3 by the actuator 13a. In this case, the actuator 13a may steer one wheel 3 or may steer multiple wheels 3. The torque sensor 13b detects, for example, the torque applied to the steering unit 4 by the driver.

[0029] As illustrated in FIG. 2, the vehicle body 2 is provided with a plurality of imaging units 15, for example, four imaging units 15a to 15d. The imaging units 15 are digital cameras incorporating imaging elements such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging units 15 can output video data at a predetermined frame rate. The imaging units 15 each have a wide-angle lens or a fisheye lens, and can capture an image in a range of, for example, 140° to 190° in the horizontal direction. The optical axis of the imaging unit 15 is set to face diagonally downward. Thus, the imaging unit 15 sequentially captures the external environment around the vehicle body 2, including the road surface on which the vehicle 1 can move and the area in which the vehicle 1 can be parked, and outputs the captured image data.

[0030] The imaging unit 15a is located, for example, at the rear end 2e of the vehicle body 2 and is provided on a wall below the door 2h of the rear trunk. The imaging unit 15b is located, for example, at the right end 2f of the vehicle body 2 and is provided on the right door mirror 2g. The imaging unit 15c is located, for example, at the front end 2c of the vehicle body 2, i.e., the front side in the vehicle longitudinal direction, and is provided on the front bumper or the like. The imaging unit 15d is located, for example, at the left end 2d of the vehicle body 2, i.e., the left end in the vehicle width direction, and is provided on the door mirror 2g as a left protrusion. The ECU 14 executes arithmetic processing and image processing based on the image data obtained by the multiple imaging units 15, and can generate an image with a wider viewing angle or generate a virtual overhead image of the vehicle 1 viewed from above. The overhead image can also be called a planar image.

[0031] Furthermore, the ECU 14 identifies, from the image data of the imaging unit 15, partition lines or the like shown on the road surface around the vehicle 1, and detects (extracts) parking spaces shown by the partition lines or the like.

[0032] As illustrated in Figs. 1 and 2, the vehicle body 2 is provided with a plurality of distance measuring units 16, 17, for example, four distance measuring units 16a to 16d and eight distance measuring units 17a to 17h. The distance measuring units 16, 17 are, for example, sonars that emit ultrasonic waves and capture the reflected waves. The sonars can also be called sonar sensors or ultrasonic detectors. The ECU 14 can measure the presence or absence of objects such as obstacles positioned around the vehicle 1 and the distance to the objects based on the detection results of the distance measuring units 16, 17. That is, the distance measuring units 16, 17 are an example of a detection unit that detects objects. The distance measuring unit 17 can be used, for example, to detect objects at a relatively short distance, and the distance measuring unit 16 can be used, for example, to detect objects at a relatively long distance that are farther away than the distance measuring unit 17. Furthermore, the distance measuring unit 17 can be used, for example, to detect objects in front of and behind the vehicle 1, and the distance measuring unit 16 can be used to detect objects to the sides of the vehicle 1.

[0033] 4, in the parking assistance system 100, the ECU 14, the monitor device 11, the steering system 13, the distance measuring units 16 and 17, the brake system 18, the steering angle sensor 19, the accelerator sensor 20, the shift sensor 21, the wheel speed sensor 22, etc. are electrically connected via an in-vehicle network 23 as an electrical communication line. The in-vehicle network 23 is configured as, for example, a CAN (controller area network).

[0034] The ECU 14 can control the steering system 13, the brake system 18, etc., by sending control signals via the in-vehicle network 23. The ECU 14 can also receive detection results from the torque sensor 13b, the brake sensor 18b, the steering angle sensor 19, the distance measuring unit 16, the distance measuring unit 17, the accelerator sensor 20, the shift sensor 21, the wheel speed sensor 22, etc., operation signals from the operation input unit 10, etc., via the in-vehicle network 23.

[0035] The ECU 14 includes, for example, a CPU 14a (Central Processing Unit), a ROM (Read Only Memory) 14b, a RAM (Random Access Memory) 14c, a display control unit 14d, a sound control unit 14e, and an SSD (Solid State Drive, flash memory) 14f.

[0036] The CPU 14a can execute various types of arithmetic processing and control, such as image processing related to images displayed on the display devices 8 and 12, determining the target position of the vehicle 1, calculating the moving route of the vehicle 1, determining whether or not there is interference with an object, and automatically controlling and canceling automatic control of the vehicle 1. The CPU 14a can read out a program installed and stored in a non-volatile storage device such as the ROM 14b, and execute arithmetic processing according to the program.

[0037] The RAM 14c temporarily stores various data used in the calculations performed by the CPU 14a. The display control unit 14d mainly performs image processing using image data obtained by the imaging unit 15 and synthesis of image data displayed on the display device 8 among the calculations performed by the ECU 14. The audio control unit 14e mainly performs processing of audio data output by the audio output device 9 among the calculations performed by the ECU 14. The SSD 14f is a rewritable non-volatile storage unit, and can store data even when the power supply of the ECU 14 is turned off. The CPU 14a, the ROM 14b, the RAM 14c, and the like can be integrated in the same package. The ECU 14 may be configured to use other logic calculation processors such as a DSP (Digital Signal Processor) or logic circuits instead of the CPU 14a. A HDD (Hard Disk Drive) may be provided instead of the SSD 14f, and the SSD 14f and the HDD may be provided separately from the ECU 14.

[0038] The brake system 18 is, for example, an anti-lock brake system (ABS) that suppresses locking of the brakes, an electronic stability control (ESC) that suppresses skidding of the vehicle 1 when cornering, an electric brake system that strengthens the braking force (performs brake assist), a brake by wire (BBW), etc. The brake system 18 applies a braking force to the wheels 3 and thus the vehicle 1 via an actuator 18a. The brake system 18 can also detect brake lock, freewheeling of the wheels 3, signs of skidding, etc. from a rotation difference between the left and right wheels 3, and execute various controls. The brake sensor 18b is, for example, a sensor that detects the position of a movable part of the brake operating unit 6. The brake sensor 18b can detect the position of a brake pedal as a movable part. The brake sensor 18b includes a displacement sensor.

[0039] The steering angle sensor 19 is a sensor that detects the amount of steering of the steering unit 4, such as a steering wheel. The steering angle sensor 19 is configured using, for example, a Hall element. The ECU 14 acquires the amount of steering of the steering unit 4 by the driver and the amount of steering of each wheel 3 during automatic steering from the steering angle sensor 19 and executes various controls. The steering angle sensor 19 detects the rotation angle of a rotating part included in the steering unit 4.

[0040] The accelerator sensor 20 is, for example, a sensor that detects the position of a movable part of the acceleration operating unit 5. The accelerator sensor 20 can detect the position of an accelerator pedal as a movable part. The accelerator sensor 20 includes a displacement sensor.

[0041] The shift sensor 21 is, for example, a sensor that detects the position of a movable part of the gear shift operation unit 7. The shift sensor 21 can detect the position of a movable part such as a lever, an arm, a button, etc. The shift sensor 21 may include a displacement sensor, or may be configured as a switch.

[0042] The wheel speed sensor 22 is a sensor that detects the amount of rotation of the wheel 3 and the number of rotations per unit time. The wheel speed sensor 22 outputs the number of wheel speed pulses indicating the detected number of rotations as a sensor value. The wheel speed sensor 22 can be configured using, for example, a Hall element. The ECU 14 calculates the amount of movement of the vehicle 1 and the like based on the sensor value acquired from the wheel speed sensor 22, and executes various controls. Note that the wheel speed sensor 22 may be provided in the brake system 18. In that case, the ECU 14 acquires the detection result of the wheel speed sensor 22 via the brake system 18.

[0043] The configurations, arrangements, electrical connection forms, and the like of the various sensors and actuators described above are merely examples, and can be set (changed) in various ways.

[0044] Next, an example of the functional configuration of the ECU 14 of the vehicle 1 according to the present embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the functional configuration of the ECU of the vehicle according to the present embodiment.

[0045] 5, the ECU 14 according to this embodiment is an example of a parking assistance device having an image acquisition unit 141, an end detection unit 142, a position information calculation unit 143, a judgment area setting unit 144, an attribute determination unit 145, a lane line judgment unit 146, an erroneous detection line removal unit 147, a route calculation unit 148, a guidance control unit 149, a display control unit 14d, etc. In this embodiment, the CPU 14a reads out and executes a program stored in a storage device such as the ROM 14b or the SSD 14f, thereby realizing various functional modules such as the image acquisition unit 141, the end detection unit 142, the position information calculation unit 143, the judgment area setting unit 144, an attribute determination unit 145, a lane line judgment unit 146, an erroneous detection line removal unit 147, a route calculation unit 148, a guidance control unit 149, and a display control unit 14d.

[0046] In this embodiment, various functional modules such as the captured image acquisition unit 141, the edge detection unit 142, the position information calculation unit 143, the judgment area setting unit 144, the belonging / non-belonging judgment unit 145, the lane line judgment unit 146, the false detection line removal unit 147, the route calculation unit 148, the guidance control unit 149, and the display control unit 14d are realized by a processor such as the CPU 14a reading and executing a program stored in a storage device such as the ROM 14b or the SSD 14f, but this is not limited to this. For example, various functional modules such as the captured image acquisition unit 141, the edge detection unit 142, the position information calculation unit 143, the judgment area setting unit 144, the belonging / non-belonging judgment unit 145, the lane line judgment unit 146, the false detection line removal unit 147, the route calculation unit 148, the guidance control unit 149, and the display control unit 14d can also be realized by independent hardware. In addition, the various functional modules such as the captured image acquisition unit 141, the end detection unit 142, the position information calculation unit 143, the judgment area setting unit 144, the belonging / non-belonging judgment unit 145, the lane marking line judgment unit 146, the false detection line removal unit 147, the route calculation unit 148, the guidance control unit 149, and the display control unit 14d are merely examples, and each functional module may be integrated or subdivided as long as it can realize similar functions.

[0047] The captured image acquisition unit 141 acquires a captured image of the road surface around the vehicle 2. The surroundings of the vehicle 2 may be, for example, the side of the vehicle 2, but is not limited thereto. As described above, in this embodiment, the imaging unit 15b is provided on the right door mirror 2g, and the imaging unit 15d is provided on the door mirror 2g as a protruding part on the left side. Therefore, the imaging unit 15b captures an image of the road surface on the right outside of the vehicle body 2 (hereinafter referred to as a "right side captured image"), and the imaging unit 15d captures an image of the road surface on the left outside of the vehicle body 2 (hereinafter referred to as a "left side captured image"). The captured image acquisition unit 141 acquires the right side captured image from the imaging unit 15b, and acquires the left side captured image from the imaging unit 15d. Here, the imaging unit 15b and the imaging unit 15d continuously capture the right side captured image and the left side captured image, respectively. Therefore, the captured image acquisition unit 141 also continuously acquires the right side captured image and the left side captured image. In this embodiment, the imaging units 15b and 15d capture the road surface around the vehicle 1 through a fisheye lens. Therefore, the right and left captured images in this embodiment are so-called fisheye images. As is well known, a fisheye image has a wider imaging range (wider angle of view), but the further away from the imaging center (center of the optical axis), the more distortion occurs, making it difficult to grasp the scene contained in the captured image. Therefore, in this embodiment, for ease of understanding, an image converted from a fisheye image into a planar image is used for explanation. Note that in the processing in the parking assistance system 100, the fisheye image may be converted into a planar image and handled, or may be handled as it is.

[0048] The edge detection unit 142 detects edges of features (hereinafter, simply referred to as "features") attached to the road surface included in the captured image. The edge detection unit 142 sets a detection area along the road surface included in the captured image to be used as an area for detecting the edges of features. It is preferable that the detection area is set longer than the vehicle length along the vehicle length direction of the vehicle 1, and longer than the vehicle width along the vehicle width direction of the vehicle 1. In this embodiment, the detection area is an area included in the captured image captured by the imaging unit 15 and is a preset area.

[0049] Here, the features are indicators such as division lines and blocks attached to the road surface. The edge detection unit 142 detects the edges of indicators such as division lines and blocks within the detection area set on the captured image described above. This detection is well known and will not be described in detail, but for example, it is possible to detect the edges of the detection area by performing edge detection using well-known edge extraction and judging whether the detected edges are indicators, blocks, etc.

[0050] In this embodiment, the edge detection unit 142 detects an edge of a feature on the vehicle 1 side that has a straight portion extending along a direction intersecting with the traveling direction of the vehicle 1. That is, the edge detection unit 142 first detects a feature in the detection area and identifies the straight portion of the detected feature. Next, the edge detection unit 142 determines whether the extension direction of the portion along the straight portion intersects with the traveling direction of the vehicle 1, and if it does intersect, in this embodiment, it detects the edge of the portion along the straight portion of the feature that is closer to the vehicle 1.

[0051] The position information calculation unit 143 calculates position information indicating the positional relationship with the vehicle 1 for the edge of the detected feature. In this embodiment, the detected edge of the feature is the edge (particularly the corner) on the vehicle 1 side of the straight part of the feature detected by the edge detection unit 142 described above. The position information indicating the positional relationship with the vehicle 1 is information indicating the edge of the feature and its relative position with respect to the vehicle 1. This relative position can be configured to indicate the coordinates of the feature on a map or coordinates set based on a predetermined position of the vehicle 1 (the center of the pair of left and right driving wheels in the vehicle width direction), for example, or can be configured to indicate the difference between the respective latitude and longitude information consisting of the latitude and longitude of the feature and the vehicle 1. In addition, the relative position with respect to the edge of the feature changes depending on the traveling of the vehicle 1, and it is preferable to calculate this based on, for example, GPS information (position information) of the vehicle 1 depending on the traveling of the vehicle 1.

[0052] In this embodiment, the location information calculated by the location information calculation unit 143 is stored in the storage unit 150. Here, the storage unit 150 may be configured to store all location information detected in the past, or, from the viewpoint of storage capacity and calculation processing load, it may be configured to store a predetermined number (e.g., 10 pieces) of the most recent location information out of all location information detected in the past.

[0053] The judgment area setting unit 144 sets a predetermined judgment area along the road surface in the captured image based on the position information of the end of the demarcation line. The position information of the end of the demarcation line is calculated by the position information calculation unit 143 and is stored in the storage unit 150 in this embodiment. The judgment area is used when determining whether or not the end of the detected feature is the end of a demarcation line that defines the parking space of the vehicle 1.

[0054] Fig. 6 is a schematic diagram showing an example of a determination region in this embodiment. Fig. 6 shows an example of a determination region 701. In the example of Fig. 6, the determination region 701 is set based on an edge 601a of an object on the ground. Setting based on the edge 601a means that the determination region is set in contact with the edge 601a. ​​The determination region is set using the position information of the edge described above.

[0055] Moreover, the judgment area 701 is set so as to extend along a first direction, which is the traveling direction of the vehicle 1, have a predetermined length in a second direction intersecting the first direction, and include an end portion, and the angle between the extension direction of the judgment area 701 and the first direction is set within a preset angle. The traveling direction of the vehicle 1 can be calculated based on the GPS information of the vehicle 1, and is shown along the vehicle length direction of the vehicle 1 in FIG. 6. Moreover, the predetermined length in the second direction intersecting the first direction means that the judgment area is set not as a line but with a predetermined width, and it is preferable to set it to, for example, about several tens of centimeters to 1 meter. In the example of FIG. 6, the end portion is set to be located at the center in the width direction of the judgment area 701. The angle between the extension direction of the judgment area and the first direction is an index indicating the parallelism between the extension direction of the judgment area and the first direction, and in this embodiment, the angle is set to 0 degrees. Therefore, the judgment area is set parallel to the first direction (traveling direction of the vehicle 1).

[0056] The belonging / non-belonging determination unit 145 determines whether an end of another feature different from the feature having an end belongs to the determination area. The feature having an end is a feature having an end used as a reference when setting the determination area 701. In the example of FIG. 6, this corresponds to the end 601a of the straight line portion 601A. The other different feature is an end of a feature different from the feature having an end used as a reference when setting the determination area 701.

[0057] In the example of Fig. 6, the end 605b of the straight line portion 605B, the end 601c of the straight line portion 601C, the end 605d of the straight line portion 605D, the end 601e of the straight line portion 601E, the end 603f of the straight line portion 603F, the end 603g of the straight line portion 603G, the end 603h of the straight line portion 603H, and the end 603i of the straight line portion 603I correspond to the end. Whether or not it belongs to the determination region 701 means whether or not it is included in the determination region 701, and such a determination of whether or not it belongs to the determination region 701 is called a belonging / non-belonging determination. Therefore, the belonging / non-belonging determination unit 145 determines whether or not an end of an object different from the object having the end used as a reference in setting the determination region 701 is included in the determination region 701.

[0058] In the example of Fig. 6, the determination region 701 is set based on the end portion 601a, and it is determined whether the end portions 605b, 601c, 605d, 601e, 603f, 603g, 603h, and 603i detected by the end portion detection unit 142 are included in the determination region 701. The above-mentioned position information is also used in this determination. In the example of Fig. 6, the end portions 605b and 605d are not included in the determination region 701, but the other end portions, that is, the end portions 601c, 601e, 603f, 603g, 603h, and 603i are included in the determination region 701.

[0059] Such a determination of whether or not the object belongs is performed by setting a determination region based on each of the ends whose position information is stored in the storage unit 150. That is, the determination of whether or not the object belongs is not only performed based on the end 601a shown in Fig. 6, but also based on the determination region set based on each of the other ends, that is, the end 605b, the end 601c, the end 605d, the end 601e, the end 603f, the end 603g, the end 603h, and the end 603i.

[0060] After the belonging / non-belonging determination is performed for each end, the demarcation line determination unit 146 determines whether or not the feature having the end used to set the determination area is a demarcation line defining a parking space for the vehicle 1, based on the determination result of the belonging / non-belonging determination unit 145. The determination result of the belonging / non-belonging determination unit 145 is transmitted to the demarcation line determination unit 146 when the belonging / non-belonging determination unit 145 performs the belonging / non-belonging determination.

[0061] Here, in this embodiment, the demarcation line determination unit 146 determines that the features having the ends included in each other's determination area in the belonging / non-belonging determination of each end including the newly detected end are demarcation lines. That is, when the belonging / non-belonging determination is performed using the determination area based on the end 601a, the end 601c is included in the determination area, and when the belonging / non-belonging determination is performed using the determination area based on the end 601c, the end 601a is included in the determination area. Note that, naturally, the determination area based on the end 601a also includes the end 601a, and the determination area based on the end 601c also includes the end 601c. Therefore, the end 601a and the end 601c are included in each other's determination area. Therefore, the demarcation line determination unit 146 determines that the feature having the end 601a and the feature having the end 601c are features defining a parking space.

[0062] In this way, based on the result of the inclusion / inclusion determination based on all the ends, in the example of Fig. 6, the demarcation line determination unit 146 determines that the ends 601a, 601c, 601e, 603f, 603g, 603h, and 603i are features that define the parking area, that is, demarcation lines. The demarcation line determination unit 146 stores the features determined to be demarcation lines and the position information of their ends in the storage unit 150 as demarcation line information.

[0063] In this embodiment, each time a new edge is detected as the vehicle 1 travels, the inclusion / inclusion determination unit 145 determines whether edges detected so far and included in the detection area belong to the determination area set based on the newly detected edge, and also determines whether other edges of the edges detected so far belong to the determination area set based on each edge. The inclusion / inclusion determination unit 145 also performs inclusion / inclusion determination on edges that are not included in the detection area but are stored in the storage unit 20.

[0064] However, as shown in the example of Fig. 6, although it is clear that the features having the ends 601a, 601c, and 601e are demarcation lines defining a parking space, the features having the ends 603f, 603g, 603h, and 603i are all short distances between the lines and cannot be parking spaces. Therefore, these features are not demarcation lines defining a parking space, but are erroneously detected lines. That is, there are cases where erroneous detection occurs even in the above-mentioned demarcation line determination.

[0065] In this embodiment, the accuracy of the lane marking determination is improved by removing such erroneous detection lines from among the lane marks determined to be lane marks. For this reason, the ECU 140 of this embodiment further includes an erroneous detection line removal unit 147 as a removal unit.

[0066] Such erroneously detected lines are caused by road marking features. These road marking features are present in front of the parking space, that is, on the vehicle side before parking assistance begins. FIG. 7 is a diagram showing an example of a road marking. The vertical lines of the road markings 603 shown in FIGS. 7(a) to (d) may be judged as demarcation lines. These road markings 603 are all present in front of the demarcation lines 601 that, as a whole, define the parking space.

[0067] For this reason, the false positive line removal unit 147 of this embodiment determines that, among the features determined to be demarcation lines by the demarcation line determination unit 146, features that are located generally forward of other demarcation lines and their ends are not demarcation lines, and deletes and excludes them from the demarcation lines stored in the memory unit 150.

[0068] Specifically, the erroneous detection line removal unit 147 of the present embodiment sets a removal range. Here, the removal range is a range including a position including the vicinity of the tip of the demarcation line having the tip on the side farthest from the vehicle 1 before the start of the parking operation among the demarcation lines and including the forward position.

[0069] Then, the erroneous detection line removal unit 147 identifies the rear end of the feature determined to be a demarcation line by the demarcation line determination unit 146. Here, position information of the rear end is stored as demarcation line information in the storage unit 150, and the erroneous detection line removal unit 147 reads position information of the rear end of each demarcation line from the storage unit 150 to identify the rear end.

[0070] The erroneous detection line detection unit 147 determines whether the rear end of the feature is included in the removal range based on the position information of the rear end and the removal range. If the erroneous detection line detection unit 147 determines that the rear end is included in the removal range, it determines that the feature determined to be a demarcation line is not a demarcation line and deletes it from the demarcation line in the storage unit 150 to exclude it from the demarcation line. On the other hand, if the erroneous detection line detection unit 147 determines that the rear end is not included in the removal range, it treats the feature determined to be a demarcation line as a demarcation line as it is, and does not delete it from the demarcation line in the storage unit 150.

[0071] FIG. 8 is a diagram showing an example of the removal range and the demarcation line of this embodiment. In the example of FIG. 8, the demarcation line determination unit 146 determines that the features having each of the straight line portions 601A, 601B, 601C, 603D, 603E, and 603F are demarcation lines, and stores the position information and the like as demarcation lines in the storage unit 150. Among the determined demarcation lines 601A, 601B, 601C, 603D, 603E, and 603F, the demarcation lines having the tips on the side farthest from the vehicle 1 before the parking operation is started are these demarcation lines 601A, 601B, and 601C. Therefore, the erroneously detected line removal unit 147 sets the removal range 801 to the range from the vicinity of the tips 601a1, 601b1, and 601c1 of the demarcation lines 601A, 601B, and 601C, respectively, in front of them. FIG. 8 shows an example in which the removal range 801 is set to be the range from near the ends 601a1, 601b1, 601c1 of the partition lines 601A, 601B, 601C to the end of the vehicle 1's travel path 803 on the parking space side before the parking operation begins, but this is not limited to this.

[0072] 8, the erroneous detection line removal unit 147 determines whether the rear ends of the demarcation lines 601A, 601B, 601C, 603D, 603E, and 603F are included in the removal range 801. As shown in FIG. 8, the removal range 801 includes the rear end 603d2 of the demarcation line 603D, the rear end 603e2 of the demarcation line 603E, and the rear end 603f2 of the demarcation line 603F.

[0073] On the other hand, a rear end 601a2 of the demarcation line 601A is not included in the removal range 801. Moreover, a rear end 601b2 of the demarcation line 601B is not included in the removal range 801. Similarly, a rear end 601c2 of the demarcation line 601C is not included in the removal range 801.

[0074] Therefore, the erroneous detection line removal unit 147 determines that the marking lines 603D, 603E, and 603F are not marking lines, and removes them from the marking line information in the storage unit 150. On the other hand, the erroneous detection line removal unit 147 maintains its determination that the marking lines 601A, 601B, and 601C are marking lines.

[0075] Returning to Fig. 5, the route calculation unit 148 determines a target parking position based on a parking space defined by a partition line, and calculates a movement route from the current position of the vehicle 1 to the target parking position. The current position of the vehicle 1 is detected based on the outputs of the wheel speed sensor 22 and the steering angle sensor 19, etc. The vehicle 1 may be provided with a GPS (Global Positioning System) receiver not shown in Fig. 3 and may detect the current position by performing GPS positioning.

[0076] The guidance control unit 149 controls each unit of the vehicle 1 so that the vehicle 1 moves to the movement target position along the movement route. The display control unit 14d is as described above.

[0077] Next, the parking assistance process performed by the parking assistance system of this embodiment configured as above will be described below. Fig. 9 is a flowchart showing the procedure of the parking assistance process of this embodiment.

[0078] First, the ECU 140 acquires various input data necessary for parking assistance, such as acquisition of an image by the captured image acquisition unit 141 (S101). Then, the ECU 140 executes a demarcation line recognition process (S102). Here, the demarcation line recognition process is executed by the end detection unit 142, the position information calculation unit 143, the judgment area setting unit 144, the belonging / non-belonging judgment unit 145, and the demarcation line judgment unit 146 as described above, and the demarcation lines are identified for each captured image.

[0079] Next, the ECU 140 executes a lane line management process (S104). The lane line management process will now be described. Fig. 10 is a flowchart showing an example of the procedure of the lane line management process of this embodiment.

[0080] First, the lane line determination unit 146 merges the lane lines recognized for each captured image (S301). As a result, position information of all lane lines and their leading and trailing ends is stored as lane line information in the storage unit 150. At the beginning of the lane line management process, the user may be allowed to select a mode in which the process is to be performed on all lane lines or on some of the lane lines.

[0081] Next, the erroneous detection line removal unit 147 sets, as the removal range, a predetermined range from near the tip of the marking line having the tip on the side farthest from the vehicle 1 before the start of the parking operation among the marking lines to the front (S302). Then, the erroneous detection line removal unit 147 selects one marking line from among the marking lines (S303).

[0082] The erroneous detection line removal unit 147 identifies the rear end of the selected lane mark (S304). Next, the erroneous detection line removal unit 147 determines whether the rear end of the lane mark is included in the removal range based on the position information (S305). Then, if the rear end of the lane mark is included in the removal range (S305: Yes), the erroneous detection line removal unit 147 determines that the selected lane mark is a erroneous detection line (S306). Then, the erroneous detection line removal unit 147 removes the lane mark from the lane mark information in the memory unit 150 by deleting the position information of the lane mark (S307).

[0083] On the other hand, if it is determined in S305 that the rear end of the demarcation line is not included in the removal range (S305: No), the selected demarcation line is not a false detection line and is determined to be a demarcation line (S308).

[0084] Next, it is determined whether processing has been completed for all lane lines (S309), and if not (S309: No), the process returns to S303, the next lane line is selected, and the processes from S304 to S308 are executed. If processing has been completed for all lane lines in S309 (S309: Yes), the lane line management process ends.

[0085] 9, when the partition line management process is completed, the route calculation unit 148 calculates a target parking position based on the parking space defined by the partition lines, and calculates a route from the current position of the vehicle 1 to the target parking position (S104). Then, the guidance control unit 149 executes guidance control of the vehicle 1 along the calculated route (S105). This completes the parking assistance process.

[0086] Thus, in this embodiment, the erroneous detection line removal unit 147 determines that, among the features determined to be demarcation lines, features that are located generally forward of other demarcation lines are not demarcation lines and removes them from the list of demarcation lines. Therefore, this embodiment can prevent erroneous detection of features such as road marks that are located generally forward of other demarcation lines as demarcation lines, and can more appropriately detect demarcation lines that define the parking space for the vehicle 1. As a result, this embodiment can provide parking assistance at a more accurate parking target position.

[0087] In addition, in this embodiment, the erroneous detection line removal unit 147 determines whether the rear end of the feature determined to be a demarcation line is included in a removal range that is a range including the front from a position including the vicinity of the tip of the demarcation line whose tip is on the farthest side from the vehicle 1 before the start of the parking operation, and if the front and rear ends are included in the removal range, the feature determined to be a demarcation line is determined to be not a demarcation line and is removed from the demarcation lines. Therefore, according to this embodiment, it is possible to prevent erroneous detection of features such as road surface marks included in a range including the front from a position including the vicinity of the tip of the demarcation line whose tip is on the farthest side from the vehicle 1 before the start of the parking operation as a demarcation line, and to more appropriately detect the demarcation line that defines the parking space of the vehicle 1. As a result, according to this embodiment, parking assistance can be provided at a more accurate parking target position.

[0088] (Modification) In the above embodiment, the removal range was set to the range from near the tip of the demarcation line whose tip is furthest from the vehicle 1 before the parking operation begins to a position just before the path along which the vehicle will travel before the parking operation begins, but the removal range is not limited to this, as long as it is a range from near the tip of the demarcation line whose tip is furthest from the vehicle 1 before the parking operation begins to a position forward.

[0089] In this modification, the erroneous detection line removal unit 147 sets the determination area set by the determination area setting unit 144 as the removal range during the demarcation line recognition process. Here, it is preferable to set the removal range as the determination area based on the end (tip) of the demarcation line having the tip on the side farthest from the vehicle 1 before the start of the parking operation among the demarcation lines. Also, as described above, the determination area is set in a predetermined range forward (i.e., the side of the vehicle 1 before the start of the parking operation) and rearward (i.e., the side opposite the vehicle 1 before the start of the parking operation) based on the end of the feature. Therefore, it includes the range from the vicinity of the tip, which is the end of the demarcation line, forward, and also includes the range from the tip to the rear.

[0090] The configuration of the vehicle 1, the configuration of the parking assistance system, and the functional configuration of the ECU 140 in this modified example are similar to those in the above embodiment.

[0091] 11 is a flowchart showing an example of a procedure for a lane line management process in the modified example. First, as in the above embodiment, the lane line determination unit 146 merges the lane lines recognized for each captured image (S301), and stores the position information of all the lane lines and their leading and trailing ends in the storage unit 150 as lane line information.

[0092] Next, the erroneous detection line removal unit 147 sets the determination area set by the determination area setting unit 144 as the removal range based on the end (leading end) of the demarcation line that has its leading end on the side farthest from the vehicle 1 before the parking operation is started (S502). The subsequent processes from S303 to S309 are performed in the same manner as in the above embodiment. As a result, for each demarcation line, it is determined whether or not the leading end and trailing end are included within the determination area as the removal range, and the included demarcation lines are excluded from the demarcation lines.

[0093] 6, when the erroneous detection line removal unit 147 sets the determination area 701 as the removal range, the leading end and trailing end of each of the straight line portion 603F, the straight line portion 603G, the straight line portion 603H, and the straight line portion 603I are included in the determination area 701. Therefore, the erroneous detection line removal unit 147 removes the demarcation lines that include each of these straight lines from the demarcation lines.

[0094] In this manner, in this modification, during the demarcation line recognition process, the erroneous detection line removal unit 147 sets the determination area set by the determination area setting unit 144 as the removal range, and determines whether the leading end and trailing end of an object determined to be a demarcation line are included in the removal range, thereby determining whether or not it is a demarcation line. Therefore, according to this modification, it is possible to more appropriately detect the demarcation lines that define the parking space of the vehicle 1, while reducing the effort required for setting the removal range, and to provide parking assistance at a more accurate parking target position.

[0095] In addition, in this modification, the erroneous detection line removal unit 147 sets, as the removal range, a determination area set in a predetermined range in the front and rear with respect to the edge of the feature as a reference. Furthermore, in this modification, the erroneous detection line removal unit 147 sets, as the removal range, a determination area set in a predetermined range in the right and left directions with respect to the edge of the feature as a reference. Therefore, according to this modification, it is possible to prevent erroneous detection of not only the area in front of the vicinity of the tip of the demarcation line, but also the area in the rear, such as road surface marks, and the predetermined range in the right and left directions, as demarcation lines, and to more appropriately detect the demarcation lines that define the parking space of the vehicle 1. As a result, according to this embodiment, parking assistance can be provided at a more accurate parking target position.

[0096] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0097] 1 vehicle 14 ECU 14a CPU 14b ROM 14c RAM 14f SSD 15 Imaging unit 141 Image acquisition unit 142 End detection unit 143 Location Information Calculation Department 144 Judgment area setting section 145 Belonging judgment part 146 Lane line detection unit 147 False positive line removal section 148 Route Calculation Unit 149 Guidance control unit 150 Storage section 701 Judgment area 801 Removal Range 803 Driving Range

Claims

1. an image acquisition unit that acquires an image of a road surface around a vehicle; an edge detection unit that detects an edge of a feature attached to the road surface included in a detection area set in the captured image; a position information calculation unit that calculates position information indicating a positional relationship between an edge of the detected feature and the vehicle; a demarcation line determination unit that determines whether or not the feature having the end portion is a demarcation line that defines a parking space for the vehicle; a removal unit which determines whether the rear end of the feature determined to be the demarcation line by the demarcation line determination unit is included in a removal range which is a range including a front area which is an area closer to the vehicle than other demarcation lines before the start of parking assistance by determining whether the rear end of the feature determined to be the demarcation line by the demarcation line determination unit is included in a removal range which is a range including a front area which is an area closer to the vehicle than other demarcation lines before the start of parking assistance, and determines that the feature determined to be the demarcation line is an feature located overall forward of the other demarcation lines and is not the demarcation line if the rear end is included in the removal range; and A parking assistance device comprising:

2. a judgment area setting unit that sets a predetermined judgment area along the road surface in the captured image based on position information of an end of the lane marking; a belonging / non-belonging determination unit that determines whether an edge of another feature different from the feature having the edge used to set the judgment region belongs to the judgment region, the demarcation line determination unit determines whether or not the feature having the end portion used to set the determination area is a demarcation line defining a parking space for the vehicle based on a determination result of the inclusion / inclusion determination unit; the removal unit sets the determination area as the removal range, and determines whether or not a rear end of the feature determined to be the lane line is included in the removal range.

2. The parking assistance device according to claim 1.

3. a judgment area setting unit that sets a predetermined judgment area along the road surface in the captured image based on position information of an end of the lane marking, the determination area setting unit sets the determination area, which is set in a predetermined range in front of the edge of the feature on a side closer to the vehicle and in a predetermined range in rear of the edge of the feature on a side farther from the vehicle, as an elimination area.

2. The parking assistance device according to claim 1.

4. the determination area setting unit sets the determination area, which is set in a predetermined range in a direction perpendicular to the division line with an edge of the feature as a reference, as an removal area.

4. The parking assistance device according to claim 3.

Images