Lane boundary line detection method and lane boundary line detection device

Abstract

In the present invention, when detecting lane boundary lines of a road by using map information including a link line (K) representing a road, point sequences of feature points (A1 to A16, B1 to B16, C1 to C16, D1 to D7) of an object indicating the lane boundary lines of a road on which a vehicle is traveling, each of the point sequences being where an approximate line generated by linear approximation forms an angle not more than a prescribed angle with respect to the link line (K), are extracted from position information, based on the link line (K), about the feature points (A1 to A16, B1 to B16, C1 to C16, D1 to D7), and lane boundary lines corresponding to the point sequences and a lane boundary line corresponding to feature points not included in the point sequences are detected.

Description

Lane boundary line detection method and lane boundary line detection device

[0001] The present invention relates to a lane boundary line detection method and a lane boundary line detection device.

[0002] A lane marking recognition device is known that extracts feature points from images taken by multiple cameras installed so that their fields of view partially overlap, converts the coordinates of the extracted feature points from the coordinates in each camera's local coordinate system to coordinates in a world coordinate system common to the multiple cameras, rotates the feature points around the vehicle so that the row of converted feature points is parallel to the Y axis of the world coordinate system, which corresponds to the longitudinal direction of the vehicle, searches for an X coordinate on the X axis of the world coordinate system, which corresponds to the lateral direction of the vehicle, where the cumulative value of the edge strength of the rotated feature points is equal to or greater than a predetermined threshold and is a maximum, extracts white line candidates composed of feature points located within a predetermined width from the searched X coordinate, and determines that the white line candidate is a white line if the difference in X coordinates between the extracted white line candidates is equivalent to the lane width (Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2017-156795

[0004] At merging points where the number of lanes on a road decreases and at branching points where the number of lanes on a road increases, lane boundary lines extending in the width direction of the road are provided. However, with the above-mentioned conventional technology, feature points corresponding to lane boundary lines extending in the width direction of the road are not aligned along the Y-axis direction of the world coordinate system, and therefore are not extracted as lane boundary line candidates, resulting in a problem that lane boundary lines at merging points and branching points cannot be accurately detected.

[0005] An object of the present invention is to provide a method and apparatus for detecting lane boundaries that can accurately detect lane boundaries when the number of lanes on a road changes due to merging or branching of lanes.

[0006] The present invention solves the above problem by, when detecting lane boundary lines of a road using map information including link lines representing the road, extracting a sequence of feature points where an approximation line generated by linear approximation forms a minor angle with the link line less than a predetermined angle from position information of feature points of an object that indicates the lane boundary lines of the road on which a vehicle is traveling, based on the link line, and detecting lane boundary lines corresponding to the sequence of points and lane boundary lines corresponding to feature points not included in the sequence of points.

[0007] According to the present invention, lane boundary lines can be accurately detected when the number of lanes on a road changes due to merging or branching of lanes.

[0008] 5 is a block diagram showing an example of an embodiment of a driving assistance device including a lane boundary line detection device according to the present invention. FIG. 6 is a plan view showing an example of a driving scene in which driving assistance is performed by the driving assistance device of FIG. 1. FIG. 7 is a plan view showing an example of feature points extracted from the driving scene of FIG. 2. FIG. 8 is a plan view when the feature points shown in FIG. 3 are expressed in a local coordinate system along link lines. FIG. 9 is a plan view showing a first section, a second section, and a third section set for the feature points of FIG. 4. FIG. 10 is a plan view showing an example of a connecting line generated in the first section of FIG. 5. FIG. 11 is a flowchart showing an example of a processing procedure in the driving assistance device of FIG. 1. FIG. 12 is a flowchart showing another example of a processing procedure in the driving assistance device of FIG. 1.

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] [Configuration of Driving Assistance Device] Figure 1 is a block diagram showing an example of an embodiment of a driving assistance device including a lane boundary detection device according to the present invention. The driving assistance device generates a driving route from the current position to a destination set by the occupant through autonomous driving control, and controls the vehicle's actuators to drive the vehicle along the driving route. Autonomous driving control is the autonomous control of the vehicle's driving operations, and driving operations include all driving operations such as acceleration, deceleration, starting, stopping, and steering.

[0011] 1, a driving assistance device 10 includes a map database 11, a navigation device 12, an imaging device 13, a distance measuring device 14, and a control device 15. The control device 15 of this embodiment also includes a lane boundary detection device 20 as a part thereof. These devices are communicably connected via a Controller Area Network (CAN) or other in-vehicle LAN, and can exchange information with each other.

[0012] The map database 11 is a storage medium that stores map information and is provided inside or outside the vehicle. The control device 15 acquires map information from the map database 11 as needed. The map information includes information on nodes corresponding to specific points on roads (such as intersections) where the vehicle's direction of travel changes, and links corresponding to road sections connecting the nodes. The node information includes location information and information on entering and exiting intersections, and the link information includes road width, road curvature radius, roadside structures, road traffic regulations, etc. The map information may be high-precision map information that includes road information for each lane, as well as information on structures, signs, traffic lights, and the like around the road.

[0013] The navigation device 12 references map information acquired from the map database 11 and generates a driving route from the current position of the vehicle detected by a positioning system (not shown) to a destination set by the occupant. The driving route includes information on at least the road on which the vehicle is traveling, the driving lane, and the direction of travel of the vehicle, and is displayed, for example, as a linear diagram. The calculated driving route is acquired by the control device 15.

[0014] The imaging device 13 is a camera equipped with an imaging element such as a CCD, and captures images of objects around the vehicle to generate an image including the objects. The imaging device 13 may be an infrared camera, a stereo camera, or the like. In order to reduce blind spots where the objects cannot be captured, multiple imaging devices 13 are provided on the front grille, side mirrors, rear bumper, etc. of the vehicle.

[0015] The distance measuring device 14 detects the relative distance and relative speed between the vehicle and an object. The distance measuring device 14 includes a laser radar, a millimeter wave radar, a LiDAR (Light Detection and Ranging) unit, etc. A plurality of distance measuring devices 14 are provided on one vehicle to prevent blind spots where an object cannot be detected.

[0016] The objects detected by the imaging device 13 and the distance measuring device 14 are objects that exist on the road and its surroundings, including lane boundaries, center lines, road markings, median strips, road signs, etc. The objects also include obstacles that may affect the travel of the vehicle, such as other automobiles (other vehicles), motorcycles, bicycles, pedestrians, etc.

[0017] The control device 15 acquires image information from the imaging device 13 and acquires position information of the object from the distance measuring device 14, and recognizes the object and the driving environment around the vehicle. The control device 15 acquires information at predetermined time intervals (for example, every 0.1 to 1 millisecond). The control device 15 may recognize the driving environment by integrating or combining the information acquired from the imaging device 13 and the distance measuring device 14.

[0018] The ranging device 14 may generate point cloud data in which information about ranging points of surrounding objects is two-dimensionally arranged in the left-right and up-down directions of the vehicle. The ranging points of the object are points on the object whose distances to the ranging device 14 are measured. The information about the ranging points may include information about the reflectance of electromagnetic waves at the ranging points in addition to position information about the ranging points (information about the coordinates of the ranging points, information about the distance from the ranging device 14 to the ranging points, etc.).

[0019] The distance measuring device 14 generates point cloud data by scanning electromagnetic waves in the left-right direction of the vehicle. For example, the distance measuring device 14 emits electromagnetic waves along the vehicle width direction, detects the reflected electromagnetic waves (reflected waves), and calculates the distance to the distance measurement point and the direction of the distance measurement point. The irradiation range of the electromagnetic waves is not particularly limited. Examples of electromagnetic waves include millimeter waves, infrared rays, and lasers.

[0020] The control device 15 controls the devices that make up the driving assistance device 10 to cooperate with each other and perform driving assistance for the vehicle. The control device 15 autonomously controls the driving behavior of the vehicle within a predetermined range using devices mounted on the vehicle. Driving behavior that is not controlled by the control device 15 is manually operated by the driver. When the driver drives the vehicle manually, the control device 15 does not perform autonomous control of the driving behavior, and the driving behavior of the vehicle is controlled by operation by the driver.

[0021] The control device 15 is, for example, a computer, and includes a CPU (Central Processing Unit) which is a processor, a ROM (Read Only Memory) in which programs are stored, and a RAM (Random Access Memory) which functions as an accessible storage device. The CPU is an operating circuit for executing the programs stored in the ROM and realizing the functions of the control device 15. Note that an MPU (Micro Processing Unit), ASIC (Application Specific Integrated Circuit), or the like may be used instead of or together with the CPU.

[0022] The control device 15 includes a lane boundary line detection device 20 as part thereof. The lane boundary line detection device 20 of this embodiment detects lane boundary lines of a road using map information including link lines representing the road. The link lines representing the road are lines that indicate the shape of the road corresponding to the road section between nodes. The link lines are, for example, lines that run along the road and are located on the center line of the road in the width direction. Information about the link lines is included in the map information (link information), and one link line is set for each road section.

[0023] Nodes are placed on the link lines at predetermined distances (for example, every 1 to 3 meters). The nodes on the link lines have position information for the nodes. Examples of position information include coordinate information in the global coordinate system used in map information, information on the angle between a predetermined direction and the direction of the link line (direction of travel), and information on the travel distance from one node to another node on the link line. The one node that is the starting point is, for example, the node closest to the start point of the travel route.

[0024] Examples of the predetermined direction include the north direction on the map and the direction along the axes of the global coordinate system. The direction of the link line is, for example, the tangent direction of the link line at a node on the link line. The travel distance between nodes is calculated as the travel distance when traveling from one node to another node along the link line.

[0025] The control device 15 has a driving assistance function that performs driving assistance for the vehicle, and the lane boundary detection device 20 has a lane boundary detection function that detects lane boundary lines as part of the driving assistance function. The ROM stores programs for realizing the driving assistance function, and the CPU executes the programs stored in the ROM to realize the driving assistance function including the lane boundary line detection function.

[0026] 1 illustrates, for convenience, the driving control unit 16, the extraction unit 21, the setting unit 22, and the detection unit 23 as functional blocks that realize the driving assistance function. The extraction unit 21, the setting unit 22, and the detection unit 23 are functional blocks that realize the lane boundary line detection function, and are included in the lane boundary line detection device 20.

[0027] The driving control unit 16 generates a driving route for the vehicle and operates actuators of the vehicle so that the vehicle travels along the driving route. The driving control unit 16 generates control signals for operating the actuators and outputs them to the actuators. When performing driving assistance, the driving control unit 16 outputs an execution instruction to the lane boundary line detection device 20 to detect lane boundary lines.

[0028] When an execution instruction is input from the driving control unit 16, the extraction unit 21 acquires information on feature points of objects present around the vehicle. Feature points are characteristic parts of an object that are used to detect the object, such as edges that capture the object's outline, corners of the object, and parts of the object where the brightness changes. The extraction unit 21 acquires an image from the imaging device 13 and extracts feature points from the image using a known algorithm such as SIFT (Scale-Invariant Feature Transform).

[0029] The extraction unit 21 may also acquire point cloud data from the distance measuring device 14, generate an image in which distance measurement points are plotted from the point cloud data, and execute a process of extracting feature points from the generated image to acquire information about the feature points. The extraction unit 21 may also incorporate information about the reflectance of electromagnetic waves into the image generated from the point cloud data, and detect road markings such as lane boundaries from the difference in reflectance between the road surface and the road markings.

[0030] The setting unit 22 sets a section of the road for the detection unit 23 to accurately detect lane boundary lines, based on the information on the feature points acquired by the extraction unit 21. The setting unit 22 sets at least a first section where the number of lanes on the road changes (increases or decreases).

[0031] The detection unit 23 detects, for each section set by the setting unit 22, lane boundary lines that correspond to the feature points extracted by the extraction unit 21. A lane boundary line is a boundary line that separates lanes on a road, and is a type of road marking.

[0032] [Function of Lane Boundary Line Detection Device] Fig. 2 is a plan view showing an example of a driving scene in which driving assistance is performed by the driving assistance device 10. In the driving scene shown in Fig. 2, a road having lanes W1 and W2 extends in the X-axis direction (left-right direction in the drawing) of a global coordinate system used in the map information, and a vehicle V is traveling in the lane W1 in the positive direction of the X-axis (from left to right in the drawing).

[0033] Lane W1 is defined by a solid line L1 and a dashed line L2, and lane W2 is defined by a dashed line L2 and a solid line L3. Furthermore, lane W2 branches at branch point J into lane W2a defined by dashed lines L2 and L4 and lane W2b defined by solid line L3 and dashed line L4. The colors of the solid and dashed lines are not particularly limited and may be white, yellow (orange), green, blue, or the like. Furthermore, the coordinate system of the map information is not limited to the global coordinate system and may be any of various coordinate systems.

[0034] 2, when a destination (not shown) of the vehicle V is set ahead in the lane W1, the driving control unit 16 outputs an instruction to the lane boundary detection device 20 (extraction unit 21) to detect lane boundary lines around the vehicle V. When the detection result of the lane boundary lines is input from the lane boundary detection device 20 (detection unit 23), the driving control unit 16 generates a driving route based on the detection result and causes the vehicle V to drive along the generated driving route.

[0035] If the map information stored in the map database 11 is high-precision map information, the driving control unit 16 detects the solid lines L1 and dashed lines L2 that define the lane W1 from the lane information of the road. On the other hand, if the map information does not include lane information, the driving control unit 16 detects the solid lines L1 and dashed lines L2 from images acquired by the imaging device 13, etc. However, if lane boundary lines are detected only from image information without relying on map information, there is a risk that lane boundary lines extending in the width direction of the road will not be detected accurately, especially in portions of the road where the number of lanes increases or decreases. Therefore, the lane boundary line detection device 20 of this embodiment performs the following processing using the functions of the extraction unit 21, setting unit 22, and detection unit 23 to accurately detect lane boundary lines even when high-precision map information (lane information) is not available.

[0036] The extraction unit 21 acquires information about feature points of objects that indicate lane boundary lines of the road on which the vehicle is traveling from sensors mounted on the vehicle. Examples of sensors include the imaging device 13 and the distance measuring device 14. The objects that indicate lane boundary lines include white solid lines, yellow solid lines, white dashed lines, etc., and the information about the feature points of the objects includes position information of the edges of the white solid lines, information about the width of the yellow solid lines, information about the length of the white dashed lines in the traveling direction, etc.

[0037] Fig. 3 is a plan view showing an example of feature points extracted from an image captured by the image capture device 13 in the driving scene shown in Fig. 2. In the driving scene shown in Fig. 2, as shown in Fig. 3, feature points A1 to A16 are extracted for the solid line L1, feature points B1 to B16 are extracted for the dashed line L2, feature points C1 to C16 are extracted for the solid line L3, and feature points D1 to D7 are extracted for the dashed line L4. Each extracted feature point is arranged on the XY plane of the global coordinate system based on the coordinate information of the feature point. Note that the number of feature points and the spacing between feature points can be set to any appropriate value within a range in which lane boundary lines can be accurately detected.

[0038] The extraction unit 21 acquires position information based on the link line from among the position information of the extracted feature points. The position information of the feature points includes information on the coordinates of the feature points in an arbitrary coordinate system, information on the distance and direction of one feature point relative to another feature point, etc. On the other hand, the position information of the feature points based on the link line includes, but is not limited to, the distance between the feature point and the link line (or node), the direction of the link line (or node) relative to the feature point, etc. The position information of the feature points based on the link line may include position information of the feature points in a local coordinate system along the link line.

[0039] As an example, the extraction unit 21 acquires position information in a coordinate system (hereinafter also referred to as an original coordinate system) of information used to extract the feature points. When the feature points are extracted from an image, the original coordinate system is the coordinate system used in the image, and when the feature points are extracted from point cloud data, the original coordinate system is the coordinate system used in the point cloud data. Next, the extraction unit 21 converts the position information in the original coordinate system into position information in a global coordinate system. For example, the extraction unit 21 converts the original coordinate system into the global coordinate system using the relative positions of the feature points with respect to the vehicle in the original coordinate system.

[0040] Next, the extraction unit 21 converts the position information in the global coordinate system into position information in a local coordinate system and acquires position information of the feature point in the local coordinate system along the link line. The local coordinate system along the link line is a coordinate system in which the position information of the feature point is expressed by the distance from the link line and the traveling distance from a predetermined node. The extraction unit 21 associates the feature point with the closest node on the link line and rotates the feature point around the associated node by the angle formed by the predetermined direction and the direction of the link line. Furthermore, the extraction unit 21 moves the rotated feature point along the link line by the traveling distance of the node at the center of rotation.

[0041] Link line K shown in Fig. 3 is an example of a link line set for the road shown in Fig. 2. Link line K is a line that follows the road and indicates the curvature of the road, etc. Nodes N1 to N18 are provided on link line K, and each of nodes N1 to N18 has information on its coordinates in the global coordinate system, information on the angle formed between the X-axis direction and the tangent direction of link line K, information on the travel distance from node N1, which is the starting point, and the like.

[0042] 3, the extraction unit 21 associates each feature point with a node on the link line K. For example, feature points A1, B1, and C1 are associated with node N1, feature points A7, B7, and C7 are associated with node N8, and feature points A10, B10, C10, and D1 are associated with node N11. Other feature points are also associated with the closest node.

[0043] Next, the extraction unit 21 rotates the feature points associated with each node by the angle formed between a predetermined direction and the direction of the link line, using information on the angle held by the node. The extraction unit 21 also moves each rotated feature point along the link line K using information on the travel distance held by the associated node.

[0044] The process of converting to the local coordinate system will be specifically described using the example of feature point B10 associated with node N11. If the angle (minor angle) between the X-axis direction and the tangent direction of link line K at node N11 is θ, then the direction (travel direction) R of link line K at node N11 is expressed by the following equation (1):

[0045]

[0046] Here, x ａ and y ａ are the X and Y coordinates of the node N11 in the global coordinate system. In this case, the coordinates P of the feature point B10 in the global coordinate system ｒａｗ is expressed by the following equation (2), the coordinates P of the feature point B10 in the global coordinate system are ｒａｗ is rotated counterclockwise by an angle θ ｎｏｄｅ is expressed by the following equation (3).

[0047]

[0048] Here, x ｂ and y ｂ are the X and Y coordinates of the feature point B10 in the global coordinate system. ｎｏｄｅ coordinate P ｗａｙ is expressed by the following equation (4).

[0049]

[0050] The travel distance s is the travel distance from node N1 to node N11.

[0051] The extraction unit 21 performs the above-described series of processes on each feature point shown in Fig. 3 to convert the coordinates of the feature point in the global coordinate system shown in Fig. 3 into coordinates in the local coordinate system shown in Fig. 4. The vertical axis of the local coordinate system in Fig. 4 represents the distance d between the feature point and the link line K, and the horizontal axis represents the traveling distance s from node N1.

[0052] The extraction unit 21 extracts a sequence of feature points (hereinafter simply referred to as a sequence of points) where an approximation line generated by linear approximation forms a minor angle with the link line that is less than a predetermined angle, based on the position information of the extracted feature points relative to the link line. The predetermined angle can be set to an appropriate value within a range in which lane boundary lines can be accurately detected, for example, 1 to 10°. To extract the sequence of points, a clustering method such as the K-means method or the Ward method is used. Alternatively, the sequence of points may be extracted using a trained model that has been trained to extract a sequence of points from the position information of the extracted feature points relative to the link line.

[0053] 4, the extraction unit 21 extracts the sequence of points consisting of feature points A1 to A16 when an approximation line generated by linear approximation from the sequence of points consisting of feature points A1 to A16 forms a minor angle with the link line K that is less than a predetermined angle. For the sequence of points consisting of feature points B1 to B16, the sequence of points consisting of feature points C1 to C7, the sequence of points consisting of feature points D1 to D7, and the sequence of points consisting of feature points C12 to C16, when an approximation line generated from the sequence of points by linear approximation forms a minor angle with the link line K that is less than a predetermined angle, the extraction unit 21 also extracts each of the sequence of points.

[0054] The detection unit 23 detects lane boundary lines corresponding to the point sequence and lane boundary lines corresponding to feature points not included in the point sequence. For each point sequence, the detection unit 23 associates the feature points that make up the point sequence and detects lane boundary lines corresponding to the associated feature points. The detection unit 23 also associates feature points that are not included in the point sequence and detects lane boundary lines corresponding to the associated feature points. Information on the detected lane boundary lines is output to the driving control unit 16.

[0055] In the example shown in Fig. 4, when a point sequence consisting of feature points A1 to A16 is extracted, the detection unit 23 associates the feature points A1 to A16 that make up the point sequence, and converts the coordinate system of the feature points A1 to A16 from the local coordinate system shown in Fig. 4 to the global coordinate system shown in Fig. 2. The detection unit 23 then estimates the shape of the lane boundary lines corresponding to the feature points A1 to A16 using a known fitting method. The detection unit 23 also associates the feature points of the other extracted point sequences, and estimates the shape of the lane boundary lines corresponding to the feature points.

[0056] When extracting the sequence of points, the extraction unit 21 may use a group of reference lines, which are a plurality of lines that form a minor angle with the link line that is less than a predetermined angle, with adjacent lines spaced a distance corresponding to the lane width of the road. The group of reference lines can be set by approximation lines generated by linear approximation from the sequence of feature points. As an example, FIG. 4 shows a group of reference lines that are made up of a plurality of parallel lines X1 to X4 that form a minor angle α with the link line K. The minor angle α is less than a predetermined angle. In the example shown in FIG. 4 , for the sequence of feature points A1 to A16, the sequence of feature points B1 to B16, the sequence of feature points C1 to C7, the sequence of feature points D1 to D7, and the sequence of feature points C12 to C16, the approximation lines generated from the sequence of points by linear approximation form minor angles with the link line K that are less than a predetermined angle. Therefore, for example, the minor angle α may be the average angle of the minor angles that each of the approximation lines forms with the link line K. Furthermore, the distance between the lines X1 and X2, the distance between the lines X2 and X3, and the distance between the lines X3 and X4 correspond to the lane widths of the lanes W1, W2, W2a, and W2b. As another example, the group of reference lines may be set as approximation lines generated by linear approximation from the sequence of characteristic points. In other words, as long as adjacent approximation lines are separated by a distance corresponding to the lane width of the road, the approximation lines may be set as the lines X1, X2, X3, and X4, respectively.

[0057] The extraction unit 21 sets predetermined sections along the link lines, and calculates a first degree of conformance, which indicates the proportion of feature points that are within a first predetermined distance from one of the reference straight lines in the width direction of the road (hereinafter simply referred to as the width direction), for each predetermined section. The predetermined section is set within a range in which the first degree of conformance can be appropriately calculated according to the processing capacity of the control device 15, and the length of one predetermined section is, for example, 2 to 5 m. The first predetermined distance can be set to an appropriate value within a range in which a point sequence can be appropriately extracted from the feature points, for example, 0.1 to 1 m.

[0058] In the example shown in Figure 4, the extraction unit 21 sets predetermined sections Y1 to Y10 along the link line K, calculates the number of feature points within a first predetermined distance in the width direction from the lines X1 to X4 for each predetermined section, and calculates the proportion of feature points within the predetermined section that are within the first predetermined distance in the width direction from one line of the group of reference lines. In this case, the extraction unit 21 adjusts the positions of the group of reference lines so that a first degree of fit equal to or greater than a fourth predetermined value is calculated for multiple predetermined sections. The fourth predetermined value can be set as appropriate within a range in which lane boundary lines can be accurately detected, and is, for example, 0.8 to 0.95.

[0059] 4 shows six feature points A1, A2, B1, B2, C1, and C2 in the predetermined section Y1. In this case, if the feature points A1 and A2 are within a range of a first predetermined distance in the width direction from the line X1, the feature points B1 and B2 are within a range of a first predetermined distance in the width direction from the line X2, and the feature points C1 and C2 are within a range of a first predetermined distance in the width direction from the line X3, the first goodness of fit for the predetermined section Y1 is calculated to be 1.

[0060] On the other hand, there are three feature points A8, B8, and C8 in the predetermined section Y5. In this case, if feature point A8 is within a range of the first predetermined distance in the width direction from line X1, feature point B8 is within a range of the first predetermined distance in the width direction from line X2, and feature point C8, which is located away from line X3, is not within a range of the first predetermined distance in the width direction from line X3, the first goodness of fit for the predetermined section Y5 is calculated to be 0.67.

[0061] If, among the feature points shown in FIG. 4, feature points other than feature points C8 to C11 are within a range of a first predetermined distance in the width direction from one straight line of the group of reference straight lines, the extraction unit 21 calculates the first conformance as 1 in the predetermined sections Y1 to Y4 and Y8 to Y10, calculates the first conformance as 0.67 in the predetermined section Y5, calculates the first conformance as 0.71 in the predetermined section Y6, and calculates the first conformance as 0.75 in the predetermined section Y7.

[0062] The setting unit 22 may set the first section based on the first degree of conformance, or may set a predetermined section for which the calculated first degree of conformance is less than a fourth predetermined value as the first section. When the position information of the feature points is expressed based on the link line, feature points corresponding to lane boundary lines extending in the traveling direction of the road are included in the point sequence, but feature points corresponding to lane boundary lines extending in the width direction of the road are not included in the point sequence. In sections where the number of lanes on a road changes, lane boundary lines extending in the width direction of the road are provided, and therefore the value of the first degree of conformance is calculated to be relatively low.

[0063] Fig. 5 is a plan view showing first sections set for the feature points in Fig. 4. In the example shown in Fig. 4, if the first conformance of the predetermined sections Y1 to Y4 and Y8 to Y10 is calculated to be 1, the first conformance of the predetermined section Y5 is calculated to be 0.67, the first conformance of the predetermined section Y6 is calculated to be 0.71, and the first conformance of the predetermined section Y7 is calculated to be 0.75, and if the fourth predetermined value is set to 0.8, the setting unit 22 sets the first section Z1 at a position corresponding to the predetermined sections Y5 to Y7 shown in Fig. 4, as shown in Fig. 5.

[0064] The setting unit 22 may set a second section that is adjacent to the first section along the link line and located closer to the first section in the direction of travel of the vehicle, and a third section that is adjacent to the first section along the link line and located further back than the first section in the direction of travel of the vehicle. The lengths of the second and third sections can be set appropriately depending on the processing capacity of the control device 15.

[0065] 5, a second section Z2 is set along the link line K, adjacent to the first section Z1, and located closer to the first section Z1 in the traveling direction of the vehicle V (closer to the vehicle V), and a third section Z3 is set along the link line K, adjacent to the first section Z1, and located further from the first section Z1 in the traveling direction of the vehicle V (farther from the vehicle V). The second section Z2 and the third section Z3 correspond to the predetermined sections Y1 to Y4 and the predetermined sections Y8 to Y10 shown in FIG. 4, respectively.

[0066] The detection unit 23 may generate a line segment corresponding to a first section, a line segment corresponding to a second section, and a line segment corresponding to a third section for each straight line in the group of reference straight lines, and calculate a second degree of conformance for each generated line segment, indicating the number of feature points within a second predetermined distance in the width direction from the line segment. The second predetermined distance may be set to an appropriate value within a range in which lane boundary lines can be accurately detected, for example, 0.1 to 1 m. The detection unit 23 detects lane boundary lines corresponding to line segments for which the second degree of conformance is calculated to be equal to or greater than a first predetermined value. On the other hand, the detection unit 23 does not detect lane boundary lines corresponding to line segments for which the second degree of conformance is calculated to be less than the first predetermined value. The first predetermined value may be set to an appropriate value within a range in which lane boundary lines can be accurately detected, for example, 1 to 5.

[0067] 5 , the detection unit 23 divides the lines X1 to X4 of the reference line group into sections to generate line segments. For example, the detection unit 23 generates, from the line X1, a line segment X1a corresponding to the second section Z2, a line segment X1b corresponding to the first section Z1, and a line segment X1c corresponding to the third section Z3, and calculates the number of feature points that are within a second predetermined distance in the width direction from each of the line segments X1a, X1b, and X1c. If the feature points A1 to A7 are within the second predetermined distance in the width direction from the line segment X1b, the feature points A8 to A11 are within the second predetermined distance in the width direction from the line segment X1a, and the feature points A12 to A16 are within the second predetermined distance in the width direction from the line segment X1c, the second conformance degrees of the line segments X1a, X1b, and X1c are calculated to be 7, 4, and 5, respectively.

[0068] 5, if the feature points other than feature points C8 to C11 are within a range of a second predetermined distance in the width direction from one straight line of the group of reference straight lines, the detection unit 23 calculates the second conformance of line segments X1a, X2a, and X3a to be 7, the second conformance of line segments X1b and X2b to be 4, and the second conformance of line segments X1c, X2c, X3c, and X4c to be 5. On the other hand, the detection unit 23 calculates the second conformance of line segment X3b to be 2, and the second conformance of line segments X4a and X4b to be 0.

[0069] When the first predetermined value is set to 3, the detection unit 23 detects lane boundary lines corresponding to the line segments X1a to X3a, X1b, X2b, and X1c to X4c, but does not detect lane boundary lines corresponding to the line segments X4a, X3b, and X4b. As shown in Fig. 6, the detection unit 23 deletes the line segments X4a, X3b, and X4b whose second conformance degree is less than 3, associates feature points within a second predetermined distance in the width direction from the line segment for each of the remaining line segments X1a to X3a, X1b, X2b, and X1c to X4c, and estimates the shape of the lane boundary lines from the associated feature points.

[0070] The detection unit 23 may generate a connecting line in the first section that corresponds to a feature point not included in the sequence of points and connects a line segment in the second section to a line segment in the third section. For example, the detection unit 23 generates an approximation line by linear approximation from a feature point not included in the sequence of points in the first section, and corrects the intercept and / or slope of the generated approximation line so that a portion of the generated approximation line that corresponds to the first section connects a line segment in the second section to a line segment in the third section.

[0071] FIG. 6 is a plan view showing an example of a connection line X5 generated in the first section Z1 of FIG. 5 . The connection line X5 corresponds to feature points C8 to C11 that are not included in the point sequence, and connects the line segment X3a and the line segment X4c. The detection unit 23 generates an approximation line from the feature points C8 to C11 by linear approximation, and corrects the intercept and slope of the approximation line so as to connect the end of the line segment X3a on the first section Z1 side with the end of the line segment X4c on the first section Z1 side. The end of the line segment X3a on the first section Z1 side is closest to the connection line X5 among the end portions of the line segments in the second section Z2, and the end of the line segment X4c on the first section Z1 side is closest to the connection line X5 among the end portions of the line segments in the third section Z3.

[0072] The detection unit 23 detects the lane boundary line corresponding to the connecting line. For example, the detection unit 23 estimates the shape of the lane boundary line of a portion Jx of the solid line L3 shown in FIG. 2 that extends in the width direction of the road and forward of the branch point J, from an equation representing the connecting line X5. The detection unit 23 may also associate feature points within a range of a second predetermined distance in the width direction from the connecting line X5, and estimate the shape of the lane boundary line from the associated feature points.

[0073] So far, we have explained the case where the number of lanes on a road increases due to lane branching, but the lane boundary detection device 20 can also perform processing similar to that described above when the number of lanes on a road decreases due to lane merging.

[0074] The extraction unit 21 may calculate a lower first degree of conformance as the number of feature points outside a first predetermined distance in the width direction from one straight line of the group of reference straight lines increases. Furthermore, if the angle formed by the link lines is equal to or greater than a predetermined angle, the extraction unit 21 may divide the predetermined section at a position where the angle is equal to or greater than the predetermined angle. The predetermined angle can be set to an appropriate value within a range that can accurately represent the position information of the feature points relative to the link lines, for example, 45 to 90 degrees.

[0075] When the difference in the first conformance between adjacent predetermined sections is equal to or greater than a second predetermined value, the setting unit 22 may set one of the adjacent predetermined sections as the first section and the other of the adjacent predetermined sections as the second section or the third section. The second predetermined value can be set to an appropriate value within a range in which the first section can be appropriately set, for example, 0.1 to 0.3. As an example, since the difference in the first conformance between the predetermined section Y4 and the predetermined section Y5 shown in FIG. 4 is 0.33, if the second predetermined value is set to 0.3, the setting unit 22 sets the predetermined section Y4 as the second section and the predetermined section Y5 as the first section.

[0076] The detection unit 23 may calculate a lower second conformance for a line segment as the number of feature points within a second predetermined distance in the width direction from the line segment decreases. Furthermore, if feature points are labeled as dashed lane boundary lines by an image analysis method such as semantic segmentation, the detection unit 23 may calculate a higher second conformance than if feature points are labeled as solid lane boundary lines. This is because fewer feature points are extracted from dashed lane boundary lines than from solid lane boundary lines.

[0077] The detection unit 23 may recognize the section between the second section and the third section as the first section if the number of line segments in the second section whose second conformance is equal to or greater than the first predetermined value differs from the number of line segments in the third section whose second conformance is equal to or greater than the first predetermined value. Furthermore, when generating a connecting line in the first section, the detection unit 23 may generate the connecting line so that a third conformance indicating the number of feature points within a third predetermined distance from the connecting line is equal to or greater than a third predetermined value. The third predetermined distance can be set to an appropriate value within a range in which lane boundary lines can be accurately detected, for example, 0.1 to 1 m. The third predetermined value can be set to an appropriate value within a range in which lane boundary lines can be accurately detected, for example, 1 to 5.

[0078] 7 and 8, the procedure for processing information by the control device 15 will be described. The process described below is executed by a processor (CPU) included in the control device 15 at predetermined time intervals (for example, every 0.1 to 1 millisecond).

[0079] FIG. 7 is an example of a flowchart showing information processing executed by the driving assistance device 10.

[0080] First, in step S1, the control device 15 extracts a sequence of feature points where the approximation line forms a minor angle with the link line that is less than a predetermined angle, and in step S2, detects lane boundary lines corresponding to the sequence of points and lane boundary lines corresponding to feature points not included in the sequence of points. In the following step S3, the control device 15 performs driving assistance using the detected lane boundary lines.

[0081] Next, FIG. 8 is a flowchart showing another example of information processing executed by the driving assistance device 10. In FIG.

[0082] First, in step S11, the control device 15 extracts feature points from the image and / or point cloud data. In step S12, the control device 15 acquires the coordinates of the feature points in the original coordinate system of the image and / or point cloud data. In the subsequent step S13, the control device 15 converts the coordinate system of the acquired coordinates from the original coordinate system to a global coordinate system. In the subsequent step S14, the control device 15 associates each feature point with the nearest node on the link line Z. In step S15, the control device 15 converts the coordinate system of the feature points from the global coordinate system to a local coordinate system. In the subsequent step S16, the control device 15 acquires the coordinates of the feature points in the local coordinate system.

[0083] In step S17, the control device 15 adjusts the position of the group of reference straight lines, and then in step S18, calculates the first degree of conformance for each predetermined section. In step S19, the control device 15 determines whether the first degree of conformance has changed by more than a second predetermined value between adjacent predetermined sections. If it is determined that the first degree of conformance has changed by less than the second predetermined value between adjacent predetermined sections, the process proceeds to step S18. On the other hand, if it is determined that the first degree of conformance has changed by more than the second predetermined value between adjacent predetermined sections, the process proceeds to step S20.

[0084] In step S20, the control device 15 determines whether or not this is the first time that the first conformance has changed by more than the second predetermined value in the adjacent predetermined section. If it is determined that this is the first time that the first conformance has changed by more than the second predetermined value in the adjacent predetermined section, the process proceeds to step S18. On the other hand, if it is determined that this is not the first time that the first conformance has changed by more than the second predetermined value in the adjacent predetermined section (i.e., it is the second time), the process proceeds to step S21.

[0085] In step S21, the control device 15 sets a first section, a second section, and a third section, and then in step S22, calculates a second degree of conformance for each line segment generated for each section. In step S23, the control device 15 deletes line segments whose second degree of conformance is less than a first predetermined value. In step S24, the control device 15 generates connecting lines connecting the line segments of the second section and the third section. In step S25, the control device 15 associates feature points with each remaining line segment and connecting line, and in step S26, converts the coordinate system from the local coordinate system to the global coordinate system. Then, in step S27, the control device 15 performs fitting for each associated feature point and estimates the shape of the lane boundary line.

[0086] According to this embodiment, when detecting lane boundary lines of a road using map information including link lines representing roads, a lane boundary line detection method and lane boundary line detection device 20 are provided that extract a point sequence of feature points where an approximation line generated by linear approximation forms a minor angle with the link line less than a predetermined angle from position information of feature points of an object that indicates lane boundary lines of the road on which a vehicle is traveling, based on the link line, and detect lane boundary lines corresponding to the point sequence and lane boundary lines corresponding to feature points not included in the point sequence. This makes it possible to accurately detect lane boundary lines when the number of lanes on a road changes due to merging or branching of lanes.

[0087] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, the position information of the feature points with respect to the link line is the position information of the feature points in a local coordinate system along the link line, which allows for more accurate detection of lane boundary lines when the number of lanes on a road changes due to merging or branching of lanes.

[0088] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, when extracting a sequence of points, a first compatibility score is calculated for each predetermined section along the link line, indicating the proportion of feature points that are within a first predetermined distance in the width direction of the road from one of a group of reference lines, which are multiple straight lines that form a minor angle with the link line less than a predetermined angle and whose adjacent straight lines are separated by a distance corresponding to the lane width of the road.The method and device then set a first section where the number of lanes on the road increases or decreases based on the first compatibility score.This enables more accurate detection of lane boundary lines when the number of lanes on a road changes due to merging or branching lanes.

[0089] In this embodiment, the lane boundary detection method and lane boundary detection device 20 define a second section that is adjacent to the first section and located closer to the vehicle in the direction of travel, along the link line, and a third section that is adjacent to the first section and located further back than the first section in the direction of travel. For each line in the group of reference lines, a line segment corresponding to the first section, a line segment corresponding to the second section, and a line segment corresponding to the third section are generated. A second compatibility score is calculated for each generated line segment, indicating the number of feature points located within a second predetermined distance in the road width direction from the line segment. A connecting line is generated in the first section that corresponds to a feature point not included in the point sequence, connecting a line segment in the second section to a line segment in the third section. The lane boundary lines corresponding to the line segments with a second compatibility score equal to or greater than the first predetermined value are detected. This allows for more accurate detection of lane boundaries when the number of lanes on a road changes due to merging or branching lanes.

[0090] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, the greater the number of feature points that are outside the range of the first predetermined distance in the road width direction from one of the reference straight lines, the lower the calculated first conformance. This makes it possible to identify the range where lane boundary lines that do not align with link lines exist.

[0091] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, if the first compatibility score between adjacent predetermined sections differs by a second predetermined value or more, one of the adjacent predetermined sections is set as the first section, and the other of the adjacent predetermined sections is set as the second section or the third section. This makes it possible to prevent unnecessary division of sections.

[0092] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, the fewer the number of feature points within a second predetermined distance from the line segment in the road width direction, the lower the calculated second compatibility score is. This makes it possible to exclude lane boundary lines that do not match the feature points.

[0093] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, when a feature point is labeled as a dashed lane boundary line, the second compatibility is calculated to be higher than when the feature point is labeled as a solid lane boundary line, thereby making it possible to prevent a situation in which a dashed line is not detected.

[0094] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, if the number of line segments in the second section whose second conformance is equal to or greater than the first predetermined value differs from the number of line segments in the third section whose second conformance is equal to or greater than the first predetermined value, the section between the second and third sections is recognized as the first section, and when generating connecting lines in the first section, the connecting lines are generated so that the third conformance, which indicates the number of feature points within a third predetermined distance from the connecting line, is equal to or greater than the third predetermined value. This makes it possible to capture changes in the shape of lane boundary lines in the first section.

[0095] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, if the angle between the link lines is equal to or greater than a predetermined angle, the predetermined section is separated at the position where the angle is equal to or greater than the predetermined angle, thereby enabling the first degree of conformance to be calculated with high accuracy.

[0096] 10... driving assistance device, 11... map database, 12... navigation device, 13... imaging device, 14... ranging device, 15... control device, 16... driving control unit, 20... lane boundary detection device, 21... extraction unit, 22... setting unit, 23... detection unit, A1 to A16, B1 to B16, C1 to C16, D1 to D7... feature points, J... branch position, Jx... portion, K... link line, L1, L3... solid line, L2, L4... dashed line, N1 to N18... node, V... vehicle, W1, W2, W2a, W2b... lane, X1 to X4... straight line, X1a to X4a, X1b to X4b, X1c to X4c... line segment, X5... connecting line, Y1 to Y10... specified section, Z1... first section, Z2... second section, Z3... third section

Claims

1. A lane boundary line detection method executed by a lane boundary line detection device that detects lane boundary lines of a road using map information including link lines that represent the road, wherein the lane boundary line detection device extracts a point sequence of feature points of an object that indicates the lane boundary lines of the road on which a vehicle is traveling, based on position information of the feature points relative to the link lines, where the approximation line generated by linear approximation forms a minor angle with the link lines that is less than a predetermined angle, and detects the lane boundary lines that correspond to the point sequence and the lane boundary lines that correspond to the feature points that are not included in the point sequence.

2. A lane boundary line detection method according to claim 1, wherein the position information of the feature points with respect to the link line is the position information of the feature points in a local coordinate system along the link line.

3. The lane boundary line detection method according to claim 1 or 2, wherein when extracting the sequence of points, the lane boundary line detection device calculates a first compatibility score for each specified section along the link line, the first compatibility score indicating the proportion of feature points that are within a first specified distance in the width direction of the road from one of a group of reference straight lines, which are a plurality of straight lines that form a minor angle with the link line that is less than the specified angle and in which adjacent straight lines are separated by a distance corresponding to the lane width of the road, and sets a first section in which the number of lanes on the road increases or decreases based on the first compatibility score.

4. The lane boundary line detection method according to claim 3, wherein the lane boundary line detection device sets a second section that is along the link line, adjacent to the first section, and located closer to the first section in the direction of travel of the vehicle, and a third section that is along the link line, adjacent to the first section, and located further back than the first section in the direction of travel; generates, for each straight line in the group of reference straight lines, a line segment that corresponds to the first section, a line segment that corresponds to the second section, and a line segment that corresponds to the third section; calculates, for each generated line segment, a second compatibility score that indicates the number of feature points that are within a second predetermined distance from the line segment in the width direction of the road; generates, in the first section, a connection line that corresponds to the feature point that is not included in the point sequence, and that connects one line segment in the second section to one line segment in the third section; and detects the lane boundary lines that correspond to the line segments whose second compatibility score is equal to or greater than a first predetermined value, and the lane boundary lines that correspond to the connection line.

5. A lane boundary line detection method as described in claim 3 or 4, wherein the lane boundary line detection device calculates the first degree of conformance to be lower the greater the number of feature points that are outside the range of the first predetermined distance in the width direction from one of the reference straight lines.

6. A lane boundary line detection method as described in claim 4, wherein, if the first conformance differs by a second predetermined value or more between adjacent specified sections, the lane boundary line detection device sets one of the adjacent specified sections as the first section and sets the other of the adjacent specified sections as the second section or the third section.

7. A lane boundary line detection method as described in claim 4 or 6, wherein the lane boundary line detection device calculates the second conformance to be lower the fewer the number of feature points within the second predetermined distance range from the line segment in the width direction.

8. A lane boundary line detection method according to any one of claims 4, 6 and 7, wherein the lane boundary line detection device calculates the second degree of conformance to be higher when the feature point is labeled as the lane boundary line with a dashed line than when the feature point is labeled as the lane boundary line with a solid line.

9. The lane boundary line detection method according to any one of claims 4 and 6 to 8, wherein, if the number of line segments in the second section whose second conformance is equal to or greater than the first predetermined value differs from the number of line segments in the third section whose second conformance is equal to or greater than the first predetermined value, the lane boundary line detection device recognizes the section between the second section and the third section as the first section, and when generating the connecting line in the first section, generates the connecting line so that a third conformance indicating the number of feature points within a third predetermined distance from the connecting line is equal to or greater than a third predetermined value.

10. A lane boundary line detection method according to any one of claims 3 to 9, wherein, when the angle formed by the link lines is equal to or greater than a predetermined angle, the lane boundary line detection device divides the predetermined section at a position where the angle is equal to or greater than the predetermined angle.

11. A lane boundary line detection device that detects lane boundary lines of a road using map information including link lines representing the road, comprising: an extraction unit that extracts a sequence of feature points of an object that indicates the lane boundary lines of the road on which a vehicle is traveling, based on position information of the feature points relative to the link lines, and extracts an approximate line generated by linear approximation that forms a minor angle with the link lines that is less than a predetermined angle; and a detection unit that detects the lane boundary lines that correspond to the sequence of points and the lane boundary lines that correspond to the feature points that are not included in the sequence of points.

Images