Lane boundary line detection method and lane boundary line detection device
Patent Information
- Application Number
- PCT/JP2025/005468
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-02-18
- Publication Date
- 2026-08-27
Smart Images
Figure JP2025005468_27082026_PF_FP_ABST
Abstract
Description
Lane boundary line detection method and lane boundary line detection device
[0001] The present invention relates to a method for detecting lane boundary lines and a device for detecting lane boundary lines.
[0002] A lane marking recognition device is known that extracts feature points from images of multiple cameras installed so that a portion of their fields of view overlap, converts the coordinates of the extracted feature points from the local coordinate system coordinates of each camera to the world coordinate system coordinates common to the multiple cameras, rotates the feature points around the vehicle so that the converted sequence of feature points is parallel to the Y-axis of the world coordinate system corresponding to the vehicle's longitudinal direction, searches for an X-coordinate on the X-axis of the world coordinate system corresponding to the vehicle's lateral direction where the cumulative value of the edge intensity of the rotated feature points is greater than or equal to a predetermined threshold and is also the maximum, extracts candidate white lines composed of feature points located within a predetermined width from the searched X-coordinate, and determines that a candidate white line is a white line if the difference in X-coordinates between the extracted candidate white lines corresponds to the lane width (Patent Document 1).
[0003] Japanese Patent Publication No. 2017-156795
[0004] The conventional technology described above has a problem in that when a vehicle travels on a curve, the feature points are dispersed in the X-axis direction, making it impossible to accurately extract candidate white lines.
[0005] The problem that this invention aims to solve is to provide a lane boundary line detection method and a lane boundary line detection device that can accurately detect lane boundary lines when a vehicle is traveling on a curve.
[0006] The present invention solves the above problem by detecting lane boundary lines that define road lanes by obtaining a change position in which at least one of the angle between the first travel trajectory of a vehicle when a vehicle travels on the road and a link line representing the shape of the road, which is registered in advance in map information, and the angle between the second travel trajectory of another vehicle when another vehicle travels on the road and the link line, changes by a predetermined angle or more; obtaining position information of feature points of an object that indicates a lane boundary line; and detecting the lane boundary line from the position information of feature points relative to the link line in a rear section behind the change position in the direction of travel of the vehicle, and in a front section that includes the change position and is ahead of the change position in the direction of travel of the vehicle.
[0007] According to the present invention, lane boundary lines can be accurately detected when a vehicle is traveling on a curve.
[0008] This is a block diagram showing one embodiment of a driver assistance system including a lane boundary detection device according to the present invention. This is a plan view showing an example of a driving scene in which driver assistance is performed by the driver assistance system of Figure 1. This is a plan view showing an example of feature points extracted in the driving scene of Figure 2. This is a plan view showing the feature points of the forward section of Figure 3 represented in a local coordinate system along the link line. This is a plan view (1) showing the feature points of the rear section of Figure 3 represented in a local coordinate system along the link line. This is a plan view (2) showing the feature points of the rear section of Figure 3 represented in a local coordinate system along the link line. This is a plan view showing an example of a line segment corresponding to a part of the lane boundary in Figure 2. This is a plan view showing another example of a line segment corresponding to a part of the lane boundary in Figure 2. This is a flowchart showing an example of a processing procedure in the driver assistance system of Figure 1.
[0009] Embodiments of the present invention will be described below with reference to the drawings. In the following description, the width direction of the road and the width direction of the lane will be assumed to coincide.
[0010] [Configuration of the Driving Assistance System] Figure 1 is a block diagram showing one embodiment of a driving assistance system including a lane boundary detection device according to the present invention. The driving assistance system generates a driving route from the current position to a destination set by the occupant through autonomous driving control, and controls the vehicle's actuators to drive the vehicle along the driving route. Autonomous driving control is the autonomous control of the vehicle's driving operations, and these driving operations include all driving operations such as acceleration, deceleration, starting, stopping, and steering.
[0011] As shown in Figure 1, the driver assistance system 10 comprises a map database 11, a navigation device 12, an imaging device 13, a distance measuring device 14, and a driver assistance device 15. The driver assistance device 15 also includes a lane boundary detection device 20 as part of it. These devices are connected via a CAN (Controller Area Network) or other in-vehicle LAN and can exchange information with each other.
[0012] The map database 11 is a storage medium that stores map information and is located inside or outside the vehicle. The driver assistance device 15 retrieves map information from the map database 11 as needed. The map information includes information on nodes corresponding to specific points on the road where the direction of travel of the vehicle changes (such as intersections) and links corresponding to road sections connecting the nodes. The node information includes location information, information on entering and exiting intersections, etc., and the link information includes width, road curvature radius, road traffic laws, etc. In addition to road information for each lane, the map information may also be high-precision map information that includes information on structures, signs, traffic lights, etc., around the road.
[0013] Furthermore, the map information includes information about link lines that represent the shape of roads (hereinafter also referred to as link line information). Link lines that represent the shape of roads (hereinafter simply referred to as link lines) are lines that represent the shape of roads corresponding to road sections between nodes, and for example, one link line is set for each road section. Link lines are lines that have a shape along the road and are located on the center line in the width direction of the road. Link lines indicate the curvature of the road, the direction in which the road extends, etc.
[0014] Nodes are placed along the link line at predetermined intervals (for example, every 1 to 3 meters). Each node on the link line has location information. This location information includes coordinates in the global coordinate system used in the map information, the angle between a predetermined direction and the direction of the link line (direction of travel), and the distance traveled from one node to another on the link line. The starting node is, for example, the node closest to the starting point of the travel route. The predetermined direction could be the north direction on the map or a direction along the axes of the global coordinate system. The direction of the link line is, for example, the tangent direction of the link line at a node on the link line. The distance traveled between nodes is calculated as the distance traveled when moving along the link line from one node to another.
[0015] The navigation device 12 refers to map information obtained from the map database 11 and generates a driving route from the vehicle's current position, detected by a positioning system (not shown), to a destination set by the occupants. The driving route includes information on at least the road the vehicle is traveling on, the lane it is traveling in, and the direction of travel, and is displayed, for example, as a linear route. The calculated driving route is acquired by the driver assistance device 15.
[0016] The imaging device 13 is a camera equipped with an image sensor such as a CCD, which captures images of objects around the vehicle and generates images that include the objects. Multiple imaging devices 13 are installed on the vehicle's front grille, side mirrors, rear bumper, etc., to suppress the occurrence of blind spots where objects cannot be captured.
[0017] The rangefinder 14 detects the relative distance and relative speed between the vehicle and the object. The rangefinder 14 includes a laser radar, millimeter-wave radar, and a LiDAR (Light Detection and Ranging) unit. Multiple rangefinders 14 are installed on a single vehicle to suppress the occurrence of blind spots where objects cannot be detected.
[0018] The driver assistance system 15 acquires image information from the imaging device 13 and positional information of objects from the distance measuring device 14, thereby recognizing objects and the driving environment around the vehicle. The objects detected by the imaging device 13 and the distance measuring device 14 are objects present on the road and its surroundings, including road lane boundaries, center lines, road markings, median strips, and road signs. The objects also include obstacles that may affect the vehicle's movement, such as other automobiles (other vehicles), motorcycles, bicycles, and pedestrians. The acquisition of information by the driver assistance system 15 is performed at predetermined time intervals (for example, every 0.1 to 1 millisecond).
[0019] The distance measuring device 14 may generate point cloud data in which information regarding distance measuring points of surrounding objects is arranged in two dimensions in the left-right and up-down directions of the vehicle. A distance measuring point of an object is a point on the object whose distance from the distance measuring device 14 has been measured. The information on the distance measuring points may include not only the position information of the distance measuring point (coordinate information of the distance measuring point, information on the distance from the distance measuring device 14 to the distance measuring point, etc.) but also information on the reflectivity of electromagnetic waves at the distance measuring point. The distance measuring device 14 generates point cloud data by scanning electromagnetic waves (millimeter waves, infrared rays, lasers, etc.) in the left-right direction of the vehicle.
[0020] The driver assistance device 15 controls and coordinates the devices that constitute the driver assistance system 10 to provide driving assistance for the vehicle. The driver assistance device 15 autonomously controls the vehicle's driving motion within a predetermined range using devices mounted on the vehicle. For driving motions not controlled by the driver assistance device 15, the driver performs manual operation. When the driver operates the vehicle manually, the driver assistance device 15 does not perform autonomous control of the driving motion, and the vehicle's driving motion is controlled by the driver's operation.
[0021] The driver assistance device 15 is, for example, a computer and comprises a CPU (Central Processing Unit) which is a processor, a ROM (Read Only Memory) in which programs are stored, and a RAM (Random Access Memory) which functions as an accessible storage device. The CPU is an operating circuit that executes the programs stored in the ROM and realizes the functions of the driver assistance device 15. Alternatively, an MPU (Micro Processing Unit), ASIC (Application Specific Integrated Circuit), etc., may be used instead of or in conjunction with the CPU.
[0022] The driver assistance device 15 includes a lane boundary detection device 20 as part of it. The driver assistance device 15 has a driver assistance function that performs driver assistance for the vehicle, and the lane boundary detection device 20 has a lane boundary detection function that detects lane boundaries as part of the driver assistance function. A program for realizing the driver assistance function is stored in the ROM, and the driver assistance function, including the lane boundary detection function, is realized when the CPU executes the program stored in the ROM. In Figure 1, the driving control unit 16, extraction unit 21, acquisition unit 22, and detection unit 23 are conveniently extracted and shown as functional blocks that realize the driver assistance function. The extraction unit 21, acquisition unit 22, and detection unit 23 are functional blocks that realize the lane boundary detection function and are included in the lane boundary detection device 20.
[0023] [Functions of the Driving Assistance System] Figure 2 is a plan view showing an example of a driving scene in which driving assistance is performed by the driving assistance system 10. In the driving scene shown in Figure 2, the road extends in the X-axis direction (left-right direction in the drawing) of the global coordinate system used in the map information. The road shown in Figure 2 has a lane W1 defined by solid lane boundary lines L1 and L2, and a lane W2 defined by solid lane boundary lines L2 and L3. Note that the coordinate system of the map information is not limited to the global coordinate system shown in Figure 2, but may also be a vehicle coordinate system centered on the vehicle's position.
[0024] In the driving scene shown in Figure 2, vehicle V1 (hereinafter also referred to as "own vehicle V1") is driving at position P1 in lane W1, and another vehicle V2 is driving at position P2 in lane W1. Vehicle V1 shown in Figure 2 is assumed to be driving in the positive X-axis direction (from left to right in the drawing) towards a destination (not shown) set ahead of lane W1 by autonomous driving control. In this case, the driving control unit 16 generates a driving path for vehicle V1 and operates the actuators (not shown) of vehicle V1 so that vehicle V1 drives along the driving path. The driving control unit 16 generates control signals to operate the actuators and outputs them to the actuators.
[0025] Furthermore, when the driving control unit 16 performs driving assistance, it outputs an execution instruction to the lane boundary detection device 20 to perform lane boundary detection. A lane boundary is a boundary line that divides (or defines) the lanes of a road, and is a type of road marking. As shown in Figure 2, the lane boundary may be a solid line or a dashed line. Also, the color of the lane boundary is not limited to white, but may be yellow, orange, green, blue, etc.
[0026] If the map information stored in the map database 11 is high-precision map information, the lane boundary detection device 20 detects lane boundaries from the road's lane information. On the other hand, if the map information does not include lane information, the lane boundary detection device 20 detects lane boundaries from images acquired by the imaging device 13. When the lane boundary detection device 20 detects lane boundaries from image information without using the lane information from the map information, it performs the following processing using the functions of the extraction unit 21, acquisition unit 22, and detection unit 23.
[0027] The extraction unit 21 acquires information on the feature points of objects present around the vehicle V1. Feature points are characteristic parts of an object used for object detection, and include edges that capture the contour of the object, corners of the object, and parts of the object where brightness changes. The extraction unit 21 acquires an image from the imaging device 13 and extracts feature points from the image using known methods such as semantic segmentation. Alternatively, the extraction unit 21 may acquire point cloud data from the distance measuring device 14, generate an image plotting distance measuring points from the point cloud data, and then perform a process to extract feature points from the generated image based on the difference in electromagnetic wave reflectivity between the road surface and road markings.
[0028] The extraction unit 21 acquires information on the feature points of objects that indicate the lane boundary lines of the road on which the vehicle V1 is traveling, from sensors mounted on the vehicle V1. Examples of sensors include an imaging device 13 and a distance measuring device 14. The objects that indicate the lane boundary lines are road markings such as solid white lines, solid yellow lines, and dashed white lines, and the information on the feature points of the objects includes information on the position of the edge of the solid white road marking, information on the width of the solid yellow road marking, and information on the length in the direction of travel of the dashed white road marking.
[0029] Figure 3 is a plan view showing an example of feature points extracted from an image captured by the imaging device 13 in the driving scene shown in Figure 2. In the driving scene shown in Figure 2, as shown in Figure 3, feature points A1 to A13 are extracted for lane boundary line L1, feature points B1 to B13 are extracted for lane boundary line L2, and feature points C1 to C13 are extracted for lane boundary line L3. Each extracted feature point is placed on the XY plane of the global coordinate system based on the coordinate information of the feature point. The number of feature points and the spacing between feature points can be set to appropriate values within a range that allows for accurate detection of lane boundaries.
[0030] The lane boundary detection device 20, through the function of the detection unit 23, determines which of the lane boundary lines L1 to L3 shown in Figure 2 corresponds to each feature point shown in Figure 3, and associates the corresponding feature point for each lane boundary line L1 to L3. When associating feature points corresponding to lane boundary lines, for example, information about link lines is used, and the position of each feature point is represented by the distance from the link line and the distance traveled along the link line from a predetermined node on the link line to the position corresponding to the feature point. As a result, the feature points corresponding to one lane boundary line are lined up in a row to form a sequence of points, so that the corresponding feature points can be associated for each lane boundary line.
[0031] An example of a link line is shown in Figure 3. The link line K shown in Figure 3 is a pre-set link line for the road shown in Figure 2, and nodes N1 to N10 are provided. Nodes N1 to N10 contain information about their coordinates in the global coordinate system, the angle between the X-axis direction and the tangential direction of the link line K, and the distance traveled from the starting node N1. In the link line K shown in Figure 3, the portion in front of the vehicle V1 in the direction of travel of the vehicle V1 is curved in the negative Y-axis direction due to the increase in the number of lanes caused by lane W2 branching into a main line and a branch line (not shown) in front of the vehicle V1 in the direction of travel. In sections where the link line K follows the shape of the road and sections where the link line K is curved in the negative Y-axis direction and has a shape different from the shape of the road, different sequences of points are formed for the same lane boundary line, and in some cases the lane boundary line cannot be accurately detected.
[0032] Therefore, when detecting a lane boundary line, the lane boundary line detection device 20 of this embodiment uses the functions of the acquisition unit 22 to acquire a position (hereinafter also called the change position) where at least one of the following changes by a predetermined angle or more: the angle formed between the driving trajectory of vehicle V1 when vehicle V1 is driving on the road (hereinafter also called the first driving trajectory) and the link line K registered in advance in the map information (hereinafter also called the first angle); and the angle formed between the driving trajectory of another vehicle V2 when another vehicle V2 is driving on the road (hereinafter also called the second driving trajectory) and the link line K registered in advance in the map information (hereinafter also called the second angle). The lane boundary detection device 20 then detects the lane boundary in relation to the link line K (hereinafter also referred to as the position information of the feature points of the object indicating the lane boundary) in relation to the link line K (hereinafter also referred to as the position information of the feature points of the feature points relative to the link line K) in a section behind (or in front of) the change position in the direction of travel of the vehicle V1 (hereinafter also referred to as the rear section) and in a section including the change position and in front (or farther away) the change position in the direction of travel of the vehicle V1 (hereinafter also referred to as the front section), by the function of the detection unit 23.
[0033] The first travel trajectory is information that shows the positions that vehicle V1 has passed through in chronological order when traveling on a certain road. The second travel trajectory is information that shows the positions that other vehicle V2 has passed through in chronological order when traveling on a certain road. The first and second travel trajectories are displayed linearly in a plan view, for example, when vehicle V1 is viewed from above. The nodes on the first travel trajectory and the nodes on the second travel trajectory each hold the position information of vehicle V1 and other vehicle V2, respectively.
[0034] The acquisition unit 22 stores, for example, information regarding the current position of vehicle V1 detected by the vehicle V1's positioning system (not shown), and information regarding the yaw rate of vehicle V1 detected by the vehicle V1's yaw rate sensor (not shown), in a non-volatile storage medium (e.g., ROM), and generates a first driving trajectory based on the stored information. The acquisition unit 22 stores image information of another vehicle V2 acquired from the imaging device 13 and position information of the other vehicle V2 acquired from the distance measuring device 14 in a non-volatile storage medium (e.g., ROM), calculates the position of the other vehicle V2 relative to vehicle V1 from the stored information, and generates a second driving trajectory based on the relative position of the other vehicle V2 relative to vehicle V1.
[0035] When the acquisition unit 22 calculates the first angle, for example, it calculates the angle formed by the tangential direction at a node of the link line K and the tangential direction at a position on the first travel trajectory corresponding to the position of the node of the link line K. When the acquisition unit 22 calculates the second angle, for example, it calculates the angle formed by the tangential direction at a node of the link line K and the tangential direction at a position on the second travel trajectory corresponding to the position of the node of the link line K. The acquisition unit 22 may calculate the first angle and the second angle at each node of the link line K, or it may calculate the first angle and the second angle at predetermined distances (for example, every 1 to 3 m). Nodes may be arranged on the first travel trajectory at predetermined distances (for example, every 1 to 3 m).
[0036] Nodes may be placed on the first and second travel trajectories at predetermined distances (for example, every 1 to 3 meters). If a node is placed on the first travel trajectory, the first angle is the angle formed by the tangential direction of the node on the link line K and the tangential direction of the node on the first travel trajectory corresponding to the node on the link line K. If a node is placed on the second travel trajectory, the second angle is the angle formed by the tangential direction of the node on the link line K and the tangential direction of the node on the second travel trajectory corresponding to the node on the link line K.
[0037] The acquisition unit 22 generates a first travel trajectory for roads that vehicle V1 has traveled on in the past. The acquisition unit 22 also generates a second travel trajectory if there is or is present of another vehicle V2 traveling around vehicle V1. Specifically, if vehicle V1 has traveled on a road in the past and there are preceding vehicles, following vehicles, and vehicles traveling alongside vehicle V1, the acquisition unit 22 generates a second travel trajectory for that road from the position information of the preceding vehicles, following vehicles, and vehicles traveling alongside vehicle V1, respectively. Furthermore, if vehicle V1 is traveling on a road that it has not traveled on in the past and there is a preceding vehicle for vehicle V1, the acquisition unit 22 generates a second travel trajectory from the position information of the preceding vehicle.
[0038] The acquisition unit 22 repeatedly calculates at least one of the first angle and the second angle while the vehicle V1 is traveling on the road, and acquires change positions where at least one of the first angle and the second angle changes by a predetermined angle or more. The predetermined angle can be set to an appropriate value within the range in which the detection unit 23 can accurately detect the lane boundary line, for example, 30 to 60°. Examples of change positions include positions in front of or behind a point where the number of road lanes increases, positions in front of or behind a point where the number of road lanes decreases, positions in front of or behind a merging point where multiple roads merge, and positions in front of or behind a branching point where a branch line branches off from the main line.
[0039] For example, if a first travel trajectory M1 shown in Figure 3 is generated from information about the current position of vehicle V1, the acquisition unit 22 calculates a first angle formed by the link line K and the first travel trajectory M1 at each of the nodes N1 to N10 of the link line K. The first angle calculated by the acquisition unit 22 is shown at the bottom of Figure 3. The horizontal axis of the graph shown at the bottom of Figure 3 indicates the position along the direction of travel on the road, and the vertical axis of the graph indicates the first angle. As shown in the graph of Figure 3, the first angle hardly changes at the positions corresponding to nodes N1 to N4 of the link line K, changes at position Px corresponding to node N5, and hardly changes at the positions corresponding to nodes N6 to N10. If the amount of change in the first angle at node N5 is greater than or equal to a predetermined angle, the acquisition unit 22 determines that position Px is a changing position.
[0040] The detection unit 23 sets a rear section and a front section based on the change position acquired by the acquisition unit 22. The rear section is, for example, the section from the current position of the vehicle V1 to the change position, and is the section on the rear side of the change position in the traveling direction of the vehicle V1 and on the front side of the current position of the vehicle V1 in the traveling direction of the vehicle V1. Further, the front section is, for example, the section from one change position to the next change position, and is the section from the change position to a position separated by a predetermined distance in the forward direction of the traveling direction of the vehicle V1 from the change position. The predetermined distance is, for example, 1 to 3 m. On the other hand, when the acquisition unit 22 does not detect a change position (or determines that there is no change position), the detection unit 23 sets a predetermined section having a predetermined length along the traveling direction of the vehicle V1. The predetermined length can be set to an appropriate value within the range where the detection unit 23 can accurately detect the lane boundary line, and is, for example, 1 to 3 m.
[0041] The detection unit 23 detects the lane boundary line from the position information of the feature points with respect to the link line K in at least one of the rear section, the front section, and the predetermined section. The detection unit 23 acquires the position information based on the link line K (that is, the position information of the feature points with respect to the link line K) among the position information of the extracted feature points. The position information of the feature points includes information on the coordinates of the feature points in an arbitrary coordinate system, information on the distance and direction of other feature points with respect to one feature point, and the like. Examples of the position information of the feature points with respect to the link line K include, but are not limited to, the distance between the feature point and the link line K (or the nodes N1 to N10 on the link line K), and the direction of the link line K with respect to the feature point. Further, the position information of the feature points with respect to the link line K includes the position information of the feature points in the local coordinate system along the link line K.
[0042] As an example, the detection unit 23 acquires the position information in the coordinate system of the information used for extracting the feature points (hereinafter, also referred to as the original coordinate system). When the feature points are extracted from the image captured by the imaging device 13, the original coordinate system is the screen coordinate system of the image. Next, the detection unit 23 converts the position information in the original coordinate system into the position information in the global coordinate system. For example, the detection unit 23 uses the camera coordinate system centered on the imaging device 13 to convert the coordinates of the feature points in the screen coordinate system into the coordinates in the world coordinate system (global coordinate system).
[0043] Next, the detection unit 23 converts the position information in the global coordinate system into the position information in the local coordinate system along the link line K, and acquires the position information of the feature points in the local coordinate system. The local coordinate system along the link line K is a coordinate system that represents the position information of the feature points by the distance from the link line K and the running distance from a predetermined position (for example, a predetermined node on the link line K). The predetermined position is, for example, a position corresponding to the starting point of the running route, and the predetermined node is, for example, the node closest to the starting point of the running route. The detection unit 23 associates the feature point with the closest node on the link line K, and rotates the associated node by the angle formed by the predetermined direction and the direction of the link line K. Further, the detection unit 23 moves the rotated feature point along the link line K by the running distance to the node at the center of rotation.
[0044] On the link line K shown in FIG. 3, nodes N1 to N10 indicated by "□" are provided. The nodes N1 to N10 have information on the coordinates in the global coordinate system, information on the running distance from the starting node N1, and the like. For example, assuming that the angle (the angle of the inferior angle) formed by the X-axis direction and the tangent direction of the link line K at one target node (hereinafter also referred to as the target node) among the nodes N1 to N10 is θ, the direction R of the link line K at the target node (that is, the traveling direction) is represented by the following formula (1).
[0045]
[0046] Here, x ａ and y ａ are the X coordinate and Y coordinate of the target node in the global coordinate system, respectively. In this case, assuming that the coordinates P ｒａｗ of the feature point associated with the target node (hereinafter also referred to as the target feature point) in the global coordinate system are represented by the following formula (2), the coordinates P ｒａｗ of the target feature point in the global coordinate system rotated by the angle θ in the counterclockwise direction are represented by the following formula (3). ｎｏｄｅ are represented by the following formula (3).
[0047]
[0048] Here, x ｂ and y ｂ are, respectively, the X coordinate and Y coordinate of the target feature point in the global coordinate system. And the coordinate P ｎｏｄｅ after moving the coordinate P ｗａｙ along the link line K by the running distance s is represented by the following formula (4). The running distance s is the running distance from the starting node (node N1 in the example shown in FIG. 3) to the target node.
[0049]
[0050] In the example shown in FIG. 3, since the acquisition unit 22 has acquired the information that the position Px is the change position, the detection unit 23 sets the rear section Z1 on the front side of the traveling direction of the vehicle V1 from the position Px and the front section Z2 including the position Px and from the position Px to the next change position, and in each of the rear section Z1 and the front section Z2, the lane boundary lines L1 to L3 are detected from the position information of the feature points A1 to A13, B1 to B13, C1 to C13 with respect to the link line K.
[0051] The detection unit 23 associates the feature points A1 to A13, B1 to B13, C1 to C13 with the nodes on the link line K. As an example, the detection unit 23 associates the feature point B5 with the node N4 having the shortest distance, and rotates the node N4 counterclockwise about the center by the angle formed by the X-axis direction and the tangent direction of the link line K at the node N4, and moves the rotated feature point B5 along the link line K by the running distance from the node N1 to the node N4.
[0052] The detection unit 23 performs the above-described series of processes on feature points A1-A6, B1-B6, and C1-C6 included in the rear section Z1 shown in Figure 3, thereby converting the coordinates of the feature points in the global coordinate system shown in Figure 3 to the coordinates in the local coordinate system along the link line K shown in Figure 4. In the local coordinate system of Figure 4, the horizontal axis represents the distance d between the feature point and the link line K, and the vertical axis represents the travel distance s from the starting node (node N1 in the example shown in Figure 3). In the rear section Z1, since the link line K has a shape that follows the road, in the local coordinate system along the link line K, the feature points A1-A6, B1-B6, and C1-C6 are aligned along the vertical axis (s-axis direction) for each corresponding lane boundary line. In the example shown in Figure 4, feature points A1-A6, B1-B6, and C1-C6 are aligned along the vertical axis at horizontal axis positions Pa, Pb, and Pc, respectively, forming a sequence of points.
[0053] The detection unit 23 searches for peak locations where a predetermined number of feature points exist, based on the positional information of feature points in the local coordinate system along the link line K, and detects lane boundary lines from the searched peak locations. The predetermined number of peak locations (i.e., the number of feature points that constitute the sequence of feature points formed at the peak locations) can be set to an appropriate value within a range in which the lane boundary lines can be accurately detected. In searching for peak locations, the detection unit 23 may generate a histogram showing the distribution of feature points from the positional information of feature points in the local coordinate system.
[0054] In the example shown in Figure 4, the detection unit 23 calculates the number of feature points in the s-axis direction along the positive d-axis direction (i.e., the direction from left to right in the drawing) from an arbitrary position (for example, the origin O), thereby detecting the peak (sequence of points) of position Pa consisting of feature points A1 to A6, the peak (sequence of points) of position Pb consisting of feature points B1 to B6, and the peak (sequence of points) of position Pc consisting of feature points C1 to C6. The detection unit 23 recognizes that lane boundary lines exist at the peak positions Pa, Pb, and Pc, and associates the feature points A1 to A6 corresponding to the lane boundary line of position Pa, the feature points B1 to B6 corresponding to the lane boundary line of position Pb, and the feature points C1 to C6 corresponding to the lane boundary line of position Pc.
[0055] Furthermore, the detection unit 23 performs the above-described series of processes on feature points A7-A13, B7-B13, and C7-C13 included in the forward section Z2 shown in Figure 3, thereby converting the coordinates of the feature points in the global coordinate system shown in Figure 3 to the coordinates in the local coordinate system along the link line K shown in Figure 5A. The vertical and horizontal axes of the local coordinate system in Figure 5A are the same as those of the local coordinate system in Figure 4. In the forward section Z2, the link line K is curved in the negative Y-axis direction (to the right with respect to the direction of travel of the vehicle V1), so as shown in Figure 5A, the feature points A7-A13, B7-B13, and C7-C13 do not form a sequence of points along the vertical axis (s-axis direction). In this case, the detection unit 23 rotates the feature points A7-A13, B7-B13, and C7-C13 (or the local coordinate system itself) clockwise so that a sequence of points is formed along the vertical axis.
[0056] Figure 5B shows the feature points A7-A13, B7-B13, and C7-C13 after rotation. In Figure 5B, the horizontal axis and vertical axis are represented as the d' axis and s' axis, respectively, to indicate that the coordinate system in Figure 5B is a rotation of the coordinate system in Figure 5A. In the example shown in Figure 5B, feature points A7-A13, B7-B13, and C7-C13 are aligned along the vertical axis (s' axis) at positions Pd, Pe, and Pf in the horizontal axis direction (d' axis direction), respectively, forming a sequence of points (peaks). The detection unit 23 recognizes that lane boundary lines exist at the peak positions Pd, Pe, and Pf, and associates feature points A7-A13 with the lane boundary line at position Pd, feature points B7-B13 with the lane boundary line at position Pe, and feature points C7-C13 with the lane boundary line at position Pf.
[0057] The detection unit 23 then estimates the shape of the lane boundary line for each associated feature point and detects the lane boundary line. Specifically, the detection unit 23 generates a line segment (hereinafter simply referred to as a line segment) which is part of the lane boundary line for each associated feature point, and estimates the shape of the lane boundary line by connecting the generated line segments. For example, the detection unit 23 performs linear regression on a plurality of associated feature points and generates line segments based on the obtained regression line. In the rear section Z1, the detection unit 23 generates line segment R1 corresponding to feature points A1 to A6, line segment R2 corresponding to feature points B1 to B6, and line segment R3 corresponding to feature points C1 to C6, and in the front section Z2, it generates line segment R4 corresponding to feature points A7 to A13, line segment R5 corresponding to feature points B7 to B13, and line segment R6 corresponding to feature points C7 to C13.
[0058] After the detection unit 23 performs association of feature points in the backward section and the forward section, it performs the reverse process of the coordinate transformation shown by equations (1) to (4) above, and transforms the coordinate system of the feature points from the local coordinate system along the link line K to the global coordinate system.
[0059] Figure 6 is a plan view showing line segments generated by the detection unit 23 in the rear section Z1 and the front section Z2. In the example shown in Figure 6, the detection unit 23 connects the line segments in the rear section Z1 and the line segments in the front section Z2 to estimate the shape of the lane boundary line. When the detection unit 23 connects the line segments in the rear section Z1 and the line segments in the front section Z2, it connects the end of the line segment in the rear section Z1 that is on the front section Z2 side (hereinafter also referred to as the front end) and the end of the line segment in the front section Z2 that is closest to the front end. In the example shown in Figure 6, the detection unit 23 connects the end of the line segment R1 that is on the front section Z2 side and the end of the line segment R4 that is on the rear section Z1 side to estimate the shape of the lane boundary line L1. Similarly, the detection unit 23 estimates the shape of the lane boundary line L2 by connecting the end of line segment R2 on the forward section Z2 side and the end of line segment R5 on the rear section Z1 side, and estimates the shape of the lane boundary line L3 by connecting the end of line segment R3 on the forward section Z2 side and the end of line segment R6 on the rear section Z1 side.
[0060] The detection unit 23 calculates the amount of change (hereinafter simply referred to as the amount of change) of at least one of the first angle and the second angle at predetermined intervals, and may divide the rear section Z1 and the front section Z2 into a plurality of sections aligned along the direction of travel of the vehicle V1 based on the amount of change. The predetermined interval can be set to an appropriate value within the range in which the detection unit 23 can accurately detect the lane boundary line, for example, an interval of 0.1 to 1 second. The detection unit 23 determines whether the amount of change is greater than or equal to the predetermined amount of change, and if it determines that the amount of change is greater than or equal to the predetermined amount of change, it divides the rear section Z1 or the front section Z2 at the position where the amount of change is greater than or equal to the predetermined amount of change, and sets a new section. The predetermined amount of change can be set to an appropriate value within the range in which the detection unit 23 can accurately detect the lane boundary line, for example, half of the predetermined angle. The detection unit 23 may also generate line segments from the position information of feature points relative to the link line K for each section divided according to the amount of change, and estimate the shape of the lane boundary line by connecting the generated line segments. In this case, the detection unit 23 connects one end of the line segment of one section to the end of the line segment of the other section adjacent to the one end, which is the end closest to the one end.
[0061] Figure 7 is a plan view showing the line segments generated by the detection unit 23 when the amount of change in the first angle at position Py exceeds a predetermined amount of change, and the forward section Z2 in Figure 6 is divided into forward section Z2a and forward section Z2b at position Py. The detection unit 23 generates line segments R4a corresponding to feature points A7 to A10, line segments R5a corresponding to feature points B7 to B10, and line segments R6a corresponding to feature points C7 to C10 in forward section Z2a, and line segments R4b corresponding to feature points A11 to A13, line segments R5b corresponding to feature points B11 to B13, and line segments R6b corresponding to feature points C11 to C13 in forward section Z2b.
[0062] For example, when connecting line segment R4a shown in Figure 7 to other line segments, the detection unit 23 connects the end of line segment R4a on the positive X-axis side (one end) to the end of line segments R4b, R5b, and R6b generated in the forward section Z2b adjacent to the forward section Z2a where line segment R4a was generated, which is the end closest to the one end (i.e., the end of line segment R4b on the negative X-axis side). The detection unit 23 also connects the end of line segment R4a on the negative X-axis side (the other end) to the end of line segments R1, R2, and R3 generated in the backward section Z1 adjacent to the forward section Z2a where line segment R4a was generated, which is the end closest to the other end (i.e., the end of line segment R1 on the positive X-axis side).
[0063] The detection unit 23 connects the line segments generated for each section so that the lane boundary line generated by connecting the line segments does not intersect with the first and second driving trajectories M1 and M2. For example, if the second driving trajectory M2 shown in Figure 7 is generated from the position information of another vehicle V2 detected by the imaging device 13 and distance measuring device 14 of vehicle V1, the detection unit 23 connects the line segments in Figure 7 so that they do not intersect with the second driving trajectory M2.
[0064] The detection unit 23 corrects the lane boundary line so that it becomes a continuous line between adjacent sections. For example, the detection unit 23 interpolates between the ends of two line segments using a spline curve so that the lane boundary line becomes a continuous line at the connection point between line segment R3 and line segment R6a in Figure 7 (i.e., at position Px).
[0065] When the acquisition unit 22 acquires the change position from the link line K and the first travel trajectory M1, it may calculate the first angle based on information regarding the travel angle (hereinafter also simply referred to as the travel angle) indicating the direction of travel of the vehicle V1 and information regarding the yaw rate of the vehicle V1. The acquisition unit 22 acquires information regarding the travel angle from a positioning system (not shown) such as GPS (Global Positioning System) and acquires information regarding the yaw rate from a yaw rate sensor (not shown). Furthermore, when the acquisition unit 22 acquires the change position from the link line K and the second travel trajectory M2, it may calculate the second angle using the position information of another vehicle V2. The acquisition unit 22 acquires the position information of the other vehicle V2 from the imaging device 13 and distance measuring device 14 of the vehicle V1.
[0066] If the first driving trajectory M1 includes a lane change section where the vehicle changes lanes from one lane to another, the acquisition unit 22 may supplement the information regarding the angle of travel in the lane change section with information regarding the angle of travel before and after the lane change section. The lane change section is, for example, the section from when the vehicle V1 starts moving in the width direction of the road from one lane to another until the movement is completed, and may also be the section from when the driving control unit 16 starts autonomous driving control of the lane change until it ends. The acquisition unit 22 replaces the information regarding the angle of travel in the lane change section of the first driving trajectory M1 with the information regarding the angle of travel detected in one of the sections adjacent to the lane change section.
[0067] [Processing in the Driving Assistance System] Referring to Figure 8, the procedure for information processing by the driving assistance device 15 will be explained. Figure 8 is an example of a flowchart showing the information processing performed in the driving assistance system 10. The processing described below is performed by the processor (CPU) of the driving assistance device 15 (lane boundary detection device 20) at predetermined time intervals (for example, every 0.1 to 1 millisecond).
[0068] First, in step S1, the lane boundary detection device 20 extracts feature points from the image captured by the imaging device 13. In step S2, the lane boundary detection device 20 calculates at least one of the first angle formed by the first travel trajectory M1 and the link line K, and the second angle formed by the second travel trajectory M2 and the link line K. In step S3, the lane boundary detection device 20 determines whether at least one of the first angle and the second angle has changed by a predetermined angle or more.
[0069] If it is determined that the first angle and the second angle have not changed by more than a predetermined angle (Step S3: No), the process proceeds to Step S4, where the lane boundary detection device 20 sets a predetermined section having a predetermined length along the direction of travel of the vehicle V1. On the other hand, if it is determined that at least one of the first angle and the second angle has changed by more than a predetermined angle (Step S3: Yes), the process proceeds to Step S5, where the lane boundary detection device 20 sets a forward section and a rear section based on the change position where at least one of the first angle and the second angle changed by more than a predetermined angle.
[0070] In step S6, the lane boundary detection device 20 acquires coordinate information of feature points in the original coordinate system of the image. In step S7, the lane boundary detection device 20 converts the coordinate system of the acquired coordinates from the original coordinate system to the global coordinate system. In step S8, the lane boundary detection device 20 associates each feature point with the nearest node on the link line and converts the coordinate system of the feature points from the global coordinate system to the local coordinate system along the link line.
[0071] In step S9, the lane boundary detection device 20 acquires the coordinates of the feature points in the local coordinate system. In step S10, the lane boundary detection device 20 detects the peak position of the feature points in the local coordinate system for each section set in step S4 or S5. In step S11, the lane boundary detection device 20 associates the feature points with each peak position in each section set in step S4 or S5.
[0072] In step S12, the lane boundary detection device 20 generates a line segment corresponding to a part of the lane boundary for each associated feature point. In step S13, the lane boundary detection device 20 converts the coordinate system of the feature points from the local coordinate system to the global coordinate system. Then, in step S14, the lane boundary detection device 20 connects the line segments generated for each section and estimates the shape of the lane boundary.
[0073] [Embodiment of the Invention] According to this embodiment, when detecting lane boundary lines L1 to L3 that define road lanes W1 and W2, the change position Px is obtained in which at least one of the angle formed by the first travel trajectory M1 of vehicle V1 when vehicle V1 travels on the road and the link line K that represents the shape of the road and is registered in advance in the map information, and the angle formed by the second travel trajectory of another vehicle V2 when another vehicle V2 travels on the road and the link line K changes by a predetermined angle or more, and the object that indicates the lane boundary lines L1 to L3 is obtained. A lane boundary detection method is provided, which acquires the positional information of feature points A1 to A13, B1 to B13, and C1 to C13, and detects the lane boundary lines L1 to L3 in the rear section Z1 on the rear side of the direction of travel of the vehicle V1 from the change position Px, and the front section Z2 which includes the change position Px and is on the front side of the direction of travel of the change position Px, based on the positional information of feature points A1 to A13, B1 to B13, and C1 to C13 with respect to the link line K. A lane boundary detection device 20 that executes the lane boundary detection method is also provided. This allows for accurate detection of the lane boundary lines L1 to L3 when the vehicle V1 is traveling on a curve. Furthermore, it allows for accurate detection of the lane boundary lines L1 to L3 even when the shape of the link line K does not conform to the shape of the lane boundary lines L1 to L3.
[0074] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, the change amount of at least one of the angle formed by the first travel trajectory M1 and the tangent to the first travel trajectory M1, and the change amount of the angle formed by the second travel trajectory M2 and the tangent to the second travel trajectory M2 is calculated at predetermined intervals. Based on the change amount, the rear section Z1 and the front section Z2 are divided into a plurality of sections aligned along the direction of travel. For each of the divided sections, line segments R1 to R6, which are part of the lane boundary line lane change section, are generated from the position information of the feature points A1 to A13, B1 to B13, and C1 to C13 with respect to the link line K. The lane boundary lines L1 to L3 are detected by connecting one end of the line segments R1 to R6 in one section with the end of the line segments R1 to R6 in the section on the side of the one end of another section adjacent to the one section, which is the end closest to the one end. This allows for more accurate detection of the shape of the lane boundary lines L1 to L3.
[0075] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, the line segments R1 to R6 generated for each section are connected so that the lane boundary lines L1 to L3 do not intersect with the first driving trajectory M1 and the second driving trajectory M2. This allows for more accurate detection of the lane boundary lines L1 to L3.
[0076] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, the lane boundary lines L1 to L3 are corrected so that they form a continuous line between adjacent sections. This allows for more accurate detection of the shape of the lane boundary lines L1 to L3.
[0077] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, when obtaining the change position from the link line K and the first travel trajectory M1, the angle formed by the link line K and the first travel trajectory M1 is calculated based on information regarding the travel angle indicating the direction of travel and information regarding the yaw rate of the vehicle V1. This allows the first angle to be calculated more accurately.
[0078] In the lane boundary line detection method and lane boundary line detection device 20 of this embodiment, when obtaining the change position from the link line K and the second travel trajectory M2, the angle between the link line K and the second travel trajectory M2 is calculated using the position information of the other vehicle V2. This allows for a more accurate calculation of the second angle.
[0079] In the lane boundary detection method and lane boundary detection device 20 of this embodiment, if the first driving trajectory M1 includes a lane change section where the vehicle changes lanes from one lane to another, the information regarding the direction of travel angle before and after the lane change section is used to supplement the information regarding the direction of travel angle in the lane change section. This makes it possible to calculate the amount of change even in the lane change section.
[0080] 10...Driving support system, 11...Map database, 12...Navigation device, 13...Imaging device, 14...Distance measuring device, 15...Driving support device, 16...Driving control unit, 20...Lane boundary detection device, 21...Extraction unit, 22...Acquisition unit, 23...Detection unit, A1-A13, B1-B13, C1-C13...Feature points, K...Link line, L1-L3...Lane boundary line, M1...First driving trajectory, M2...Second driving trajectory, N1-N10...Nodes, P1, P2, Pa, Pb, Pc, Pd, Pe, Pf, Px, Py...Position, R1-R6, R4a, R4b, R5a, R5b, R6a, R6b...Line segment, V1...Vehicle, V2...Other vehicles, W1, W2...Lane, Z1...Rear section, Z2, Z2a, Z2b...Forward section
Claims
1. A lane boundary line detection method performed by a lane boundary line detection device for detecting lane boundary lines that define road lanes, wherein the lane boundary line detection device acquires a change position in which at least one of the angle formed between the first travel trajectory of the vehicle when the vehicle travels on the road and a link line representing the shape of the road, which is registered in advance in map information, and the angle formed between the second travel trajectory of the other vehicle when another vehicle travels on the road and the link line changes by a predetermined angle or more; acquires position information of a feature point of an object indicating the lane boundary line; and detects the lane boundary line from the position information of the feature point relative to the link line in a rear section behind the change position in the direction of travel of the vehicle and in a front section including the change position and in front of the change position in the direction of travel of the vehicle.
2. The lane boundary line detection method according to claim 1, wherein the lane boundary line detection device calculates at least one change in the angle between the first travel trajectory and the tangent to the first travel trajectory and the angle between the second travel trajectory and the tangent to the second travel trajectory at predetermined intervals, divides the rear section and the front section into a plurality of sections aligned along the direction of travel based on the change in the change in each section, generates a line segment which is part of the lane boundary line from the position information of the feature point relative to the link line for each of the divided sections, and detects the lane boundary line by connecting one end of the line segment in one section with the end of the line segment in the section on the side of the one end of another section adjacent to the one section, which is the end closest to the one end.
3. The lane boundary line detection method according to claim 2, wherein the lane boundary line detection device connects the line segments generated for each section so that the lane boundary line does not intersect with the first and second driving trajectories.
4. The lane boundary detection method according to claim 2 or 3, wherein the lane boundary detection device corrects the lane boundary so that the lane boundary becomes a continuous line between adjacent sections.
5. The lane boundary line detection method according to any one of claims 1 to 4, wherein when the lane boundary line detection device obtains the change position from the link line and the first travel trajectory, it calculates the angle formed between the link line and the first travel trajectory based on information regarding the direction of travel angle and information regarding the yaw rate of the vehicle.
6. The lane boundary line detection method according to any one of claims 1 to 5, wherein when the lane boundary line detection device obtains the change position from the link line and the second travel trajectory, it calculates the angle between the link line and the second travel trajectory using the position information of the other vehicle.
7. The lane boundary detection method according to any one of claims 1 to 6, wherein, if the first driving trajectory includes a lane change section in which the vehicle changes lanes from one lane to another, the lane boundary detection device supplements information regarding the direction of travel angle in the lane change section from information regarding the direction of travel angle before and after the lane change section.
8. A lane boundary line detection device for detecting lane boundary lines that define the lanes of a road, comprising: an acquisition unit that acquires a change position in which at least one of the angle formed between the first travel trajectory of a vehicle when the vehicle travels on the road and a link line representing the shape of the road, which is registered in advance in map information, and the angle formed between the second travel trajectory of another vehicle when another vehicle travels on the road and the link line changes by a predetermined angle or more; and a detection unit that acquires position information of a feature point of an object indicating the lane boundary line, and detects the lane boundary line from the position information of the feature point relative to the link line in a rear section behind the change position in the direction of travel of the vehicle, and in a front section including the change position and in front of the change position in the direction of travel of the vehicle.