Road marking detection method and road marking detection device

Abstract

The present invention provides a road marking detection method and a road marking detection device (20). In a case where a road marking of a road is detected by using map information including a link line (Z) representing the road, information of a feature point of an object indicating the road marking of the road on which a vehicle (V) is traveling is acquired from a sensor mounted on the vehicle (V), and the road marking extending in the width direction of the road is detected from position information of the feature point with respect to the link line (Z).

Description

Road marking detection method and road marking detection device

[0001] The present invention relates to a road marking detection method and a road marking detection device.

[0002] A lane marking recognition device is known that extracts feature points that make up white line candidates from camera images, converts the coordinates of the feature points from the local coordinate system of each camera to a world coordinate system common to multiple cameras, rotates the feature points around the vehicle so that the row of converted feature points is parallel to the Y axis of the world coordinate system, extracts the X coordinate where the cumulative value of the edge strength of the feature points is equal to or greater than a predetermined threshold and is a maximum, and detects the white line from the extracted X coordinate (Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2017-156795

[0004] In the above-mentioned conventional technology, when detecting road markings such as stop lines and pedestrian crossings that extend in the width direction of the road (hereinafter simply referred to as the width direction), feature points are rotated around the vehicle and the width direction in which the row of feature points is parallel to the Y axis is searched for. Therefore, with the above-mentioned conventional technology, when the traveling direction of the vehicle changes significantly, the width direction of the road cannot be searched appropriately, and road markings that extend in the width direction cannot be accurately detected.

[0005] The problem to be solved by the present invention is to provide a road marking detection method and a road marking detection device that can accurately detect road markings extending in the width direction of a road.

[0006] The present invention solves the above problem by detecting road markings on a road using map information including link lines that represent the road, by detecting road markings that extend in the width direction of the road from position information, relative to link lines, of feature points of objects that indicate road markings on the road on which a vehicle is traveling.

[0007] According to the present invention, road markings extending in the width direction of a road can be accurately detected.

[0008] 1 is a block diagram showing an example of an embodiment of a driving assistance device including a road sign detection device according to the present invention. FIG. 1 is a plan view showing an example of a driving scene in which driving assistance is performed by the driving assistance device of FIG. 1. FIG. 2 is a plan view showing an example of feature points extracted from the driving scene of FIG. 2. FIG. 3 is a plan view showing an example of a link line in the driving scene of FIG. 2. FIG. 4 is a plan view showing another example of a link line of this embodiment. FIG. 5 is an example of a histogram showing the distribution of feature points of FIG. 4. FIG. 6 is another example of a histogram showing the distribution of feature points of FIG. 4. FIG. 6 is yet another example of a histogram showing the distribution of feature points of FIG. 4. FIG. 7 is yet another example of a histogram showing the distribution of feature points of FIG. 4. FIG. 7 is a plan view showing yet another example of a link line of this embodiment. FIG. 10 is a flowchart showing an example of a processing procedure in the driving assistance device of FIG. 1. FIG. 11 is a flowchart showing another example of a processing procedure in the driving assistance device of FIG. 1 (part 1). FIG. 12 is a flowchart showing another example of a processing procedure in the driving assistance device of FIG. 1 (part 2).

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following description is based on the assumption that vehicles are driven on the left side of the road in countries where left-hand traffic regulations apply. In countries where right-hand traffic regulations apply, the left and right in the following description should be interpreted as symmetrical.

[0010] [Configuration of Driving Assistance Device] FIG. 1 is a block diagram showing a driving assistance device 10 according to the present invention. The driving assistance device 10 is a group of devices that perform driving assistance for a vehicle. Driving assistance includes driving a vehicle using autonomous driving control and providing information to the driver of the vehicle to assist the driver in driving operations. As an example, the driving assistance device 10 generates a driving route to a destination set by an occupant and controls the vehicle's actuators to drive the vehicle along the driving route. As another example, the driving assistance device 10 notifies the driver to stop the vehicle at a stop line when the vehicle approaches the stop line. The driving assistance device 10 may be an in-vehicle system, or some of its components may be provided outside the vehicle.

[0011] Autonomous driving control refers to autonomously controlling the driving behavior of a vehicle using a vehicle control device, and driving behavior includes all driving behaviors such as acceleration, deceleration, starting, stopping, and steering. Autonomously controlling driving behavior means that the control device controls the driving behavior using the vehicle's devices. The control device controls these driving behaviors within a predetermined range, and driving behaviors that are not controlled by the control device are manually operated by the driver. When the vehicle is driven manually by the driver without autonomous driving control, the control device does not perform autonomous control of the driving behavior, and the vehicle's driving behavior is controlled by the driver's operation.

[0012] As shown in Fig. 1, the driving assistance device 10 includes a map database 11, a navigation device 12, an imaging device 13, a distance measuring device 14, and a control device 15. The control device 15 of this embodiment also includes a road sign detection device 20 as part thereof. These devices are connected via a Controller Area Network (CAN) or other in-vehicle LAN, and can exchange information with each other. Information is exchanged with devices provided outside the vehicle via a network such as the Internet or a local area network (LAN).

[0013] The map database 11 is a storage medium storing map information and is provided inside or outside the vehicle. The control device 15 acquires map information from the map database 11 as needed. The map information is used for generating a driving route, etc., and includes information on nodes corresponding to specific points on roads where the vehicle's direction of travel changes (intersections, branching points, etc.) and information on links corresponding to road sections connecting the nodes. Node information includes location information (e.g., latitude and longitude) and information on entering and exiting intersections and branching points. Link information includes road width, road curvature radius, road shoulder structures, road traffic regulations (e.g., traffic direction), merging points, branching points, etc. The map information may be high-precision map information (HD map) that can grasp the movement trajectory for each lane.

[0014] The navigation device 12 is a device that references map information and generates a driving route from the current position of the vehicle detected by a positioning system (not shown) to a set destination. The navigation device 12 uses node and link information in the map information to search for a driving route for the vehicle to reach the destination. The driving route includes at least information on the roads the vehicle will travel on and the vehicle's direction of travel, and is displayed, for example, by nodes and link lines.

[0015] The imaging device 13 is a device that captures images of objects around the vehicle, and examples thereof include a camera equipped with an imaging element such as a CCD, an ultrasonic camera, an infrared camera, etc. In order to reduce blind spots when detecting objects, multiple imaging devices 13 are installed on the front grille of the vehicle, below the left and right door mirrors, near the rear bumper, etc.

[0016] The ranging device 14 is a device that acquires the relative distance and relative speed between the ranging device 14 and an object, and examples thereof include laser radar, millimeter-wave radar, and LiDAR (Light Detection and Ranging) units. To reduce blind spots when detecting an object, multiple ranging devices 14 are installed at the front, right and left sides, and rear of the vehicle. The ranging device 14 may also generate point cloud data in which information about ranging points of surrounding objects is two-dimensionally arranged in the left-right and up-down directions of the vehicle. The ranging points of an object are points on the object whose distance to the ranging device 14 is measured. The information about the ranging points includes, in addition to position information of the ranging points, the reflectivity of electromagnetic waves at the ranging points, etc.

[0017] The object is an object that exists on or around the road, and includes lane boundaries, center lines, road markings, medians, guardrails, curbs, road signs, traffic lights, crosswalks, etc. The object also includes obstacles that may affect the travel of a vehicle, such as automobiles other than the host vehicle, motorcycles, bicycles, and pedestrians. In this embodiment, the object is particularly an object that indicates road markings. The control device 15 acquires the detection results of the imaging device 13 and the distance measuring device 14 at predetermined time intervals (for example, every 0.1 to 1 millisecond).

[0018] The control device 15 is a device that performs driving assistance by controlling and cooperating with the constituent devices of the driving assistance device 10. The control device 15 is, for example, a computer, and includes a CPU (Central Processing Unit) that is a processor, a ROM (Read Only Memory) that stores programs, and a RAM (Random Access Memory) that functions as an accessible storage device. The CPU is an operating circuit that executes the programs stored in the ROM and realizes the functions of the control device 15.

[0019] The control device 15 has a driving assistance function that executes driving assistance for the vehicle. The control device 15 also includes a road sign detection device 20 as a part thereof, and the road sign detection device 20 has a road sign detection function that detects road signs as part of the driving assistance function. The ROM stores programs for realizing the driving assistance function, and the CPU executes the programs stored in the ROM to realize the driving assistance function including the road sign detection function.

[0020] 1 illustrates, for convenience, the driving control unit 16, the feature acquisition unit 21, the position acquisition unit 22, and the sign detection unit 23 as functional blocks that realize the driving assistance function. Of these functional blocks, the feature acquisition unit 21, the position acquisition unit 22, and the sign detection unit 23 are functional blocks that realize the road sign detection function, and are included in the road sign detection device 20. The functions of each functional block will be described below with reference to FIG. 2.

[0021] [Function of Road Sign Detection Device] FIG. 2 is a plan view showing an example of a driving scene in which the control device 15 executes autonomous driving control using the driving assistance function. In the driving scene shown in FIG. 2, a two-lane road with two-way traffic extends in the Y-axis direction (the vertical direction of the drawing) of a global coordinate system used in map information. The coordinate system of the map information is not limited to the global coordinate system, and various coordinate systems may be used. Vehicles traveling on lanes L1 and L2 travel in the positive Y-axis direction (from bottom to top of the drawing), and vehicles traveling on lanes L3 and L4 travel in the negative Y-axis direction (from top to bottom of the drawing). Additionally, a crosswalk B is provided on the road shown in FIG. 2, and stop lines A1 and A2 are provided just before the crosswalk B in the direction of travel.

[0022] 2, the vehicle V is traveling at a position Pv on the lane L1 and is traveling to a destination (not shown) ahead on the lane L1. In this case, the control device 15 generates a traveling route along the lane L1 from the position Pv to the destination, using the function of the traveling control unit 16. Then, the control device 15 autonomously controls the traveling operation of the vehicle V so that the vehicle V travels along the traveling route.

[0023] If the map information stored in the map database 11 is high-precision map information, the stop line A1 and the crosswalk B can be recognized from the road information, but if the map information does not include road marking information, the stop line A1 and the crosswalk B will be detected from images acquired by the imaging device 13, etc. When detecting road markings that extend in the width direction of the road, such as the stop line A1 and the crosswalk B, from an image, it is necessary to search the image in the width direction of the road, and if the vehicle's direction of travel changes significantly, there is a risk that the road markings will not be accurately recognized. Therefore, the control device 15 of this embodiment detects the stop line A1 and the crosswalk B from position information of feature points relative to link lines, using the processing described below.

[0024] 1 has a function of generating a driving route for the vehicle V and activating actuators of the vehicle V so that the vehicle V travels along the driving route. When performing driving assistance using the functions of the driving control unit 16, the control device 15 determines whether to use map information including link lines representing roads. If it is determined that driving assistance should be performed using map information including link lines representing roads, the control device 15 instructs the road sign detection device 20 to detect road markings using map information including link lines representing roads in order to generate a driving route X.

[0025] The feature acquisition unit 21 has a function of acquiring feature points of objects around the vehicle V. Feature points are characteristic parts of an object that are used to detect the object, such as edges that capture the object's outline, corners of the object, and parts of the object where brightness changes. The road sign detection device 20 acquires images from the imaging device 13 using the function of the feature acquisition unit 21 and extracts feature points from the images using a known algorithm such as SIFT (Scale-Invariant Feature Transform). The road sign detection device 20 may also acquire point cloud data from the distance measurement device 14 and acquire feature points from the point cloud data. For example, the road sign detection device 20 generates an image in which distance measurement points are plotted from the point cloud data and performs feature point extraction processing on the generated image.

[0026] The road marking detection device 20 of this embodiment acquires, in particular, information on feature points of objects indicating road markings extending in the width direction of the road on which the vehicle V is traveling, from a sensor mounted on the vehicle V. In this embodiment, the objects indicating road markings are white solid lines that are stop lines, white solid lines that are pedestrian crossings, etc., and the information on feature points of the objects includes position information of the edges of the white solid lines, information on the width of the white solid lines, information on the length of the white solid lines in the traveling direction, etc. Examples of sensors include an imaging device 13 and a distance measuring device 14.

[0027] FIG. 3 is a plan view showing an example of feature points extracted from an image acquired by the image capture device 13 in the driving scene shown in FIG. 2. The coordinate system shown in FIG. 3 is a global coordinate system. In the driving scene shown in FIG. 2, as shown in FIG. 3, feature points indicated by dashed lines G1 to G5 are extracted from the lane boundary lines of lanes L1 to L4, feature points indicated by dashed lines G6 and G7 are extracted from stop lines A1 and A2, and a feature point indicated by dashed line G8 is extracted from the crosswalk B. Each extracted feature point is arranged (plotted) on the XY plane of FIG. 3 based on the coordinate information of the feature point. Note that the number of feature points and the spacing between feature points can be set to appropriate values ​​depending on the feature point extraction process.

[0028] The position acquisition unit 22 has a function of acquiring position information of feature points relative to link lines. A link line is, for example, a line along the road that indicates the shape of the road corresponding to the road section between nodes. As an example, the link line is located on the center line in the width direction of the road, and one link line is set for each road section between nodes. Information about the link lines is included in the information about the links.

[0029] Nodes are placed on the link lines at predetermined distances (for example, every 1 to 3 meters). The nodes on the link lines have position information for the nodes. Examples of position information include coordinate information in the global coordinate system used in map information, information on the angle between a predetermined direction and the direction of the link line (direction of travel), and information on the travel distance from one node to another node on the link line. For example, the node closest to the start point of travel route X is set as the one node serving as the starting point.

[0030] Examples of the predetermined direction include the north direction on the map and the direction along the axes of the global coordinate system. The direction of the link line is, for example, the tangent direction of the link line at a node on the link line. The travel distance between nodes is calculated as the travel distance when traveling from one node to another node along the link line.

[0031] FIG. 4 is a plan view showing an example of link lines set for the road shown in FIG. 2. Link line Z shown in FIG. 4 is a line along the road located on the center line of the road in the width direction, and indicates the curvature of the road (lanes L1 to L4). Multiple nodes (shown as black diamonds) are provided on link line Z, and each node has information such as its coordinates in the global coordinate system, the angle formed between the Y-axis direction and the tangent direction of link line Z, and the travel distance from the node that is the starting point. Note that, for the sake of explanation, feature points indicated by dashed lines G1 to G5 in FIG. 3 are omitted in FIG. 4.

[0032] The road sign detection device 20 acquires position information in a coordinate system (hereinafter also referred to as the original coordinate system) of the information used to extract the feature points, using the function of the position acquisition unit 22. If the feature points are extracted from an image, the original coordinate system is the coordinate system used for the image, and if the feature points are extracted from point cloud data, the original coordinate system is the coordinate system used for the point cloud data. Next, the road sign detection device 20 converts the position information in the original coordinate system into position information in the global coordinate system. As an example, the road sign detection device 20 converts the original coordinate system into the global coordinate system using the relative position of the feature points with respect to the vehicle V in the original coordinate system.

[0033] The road sign detection device 20 then converts the position information in the global coordinate system into position information in the local coordinate system, and acquires position information of the feature point in the local coordinate system along the link line Z. The local coordinate system along the link line Z is a coordinate system in which the position information of the feature point is expressed by the distance from the link line Z and the traveling distance from a predetermined node. The road sign detection device 20 associates the feature point with the closest node on the link line Z, and rotates the feature point around the associated node by the angle formed by the predetermined direction and the direction of the link line. Furthermore, the road sign detection device 20 moves the rotated feature point along the link line Z by the traveling distance of the node at the center of rotation.

[0034] The position information of a feature point is information relating to the position of the feature point, and is not particularly limited. Examples include information on the coordinates of the feature point in an arbitrary coordinate system such as a local coordinate system, information on the distance and direction of one feature point relative to another feature point, and information on the distance between the feature point and the link line Z. Note that the position information of the feature point relative to the link line does not necessarily have to be position information of the feature point in a local coordinate system along the link line Z, and is not particularly limited as long as it is position information based on the link line Z.

[0035] In the example shown in FIG. 4 , the coordinates of each feature point in the global coordinate system have been acquired, so the road sign detection device 20 associates each feature point with a node on the link line Z. As an example, feature point C1 shown in FIG. 4 is associated with node N1. As another example, feature point C2 is associated with node N2. Next, the road sign detection device 20 rotates each feature point by an angle formed by a predetermined direction and the direction of the link line, using information held by the associated node about the angle. Then, the road sign detection device 20 moves each rotated feature point along the link line Z, using information about the travel distance held by the associated node.

[0036] The process of converting to the local coordinate system will be described in detail with reference to FIG. 5 . FIG. 5 is a plan view showing another example of a link line according to this embodiment. As shown in FIG. 5 , the link line Za curves to the right, and the angle θ formed between the Y-axis direction D1 and the tangent direction D2 of the link line Z at node N3 is assumed to be θ. Also, assume that feature point C3 is associated with node N3. In this case, the direction (travel direction) R of the link line Za at node N3 is expressed by the following equation (1): Here, x ａ and y ａ are the X and Y coordinates of the node N3 in the global coordinate system. In this case, the coordinates P of the feature point C3 in the global coordinate system ｒａｗ is expressed by the following equation (2), the coordinates P of the feature point C3 in the global coordinate system are ｒａｗ is rotated counterclockwise by an angle θ ｎｏｄｅ is expressed by the following equation (3). Here, xｂ and y ｂ are the X and Y coordinates of the feature point C3 in the global coordinate system. ｎｏｄｅ coordinate P ｗａｙ is expressed by the following equation (4). The travel distance s is the travel distance from the starting node to node N3.

[0037] The road sign detection device 20 performs the above-described series of processes on each of the feature points shown in Fig. 4 to convert the coordinates of the feature points in the global coordinate system shown in Fig. 4 into coordinates in the local coordinate system along the link line Z. Fig. 6 is a diagram showing an example of the distribution of feature points in the local coordinate system, with the vertical axis representing the distance d between the feature point and the link line Z and the horizontal axis representing the traveling distance s from the starting node.

[0038] The marking detection unit 23 has a function of detecting road markings extending in the width direction of the road from position information of feature points relative to the link line Z (for example, in a local coordinate system along the link line Z). There are no particular limitations on road markings extending in the width direction of the road, and examples include stop lines, crosswalks, speed reduction marks, and guidance strips.

[0039] Because the feature points of road markings extending in the width direction of the road are aligned perpendicular to the s-axis, in a local coordinate system in which the s-axis is the vertical axis, road markings cannot be detected by searching for peak positions in the d-axis direction. Therefore, the road marking detection device 20 uses a local coordinate system in which the vertical axis is the d-axis and the horizontal axis is the s-axis.

[0040] To search for peak positions in the local coordinate system shown in FIG. 6 , the road sign detection device 20 generates a histogram showing the distribution of feature points relative to a position on the link line from the position information of the feature points relative to the link line. The generated histogram is then used to search for peak positions on the link line where a predetermined number of feature points or more are present and adjacent positions have fewer feature points than the predetermined number, and road stop lines are detected from the peak positions. Alternatively or additionally, the road sign detection device 20 may search for peak ranges on the link line where multiple consecutive positions each have a predetermined number of feature points or more are present, and detect road crosswalks from the start and end positions of the peak ranges. The predetermined number can be set to an appropriate value within a range that allows accurate detection of road markings, depending on the settings of the feature point extraction process. The histogram, for example, is a histogram showing the distribution relative to the position of the node closest to the feature point on the link line.

[0041] 6, if the predetermined number is set to 4, the road sign detection device 20 searches for the number of feature points in the horizontal direction and detects peak positions X1 and X2 where there are five feature points and no feature points adjacent to them. The road sign detection device 20 then recognizes that there are stop lines at positions corresponding to peak positions X1 and X2, and detects stop lines A1 and A2 shown in FIG.

[0042] The road sign detection device 20 also searches for the number of feature points in the horizontal direction, and detects a range where there are four consecutive positions with four or more feature points as a peak range Y. Next, it obtains the start position Px and end position Py of the peak range Y. It then calculates the length of the peak range Y along the road from the difference between the start position Px and the end position Py, and recognizes the peak range Y as a crosswalk if the calculated length is equal to or greater than a certain length (for example, 3 m). As a result, the road sign detection device 20 recognizes that a crosswalk exists in the range corresponding to the peak range Y, and detects the crosswalk B shown in FIG. 2 .

[0043] On the other hand, if the length of peak range Y along the road is less than a certain length, the road marking detection device 20 recognizes that peak range Y is not a crosswalk. In this case, the road marking detection device 20 recognizes that peak range Y corresponds to a plurality of stop lines arranged in succession. Alternatively or additionally, the road marking detection device 20 may change the predetermined number and search for peak range Y again.

[0044] The road sign detection device 20 outputs information about the detected road signs to the driving control unit 16. The control device 15 generates a driving route using the road sign information acquired from the road sign detection device 20, and causes the vehicle V to drive along the driving route by autonomous driving control.

[0045] If the road is a one-way road, the road surface marking detection device 20 may acquire position information for the link line on both sides of the link line Z. Alternatively or in addition, if the road is a two-way road, the road surface marking detection device 20 may acquire position information for the link line on the side of the link line Z on which the target vehicle for driving assistance using the detected road marking is traveling.

[0046] As an example, the road shown in Fig. 2 has two-way traffic between lanes L1, L2 and lanes L3, L4, and the target vehicle for driving assistance is vehicle V, so the road sign detection device 20 acquires the coordinates of feature points in the local coordinate system on the side of link line Z on which vehicle V is traveling. In other words, the road sign detection device 20 acquires the coordinates of feature points on the left side of link line Z shown in Fig. 4, which corresponds to lanes L1, L2 on which vehicle V can travel, and generates the histogram shown in Fig. 7.

[0047] 7, unlike the histogram shown in Fig. 6, only road markings that may affect the travel of the vehicle V are detected. Specifically, the stop line A1 corresponding to the peak position X1 and the pedestrian crossing B corresponding to the peak range Ya are detected.

[0048] The road sign detection device 20 may generate a histogram for feature points that exist within a range of a predetermined distance from the link line Z. The predetermined distance can be set to an appropriate value within a range in which a pedestrian crossing on a road can be appropriately detected, for example, 3 to 6 m. If the predetermined distance M is approximately 3.5 m, the road sign detection device 20 acquires the coordinates of the feature points that are within the predetermined distance M from the link line Z, out of the feature points shown in FIG. 4, and generates the histogram shown in FIG. 8.

[0049] In the histogram shown in Fig. 8, similar to the histogram shown in Fig. 6, the stop line A1 corresponding to the peak position X1a, the stop line A2 corresponding to the peak position X2a, and the crosswalk B corresponding to the peak range Yb are detected. This allows road markings extending in the width direction to be accurately detected regardless of the road width. Furthermore, the detection of unnecessary stop lines can be suppressed, reducing the amount of calculation. Note that, since the histogram shown in Fig. 8 acquires fewer coordinates of feature points than the histogram shown in Fig. 6, a smaller predetermined number (e.g., 2) is set than in the histogram shown in Fig. 6.

[0050] Furthermore, when vehicle V is traveling at a position relatively far from the link line, in order to suppress detection of road markings at positions relatively close to the link line and reduce the amount of calculation, road marking detection device 20 may calculate the separation distance between link line Z and the target vehicle for driving assistance that uses the detected road markings, and acquire position information for the link line within a predetermined range from a reference position that is the separation distance away from link line Z in the direction of the target vehicle. The predetermined range can be set to an appropriate value within a range that allows appropriate detection of road markings for the lane in which the target vehicle for driving assistance is traveling, and is, for example, 2 to 4 m.

[0051] As an example, in Fig. 4 , the separation distance between link line Z and vehicle V, which is the target vehicle for driving assistance, is separation distance J, and the reference position that is separated from link line Z in the direction of vehicle V by separation distance J is reference position Ps. If the width of predetermined range K is approximately 3 m in the width direction of the road, the road sign detection device 20 acquires the coordinates of the feature points within predetermined range K that extend in the traffic direction of the road, out of the feature points shown in Fig. 4 , and generates the histogram shown in Fig. 9 .

[0052] Unlike the histogram shown in Fig. 6, the histogram shown in Fig. 9 detects only road markings within a predetermined range K corresponding to lane L1. Specifically, the stop line A1 corresponding to peak position X1 and the pedestrian crossing B corresponding to peak range Ya are detected. Note that the histogram shown in Fig. 9 obtains fewer coordinates of feature points than the histogram shown in Fig. 6, so a smaller predetermined number (e.g., 2) is set than for the histogram shown in Fig. 6.

[0053] If multiple peak positions are detected, the road surface marking detection device 20 may detect the stop line from the peak position with the most characteristic points. As an example, suppose that to the left of the traffic lane L1 shown in FIG. 2 , there is a narrower bicycle lane than the traffic lane L1, and a bicycle stop line is provided near the crosswalk B. In this case, the road surface marking detection device 20 may detect a peak position X1 corresponding to the stop line A1 and a peak position corresponding to the bicycle stop line, but it detects the stop line from the peak position with more characteristic points. In other words, because the bicycle stop line is shorter than the stop line A1 and has fewer characteristic points, it detects the stop line A1 from the peak position X1 with the most characteristic points.

[0054] The pavement sign detection device 20 may calculate the widthwise length of the crosswalk from the peak value in the peak range. For example, in the histogram shown in FIG. 7 , the pavement sign detection device 20 calculates the widthwise length of the crosswalk B as 2E from the peak value E in the peak range Ya, based on the assumption that the crosswalk B is line-symmetrical with respect to the link line Z. Furthermore, the pavement sign detection device 20 may set the predetermined number when traveling around an intersection to be smaller than the predetermined number when traveling outside the intersection. This can reduce erroneous detection of stop lines outside intersections.

[0055] For example, before acquiring position information in the local coordinate system, the road surface sign detection device 20 may determine whether the angle formed between a link line ahead of a certain node on the link line in the traveling direction and a link line behind the node in the traveling direction is equal to or greater than a predetermined angle. If the road surface sign detection device 20 determines that the angle formed between a link line ahead of a certain node in the traveling direction and a link line behind the node in the traveling direction is equal to or greater than the predetermined angle, it corrects the shape of the link line Z at the node and before and after it. The predetermined angle can be set to any appropriate value within a range in which conversion to the local coordinate system can be appropriately performed.

[0056] For example, within an intersection, the link line Zb may bend at a right angle, as shown in FIG. 10 . In this case, because the link line Zb bends at a right angle at node N5 within the intersection, the road sign detection device 20 determines whether the angle formed by the direction of the link line between nodes N4 and N5 and the direction of the link line between nodes N5 and N6 is 90°, and whether 90° is equal to or greater than a predetermined angle. If it is determined that 90° is equal to or greater than the predetermined angle, the road sign detection device 20 corrects the shape of the link line Zb at node N5 and the nodes N4 and N6 before and after it. On the other hand, if it is determined that 90° is less than the predetermined angle, the road sign detection device 20 does not correct the link line Zb.

[0057] A method for correcting the shape of the link line Zb may involve, for example, generating a curve F such as a Bezier curve at the right-angle bend of the link line Zb, and moving the nodes N4, N5, and N6 to the positions of the nodes N4a, N5a, and N6a on the curve F, respectively, or replacing the nodes N4, N5, and N6 with the nodes N4a, N5a, and N6a on the curve F, respectively. The nodes N4a, N5a, and N6a are the points on the curve F that are closest to the nodes N4, N5, and N6, respectively. Alternatively, instead of the curve F, an inscribed circle tangent to the nodes N4 and N6 may be generated, and the nodes N4, N5, and N6 may be moved to the points on the inscribed circle that are closest to the nodes N4, N5, and N6, respectively. Furthermore, an approximate curve may be generated by polynomial approximation using the coordinates of the nodes N4, N5, and N6, and the nodes N4, N5, and N6 may be moved to the points on the approximate curve that are closest to the nodes N4, N5, and N6.

[0058] 11 to 13, the procedure for processing information by the control device 15 will be described. The process described below is executed by a processor (CPU) included in the control device 15 at predetermined time intervals (for example, every 0.1 to 1 millisecond).

[0059] FIG. 11 is an example of a flowchart showing information processing executed by the driving assistance device 10.

[0060] First, in step S1, the feature acquisition unit 21 acquires information about feature points of an object that indicates a road marking from an image or the like, and then in step S2, the position acquisition unit 22 acquires position information of the feature points relative to the link lines. In step S3, the marking detection unit 23 detects road markings from the position information of the feature points relative to the link lines, and then in step S4, the driving control unit 16 performs driving assistance using the detected road markings.

[0061] Next, FIGS. 12 and 13 are flowcharts showing other examples of information processing executed by the driving assistance device 10. In FIG.

[0062] 12, feature points are extracted from the image and / or point cloud data by the function of the feature acquisition unit 21. In step S12, coordinates of the feature points in the original coordinate system of the image and / or point cloud data are acquired by the function of the position acquisition unit 22. In the subsequent step S13, the coordinate system of the acquired coordinates is converted from the original coordinate system to the global coordinate system. In the subsequent step S14, each feature point is associated with the nearest node on the link line Z.

[0063] In step S15, the coordinate system is transformed from the global coordinate system to the local coordinate system, and in the following step S16, the coordinates of the feature points in the local coordinate system are acquired. In step S17, it is determined whether the road to which the node belongs is a one-way street. If it is determined that the road to which the node belongs is not a one-way street, the process proceeds to step S18, where the coordinates of the feature points on the oncoming lane side of the link line Z are excluded, and the process proceeds to step S19 in Figure 13. On the other hand, if it is determined that the road to which the node belongs is a one-way street, the process proceeds to step S19.

[0064] 13, the position acquisition unit 22 acquires the coordinates of feature points located within a range of a predetermined distance M from the link line Z, and then in step S20, a histogram is generated with the horizontal axis representing the travel distance s and the vertical axis representing the distance d from the link line Z. In step S21, the sign detection unit 23 searches for a peak range from the histogram, and in step S22, the crosswalk B is detected from the start position Px and end position Py of the peak range. In step S23, the width of the crosswalk B is calculated from the peak value E of the peak range.

[0065] [Embodiments of the Present Invention] According to this embodiment, there is provided a road marking detection method executed by a road marking detection device 20 that detects road markings on a road using map information including link lines that represent the road, in which the road marking detection device 20 acquires information on feature points of objects that indicate the road markings on the road on which a vehicle V is traveling from a sensor mounted on the vehicle V, and detects the road markings extending in the width direction of the road from position information of the feature points relative to the link lines. This makes it possible to accurately detect road markings extending in the width direction of the road.

[0066] In the road marking detection method of this embodiment, the road marking detection device 20 may generate a histogram indicating the distribution of the feature points with respect to positions on the link line from the position information for the link line, search for peak positions on the link line from the histogram where a predetermined number of feature points or more exist and where adjacent positions have fewer than the predetermined number of feature points, and detect stop lines on the road from the peak positions. This allows stop lines to be detected accurately.

[0067] In the road marking detection method of this embodiment, the road marking detection device 20 may acquire the position information with respect to the link line on both sides of the link line if the road is one-way, and may acquire the position information with respect to the link line on the side of the link line on which a target vehicle for driving assistance using the road marking is traveling if the road is two-way. This makes it possible to accurately detect stop lines that may affect the traveling of the target vehicle.

[0068] In the road marking detection method of this embodiment, the road marking detection device 20 may generate the histogram for the feature points that exist within a range from the link line to a predetermined distance M. This makes it possible to accurately detect road markings that extend in the width direction, regardless of the road width.

[0069] In the road marking detection method of this embodiment, the road marking detection device 20 may calculate a separation distance J between the link line and a target vehicle for driving assistance using the road marking, and acquire the position information with respect to the link line within a predetermined range K from a reference position Ps that is separated from the link line in the direction of the target vehicle by the separation distance J. This makes it possible to accurately detect stop lines that may affect the travel of the vehicle V at intersections where the positions of stop lines differ for each lane.

[0070] In the road marking detection method of this embodiment, if multiple peak positions are detected, the road marking detection device 20 may detect the stop line from the peak position that has the most feature points. This makes it possible to prevent a stop line for a bicycle lane next to a roadway from being mistakenly recognized as a stop line for the vehicle V.

[0071] In the road marking detection method of this embodiment, the road marking detection device 20 may generate a histogram indicating the distribution of the feature points with respect to positions on the link line from the position information for the link line, search for a peak range on the link line from the histogram where multiple consecutive positions contain a predetermined number or more of the feature points, and detect a crosswalk on the road from the start position and end position of the peak range. This allows for accurate detection of crosswalks.

[0072] In the road marking detection method of this embodiment, the road marking detection device 20 may calculate the width of the crosswalk from the peak value in the peak range. This makes it possible to accurately detect the width of the crosswalk.

[0073] In the road marking detection method of this embodiment, the road marking detection device 20 may set the predetermined number when traveling around an intersection to be smaller than the predetermined number when traveling outside the intersection, thereby preventing erroneous detection of stop lines outside intersections.

[0074] In the road sign detection method of this embodiment, the road sign detection device 20 determines whether the angle formed by the link line ahead of the node on the link line in the direction of travel and the link line behind the node in the direction of travel is equal to or greater than a predetermined angle, and if it determines that the angle formed by the link line ahead of the node in the direction of travel and the link line behind the node in the direction of travel is equal to or greater than the predetermined angle, the road sign detection device 20 may correct the shape of the link line at the node and before and after it. This makes it possible to more accurately calculate the distance between the link line and the feature point.

[0075] Furthermore, according to this embodiment, there is provided a road sign detection device 20 that detects road markings on a road using map information including link lines representing the road, the road sign detection device 20 including: a feature acquisition unit 21 that acquires information on feature points of objects that indicate the road markings on the road on which a vehicle V is traveling from a sensor mounted on the vehicle V; and a marking detection unit 23 that detects the road markings extending in the width direction of the road from position information of the feature points relative to the link lines Z. This makes it possible to accurately detect road markings extending in the width direction of the road.

[0076] 10...driving assistance device, 11...map database, 12...navigation device, 13...imaging device, 14...distance measuring device, 15...control device, 16...driving control unit, 20...road marking detection device, 21...feature acquisition unit, 22...position acquisition unit, 23...marking detection unit, A1, A1a, A1b, A2, A3...stop line, B, Ba...pedestrian crossing, C1, C2, C3, C4...feature point, D1...Y-axis direction, D2...tangential direction, E...peak value, F...curve, G1, G2, G3, G4, G5, G6, G7, G8...dashed lines, J...separation distance, K...predetermined range, L1, L2, L3, L4...lane, M...predetermined distance, N1, N2, N3, N4, N5, N6...node, Pv...position, Ps...reference position, Px...start position, Py...end position, V...vehicle, X1, X1a, X2, X2a...peak position, Y, Ya, Yb...peak range, Z, Za, Zb...link line

Claims

1. In a road marking detection method performed by a road marking detection device that detects road markings using map information, The aforementioned road marking detection device is, The link lines representing the roads included in the aforementioned map information are obtained, Information on the characteristic points of objects representing the road markings on the road on which the vehicle is traveling is acquired from sensors mounted on the vehicle. A road marking detection method for detecting road markings extending in the width direction of the road from the positional information of the feature points relative to the link line.

2. The aforementioned road marking detection device is, From the position information of the link line, a histogram is generated showing the distribution of the feature points with respect to their positions on the link line. From the histogram, a peak position on the link line is searched for where there are a predetermined number or more of the feature points, and where the number of feature points at adjacent positions is less than the predetermined number. The road marking detection method according to claim 1, wherein the stop line of the road is detected from the aforementioned peak position.

3. The aforementioned road marking detection device is, If the aforementioned road is one-way, the position information for the link line is acquired on both sides of the link line. The road marking detection method according to claim 2, wherein, if the road is a two-way road, the position information with respect to the link line is acquired on the side of the link line on which the vehicle subject to driving assistance using the road marking is traveling.

4. The road marking detection method according to claim 2 or 3, wherein the road marking detection device generates the histogram for the feature points that exist within a predetermined distance from the link line.

5. The aforementioned road marking detection device is, The distance between the aforementioned link line and the vehicle targeted for driving assistance using the aforementioned road markings is calculated. A road marking detection method according to any one of claims 2 to 4, wherein the position information with respect to the link line is acquired within a predetermined range from a reference position that is separated by the distance from the link line in the direction of the target vehicle.

6. The road marking detection method according to any one of claims 2 to 5, wherein the road marking detection device detects a plurality of peak positions, and detects the stop line from the peak position where the most feature points are located.

7. The aforementioned road marking detection device is, From the position information of the link line, a histogram is generated showing the distribution of the feature points with respect to their positions on the link line. From the histogram, search for a peak range on the link line where multiple locations with a predetermined number or more of the feature points are consecutive. A road marking detection method according to any one of claims 1 to 6, wherein a pedestrian crossing on the road is detected from the start position and end position of the peak range.

8. The road marking detection method according to claim 7, wherein the road marking detection device calculates the length of the pedestrian crossing in the width direction from the peak value of the peak range.

9. The road marking detection method according to any one of claims 2 to 8, wherein the road marking detection device sets the predetermined number when driving around an intersection to be smaller than the predetermined number when driving outside the vicinity of an intersection.

10. The aforementioned road marking detection device is, It is determined whether the angle formed by the link line forward in the direction of travel from the node on the link line and the link line backward in the direction of travel from the node is greater than or equal to a predetermined angle. A road marking detection method according to any one of claims 1 to 9, wherein if it is determined that the angle formed by the link line forward of the node in the direction of travel and the link line backward of the node in the direction of travel is greater than or equal to the predetermined angle, the shape of the link line at the node and before and after it is corrected.

11. A road marking detection device that detects road markings using map information, The link lines representing the roads included in the aforementioned map information are obtained, A feature acquisition unit acquires information on the feature points of objects representing the road markings on the road the vehicle is traveling on from sensors mounted on the vehicle, A road marking detection device comprising: a marking detection unit that detects the road markings extending in the width direction of the road from the positional information of the feature points relative to the link line.