Sidewalk edge detection device, sidewalk edge detection method, and sidewalk edge detection program

Abstract

This sidewalk edge detection device acquires a set of lines corresponding to a sidewalk edge from point cloud data as sidewalk edge information, and comprises: a candidate point detection unit that detects each sidewalk edge candidate point corresponding to a candidate for a sidewalk edge from the point cloud data; a candidate point clustering unit that clusters each sidewalk edge candidate point; an adjacent cluster reduction unit that reduces clusters on the basis of the distributions of points in adjacent clusters using a predetermined cluster reduction method; a line fitting unit that fits one or more straight or curved lines to one or more clusters and outputs the fitted lines as sidewalk edge candidates; a line connecting unit that connects some of the fitted lines using a predetermined analysis method; and an information output unit that outputs calculated lines as the sidewalk edge information.

Description

[Technical field]

[0001] The disclosed technology relates to a sidewalk edge detection device, a sidewalk edge detection method, and a sidewalk edge detection program, and in particular to a technology for extracting points corresponding to sidewalk edges from point cloud data and expressing them as lines. [Background technology]

[0002] There is a device called MMS (Mobile Mapping System) that acquires shape data of three-dimensional space by mounting various measuring devices on a vehicle. By using LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) as a measuring device, it is possible to acquire point cloud data, and by combining this with position information acquired by GPS (Global Positioning System) etc., it is possible to acquire three-dimensional point cloud information of the space.

[0003] By analyzing the acquired 3D point cloud information, it is possible to acquire the exact positions of structures on the ground and construct a highly accurate 3D map. For example, there is a technology shown in Non-Patent Document 1 for detecting the sidewalk boundary, which indicates the step caused by the curb between the roadway and the sidewalk.

[0004] By applying the technology in Non-Patent Document 1 to point clouds measured from vehicles parked on the street or from roads without fine undulations, it is possible to detect points corresponding to the sidewalk edges with high accuracy, apply lines by curve fitting, and obtain lines representing the sidewalk edges. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] ZAI, Dawei, et al. 3-D road boundary extraction from mobile laser scanning data via supervoxels and graph cuts. IEEE Transactions on Intelligent Transportation Systems, 2017, 19.3: 802-813. Summary of the Invention [Problem to be solved by the invention]

[0006] In the technology shown in Non-Patent Document 1, when there is an object that blocks the laser from the LiDAR, such as a vehicle parked on the road as shown in FIG. 11A, the point cloud of the vehicle part is erroneously detected as the sidewalk edge, and an incorrect line is acquired. In addition, when the road surface is rough and has small undulations, the points extracted from the raised part are recognized as steps, and the line is applied to a position different from the actual sidewalk edge, which causes a decrease in accuracy. (1) in FIG. 11B is an image of the acquired point cloud, and the higher the point, the larger it is displayed. (2) in FIG. 11B is an image of the result of applying a line to the sidewalk edge.

[0007] The disclosed technology has been made in consideration of the above-mentioned circumstances, and performs processing using point cloud data to remove erroneously detected lines when there are obstructions such as parked vehicles on the roadway, and to robustly analyze elevation differences even when the roadway has small undulations. The purpose of the disclosure is to provide a sidewalk edge detection device, a sidewalk edge detection method, and a sidewalk edge detection program that are capable of acquiring lines corresponding to the sidewalk edge with high accuracy. [Means for solving the problem]

[0008] A first aspect of a sidewalk edge detection device of the present disclosure is a sidewalk edge detection device that acquires a set of lines corresponding to sidewalk edges from point cloud data as sidewalk edge information, and is configured to include a candidate point detection unit that detects each point of sidewalk edge candidates that correspond to sidewalk edge candidates from the point cloud data, a candidate point clustering unit that clusters each point of the sidewalk edge candidates, an adjacent cluster reduction unit that reduces clusters using a predetermined cluster reduction method based on the distribution of points within adjacent clusters, a line fitting unit that fits one or more straight lines or curves to one or more of the clusters and outputs the fitted lines as the sidewalk edge candidates, a line connecting unit that connects parts of the fitted lines using a predetermined analysis method, and an information output unit that outputs the calculated lines as the sidewalk edge information.

[0009] Moreover, a second aspect of the sidewalk edge detection device is a sidewalk edge detection device that acquires a set of lines corresponding to sidewalk edges from point cloud data as sidewalk edge information, and is configured to include a point cloud density reduction unit that downsamples the point cloud data to reduce its density, a candidate point detection unit that detects each point of sidewalk edge candidates that correspond to sidewalk edge candidates from the reduced point cloud data, a candidate point clustering unit that clusters each point of the sidewalk edge candidates, an adjacent cluster reduction unit that reduces clusters using a predetermined cluster reduction method based on the distribution of points within adjacent clusters, a line fitting unit that fits one or more straight lines or curves to one or more of the clusters and outputs the fitted lines as the sidewalk edge candidates, a line connecting unit that connects parts of the fitted lines using a predetermined analysis method, and an information output unit that outputs the calculated lines as the sidewalk edge information.

[0010] The sidewalk edge detection method disclosed herein is a sidewalk edge detection method in which a computer obtains a set of lines corresponding to sidewalk edges from point cloud data as sidewalk edge information, and causes the computer to execute the following processes: detect each point of sidewalk edge candidates corresponding to sidewalk edge candidates from the point cloud data, cluster each point of the sidewalk edge candidates, reduce clusters using a predetermined cluster reduction method based on the distribution of points within adjacent clusters, fit one or more straight lines or curves to one or more of the clusters, output the fitted lines, connect parts of the fitted lines, and output the calculated lines as the sidewalk edge information.

[0011] The sidewalk edge detection program disclosed herein is a sidewalk edge detection program that acquires a set of lines equivalent to sidewalk edges from point cloud data as sidewalk edge information, and causes a computer to execute the following processes: detect each point of sidewalk edge candidates that correspond to sidewalk edge candidates from the point cloud data, cluster each point of the sidewalk edge candidates, reduce clusters using a predetermined cluster reduction method based on the distribution of points within adjacent clusters, fit one or more straight lines or curves to one or more of the clusters, output the fitted lines, connect parts of the fitted lines, and output the calculated lines as the sidewalk edge information. Effect of the Invention

[0012] According to the disclosed technology, a line corresponding to the sidewalk edge can be obtained with high accuracy. [Brief description of the drawings]

[0013] [Figure 1] 2 is a block diagram showing a hardware configuration of the sidewalk edge detection device. FIG. [Diagram 2] 2 is a block diagram showing a functional configuration of a sidewalk edge detection device. FIG. [Diagram 3] In the block diagram showing the functional configuration of the sidewalk edge detection device, a point cloud density reduction unit is added. [Figure 4] 4 is a flowchart showing a flow of sidewalk edge detection by the sidewalk edge detection device. [Figure 5A]FIG. 5A is an example of a clustering process. [Figure 5B] FIG. 5B is an example regarding the clustering process. [Figure 5C] FIG. 5C is an example regarding the clustering process. [Figure 5D] FIG. 5D is an example of a clustering process. [Figure 6A] FIG. 6A is an example regarding adjacent cluster reduction processing. [Figure 6B] FIG. 6B is an example regarding adjacent cluster reduction processing. [Figure 6C] FIG. 6C is an example regarding adjacent cluster reduction processing. [Figure 7A] FIG. 7A is an example of a classification process. [Figure 7B] FIG. 7B is an example of a classification process. [Figure 7C] FIG. 7C is an example of a classification process. [Figure 7D] FIG. 7D is an example of a classification process. [Figure 8A] FIG. 8A is an example of a line fit to a straight segment. [Figure 8B] FIG. 8B is an example of fitting a line to a curved segment. [Figure 9A] FIG. 9A shows an example of lines to be connected. [Figure 9B] FIG. 9B shows an example of lines that are not targets for connection. [Figure 10] 13 is a subroutine for detecting candidate sidewalk edge points. [Figure 11A] FIG. 11A is an image of sidewalk edge detection. [Figure 11B] FIG. 11B is an image of the point cloud and an image of the line fit to the sidewalk edge. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. In addition, the dimensional ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0015] The configuration of this embodiment will be described below.

[0016] FIG. 1 is a block diagram showing the hardware configuration of a sidewalk edge detection device 100. As shown in FIG.

[0017] 1, the sidewalk edge detection device 100 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0018] The CPU 11 is a central processing unit, and executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 14, and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculation processes according to the program stored in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores a sidewalk edge detection program.

[0019] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system, and various data.

[0020] The input unit 15 includes a pointing device such as a mouse, and a keyboard, and is used to perform various input operations.

[0021] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may function as the input unit 15 by adopting a touch panel system.

[0022] The communication interface 17 is an interface for communicating with other devices such as terminals. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI, or a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used.

[0023] Next, a description will be given of each functional configuration of the sidewalk edge detection device 100. Fig. 2 is a block diagram showing the functional configuration of the sidewalk edge detection device of this embodiment. Each functional configuration is realized by the CPU 11 reading out the sidewalk edge detection program stored in the ROM 12 or the storage 14, expanding it in the RAM 13, and executing it.

[0024] As shown in FIG. 2, the sidewalk edge detection device 100 includes a candidate point detection unit 110, a candidate point clustering unit 112, an adjacent cluster reduction unit 114, a line fitting unit 116, a line connection unit 118, and an information output unit 120. The sidewalk edge detection device 100 may further include a point cloud density reduction unit 108. FIG. 3 is a block diagram showing a functional configuration in which the point cloud density reduction unit 108 is added to the sidewalk edge detection device 100. In the following description, it is assumed that the sidewalk edge detection device 100 includes the point cloud density reduction unit 108. However, if the sidewalk edge detection device 100 does not include the point cloud density reduction unit 108, the point cloud data that is not downsampled is input to the processing of each unit. Whether or not to include the point cloud density reduction unit 108 may be determined depending on the noise condition of the sampled point cloud data.

[0025] The sidewalk edge detection device 100 accepts input of point cloud data. The point cloud data is three-dimensional point cloud information acquired by a Lidar or the like and information on the position trajectory of a measuring instrument that measured the point cloud. It is assumed that the measuring instrument moves along the road generally in the direction of the sidewalk edge to perform measurements. Below, an overview of the processing of each unit will be explained, and the details of the processing will be explained in the later explanation of the operation.

[0026] The point cloud density reduction unit 108 reduces the point density of the input point cloud data by downsampling, and outputs the reduced point cloud data. Note that for each partial space obtained by dividing the three-dimensional space in the point cloud data, if the point density is below a certain level, if the point is at a height above a certain level from the position of the measuring instrument, or if the point is at a distance above a certain level from the position information of the measuring instrument, it is possible to delete points in the partial space based on the point density or position information, and not use them in subsequent processing.

[0027] The candidate point detection unit 110 receives point cloud data, extracts only points thought to correspond to sidewalk edges using a sidewalk edge candidate point detection technique, and outputs each sidewalk edge candidate point. Any technique can be used for the sidewalk edge candidate point detection technique, and for example, the sidewalk edge candidate point detection technique described in Non-Patent Document 1 can be used. A detailed processing routine for detecting sidewalk edge candidate points will be described later.

[0028] The candidate point clustering unit 112 receives a group of points that are sidewalk edge candidates, clusters the points of the sidewalk edge candidates so that points aligned in the same direction are in the same cluster, and outputs point information for each cluster.

[0029] The adjacent cluster reduction unit 114 receives point information for each cluster, calculates the principal component direction of the points in the cluster, and searches for clusters adjacent to the cluster. The adjacent cluster reduction unit 114 uses a cluster reduction method to compare the distribution of points in adjacent clusters, and eliminates clusters in which points are not distributed in a linear, elongated manner from sidewalk edge candidates, thereby reducing the number of clusters.

[0030] The line fitting unit 116 receives information on the sidewalk edge candidate points and information on the position trajectory of the measuring instrument, determines whether to fit the neighboring sidewalk edge candidate points as straight lines or curved lines according to the shape of the position trajectory, and outputs the fitted lines. The line fitting unit 116 fits the straight line and curved line parts with one or more lines so as to reduce the position error with the sidewalk edge candidate points, and outputs the lines as the sidewalk edge candidates.

[0031] The line connection unit 118 uses a predetermined analysis method to connect lines with high continuity among the lines that have been fitted as sidewalk edge candidates. The analysis method uses a method of analyzing the continuity of nearby lines using information such as the distance between the end points, the difference in line direction at the end points, the intersection of lines, the length of a perpendicular line dropped from the end point of one line to the other line, and the length of the lines. The line connection unit 118 may analyze the length of the lines after connection and exclude lines with a length equal to or less than a certain value from the sidewalk edge candidates. The line connection unit 118 may also analyze the continuity by combining a plurality of these conditions. In this manner, the line connection unit 118 connects parts of the fitted lines based on the analysis result of the continuity.

[0032] The information output unit 120 outputs the calculated line as sidewalk edge information. The calculated line is a line fitted by the processing of the line fitting unit 116, or a line connecting the lines fitted by the processing of the line connecting unit 118.

[0033] Next, a description will be given of the operation of the sidewalk edge detection device 100. Fig. 4 is a flowchart showing the flow of the sidewalk edge detection process by the sidewalk edge detection device 100. The CPU 11 reads out a sidewalk edge detection program from the ROM 12 or the storage 14, expands it in the RAM 13, and executes it to perform the sidewalk edge detection process.

[0034] In step S100, the CPU 11 receives point cloud data as an input. The point cloud data includes three-dimensional point cloud information and information on the position trajectory of the measuring instrument that measured the point cloud.

[0035] The processes of steps S102 and S104 are executed by the CPU 11 as processes of the point cloud density reducing unit 108.

[0036] In step S102, the CPU 11 downsamples the points of the point cloud data to reduce the point density.

[0037] In downsampling, for example, each axis of a three-dimensional space is divided at regular intervals to create voxels, the center of gravity of the points in the voxel is set as the representative point of that voxel, and other points are deleted to reduce the point density.

[0038] Next, the process of step S104 is executed by the CPU 11 in Loop A1 as the process of the point cloud density reducing unit 108. In Loop A1, the loop process is executed for each point of the point cloud data after the point density is deleted.

[0039] In step S104, if the density of surrounding points is equal to or less than a threshold, the CPU 11 deletes the point, and excludes it from the processing targets of each subsequent step. For each point after the deletion, points within a certain distance of the point may be counted as surrounding points, and if the density is equal to or less than the threshold, the point may be determined to be a location with a large measurement error and deleted. Also, if the relative altitude of a point is equal to or greater than a threshold in comparison with the position information of a nearby measuring instrument, the point may be determined to be not a sidewalk edge and deleted. Also, if the distance from the position information of a nearby measuring instrument is equal to or greater than a threshold, the point may be determined to be not a sidewalk edge and deleted, or a point on the left or right side in comparison with the traveling direction of a nearby measuring instrument may be excluded from the sidewalk edge detection and deleted.

[0040] The process of step S106 is executed by the CPU 11 as the process of the candidate point detection unit 110.

[0041] In step S106, the CPU 11 detects sidewalk edge candidate points from the reduced point cloud data. The details of the detection process will be described later as a subroutine.

[0042] Next, step S108 is executed by the CPU 11 in Loop B1 and Loop B2 as processing of the candidate point clustering unit 112. In Loop B1, loop processing is performed for each point of the point cloud data, and in Loop B2, loop processing is performed for the peripheral points of the point.

[0043] In step S108, the CPU 11 inputs a group of points that are candidates for sidewalk edges, and calculates weighting values ​​for the peripheral points of the input points based on the principal component vectors of the angles made by line segments connecting the points, such that points lined up in the same direction are grouped in the same cluster (groups of points lined up in parallel are grouped in different clusters).

[0044] In step S109, the CPU 11 performs clustering on the points of the sidewalk edge candidates based on the calculated weight values, and outputs point information for each cluster.

[0045] Examples of the clustering process in steps S108 and S109 are shown in Figs. 5A to 5D. Fig. 5A shows a group of points that are input sidewalk edge candidates. In this process, first, for each candidate point, n neighboring points (peripheral points) are extracted and subjected to principal component analysis, and the first principal component is set as the principal component vector of the point. Fig. 5B shows the principal component vectors calculated from the neighboring points of each point. Next, in this process, a graph is constructed that connects the neighboring points of all the candidate points. Fig. 5C shows the edge weights at the time of clustering of the constructed graph. The edge weights of the graph are defined so that they become larger as the principal component vectors face the same direction and are continuous in the direction of the principal component vectors. For example, as shown in Fig. 5, the weights of points A and B are set to the weight value w in the following equation (1): A,B Figure 5D shows the clustering results, showing that points in the same direction are in the same cluster, and sets of points that are parallel to each other are in different clusters.

number

[0046] Where θ Ais the angle between the principal component vector of point A and the line segment connecting point A and point B, expressed in degrees from 0 to 90 degrees. θ B is a value similarly expressed for point B. Graph clustering is applied to the graph to cluster the candidate points. Any method can be used for graph clustering, and for example, the graph clustering method shown in Non-Patent Document 2 can be used. [Non-Patent Document 2] FELZENSZWALB, Pedro F.; HUTTENLOCHER, Daniel P. Efficient graph-based image segmentation. International journal of computer vision, 2004, 59.2: 167-181.

[0047] Next, the processes of steps S110 to S116 are executed by the CPU 11 as processes of the adjacent cluster reduction unit 114. The processes of steps S110 and S112 are performed in Loop C1 and Loop C2. The processes of the loop are performed for each cluster in Loop C1, and for the neighboring clusters of the cluster in Loop C2. The processes of steps S110 to S116 are an example of a predetermined cluster reduction method of the present disclosure.

[0048] In step S110, the CPU 11 receives information on the points of the cluster, calculates the principal component directions (principal component vectors) of the points in the cluster, and searches for clusters adjacent to the cluster.

[0049] In step S112, the CPU 11 judges whether or not the neighboring clusters found for the cluster in question are adjacent to the cluster in question. If it is judged that the neighboring clusters are adjacent to the cluster in question, the process proceeds to step S114, and if it is judged that the neighboring clusters are not adjacent to the cluster in question, the process proceeds to step S116.

[0050] In step S114, the CPU 11 updates the cluster scores of the surrounding clusters that are determined to be adjacent to each other.

[0051] Examples of the adjacent cluster reduction process in steps S110 to S114 are shown in Figs. 6A to 6C. Fig. 6A shows an example of each cluster created by the candidate point clustering unit 112, and of the line segments determined by connecting the end points of the clusters. In this process, the principal component vector of the cluster is calculated from the set of points included in the cluster, and the points at both ends in the direction of the principal component vector of the cluster are set as end points and are set as line segments corresponding to the cluster. Next, in this process, it is determined whether clusters existing around the cluster are adjacent. Fig. 6B shows adjacent columns and non-adjacent columns. If the foot of a perpendicular line drawn from the midpoint of the line segment of the surrounding cluster to a straight line connecting the two end points of the cluster is included between the two end points, it is determined that the two clusters are adjacent. Each cluster has a score that indicates the slenderness of the points that belong to it. For example, S in the following formula (2) is used as the score of cluster A. A It can be calculated as follows.

number

[0052] However, A is the difference between the maximum and minimum values ​​when the points belonging to cluster A are projected onto an axis parallel to the principal component vector (length l), and w A represents the difference between the maximum and minimum values ​​(width w) when the points belonging to cluster A are projected onto an axis perpendicular to the principal component vector. Figure 6C shows the relationship between length l and width w in score calculation.

[0053] In step S116, the score values ​​of adjacent clusters are compared, and points belonging to the cluster with the smaller score value are deleted from the sidewalk edge candidates. In other words, the cluster is deleted. For example, for adjacent clusters A and B, A and S. B The values ​​of are compared, and the points belonging to the smaller cluster are deleted from the sidewalk edge candidates. Note that if there are multiple adjacent clusters for a given cluster, it may be decided to delete a cluster whose score is smaller than that of the majority of the adjacent clusters. In this way, the distribution of points within the clusters is compared.

[0054] Next, the CPU 11 executes the processes of steps S118 to S122 as the processes of the line fitting unit 116. The processes of step S120 are performed in Loop D1 and Loop D2, and the processes of step S122 are performed in Loop E1. The processes of loops are performed for each straight line portion in Loop D1, and for each cluster corresponding to the portions in Loop D2. The processes of loops are performed for each curved line portion in Loop E1. The processes of steps S118 to S122 are an example of a predetermined analysis method of the present disclosure.

[0055] In step S118, the CPU 11 classifies the information on the position trajectory of the measuring instrument into a straight line portion and a curved line portion as the type of line to be fitted to the subset of the sidewalk edge candidate points.

[0056] An example of the classification process in step S118 is shown in FIG. 7. FIG. 7A is an example of the position trajectory of a measuring device. In the position trajectory, the positions of the measuring device at regular intervals are set as measurement points, and the direction connecting the positions of the measurement points at a certain time and the measurement points at the next time is set as the moving direction of the measurement points. The measurement points are clustered based on the moving direction, and the measurement points before and after the point where the clusters change when the measurement points are viewed in chronological order are defined as curved parts, and the other parts are defined as straight parts. FIG. 7B shows the clustering of the position trajectory based on the moving direction. Note that different curved parts and straight parts are defined for each cluster of measurement points. FIG. 7C shows the classification of straight parts and curved parts. Then, the type of line to be applied is determined depending on whether the nearest measurement point for each sidewalk edge candidate point is a straight part or a curved part. FIG. 7D shows whether the sidewalk edge candidate point is classified as a straight part or a curved part.

[0057] In step S120, the CPU 11 fits a straight line to the points in each cluster corresponding to each straight line portion.

[0058] In step S122, the CPU 11 fits a curve to the points of the entire cluster for each cluster corresponding to the curve portion.

[0059] An example of the line fitting process is shown in FIG. 8. FIG. 8A shows fitting of a line to a straight line portion. The straight line portion is fitted with one line segment for each cluster of candidate points assigned by the candidate point clustering unit. (1) Sidewalk edge candidate points belonging to the same cluster of position trajectories. (2) Line fitting for each candidate point cluster. (3) Line fitting result. Note that if points in a cluster of candidate points are divided into two or more straight or curved parts and associated with each other, fitting is performed as separate lines. The line segment to be fitted may be determined by any method, for example, as a line segment connecting both end points of the principal component vector direction, or may be determined by Hough transform. In addition, for a straight line connecting any candidate points, a value according to the distance between each point and the straight line is added up to calculate the fitting error, and the line segment with the smallest error may be determined.

[0060] FIG. 8B shows fitting of a line to a curved portion. Points corresponding to the curved portion are fitted as one circular arc regardless of the cluster of the candidate points. (1) Sidewalk edge candidate points belonging to the same cluster of position trajectories. (2) Fitting of a curve and a straight line to the entire candidate point cluster. The second and third from the top show two lines fitted to the boundary of the thick-framed points. (3) Line fitting result. Note that fitting may be performed as a combination of multiple line segments and arcs with one or more of the candidate points as division points and the division points as boundaries. The circular arc to be fitted to the curved portion may be determined by any method. For example, among circles of any radius that pass through both end points of the constituent points of the curved portion, the circle with the smallest error between the circular arc and the constituent points may be determined as the circular arc divided by the central angle corresponding to both end points. In addition, a smooth connection at the division points may be a constraint.

[0061] The processes of steps S124 and S126 are executed by the CPU 11 as processes of the line connection unit 118. The process of step S124 is performed in LoopF1 and LoopF2. The process of step S126 is performed in LoopG1. Loop processing is performed for each line in LoopF1, and for the surrounding lines of the line in LoopF2. Loop processing is performed for each line in LoopG1.

[0062] In step S124, the CPU 11 connects lines of sidewalk edge candidates that have high continuity. Continuity is defined by the distance between the end points, the difference in line direction at the end points, the intersection of the lines, the length of the perpendicular line dropped from the end point of one line to the other line, the length of the line, etc. The continuity of the fitted lines is judged based on any one of these conditions or a combination of a plurality of these conditions, and it is determined whether the lines are to be connected or not.

[0063] In step S126, the CPU 11 excludes, from the sidewalk edge candidates, any line that is equal to or shorter than a predetermined length (shorter line) among the connected lines.

[0064] FIG. 9 shows examples of lines to be connected and lines that are not to be connected. FIG. 9A shows examples of lines to be connected, and FIG. 9B shows examples of lines that are not to be connected, (B1) to (B5). The example shown in FIG. 9A shows examples of lines with high continuity before and after connection. The examples in FIG. 9B are (B1) a case where the distance between the end points is long, and (B2) a case where the difference in direction of the lines at both end points deviates from 180°. Also, the examples in FIG. 9B are (B3) a case where lines intersect (including a multi-level intersection), (B4) a case where the distance from the end point of one line to the foot of a perpendicular line dropped to the other line is long, and (B5) a case where the length of the line is short.

[0065] In step S128, the CPU 11 outputs the line calculated by the processes in steps S100 to S126 as sidewalk edge information.

[0066] Next, the subroutine for detecting sidewalk edge candidate points in step S106 will be described with reference to FIG.

[0067] In step S200, the CPU 11 receives the reduced point cloud data as an input.

[0068] In step S202, the CPU 11 clusters the points based on the coordinates of the points in the point cloud data, the normal direction calculated from the surrounding points of the points, and the reflection intensity of the laser irradiated from the measuring device. As a result, points that are close to each other, form the same local plane, and are homogeneous in color and material form a cluster.

[0069] In LoopH1, loop processing (S204) is performed for each cluster.

[0070] In step S204, for each calculated cluster, CPU 11 leaves only the points that exist on the outer edge of the cluster and deletes the other points from the cluster. For detecting the points on the outer edge of the cluster, for example, the alpha shape calculation algorithm shown in Non-Patent Document 3 can be used. [Non-Patent Document 3] EDELSBRUNNER, Herbert; KIRKPATRICK, David; SEIDEL, Raimund. On the shape of a set of points in the plane. IEEE Transactions on information theory, 1983, 29.4: 551-559.

[0071] The sidewalk edge is assumed to be composed of vertical planes, while the road and sidewalk on both sides of the sidewalk edge are assumed to be composed of horizontal planes. Therefore, points other than clusters that are located on the boundary between two planes with different angles are removed based on the normal direction of each cluster. Specifically, the normal direction calculated from the points included in each cluster is compared with the normal directions of the surrounding adjacent clusters to calculate the angle difference, and if the angle difference with the normals of all surrounding clusters is within a threshold, it is determined that the cluster is not located on the boundary between two planes, and the cluster is removed.

[0072] In addition, even if there is a neighboring cluster where the angle difference is equal to or greater than the threshold, many of the points on the outer edge of the cluster are not located on the boundary between two planes and belong to only one plane. Therefore, a threshold judgment is performed between the line segments of the cluster based on the angle difference of the normals.

[0073] In LoopI1, loop processing (S206 to S218) is performed for each cluster. In LoopI2, loop processing (S206 to S210) is performed for the surrounding clusters. In LoopJ1 nested in LoopI2, loop processing (S210) is performed for the constituent points of the cluster.

[0074] In step S206, the CPU 11 calculates the angle difference between each of the constituent points of both the target cluster and the surrounding clusters and the center of gravity of the cluster.

[0075] In step S208, CPU 11 judges whether the angle between the line segment connecting the center of gravity of the cluster and the center of gravity of the surrounding cluster, whose angle difference is equal to or greater than the threshold, and the line segment connecting the center of gravity of the cluster and the constituent point is equal to or greater than the threshold. If it is judged to be equal to or greater than the threshold, the process proceeds to step S210, and if it is judged not to be equal to or greater than the threshold, the process proceeds to step S212.

[0076] In step S210, if the difference is equal to or greater than the threshold, the CPU 11 determines that the point belongs to only one surface, and deletes the point.

[0077] In step S212, the CPU 11 determines whether the angular difference between the cluster and all surrounding clusters is within a threshold value. If it is determined that the angular difference is within the threshold value, the process proceeds to step S214, and if it is determined that the angular difference is not within the threshold value, the process proceeds to step S216.

[0078] In step S214, the CPU 11 determines that the angular difference from all surrounding clusters is within a threshold value and deletes the cluster.

[0079] In step S216, the CPU 11 judges whether the elevation difference between the cluster and all surrounding clusters is within a threshold value. If it is judged that the elevation difference is not within the threshold value, the process proceeds to step S218, and if it is judged that the elevation difference is within the threshold value, the process proceeds to step S220.

[0080] In step S218, CPU 11 deletes the cluster because the difference in elevation between the cluster and the neighboring cluster is not within the threshold. In other words, if there is a neighboring cluster whose difference in elevation between the cluster and the neighboring cluster is equal to or greater than the threshold, the cluster is determined to be an area that is not a sidewalk edge and has a step, and the cluster is deleted.

[0081] In step S220, the CPU 11 performs two-class classification into sidewalk edge candidate points and other points so that points aligned parallel to the position trajectory of the measuring instrument are classified into the same class. When a lidar mounted on an MMS or the like is used as a measuring instrument to travel along a roadway and measure a point cloud, it is assumed that the sidewalk edge exists so as to be approximately parallel to the position trajectory of the measuring instrument. Objects existing parallel to the position trajectory of the measuring instrument include objects other than the sidewalk edge, such as buildings and structures on the sidewalk.

[0082] Therefore, in step S222, the CPU 11 performs clustering on all points included in the class of sidewalk edge candidate points based on the relative heights from nearby measurement position trajectory points.

[0083] In step S224, the CPU 11 adopts only the points included in the cluster whose average relative altitude is closest to 0, and deletes the points included in the other clusters. This makes it possible to exclude points other than objects that exist at the same height as the roadway.

[0084] As described above, the sidewalk edge detection device 100 of this embodiment performs processing using point cloud data to remove erroneously detected lines when there are obstructions such as parked vehicles on the roadway, and to robustly analyze elevation differences even when the roadway has small undulations. This makes it possible to obtain lines corresponding to the sidewalk edge with high accuracy.

[0085] With the technology disclosed herein, points detected as sidewalk edge candidates that are lined up in the same direction are grouped into a cluster, and for adjacent clusters, the main cluster is retained and the others are eliminated, thereby eliminating sidewalk edge candidate points that are erroneously detected due to parked vehicles on the road or road undulations, and making it possible to fit a line corresponding to the sidewalk edge with high accuracy.

[0086] In addition, the technology disclosed herein downsamples the point cloud before processing to smooth out small undulations, thereby avoiding erroneous detection of candidate sidewalk edge points caused by undulations, and further reducing the number of points to be processed, enabling faster detection.

[0087] Furthermore, with the technology disclosed herein, after fitting lines to the sidewalk edge candidate points, the lines are connected according to a continuity determination, and the shorter of the connected lines are excluded from the sidewalk edge candidates, thereby reducing false detections of sidewalk edge candidates that could not be reduced by the adjacent cluster reduction unit due to blind spots caused by on-street parked vehicles and therefore not being able to be reduced, thereby enabling sidewalk edges to be detected with high accuracy.

[0088] In the above embodiment, the sidewalk edge detection process executed by the CPU by reading the software (program) may be executed by various processors other than the CPU. Examples of the processor in this case include a dedicated electric circuit that is a processor having a circuit configuration designed exclusively for executing a specific process, such as a programmable logic device (PLD) such as a field-programmable gate array (FPGA) whose circuit configuration can be changed after manufacture, a graphics processing unit (GPU), and an application specific integrated circuit (ASIC). The sidewalk edge detection process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, and a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0089] In the above embodiment, the sidewalk edge detection program is pre-stored (installed) in the storage 14, but the present invention is not limited to this. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0090] The following supplementary notes are further disclosed regarding the above embodiment.

[0091] (Additional note 1) Memory, at least one processor coupled to the memory; Including, The processor, Detecting each point of a sidewalk edge candidate corresponding to a sidewalk edge candidate from the point cloud data; Clustering each point of the sidewalk edge candidates; Based on the distribution of points in the adjacent clusters, a predetermined cluster reduction method is used to reduce the clusters; fitting one or more lines or curves to one or more of the clusters and outputting the fitted lines; connecting a portion of the fitted line; The calculated line is output as the sidewalk edge information. A sidewalk edge detection device configured as follows.

[0092] (Additional note 2) A non-transitory storage medium storing a program executable by a computer to execute a sidewalk edge detection process, Detecting each point of a sidewalk edge candidate corresponding to a sidewalk edge candidate from the point cloud data; Clustering each point of the sidewalk edge candidates; Based on the distribution of points in the adjacent clusters, a predetermined cluster reduction method is used to reduce the clusters; fitting one or more lines or curves to one or more of the clusters and outputting the fitted lines; connecting a portion of the fitted line; The calculated line is output as the sidewalk edge information. Non-transitory storage medium. [Explanation of symbols]

[0093] 100 Sidewalk edge detection device 108 Point cloud density reduction part 110 Candidate point detection unit 112 Candidate Point Clustering Section 114 Adjacent Cluster Reduction Section 116 Line Fitting Section 118 Line connection part 120 Information output section

Claims

1. A sidewalk edge detection device that acquires a set of lines corresponding to sidewalk edges from point cloud data as sidewalk edge information, a candidate point detection unit that detects each point of a sidewalk edge candidate corresponding to a sidewalk edge candidate from the point cloud data; a candidate point clustering unit that clusters each point of the sidewalk edge candidates; an adjacent cluster reduction unit that reduces clusters based on a distribution of points in adjacent clusters using a predetermined cluster reduction method; a line fitting unit that fits one or more straight lines or curves to one or more of the clusters and outputs the fitted lines as sidewalk edge candidates; a line connection unit that connects a part of the fitted lines by a predetermined analysis method; an information output unit that outputs the calculated line as the sidewalk edge information; Including, The candidate point clustering unit calculates a principal component vector of each of the sidewalk edge candidate points from its positional relationship with neighboring points, and clusters the points based on the angle between the line segment connecting the point and the neighboring points and the principal component vector, thereby clustering a set of points that are lined up in parallel into a separate cluster.

2. A sidewalk edge detection device that acquires a set of lines corresponding to sidewalk edges from point cloud data as sidewalk edge information, a point cloud density reduction unit that downsamples the point cloud data to reduce density; a candidate point detection unit that detects each point of a sidewalk edge candidate corresponding to a sidewalk edge candidate from the reduced point cloud data; a candidate point clustering unit that clusters each point of the sidewalk edge candidates; an adjacent cluster reduction unit that reduces clusters based on a distribution of points in adjacent clusters using a predetermined cluster reduction method; a line fitting unit that fits one or more straight lines or curves to one or more of the clusters and outputs the fitted lines as sidewalk edge candidates; a line connection unit that connects a part of the fitted lines by a predetermined analysis method; an information output unit that outputs the calculated line as the sidewalk edge information; Including, The candidate point clustering unit calculates a principal component vector of each of the sidewalk edge candidate points from its positional relationship with neighboring points, and clusters the points based on the angle between the line segment connecting the point and the neighboring points and the principal component vector, thereby clustering a set of points that are lined up in parallel into a separate cluster.

3. The sidewalk edge detection device according to claim 2 , wherein the point cloud density reduction unit deletes each point of the downsampled point cloud data when a surrounding point density is equal to or less than a threshold value, and excludes the point from processing.

4. 3. The sidewalk edge detection device according to claim 1, wherein the adjacent cluster reduction unit, in the cluster reduction method, calculates a principal component vector of the cluster from a set of points included in the cluster, determines two endpoints from the direction of the principal component vector of the cluster, determines whether the clusters are adjacent based on whether a foot of a perpendicular line drawn from the midpoint of a surrounding cluster to a line connecting the two endpoints of the cluster is included between the two endpoints, and determines which clusters to reduce based on a score calculated from the lengths and widths of the clusters between the adjacent clusters.

5. 3. The sidewalk edge detection device according to claim 1, wherein the line fitting unit performs clustering on a position trajectory of a measurement device when measuring the point cloud data based on a traveling direction, and determines whether to apply a straight line or a curved line to a subset of the point cloud based on a position of a cluster switching point.

6. 3. The sidewalk edge detection device according to claim 1 or 2, wherein the line connection unit, in the analysis method, judges continuity of the fitted line and surrounding lines based on one or more conditions of the distance between the endpoints, the difference in line direction at the endpoints, the intersection of the lines, the length of a perpendicular line dropped from the endpoint of one line to the other line, or the length of the lines, and decides whether to connect the lines.

7. 3. The sidewalk edge detection device according to claim 1, wherein the line connection unit excludes short lines having a length equal to or less than a predetermined length from the sidewalk edge candidates, from among the connected lines.

8. A sidewalk edge detection method in which a computer acquires a set of lines corresponding to sidewalk edges from point cloud data as sidewalk edge information, the method comprising: On the computer, Detecting each point of a sidewalk edge candidate corresponding to a sidewalk edge candidate from the point cloud data; Clustering each point of the sidewalk edge candidates; Based on the distribution of points in the adjacent clusters, a predetermined cluster reduction method is used to reduce the clusters; fitting one or more lines or curves to one or more of the clusters and outputting the fitted lines; connecting a portion of the fitted line; The calculated line is output as the sidewalk edge information. In the clustering process, a principal component vector of each point of the sidewalk edge candidate is calculated based on a positional relationship with neighboring points, and the points are clustered based on an angle between the line segment connecting the point and the neighboring points and the principal component vector, thereby clustering a set of points arranged in parallel into a separate cluster. A sidewalk edge detection method that causes processing to be performed.

9. A sidewalk edge detection program that acquires a set of lines corresponding to sidewalk edges from point cloud data as sidewalk edge information, On the computer, Detecting each point of a sidewalk edge candidate corresponding to a sidewalk edge candidate from the point cloud data; Clustering each point of the sidewalk edge candidates; Based on the distribution of points in the adjacent clusters, a predetermined cluster reduction method is used to reduce the clusters; fitting one or more lines or curves to one or more of the clusters and outputting the fitted lines; connecting a portion of the fitted line; The calculated line is output as the sidewalk edge information. In the clustering process, a principal component vector of each point of the sidewalk edge candidate is calculated based on a positional relationship with neighboring points, and the points are clustered based on an angle between the line segment connecting the point and the neighboring points and the principal component vector, thereby clustering a set of points arranged in parallel into a separate cluster. A sidewalk edge detection program that performs processing.

Images