Road surface state detection device and road surface state detection program

Abstract

To appropriately detect both a region type damage and a linear type damage.SOLUTION: A normal paved surface estimating unit 103A that estimates a region-type normal paved surface d11 by fitting to each first section F1 in the point cloud for each scanning line acquired from the point cloud generating device 2, and estimates a line-type normal paved surface d11 by fitting to each second section d21 shorter than the first section F2, and determines region-type damage for each first determination section d11 shorter than the first section F1 based on first deformation displacement amounts that are differences between the point cloud for each scanning line and the region-type normal paved surface; d12, and a linear type damage determination unit F2 that determines linear type damage for each second determination section d21 shorter than the second section d22, based on a second deformation displacement amount that is a differential between the point cloud for each scan line and the linear type normal paved surface 104B.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a road surface condition detection device and a road surface condition detection program for appropriately detecting road surface conditions. [Background technology]

[0002] Generally, as part of road maintenance work, daily inspections are carried out by patrols according to traffic volume, as damage to pavement such as potholes, cracks, and ruts can be factors in traffic accidents, etc. Potholes in particular require prompt repair, so abnormal areas are identified and repaired through visual inspections by patrols, but this amounts to after-the-fact maintenance.

[0003] On the other hand, in recent years, the use of point clouds acquired by MMS (Mobile Mapping System) has been increasing in order to improve the efficiency and sophistication of road maintenance work.

[0004] Patent Document 1 discloses technology related to a road surface condition inspection system that sets a reference depth (e.g., 6 mm) as the lower limit value of the crack to be detected and determines whether the depth of each measurement point H(j) from the road surface height (height of the measurement line L) is equal to or greater than the reference depth.

[0005] Patent Document 2 discloses technology relating to a system that includes a reference line setting means, a reference area setting means, a reference road surface model creation means, a measured road surface model creation means, and a difference extraction means, and that detects unevenness of a paved road surface based on the differences. The reference road surface model creation means creates a model using only measurement points within the lane marks. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-141624 [Patent Document 2] Japanese Patent Application Laid-Open No. 2015-031018 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the technology described in Patent Document 1 has the problem that when the reference depth is set to a few millimeters, it is affected by minute irregularities on the road surface, resulting in the false extraction of many points. Furthermore, when the reference depth is large, shallow cracks cannot be extracted, making parameter setting difficult, especially when targeting data over a long section. Furthermore, the technology described in Patent Document 1 has difficulty detecting regional damage, such as rutting, potholes, and sloping, which are damage that occurs over a relatively wide area.

[0008] Road damage is influenced not only by water but also by the roadbed and subgrade, so damage can occur on lane marks. However, the technology described in Patent Document 2 creates a reference road surface model using only measurement points within the lane marks, so if there is damage on the lane marks, there is a possibility that an incorrect reference road surface model will be estimated. In addition, since points other than the lane marks are not used, there is a problem that it is difficult to reproduce a fine reference road surface model. Furthermore, it is difficult to detect linear damage that occurs in a relatively narrow area, such as cracks or pavement joints.

[0009] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a road surface condition detection device and a road surface condition detection program that can appropriately detect both area-type damage and linear-type damage. [Means for solving the problem]

[0010] In order to achieve the above object, a first feature of the road surface condition detection device according to the present invention is to: A road surface condition detection device that acquires point cloud data from a point cloud data generation device that generates three-dimensional coordinates of a road as point cloud data for each scan line based on reflected light of laser light irradiated onto the road along which a mobile object is traveling, and detects a road surface condition of the road based on the acquired point cloud data, a normal pavement surface estimation unit that estimates an area-type normal pavement surface by fitting each first section in the point cloud data for each scan line acquired from the point cloud data generation device, and estimates a line-type normal pavement surface by fitting each second section that is shorter than the first section; a damage determination unit that determines area-type damage for each first determination section that is shorter than the first section based on a first deformation displacement amount, which is the difference between the point cloud data for each scan line and the area-type normal pavement surface, and that determines linear-type damage for each second determination section that is shorter than the second section based on a second deformation displacement amount, which is the difference between the point cloud data for each scan line and the linear-type normal pavement surface; The reason is that it is equipped with the following.

[0011] A second feature of the road surface condition detection device according to the present invention is that The normal pavement surface estimation unit calculates the curvature based on the angle between a straight line connecting an arbitrary point of interest in the point cloud data for each scan line to a first neighboring point, which is a point in the point cloud data located a predetermined proximity distance away on one side, and a straight line connecting the point of interest to a second neighboring point, which is a point in the point cloud data located a predetermined proximity distance away on the other side; calculates the proportion of the curvature that is below a predetermined curvature threshold when the proximity distance is changed within a predetermined range as the degree of non-damage; and estimates the area-type normal pavement surface and the line-type normal pavement surface by performing fitting using the degree of non-damage as a weight.

[0012] A third feature of the road surface condition detection device according to the present invention is The system further includes a polygon generation unit that extracts point cloud data in which the first deformation displacement amount is equal to or greater than a predetermined first damage threshold as a damage point set, and generates a damage polygon by clustering the extracted damage point set.

[0013] A fourth feature of the road surface condition detection device according to the present invention is The polygon generation unit extracts point cloud data in which the first deformation displacement amount is equal to or greater than a predetermined first damage threshold as a set of damage candidate points, performs a morphological operation on the extracted set of damage candidate points to extract it as a set of damage points, and generates a damage polygon by clustering the extracted set of damage points.

[0014] A fifth feature of the road surface condition detection device according to the present invention is a storage means for storing a region-type damage table that associates the type of region-type damage with the characteristics of the region-type damage; a type determination unit that determines a type of area-type damage corresponding to the generated damage polygon based on the area-type damage table and attribute information of the generated damage polygon; The advantage is that it further features:

[0015] A sixth feature of the road surface condition detection device according to the present invention is The system further includes a polyline generation unit that extracts a set of linear damage candidate points by applying adaptive threshold processing to the second deformation displacement amount, generates damage candidate clusters by clustering the set of linear damage candidate points using a region growing method, extracts linear damage clusters by applying a random forest to the generated damage candidate clusters, and generates linear damage polylines by constructing a minimum spanning tree from Delaunay triangles generated using representative points of the extracted linear damage clusters.

[0016] A first feature of the road surface condition detection program according to the present invention is to A road surface condition detection program executed by a road surface condition detection device that acquires point cloud data from a point cloud data generation device that generates three-dimensional coordinates of a road as point cloud data for each scan line based on reflected light of a laser beam irradiated onto the road along which a mobile object travels, and detects the road surface condition of the road based on the acquired point cloud data, a normal pavement surface estimation step of estimating an area-type normal pavement surface by fitting each first section in the point cloud data for each scan line acquired from the point cloud data generation device, and estimating a line-type normal pavement surface by fitting each second section that is shorter than the first section; The road surface condition detection device is caused to execute a damage determination step of determining area-type damage for each first determination section shorter than the first section based on a first deformation displacement amount, which is the difference between the point cloud data for each scan line and the area-type normal pavement surface, and determining linear-type damage for each second determination section shorter than the second section based on a second deformation displacement amount, which is the difference between the point cloud data for each scan line and the linear-type normal pavement surface. [Effects of the Invention]

[0017] According to the road surface condition detection device and road surface condition detection program of the present invention, both area-type damage and linear-type damage can be appropriately detected. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a schematic diagram showing the schematic configuration of a road surface condition detection system according to an embodiment of the present invention; [Figure 2] 1 is an explanatory diagram that schematically explains the generation of point cloud data by a point cloud data generating device 2 included in a road surface condition detection system that is one embodiment of the present invention. FIG. [Figure 3] FIG. 3(a) is an example of point cloud data before smoothing processing in a road surface condition detection system that is one embodiment of the present invention, and FIG. 3(b) is an example of point cloud data after smoothing processing in a road surface condition detection system that is one embodiment of the present invention. [Figure 4] 1 is an explanatory diagram illustrating processing by a normal pavement surface estimation unit 103 in a road surface condition detection system that is one embodiment of the present invention. FIG. [Figure 5]Figure 5(a) is an explanatory diagram illustrating processing by an area-type normal pavement surface estimation unit provided in a road surface condition detection system that is one embodiment of the present invention, and Figure 5(b) is an explanatory diagram illustrating processing by a line-type normal pavement surface estimation unit provided in a road surface condition detection system that is one embodiment of the present invention. [Figure 6] 6A and 6B are explanatory diagrams illustrating processing by a polygon generation unit included in a road surface condition detection system according to an embodiment of the present invention. Fig. 6A shows the state in which damage candidate points are extracted, Fig. 6B shows the results of a morphology opening operation, Fig. 6C shows the results of a morphology closing operation, Fig. 6D shows the clustering results, and Fig. 6E shows a damage polygon. [Figure 7] 1 is a diagram showing an example of an area-type damage table stored in a table storage unit included in a road surface condition detection system according to an embodiment of the present invention. [Figure 8] 8A and 8B are explanatory diagrams illustrating processing by a polyline generation unit included in a road surface condition detection system according to an embodiment of the present invention. Fig. 8A shows an example of a damage candidate point set, Fig. 8B shows an example of a damage candidate cluster, and Fig. 8C shows an example of an oriented bounding box (OBB). [Figure 9] 9(a) and 9(b) are explanatory diagrams illustrating processing by a polyline generation unit included in a road surface condition detection system according to an embodiment of the present invention. Fig. 9(a) is a diagram illustrating an example of extraction of an initial representative point set, Fig. 9(b) is a diagram illustrating an example of selection of a locally deepest point, Fig. 9(c) is a diagram illustrating an example of graph construction, and Fig. 9(d) is a diagram illustrating an example of polyline generation. Here, initial representative points are indicated by black circles, damage cluster constituent points are indicated by dotted hatching, reselected representative points are indicated by vertical hatching, and other points are indicated by white circles. [Figure 10] FIG. 2 is an explanatory diagram illustrating the extraction process of an initial representative point set using Farthest Point Sampling (FPS) by a polyline generation unit 108 included in the road surface condition detection system according to one embodiment of the present invention. [Figure 11]11A to 11F are explanatory diagrams illustrating the process of selecting the local deepest point by the polyline generation unit included in the road surface condition detection system according to one embodiment of the present invention. Figures 11A to 11C show the process when replacement is performed, and Figures 11D to 11F show the process when no replacement is performed. [Figure 12] 12(a) to 12(c) are diagrams illustrating the construction of a graph by the polyline generation unit 108 provided in the road surface condition detection system of one embodiment of the present invention, and FIGS. 12(d) to 12(e) are diagrams illustrating the generation of a linear damage polyline by the polyline generation unit provided in the road surface condition detection system of one embodiment of the present invention. [Figure 13] 10A and 10B are diagrams showing the estimation results of a normal pavement surface in the road surface condition detection system according to one embodiment of the present invention. [Figure 14] 10 is a diagram showing the detection result of the road surface condition displayed on the display control means of the road surface condition detection system according to one embodiment of the present invention. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The same or equivalent parts and components are designated by the same or equivalent reference numerals throughout the drawings. However, it should be noted that the drawings are schematic and may differ from the actual product. Furthermore, the drawings may include parts with different dimensional relationships and ratios.

[0020] Furthermore, the embodiments shown below are merely examples of devices that embody the technical concept of the present invention, and the technical concept of the present invention does not limit the arrangement of each component to that shown below. Various modifications can be made to the technical concept of the present invention within the scope of the claims.

[0021] <Configuration of road surface condition detection device> A road surface condition detection system including a road surface condition detection device according to one embodiment of the present invention will be described below.

[0022] FIG. 1 is a schematic diagram showing the schematic configuration of a road surface condition detection system according to one embodiment of the present invention.

[0023] As shown in FIG. 1, the road surface condition detection system 100 includes a road surface condition detection device 1, a point cloud data generation device 2, a cart (mobile body) 3, and a monitor 5.

[0024] The point cloud data generation device 2 is mounted on a carriage 3 that travels on a road surface RN. While the carriage 3 is traveling, the point cloud data generation device 2 irradiates a laser beam onto the road surface RN of the road on which the carriage 3 is traveling. Then, based on the reflected light of the irradiated laser beam, the point cloud data generation device 2 generates three-dimensional coordinates of the road surface RN as point cloud data. This will be described in detail later.

[0025] The road surface condition detection device 1 acquires point cloud data from the point cloud data generation device 2 and detects the road surface condition based on the acquired point cloud data. For example, the road surface condition detection device 1 detects area-type damage such as rutting, potholes, and rutting, which is damage over a relatively wide area, and also detects linear-type damage such as cracks and pavement joints, which is damage that is linear in a relatively narrow area.

[0026] The monitor 5 displays various screens, such as a screen that displays area-type damage or linear-type damage detected by the road surface condition detection device 1.

[0027] <Generation of point cloud data by point cloud data generator 2> FIG. 2 is an explanatory diagram that schematically explains the generation of point cloud data by the point cloud data generating device 2 provided in the road surface condition detection system 100 according to one embodiment of the present invention.

[0028] As shown in FIG. 2, the point cloud data generating device 2 includes a laser scanner 21, a Global Navigation Satellite System (GNSS) 22, an Inertial Measurement Unit (IMU) 23, a data collecting means 24, and a temporary data storage unit 25.

[0029] The laser scanner 21 is provided behind the carriage 3 and emits laser light toward the road surface RN. The reach of the emitted laser light is, for example, a maximum of 200 m, and the scanner rotates at a cycle of 200 Hz and emits laser light at 1000 kHz. The laser scanner 21 receives reflected light of the emitted laser light, and acquires laser data (Time, θ, L, I) indicating the measurement time of reception, the angle and distance to the road surface RN, and reflection intensity based on the received reflected light.

[0030] The GNSS 22 acquires GNSS data (position coordinates) using signals emitted from artificial satellites.

[0031] The IMU 23 can detect the attitude of the point cloud data generation device 2. The attitudes acquired by the IMU 23 are matched with GNSS data and supplied to the data collection means 24 as POS (Position and Orientation System) data. That is, the POS data includes data (Time, xi, yi, zi, Roll, Pitch, Yaw) that indicates the reference position and attitude (measurement time, position coordinates, tilt) of the IMU 23, which is a combination of GNSS data (position coordinates) from the GNSS 22 and the detected attitude. Zi is the altitude value from ground level at which the IMU 23 is attached.

[0032] The position coordinates of the laser scanner 21 relative to the IMU 23 can be expressed as (Δx, Δy, Δz).

[0033] The data collection means 24 generates a set of three-dimensional coordinates indicating the positions of features as point cloud data P expressed by the following (Equation 1) based on the POS data acquired from the IMU 23 and the laser data acquired from the laser scanner 21. Point Pi at acquisition time ti is represented by the coordinates (x p i ,y p i ,z p i ) and the reflection intensity Ii. The travel trajectory Q is expressed by the following (Equation 2), and q j is the acquisition time t j coordinates of the point cloud data generator 2 indicated by the plane rectangular coordinates and elevation at j are the coordinate axes (Roll, Pitch, Yaw) that represent the vehicle posture. Here, the point cloud data is divided directly above the point cloud data generating device 2, and the sequence of points arranged in the order of acquisition time is called a scan line, which is the sequence of points for one scan rotation of the laser scanner 21. P={P i =(p i ,t i ,I i )|p i =(x p i ,y p i ,z p i ),i=1,…,n,t i <t i +1} (Formula 1) Q={Q j =(q j ,c j ,t j )|j=1,…,s,t j <t j +1} (Equation 2) P i : point i Coordinates in the plane rectangular coordinate system, t i :Time, Ii:Reflection intensity,Q j : coordinates at point j, c j :Cart posture, t j :time

[0034] The data temporary storage unit 25 temporarily stores the point cloud data P generated by the data collection means 24. The stored point cloud data P is supplied to the road surface condition detection device 1. Here, the point cloud data P stored in the data temporary storage unit 25 is supplied to the road surface condition detection device 1 in real time, but this is not limiting. After the point cloud data P is temporarily stored, when the point cloud data generation device 2 and the road surface condition detection device 1 are connected, the point cloud data P may be supplied from the data temporary storage unit 25 to the road surface condition detection device 1.

[0035] <Configuration of road surface condition detection device 1> Returning to Figure 1, the road surface condition detection device 1 has a point cloud data acquisition unit 101, a point cloud data storage unit 102, a normal pavement surface estimation unit 103, a damage determination unit 104, a polygon generation unit 105, a table storage unit 106, a type determination unit 107, a polyline generation unit 108, and a display control means 111.

[0036] The point cloud data acquisition unit 101 acquires the point cloud data P supplied from the data temporary storage unit 25 of the point cloud data generation device 2, and stores the acquired point cloud data P in the point cloud data storage unit .

[0037] The point cloud data storage unit 102 is configured by, for example, a hard disk drive, and stores the point cloud data P.

[0038] The normal pavement surface estimation unit 103 performs smoothing processing by applying a smoothing filter to remove measurement noise from the point cloud data P for each scan line that the point cloud data acquisition unit 101 acquires from the point cloud data generation device 2. Specifically, the normal pavement surface estimation unit 103 applies Taubin smoothing to the point cloud data P for each scan line, which repeats a Gaussian smoothing step while alternating between positive and negative scale factors. The Gaussian smoothing step may be repeated approximately 50 times.

[0039] FIG. 3(a) is an example of point cloud data P before smoothing processing in road surface condition detection system 100, which is one embodiment of the present invention, and FIG. 3(b) is an example of point cloud data P after smoothing processing in road surface condition detection system 100, which is one embodiment of the present invention.

[0040] By performing the smoothing process, the normal pavement surface estimation unit 103 can reduce measurement noise while leaving the damaged shapes of the cracks h1 and h2 as shown in FIG. 3(b).

[0041] FIG. 4 is an explanatory diagram illustrating the processing performed by the normal pavement surface estimation unit 103 in the road surface condition detection system 100 according to one embodiment of the present invention.

[0042] The normal pavement surface estimation unit 103 estimates a first neighboring point P, which is the furthest point in the point cloud data P that is a predetermined neighboring distance r away from an arbitrary point of interest Pi in the point cloud data P for each scan line. s r A straight line L1 connecting the point of interest Pi and the second neighboring point P, which is the furthest point in the point cloud data that is a predetermined neighboring distance r away from the point of interest Pi, is connected to the line L1 connecting the point of interest Pi and the second neighboring point P, which is the furthest point in the point cloud data that is a predetermined neighboring distance r away from the point of interest Pi. e r The degree of curvature ρ is calculated based on the angle between the line L1 and the line L2. r i Specifically, the curvature ρ r i is the angle obtained by subtracting the minimum angle between the lines L1 and L2 from 180°.

[0043] Specifically, the normal pavement surface estimation unit 103 calculates the degree of curvature ρ when the proximity distance r is changed within a predetermined range using the following (Equation 3): r i The proportion of curvatures that are less than or equal to a predetermined curvature threshold ρth is defined as the undamaged degree ω i The degree of curvature ρ r i is the first neighboring point P s r and the second nearest point P e rThis is a local gradient change calculated from the road gradient and varies depending on the distance (scale) when selecting nearby points. In areas without damage, the curvature ρ r i The value of decreases, and the curvature ρ r i Therefore, as shown in (Equation 3), the undamaged degree ω at the target point Pi i The curvature ρ when the proximity distance r is changed within a predetermined range r i is a predetermined curvature threshold τ ρ It is calculated as the percentage that is less than or equal to the

[0044]

number

[0045] where B i (r) is defined as follows (Equation 4). R is a set of neighbor point selection distances. The first neighbor point P s r and the second nearest point P e r is the farthest point in the front and back directions along the scan line within a certain distance r from the point of interest Pi. The angle formed by the lines L1 and L2 is the absolute value of the smaller angle. i is close to 1 in the flat area without damage, and is small in the damaged area and its surroundings.

[0046]

number

[0047] The normal pavement surface estimation unit 103 fits a cubic curve using the weighted moving least squares method. The normal pavement surface estimation unit 103 includes an area-type normal pavement surface estimation unit 103A and a line-type normal pavement surface estimation unit 103B.

[0048] Figure 5(a) is an explanatory diagram illustrating processing by the area-type normal pavement surface estimation unit 103A provided in the road surface condition detection system 100, which is one embodiment of the present invention, and Figure 5(b) is an explanatory diagram illustrating processing by the line-type normal pavement surface estimation unit 103B provided in the road surface condition detection system 100, which is one embodiment of the present invention.

[0049] As shown in FIG. 5(a), the area-type normal pavement surface estimation unit 103A estimates the degree of undamaged ω for each first section d11 using the following (Equation 5) in the point cloud data for each scan line that the point cloud data acquisition unit 101 has acquired from the point cloud data generation device 2 and has undergone smoothing processing. i A polynomial curve f(x) is estimated by fitting a cubic curve with weighting, and the estimated f(x) is set as the area-type normal pavement surface F1 within the range of the first section d11.

[0050] 5(b), the linear normal pavement surface estimation unit 103B calculates the degree of undamaged ω for each second section d21, which is shorter than the first section d11, using the following (Equation 5) in the point cloud data for each scan line that the point cloud data acquisition unit 101 acquired from the point cloud data generation device 2 and smoothed. i The polynomial curve f(x) is estimated by fitting a cubic curve with weighting, and the estimated f(x) is estimated as the linear normal pavement surface F2 in the range of the second section d21.

[0051]

number

[0052] S: a set of points in the first interval d11 or the second interval d21, ω i : Undamaged degree of point i, Z i : height of point i, f(x): cubic polynomial curve of x The damage determination unit 104 determines area-type damage and linear-type damage and includes an area-type damage determination unit 104A and a linear-type damage determination unit 104B.

[0053] The area-type damage determination unit 104A calculates the first deformation displacement amount Δz, which is the difference between the point cloud data P for each scan line and the area-type normal pavement surface F1. 1(i) Area-type damage is determined for each first determination section d12 that is shorter than the first section d11 based on the above formula. Here, the first determination section d12 is preset to a value that is sufficiently larger than rutting.

[0054] The linear damage determination unit 104B calculates the second deformation displacement amount Δz, which is the difference between the point cloud data P for each scan line and the linear normal pavement surface F2. 2(i) Based on this, linear damage is judged for each second judgment section d22 that is shorter than the second section d21. The second judgment section d22 is set to a value that is sufficiently smaller than rutting and sufficiently larger than cracking by performing adaptive threshold processing.

[0055] Fig. 6 is an explanatory diagram illustrating processing by the polygon generation unit 105 included in the road surface condition detection system 100, which is one embodiment of the present invention. Fig. 6(a) shows the state in which damage candidate points have been extracted, Fig. 6(b) shows the result of the morphology opening operation, Fig. 6(c) shows the result of the morphology closing operation, Fig. 6(d) shows the clustering result, and Fig. 6(e) shows the damage polygon.

[0056] As shown in FIG. 6(a), the polygon generating unit 105 generates the first deformation displacement amount Δz 1(i) The absolute value of the first damage threshold Δz th1 The point cloud data described above is extracted as a set of damage candidate points. Area-type damage includes concave damage such as rutting and podholes, which are negative values ​​relative to the area-type normal pavement surface F1, and convex damage such as deviation, which are positive values ​​relative to the area-type normal pavement surface F1.

[0057] In the example shown in FIG. 6(a), the points indicated by black circles are the points where the first deformation displacement amount Δz1(i) is greater than or equal to the predetermined first damage threshold Δz th1 The points marked with white circles are considered to be damage candidate points judged to be equal to or greater than the first deformation displacement amount Δz 1(i) is the first damage threshold Δz th1The points are those that are judged to be less than the damage candidate point set. A set of damage candidate points is called a damage candidate point set. In this example, an isolated point pd1 is generated that is not included in the damage candidate point set and is surrounded by a white circle.

[0058] The polygon generator 105 performs a morphological opening operation on the extracted damage candidate point set. As a result, it is possible to extract a damage candidate point set from which the isolated point pd1 has been removed, as shown in Figure 6(b). However, small holes pd2 and pd3 remain in the damage candidate point set.

[0059] Therefore, the polygon generator 105 performs a morphological closing operation on the damage point candidate set from which the isolated point pd1 has been removed, thereby extracting a damage point candidate set from which small holes pd2 and pd3 have been removed, as shown in Figure 6(c).

[0060] Next, the polygon generation unit 105 clusters the damage point sets using a region growing method to obtain a damage cluster for each damaged region. In the example shown in Figure 6(d), a damage cluster containing damage point pd4 and a damage cluster containing damage point pd5 are obtained. The damage cluster containing damage point pd4 is represented by dot hatching, and the damage cluster containing damage point pd5 is represented by vertical hatching. These processes are performed in time grid space to accommodate changes in scan line spacing due to point cloud density and the traveling speed of the bogie 3. The time grid space is a space consisting of a scan line number and a number indicating the position of the point on the scan line. Note that this process is performed separately for concave and convex damage candidate sets. The kernel size for the morphology opening and closing operations is 3 × 3, and contraction and expansion are each performed once. In addition, the neighbor point search range in the region growing method is limited to adjacent points.

[0061] Next, the polygon generation unit 105 applies, for example, AlphaShape, which is a generalized convex hull method, to the damage cluster to generate damaged area polygons. As a result, damaged polygons PP1 and PP2 are generated as shown in FIG. 6(e).

[0062] The table storage unit 106 is configured by a storage medium such as a hard disk drive, and stores an area-type damage table that associates the type of area-type damage with the characteristics of the area-type damage.

[0063] FIG. 7 is a diagram showing an example of an area-type damage table stored in the table storage unit 106 included in the road surface condition detection system 100 according to one embodiment of the present invention.

[0064] As shown in Figure 7, the area-type damage table shows the damage type determination conditions for area-type damage, and for each type of area-type damage such as ruts, potholes, other concave shapes, and convex shapes, the characteristics of the area-type damage such as maximum / minimum deformation displacement, average deformation displacement, area, area-to-perimeter ratio, and distance from the lane center line are associated and stored.

[0065] The type determination unit 107 determines the type of area-type damage corresponding to the generated damage polygon based on the area-type damage table stored in the table storage unit 106 and the attribute information of the damage polygon generated by the polygon generation unit 105. For example, if the attribute information of the damage polygon includes a maximum / minimum deformation displacement of -3 (mm) and a distance from the lane center line of 1.0 (m), the type determination unit 107 determines that this damage polygon is a rut based on the area-type damage table.

[0066] The polyline generator 108 generates a linear damage polyline having branches, which has damage depth information.

[0067] Fig. 8 is an explanatory diagram illustrating processing by the polyline generation unit 108 included in the road surface condition detection system 100 according to one embodiment of the present invention. Fig. 8(a) shows an example of a damage candidate point set, Fig. 8(b) shows an example of a damage candidate cluster, and Fig. 8(c) shows an example of an oriented bounding box (OBB).

[0068] The polyline generating unit 108 calculates the second deformation displacement amount Δz 2(i) A set of linear damage candidate points is extracted by applying adaptive threshold processing to the image.

[0069] On road pavement surfaces, there are minute irregularities other than linear damage, such as minute peeling of the asphalt pavement surface layer. Therefore, when extracting linear damage by threshold processing of the amount of deformation, it is necessary to set a threshold according to the state of the road pavement surface. Therefore, the polyline generation unit 108 calculates the amount of deformation in the Time Grid space as d st When this is done, the amount of deformation in the Time Grid space, d st By applying adaptive threshold processing based on the median G(s,t) of the above equations, points that satisfy the following (Equation 6) are set as linear damage candidate points, and a set of linear damage candidate points is extracted.

[0070]

number

[0071] As a result, as shown in FIG. 8(a), a set of linear damage candidate points including the linear damage candidate point pd6 indicated by a black circle is extracted.

[0072] The polyline generating unit 108 generates damage candidate clusters by clustering a set of linear damage candidate points using a region growing method.

[0073] As a result, as shown in Figure 8(b), it is possible to generate a damage candidate cluster including the linear damage candidate point pd7, a damage candidate cluster including the linear damage candidate point pd8, and a damage candidate cluster including the linear damage candidate point pd9. Here, the damage candidate cluster including the linear damage candidate point pd7 is represented by vertical hatching, the damage candidate cluster including the linear damage candidate point pd8 is represented by dotted hatching, and the damage candidate cluster including the linear damage candidate point pd9 is represented by a double circle.

[0074] The polyline generation unit 108 generates an oriented bounding box (OBB) on a horizontal plane from a set of linear damage candidate points that make up the obtained damage candidate cluster. The polyline generation unit 108 generates an oriented bounding box (OBB) on a horizontal plane from a set of linear damage candidate points that make up the obtained damage candidate cluster. The polyline generation unit 108 generates an oriented bounding box (OBB) on a horizontal plane from a set of linear damage candidate points that make up the damage candidate cluster. l The area of ​​the Oriented Bounding Box (OBB) is shorter than the area threshold τ a If it is larger, it is determined that the damage is not linear, and the corresponding damage candidate cluster is deleted.

[0075] As a result, as shown in FIG. 8(c), oriented bounding boxes OBB1, OBB2, and OBB3 can be obtained from which damaged candidate clusters have been eliminated.

[0076] Damage candidate clusters may include manholes, drainage basins, and other structures in addition to the linear damage targeted for extraction. Therefore, the polyline generator 108 extracts linear damage clusters by applying a random forest to the generated damage candidate clusters. Specifically, the polyline generator 108 extracts linear damage clusters by applying a random forest to the damage candidate clusters using two objective variables, linear damage clusters and non-linear damage clusters, and explanatory variables such as the area and aspect ratio of the oriented bounding box (OBB), the damage point occupancy rate, the mean / median / standard deviation / first quartile / third quartile of the amount of deformation, the mean / median of the reflection intensity, the mean / median / standard deviation of the degree of undamage, average linearity, average omnivariance, and average eigenentropy.

[0077] Based on the extracted linear damage cluster, the polyline generation unit 108 uses Farthest Point Sampling (FPS) to generate a polyline at a fixed distance τ f A linear damage polyline is generated by extracting an initial representative point set with the above point spacing and constructing a minimum spanning tree from the Delaunay triangles generated using the extracted initial representative point set. Here, the endpoints of the linear damage are accurately represented, and the locally deepest points are used as vertices. To simplify the representation, a linear damage polyline is generated that consists of line segments of a certain length or more.

[0078] 9A and 9B are explanatory diagrams illustrating processing by the polyline generation unit 108 included in the road surface condition detection system 100, which is one embodiment of the present invention. Fig. 9A is a diagram illustrating an example of extraction of an initial representative point set, Fig. 9B is a diagram illustrating an example of selection of a locally deepest point, Fig. 9C is a diagram illustrating an example of graph construction, and Fig. 9D is a diagram illustrating an example of polyline generation. Here, initial representative points are indicated by black circles, damage cluster constituent points are indicated by dotted hatching, reselected representative points are indicated by vertical hatching, and other points are indicated by white circles.

[0079] As shown in FIG. 9(a), the polyline generating unit 108 uses Farthest Point Sampling (FPS) to generate a polyline at a certain distance τ from the linear damage cluster constituent points (PC1, PC2, . . . ). f An initial representative point set (PS1, PS2...) having the above point intervals is extracted.

[0080] 10 is an explanatory diagram illustrating the extraction process of an initial representative point set using Farthest Point Sampling (FPS) by the polyline generation unit 108 included in the road surface condition detection system 100, which is one embodiment of the present invention. Here, the initial representative points are indicated by black circles, and the damage cluster constituent points are indicated by dotted hatching.

[0081] As shown in FIG. 10(a), the polyline generating unit 108 selects the point P'0 having the smallest X coordinate as the initial representative point.

[0082] As shown in FIG. 10(b), the polyline generating unit 108 calculates the distances from the initial representative point P'0 to the respective damage points P'1 to P'5 as d1 to d7.

[0083] As shown in FIG. 10(c), the polyline generating unit 108 determines whether the distance to the farthest damaged point P'4 is greater than the threshold value τ d If this is the case, the damage point P'4 is selected as the initial representative point.

[0084] As shown in FIG. 10(d), the polyline generating unit 108 calculates the distance from the initial representative point P'4 to each of the damage points P'1 to P'3 and P'5 by a threshold value τ d In the example shown in FIG. 10(d), the distance d2 from the initial representative point P'4 to the damaged point P'2, the distance d3 from the initial representative point P'4 to the damaged point P'3, and the distance d7 from the initial representative point P'4 to the damaged point P'7 are compared with the threshold value τ d Since it is shorter, the polyline generating unit 108 updates d2, d3, and d7.

[0085] On the other hand, as shown in FIG. 10(e), the polyline generating unit 108 generates a polyline when the distance to the farthest point is less than the threshold value τd In the example shown in FIG. 10(e), the distance d5 from the initial representative point P'4 to the damaged point P'5 is equal to or greater than the threshold τ d As described above, the polyline generating unit 108 selects the damaged point P'5 as the initial representative point.

[0086] As shown in Fig. 10(f), the polyline generator 108 repeatedly executes the processes shown in Fig. 10(d) and Fig. 10(e) until there are no more non-representative points that satisfy the conditions in Fig. 10(e). As a result, the damage points P'0, P'4, P'5, P'5, and P'7 are selected as initial representative points.

[0087] Next, in order to set the initial representative point as the locally deepest point, the polyline generating unit 108 sets each initial representative point at a fixed distance r p In the example shown in FIG. 9(b), the polyline generating unit 108 replaces the initial representative point PS1 with the point with the smallest deformation displacement within the l Similarly, the polyline generating unit 108 replaces the initial representative point PS2 with the damage point PR1 that is the point with the smallest amount of deformation displacement within the l The point with the smallest deformation displacement is replaced by the damage point PR2.

[0088] However, in order to preserve the initial representative point of the damaged edge, the maximum angle e of the neighboring points shown in the following (Equation 7) is k If is equal to or smaller than π / 2, the polyline generating unit 108 does not replace it and sets it as the representative point as it is.

[0089]

number

[0090] Fig. 11 is an explanatory diagram illustrating the process of selecting the local deepest point by the polyline generation unit 108 included in the road surface condition detection system 100 according to one embodiment of the present invention. Fig. 11(a) to (c) show the process when replacement is performed, and Fig. 11(d) to (f) show the process when no replacement is performed.

[0091] As shown in FIG. 11(a) to FIG. 11(c), when the initial representative point P′5 is focused on, the initial representative point P′5 and its fixed distance τ l In the example shown in FIG. 11(b), the maximum angle e between the neighboring points P'1, P'5, and P'6 is k is greater than or equal to π / 2, so the initial representative point P'5 is placed at a fixed distance τ l The point with the smallest deformation displacement is replaced by the initial representative point P'2.

[0092] On the other hand, as shown in FIG. 11(d) to FIG. 11(f), when attention is focused on the initial representative point P'0, the initial representative point P'0 and its fixed distance τ l In the example shown in FIG. 11(e), the maximum angle e between the points P'1, P'0, and P'5 is k is less than π / 2, no replacement is performed for the initial representative point P'0.

[0093] Next, the polyline generation unit 108 constructs a graph of a linear polyline by Delaunay triangulation generated using the extracted set of initial representative points. In the example shown in Fig. 9(c), an initial graph G1 of a linear polyline is constructed by the initial representative points PS3 to PS5 and the damage points PR1 and PR2 replaced by the initial representative points.

[0094] Then, as shown in Figure 9(d), the polyline generator 108 generates a linear damage polyline PL1 by constructing a minimum spanning tree (MST) of the constructed initial graph G1. In generating the minimum spanning tree (MST), in order to retain edges along the linear damage, the representative value of the corresponding damage for each edge is calculated, and its inverse is used as the weight for constructing the MST. As shown in Figure 9(c), since the edges of the Delaunay triangle do not correspond to point clouds other than the endpoints, here, the point set on the shortest path between the endpoints in the time grid space is obtained, and the third quartile of the absolute value of the deformation displacement of the point set is used as the representative value of the damage.

[0095] The MST is constructed using the Prim method, and the third quartile of the deformation displacement is assigned to the MST line segments as information on the depth of the damage, and the data is output as a linear damage polyline.

[0096] Figures 12(a) to 12(c) are diagrams illustrating the construction of a graph by the polyline generator 108 included in the road surface condition detection system 100, which is one embodiment of the present invention, and Figures 12(d) to 12(e) are diagrams illustrating the generation of a linear damage polyline by the polyline generator 108 included in the road surface condition detection system 100, which is one embodiment of the present invention. Here, representative points are shown as black circles, damage points are shown as dotted hatching, and other points are shown as white circles.

[0097] As shown in Figure 12(a), an initial graph G2 is generated, and the polyline generation unit 108 extracts paths RT1 to RT5 shown in Figure 12(b) by searching for the shortest path between representative points in the initial graph G2 using the Dijkstra algorithm.

[0098] The polyline generating unit 108 then sets the inverse of the representative depth as the weight. The third quartile of the absolute value of the amount of deformation is used as the representative depth. In the example shown in Figure 12(c), routes RT1 to RT5 are ranked in descending order of weight.

[0099] As shown in Fig. 12(d), the polyline generation unit 108 constructs a minimum spanning tree (MST) from the weighted graph G3 using the Prim method, minimizing the sum of edge costs, and removes polylines whose total length is less than a certain length. Then, as shown in Fig. 12(e), the polyline generation unit 108 assigns a representative depth to each polyline to generate it as a linear damage polyline PL2.

[0100] The display control means 111 causes the monitor 5 to display the damage polygons generated by the polygon generation unit 105 as area-type damage for each type determined by the type determination unit 107. For example, the display color of the area-type damage may be changed for each type determined by the type determination unit 107. Furthermore, the display control means 111 causes the monitor 5 to display the linear damage polylines generated by the polyline generation unit 108 as linear damage.

[0101] Fig. 13 is a diagram showing the estimation results of a normal pavement surface in the road surface condition detection system 100, which is one embodiment of the present invention. Fig. 14 is a diagram showing the detection results of road surface conditions displayed on the display control means 111 of the road surface condition detection system 100, which is one embodiment of the present invention. Fig. 14(a) is a diagram showing the detection results of road surface conditions for area-type damage, and Fig. 14(b) is a diagram showing the detection results of road surface conditions for line-type damage.

[0102] As shown in FIG. 13, the normal pavement estimation unit 103 estimates an area-type normal pavement surface F1 and a line-type normal pavement surface F2.

[0103] By using the area-type normal pavement surface F1, it is possible to detect area-type damage that is relatively widespread, such as rutting Aw1, Aw2 and convex damage At1, At2. In addition, by using the linear-type normal pavement surface F2, it is possible to detect linear damage that is relatively narrow, such as cracks h3, h4.

[0104] As shown in Figure 14(a), as a result of detecting the road surface condition of area-type damage, area-type damage such as ruts DR1, potholes DR2, other concave damage DR3, and convex damage DR4 is appropriately detected and displayed on the monitor 5 by the display control means 111.

[0105] Furthermore, as shown in FIG. 14(b), as a result of detecting the road surface condition of linear damage, linear damage such as cracks DP1 and DP2 is appropriately detected and displayed on the monitor 5 by the display control means 111.

[0106] In this way, the road surface condition detection system 100 according to one embodiment of the present invention includes a normal pavement estimation unit 103 that estimates an area-type normal pavement surface F1 by fitting to each first section d11 in the point cloud data for each scan line acquired from the point cloud data generation device 2, and estimates a linear-type normal pavement surface F2 by fitting to each second section d21 that is shorter than the first section d11; and a normal pavement estimation unit 103 that estimates a linear-type normal pavement surface F2 by fitting to each second section d21 that is shorter than the first section d11 based on a first deformation displacement amount that is the difference between the point cloud data for each scan line and the area-type normal pavement surface F1. The system is equipped with a damage determination unit 104 that determines area-type damage for each first determination section d12 that is shorter than the first section d21, and that determines linear damage for each second determination section d22 that is shorter than the second section d21 based on the second deformation displacement amount, which is the difference between the point cloud data for each scan line and the linear normal pavement surface F2.This makes it possible to easily and appropriately detect both area-type damage, which is damage over a relatively wide area such as rutting, potholes, and sagging, and linear damage, which is damage in a line over a relatively narrow area such as cracks and pavement joints, using the same algorithm.

[0107] The above-described embodiment can also be realized by executing a road surface condition detection program installed in a computer. [Explanation of symbols]

[0108] 1. Road surface condition detection device 2. Point cloud data generator 3. Cart (mobile) 5 monitors 21 Laser scanner 24 Data Collection Methods 25 Temporary data storage unit 100 Road Condition Detection System 101 Point cloud data acquisition unit 101 Point cloud data acquisition unit 102 Point cloud data storage unit 103 Normal pavement surface estimation section 103A Area-based normal pavement surface estimation unit 103B Linear normal pavement surface estimation section 104 Damage determination section 104A Area type damage determination section 104B Linear damage determination section 105 Polygon Generation Unit 106 Table storage unit 107 Type determination unit 108 Polyline generation section 111 Display control means

Claims

1. A road surface condition detection device that acquires point cloud data from a point cloud data generation device that generates three-dimensional coordinates of a road as point cloud data for each scan line based on reflected light of laser light irradiated onto the road along which a mobile object is traveling, and detects a road surface condition of the road based on the acquired point cloud data, a normal pavement surface estimation unit that estimates an area-type normal pavement surface by fitting each first section in the point cloud data for each scan line acquired from the point cloud data generation device, and estimates a line-type normal pavement surface by fitting each second section that is shorter than the first section; a damage determination unit that determines area-type damage for each first determination section that is shorter than the first section based on a first deformation displacement amount, which is the difference between the point cloud data for each scan line and the area-type normal pavement surface, and that determines linear-type damage for each second determination section that is shorter than the second section based on a second deformation displacement amount, which is the difference between the point cloud data for each scan line and the linear-type normal pavement surface; A road surface condition detection device comprising:

2. The normal pavement surface estimation unit The curvature is calculated based on the angle formed by a straight line connecting an arbitrary point of interest in the point cloud data for each scan line with a first neighboring point, which is one point of the point cloud data located a predetermined proximity distance away on one side, and a straight line connecting the point of interest with a second neighboring point, which is one point of the point cloud data located a predetermined proximity distance away on the other side, and the proportion of the curvature that is equal to or less than a predetermined curvature threshold when the proximity distance is changed within a predetermined range is calculated as the non-damage degree, and fitting is performed using the non-damage degree as a weight to estimate the area-type normal pavement surface and the line-type normal pavement surface.

2. The road surface condition detection device according to claim 1.

3. a polygon generation unit that extracts point cloud data in which the first deformation displacement amount is equal to or greater than a predetermined first damage threshold as a damage point set, and generates a damage polygon by clustering the extracted damage point set; 2. The road surface condition detection device according to claim 1, further comprising:

4. The polygon generation unit Point cloud data in which the first deformation displacement amount is equal to or greater than a predetermined first damage threshold is extracted as a damage candidate point set, a morphology operation is performed on the extracted damage candidate point set to extract it as a damage point set, and a damage polygon is generated by clustering the extracted damage point set.

4. The road surface condition detection device according to claim 3.

5. a storage means for storing a region-type damage table that associates the type of region-type damage with the characteristics of the region-type damage; a type determination unit that determines a type of area-type damage corresponding to the generated damage polygon based on the area-type damage table and attribute information of the generated damage polygon; 5. The road surface condition detection device according to claim 4, further comprising:

6. a polyline generation unit that extracts a set of linear damage candidate points by applying adaptive threshold processing to the second deformation displacement amount, generates damage candidate clusters by clustering the set of linear damage candidate points using a region growing method, extracts linear damage clusters by applying a random forest to the generated damage candidate clusters, and generates linear damage polylines by constructing a minimum spanning tree from a Delaunay triangulation generated using representative points of the extracted linear damage clusters; 2. The road surface condition detection device according to claim 1, further comprising:

7. A road surface condition detection program executed by a road surface condition detection device that acquires point cloud data from a point cloud data generation device that generates three-dimensional coordinates of a road as point cloud data for each scan line based on reflected light of a laser beam irradiated onto the road along which a mobile object travels, and detects the road surface condition of the road based on the acquired point cloud data, a normal pavement surface estimation step of estimating an area-type normal pavement surface by fitting each first section in the point cloud data for each scan line acquired from the point cloud data generation device, and estimating a line-type normal pavement surface by fitting each second section that is shorter than the first section; a damage determination step for determining area-type damage for each first determination section shorter than the first section based on a first deformation displacement amount, which is the difference between the point cloud data for each scan line and the area-type normal pavement surface, and determining linear-type damage for each second determination section shorter than the second section based on a second deformation displacement amount, which is the difference between the point cloud data for each scan line and the linear-type normal pavement surface; a road surface condition detection program for causing the road surface condition detection device to execute the above steps;

Images