Road surface roughness detection method and system based on binocular vision

Abstract

The present application relates to the technical field of road surface flatness detection, and specifically to a road surface flatness detection method and system based on binocular vision, the method comprising: using the point cloud data acquisition module of the system to acquire road surface point cloud data of the road to be tested; analyzing the characteristics of the road surface point cloud data, and constructing the comprehensive characteristic value of each supervoxel in the road surface point cloud data; using the comprehensive characteristic values ​​of all supervoxels in the road surface point cloud data to distinguish supervoxels, and performing downsampling processing according to the distinguished supervoxels; performing plane fitting on the downsampled road surface point cloud data, and outputting the fitting plane and outlier point set; taking the average of the shortest distances from all outliers in the outlier point set to the fitting plane as the road surface flatness of the road to be tested. The present application aims to use different downsampling methods for different areas in the road surface point cloud data to improve the accuracy of road surface flatness detection.

Description

Technical Field

[0001] The present application relates to the technical field of road surface smoothness detection, and in particular to a road surface smoothness detection method and system based on binocular vision. Background Art

[0002] Road surfaces are very likely to suffer varying degrees of damage under the long-term effects of various natural factors and vehicle loads. When the damage accumulates to a certain extent, the road surface smoothness will be reduced. If the road surface smoothness is not detected and maintained in time, the road surface damage will become increasingly serious, which will lead to the road being unable to be used normally.

[0003] Compared with the traditional road surface roughness detection technology based on laser sensors, the road surface roughness detection technology based on binocular vision can quickly detect road surface roughness at a lower cost. For example, the Chinese patent publication No. CN115797338B, a panoramic road surface multi-performance index calculation method and system based on binocular vision, calculates road surface roughness based on road surface point cloud data acquired by a binocular camera. Road surface point cloud data usually needs to be preprocessed before use, including downsampling and filtering. Downsampling is a key step in point cloud preprocessing, which can effectively reduce the influence of redundant data contained in the road surface point cloud data acquired by the binocular camera on the plane fitting of subsequent road surface point cloud data, thereby improving the calculation accuracy of road surface roughness.

[0004] However, the existing point cloud downsampling method usually adopts a method to downsample the collected point cloud data, and the manhole covers, rain grates and other accessories on the road surface will also form mutations in the acquired road surface point cloud data, making this single point cloud downsampling method easily lead to the downsampled road surface point cloud data retaining more road surface detail information in the said accessory area and losing more road surface detail information in the road surface damaged pothole area, thereby affecting the accuracy of subsequent road surface flatness detection. Summary of the invention

[0005] In order to solve the above technical problems, this application provides a road surface flatness detection method and system based on binocular vision. The technical solutions adopted are as follows:

[0006] In a first aspect, an embodiment of the present application provides a road surface flatness detection method based on binocular vision, the method is implemented by a road surface flatness detection system based on binocular vision, and the method comprises the following steps:

[0007] Step 1: Use the point cloud data acquisition module of the system to obtain the road surface point cloud data of the road to be tested; wherein the positive direction of the Z axis of the road surface point cloud data is the direction perpendicular to the upward direction of the binocular camera lens plane;

[0008] Step 2: Analyze the characteristics of the road point cloud data and construct the comprehensive characteristic value of each supervoxel in the road point cloud data; specifically:

[0009] S1, using the difference between the Z coordinate value of each voxel in the road point cloud data and the mean Z coordinate value of all voxels, screen out abnormal voxels; perform clustering on all abnormal voxels, obtain the minimum bounding box of each cluster obtained by clustering, and form all voxels in the cluster into a voxel set;

[0010] S2, obtaining a first eigenvalue of each voxel set based on the mean distance of the ESF histogram between each voxel set and all remaining voxel sets and the mean difference of the minimum bounding box volume;

[0011] S3, dividing each voxel set into multiple supervoxels, analyzing the standard deviation of the curvature of all voxels in each supervoxel and the distance between each supervoxel and the voxel set to which it belongs, and obtaining the second eigenvalue of each voxel set;

[0012] S4, dividing the road point cloud data into a plurality of supervoxels, and forward fusing the first eigenvalue and the second eigenvalue in the voxel set to which each voxel in the supervoxel belongs, taking the average level of the result as the comprehensive eigenvalue of the supervoxel;

[0013] Step 3: Use the comprehensive characteristic values ​​of all supervoxels in the road point cloud data to distinguish supervoxels, and perform downsampling processing according to the distinguished supervoxels; perform plane fitting on the downsampled road point cloud data, and output the fitting plane and the outlier point set; take the average of the shortest distances from all outliers in the outlier point set to the fitting plane as the road surface flatness of the road to be tested.

[0014] Preferably, in step S1, the method for screening abnormal voxels is: obtaining the segmentation threshold m of all the differences calculated from all voxels in the road surface point cloud data; and recording the voxels in the road surface point cloud data whose differences are greater than or equal to the segmentation threshold m as abnormal voxels.

[0015] Preferably, in step S1, the clustering distance for clustering all abnormal voxels is the spatial coordinate distance between the abnormal voxels.

[0016] Preferably, in step S2, the ESF histogram is obtained by extracting each voxel set in the road surface point cloud data by using a point cloud global feature ESF algorithm.

[0017] Preferably, in step S2, the first eigenvalue of each voxel set is determined by the product of the distance mean of the ESF histogram calculated for the corresponding voxel set and the difference mean of the minimum bounding box volume.

[0018] Preferably, in step S3, the distance between each supervoxel and the voxel set to which it belongs is: the distance between the coordinates of the center point of each supervoxel and the coordinates of the center point of the voxel set to which it belongs.

[0019] Preferably, in step S3, the method for obtaining the second eigenvalue of each voxel set is:

[0020] Forward fusion of the standard deviation calculated for each supervoxel in each voxel set and the distance between each supervoxel and the voxel set to which it belongs;

[0021] The average level of the forward fusion results of all supervoxels in each voxel set is used as the second eigenvalue of each voxel set.

[0022] Preferably, in step S4, for a voxel in the supervoxel that does not belong to any voxel set, the voxel set to which the voxel belongs is determined as the voxel set corresponding to the maximum value in the result of forward fusion of the first eigenvalue and the second eigenvalue in the road surface point cloud data.

[0023] Preferably, the method of distinguishing supervoxels by using the comprehensive characteristic values ​​of all supervoxels in the road surface point cloud data, and performing downsampling processing respectively according to the distinguished supervoxels includes:

[0024] Obtain a segmentation threshold w of the second comprehensive characteristic values ​​of all supervoxels in the road surface point cloud data;

[0025] All supervoxels in the road point cloud data with comprehensive eigenvalues ​​less than or equal to the segmentation threshold w are downsampled using the point cloud geometric sampling method; all supervoxels in the road point cloud data with comprehensive eigenvalues ​​greater than the segmentation threshold w are downsampled using the point cloud body centroid sampling method.

[0026] In the second aspect, another embodiment of the present application further provides a road surface flatness detection system based on binocular vision, which implements the above-mentioned road surface flatness detection method based on binocular vision. The system includes a point cloud data acquisition module, a point cloud data processing module, and a road surface flatness calculation module:

[0027] The point cloud data acquisition module is used to collect road point cloud data in real time by installing a binocular camera on the test vehicle, and input the road point cloud data into the point cloud data processing module;

[0028] The point cloud data processing module is used to process the data acquired by the point cloud data acquisition module. The processing method adopts step 2 of the road surface roughness detection method based on binocular vision, and the processed data is input into the road surface roughness calculation module;

[0029] The road surface flatness calculation module is used to analyze the road surface flatness of the data output by the point cloud data processing module, and the analysis method adopts step three of the road surface flatness detection method based on binocular vision.

[0030] This application has at least the following beneficial effects:

[0031] 1. This application obtains comprehensive feature values ​​by analyzing the features of road point cloud data, which can effectively identify the point cloud data corresponding to the road damage pothole area in the road point cloud data to be tested, and avoid the possibility that the manhole cover, rain grate and other accessory areas in the road to be tested are identified as road damage pothole areas;

[0032] 2. The present application utilizes comprehensive eigenvalues ​​to select different downsampling methods for each supervoxel in the road surface point cloud data, which can effectively retain the detail information in the road surface damaged pothole area in the road surface point cloud data, and suppress the detail information in the road attachment area in the road surface point cloud data, thereby enabling the road surface point cloud data after downsampling to truly reflect the actual damage condition of the road surface and improve the accuracy of road surface flatness detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 A flow chart of a method for detecting road surface flatness based on binocular vision provided in one embodiment of the present application;

[0035] Figure 2 A flow chart of a method for constructing a comprehensive feature value of each supervoxel in road point cloud data provided by an embodiment of the present application;

[0036] Figure 3 A block diagram of a point cloud data acquisition module provided for one embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following is a detailed description of the road surface flatness detection method and system based on binocular vision proposed in the present application, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0038] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0039] The specific scheme of the road surface flatness detection method and system based on binocular vision provided by the present application is described in detail below with reference to the accompanying drawings.

[0040] Example 1

[0041] An embodiment of the present application provides a method for detecting road surface flatness based on binocular vision, for details, see Figure 1 , the method comprises the following steps:

[0042] Step 1: Use the point cloud data acquisition module of the system to obtain the road surface point cloud data of the road to be tested; wherein the positive direction of the Z axis of the road surface point cloud data is the upward direction perpendicular to the plane of the binocular camera lens.

[0043] Step 2: Analyze the characteristics of road point cloud data and construct the comprehensive feature value of each supervoxel in the road point cloud data.

[0044] Since the purpose of road surface roughness detection is to evaluate the degree of road surface damage by detecting the road surface roughness, this application ensures that the point cloud data after downsampling processing can truly reflect the actual damage condition of the road surface by retaining the detailed information in the road surface damaged pothole area in the road surface point cloud data as much as possible, thereby improving the accuracy of subsequent road surface roughness detection.

[0045] At the same time, there are usually road appendages such as manhole covers and rain grates on the road surface. The shapes and heights of these appendages have nothing to do with the flatness of the road surface, but they will form mutations in the point cloud data, thereby interfering with the subsequent calculation of the flatness of the road surface. Therefore, this application improves the subsequent detection accuracy of the flatness of the road surface by suppressing the detailed information in these road appendage areas in the road point cloud data.

[0046] Accordingly, in this application, the flowchart of the method for constructing the comprehensive feature value of each supervoxel in the road surface point cloud data is as shown in the attached figure. Figure 2 As shown, specifically:

[0047] S1, using the difference between the Z coordinate value of each voxel in the road point cloud data and the mean Z coordinate value of all voxels, screen out abnormal voxels; cluster all abnormal voxels, and obtain the minimum bounding box of each cluster obtained by clustering, and form all the voxels in it into a voxel set.

[0048] Specifically, since the road surface is flat under normal circumstances, the road surface usually has a relatively consistent Z coordinate value in the road surface point cloud data. The road surface is usually damaged and has potholes due to various natural factors and the long-term action of vehicle loads. There is usually a certain height difference between the road surface and the normal road surface, and the road accessories such as manhole covers and rain grates on the road surface usually have a certain height difference from the normal road surface. The manhole covers and rain grates will sink under the long-term action of vehicle loads, resulting in a large difference in Z coordinate values ​​between the corresponding voxels in the road surface point cloud data of the damaged pothole areas and the road accessory areas and the normal road surface areas.

[0049] Based on the above analysis, as a preferred implementation, the present application uses the difference between the Z coordinate value of each voxel in the road point cloud data and the average Z coordinate value of all voxels to screen out abnormal voxels. In this embodiment, specifically:

[0050] The average Z coordinate value of all voxels in the road surface point cloud data is calculated as the height value z1 of the normal road surface in the road surface point cloud data, and the absolute value of the difference between the Z coordinate value and the height value z1 of each voxel in the road surface point cloud data is calculated as the road surface height difference corresponding to each voxel.

[0051] The road surface height difference of all voxels in the road surface point cloud data is used as the input of the maximum inter-class variance algorithm, and the output segmentation threshold m is recorded as the first threshold. The voxels in the road surface point cloud data whose road surface height difference is less than the first threshold are recorded as normal voxels, which are used to characterize the voxels corresponding to the normal road surface area in the road to be tested in the road surface point cloud data, and the remaining voxels are recorded as abnormal voxels. The maximum inter-class variance algorithm is a well-known technology, and the specific process will not be repeated here. In other embodiments, other threshold segmentation algorithms can also be used to obtain the segmentation threshold.

[0052] Furthermore, as a preferred implementation, the present application performs clustering processing on all abnormal voxels, obtains the minimum bounding box of each cluster obtained by clustering, and forms all voxels within it into a voxel set. Specifically in this embodiment:

[0053] A point cloud Euclidean clustering algorithm is used to cluster all abnormal voxels in the road surface point cloud data. The neighbor search radius, the minimum number of cluster points, and the maximum number of cluster points in the point cloud Euclidean clustering algorithm are set to 2 cm, 100, and 25,000, respectively, in this embodiment. The clustering distance is the spatial coordinate distance between abnormal voxels. Multiple voxel point clusters in the road surface point cloud data are output. The point cloud Euclidean clustering algorithm is a well-known technology, and the specific process is not repeated here. In other embodiments, other suitable methods such as density clustering algorithm can also be used for clustering processing.

[0054] Since the road surface damaged pothole area or road accessory areas such as manhole covers and rain grates in the road to be tested usually have a continuous surface, the minimum bounding box of each voxel point cluster in the road surface point cloud data is extracted respectively, and the set composed of all voxels in each obtained minimum bounding box is used as each voxel set of the road surface point cloud data, which is used to characterize the set composed of all voxels corresponding to each road surface damaged pothole area or the road accessory area on the road surface to be tested in the road surface point cloud data. The extraction of the minimum bounding box of the point cloud data is a well-known technology, and the specific process is not repeated here.

[0055] S2: Obtain a first eigenvalue of each voxel set based on a mean distance of an ESF histogram between each voxel set and all remaining voxel sets and a mean difference of minimum bounding box volumes.

[0056] Since the damaged potholes on the road surface are caused by various natural factors and the long-term effects of different traffic loads, the damaged potholes on the road surface not only have inconsistent surface shapes, but also different sizes. On the same road surface, manhole covers, rain grates and other road accessories usually use unified specifications. For example, manhole covers are unified in round shape, the inner diameter of the inspection manhole cover base is unified in 700mm, rainwater grates are unified in rectangular shape, and the inner opening size of the rainwater grate base is unified in 300mm×550mm.

[0057] Based on this, this embodiment uses the point cloud global feature ESF algorithm to extract the ESF histogram of each voxel set in the road surface point cloud data, which is used to describe the surface shape characteristics of the road surface area corresponding to each voxel set. The default value of the ESF iteration number in the algorithm is 2000. The point cloud global feature ESF algorithm is a well-known technology, and the specific process will not be repeated here.

[0058] The volume of the minimum bounding box of each voxel set in the road surface point cloud data is calculated respectively as the point cloud size of each voxel set, which is used to characterize the surface size of the road surface area corresponding to each voxel set.

[0059] Based on the above analysis, as a preferred implementation, the present application obtains the first eigenvalue of each voxel set based on the mean distance of the ESF histogram between each voxel set and all remaining voxel sets and the mean difference of the minimum bounding box volume. In this embodiment, it is specifically:

[0060] Taking the i-th voxel set of road point cloud data as an example, the first eigenvalue W1 of the i-th voxel set is obtained to evaluate whether the road surface area corresponding to the i-th voxel set has the characteristics of uniform specifications of road accessory areas such as manhole covers and rainwater grates on the road. The specific calculation relationship is:

[0061] , where d(i) represents the mean of the distances between the ESF histogram of the ith voxel set and the ESF histograms of the remaining voxel sets in the road surface point cloud data; v(i) represents the mean of the absolute values ​​of the differences between the point cloud sizes of the ith voxel set and the point cloud sizes of the remaining voxel sets in the road surface point cloud data; wherein, the Bhattacharyya distance is used to calculate the distance between the ESF histograms, and in other embodiments, other methods for analyzing the distance between the ESF histograms may also be used.

[0062] The less similar the surface shape distribution characteristics and the closer the surface size are between the road pavement area corresponding to the i-th voxel set and the road pavement areas corresponding to the other voxel sets in the road surface point cloud data, that is, the larger the first eigenvalue of the i-th voxel set, the less the road pavement area corresponding to the i-th voxel set has the uniform specification characteristics of road accessory areas such as manhole covers and rain gratings on the road, and the more likely the road pavement area corresponding to the voxel set is to be a damaged pothole area in the road.

[0063] In other embodiments, the calculation relationship of the first eigenvalue of the i-th voxel set may also be: , which is set by the implementer.

[0064] S3, dividing each voxel set into multiple supervoxels, analyzing the standard deviation of the curvature of all voxels in each supervoxel and the distance between each supervoxel and the voxel set to which it belongs, and obtaining the second eigenvalue of each voxel set.

[0065] Furthermore, the surface of the damaged pothole area on the road surface usually has local uneven areas such as potholes and bumps, and the distribution of such local uneven areas is random, while the potholes, bumps and other local uneven areas on the road surface where road accessories such as manhole covers and rain grates are located are usually distributed around the manhole covers and rain grates. This is because manhole covers and rain grates are usually made of cast iron, and their surfaces are less likely to be damaged compared to the road surface.

[0066] Based on the above analysis, as a preferred implementation, the present application divides each voxel set into multiple supervoxels, analyzes the standard deviation of the curvature of all voxels in each supervoxel and the distance between each supervoxel and its voxel set, and obtains the second eigenvalue of each voxel set.

[0067] The method for obtaining the second eigenvalue of each voxel set is as follows: forwardly fuse the standard deviation calculated for each supervoxel in each voxel set and the distance between each supervoxel and the voxel set to which it belongs; and use the average level of the forward fusion results of all supervoxels in each voxel set as the second eigenvalue of each voxel set.

[0068] It can be understood that fusion can be divided into forward fusion and reverse fusion. This embodiment adopts the forward fusion method. Forward fusion is a fusion method such as addition and multiplication between data. The specific forward fusion method is determined by the implementer according to the actual situation. The application does not impose any special restrictions.

[0069] The process of obtaining the second eigenvalue of each voxel set in this embodiment is specifically as follows:

[0070] Taking the i-th voxel set as an example, the voxel set is segmented using the supervoxel cloud connectivity segmentation algorithm (VCCS) to obtain multiple supervoxels of the voxel set, wherein the voxel resolution, seed resolution, color weight, spatial resolution, and normal vector weight in the VCCS algorithm are respectively set to 5 cm, 0.1, 1, 1, and 4 in this embodiment, and are specifically set by the implementer in other embodiments. In addition, the VCCS algorithm is a well-known technology and will not be described in detail. In other embodiments, other methods can also be used to achieve supervoxel segmentation.

[0071] Taking the j-th supervoxel in the i-th voxel set as an example, the curvature of each voxel in the j-th supervoxel is calculated respectively to characterize the local surface features of each voxel, and the standard deviation of all the curvatures obtained is recorded as the local roughness of the j-th supervoxel, which is used to characterize the local roughness of potholes and bumps on the surface of the road surface area corresponding to the supervoxel. The curvature calculation of point cloud data is a well-known technology, and the specific process will not be repeated here.

[0072] The coordinates of the center points of the i-th voxel set and its j-th supervoxel are calculated respectively, and the distance between the two center point coordinates is recorded as the distance between the j-th supervoxel and its voxel set, which is used to characterize the extent to which the road surface area corresponding to the j-th supervoxel is located around the road surface area corresponding to the voxel set to which it belongs.

[0073] Furthermore, the local unevenness of all supervoxels in the i-th voxel set and the distance to the voxel set to which it belongs are used to determine the second eigenvalue of the i-th voxel set, which is used to evaluate whether the road surface area corresponding to the voxel set has the characteristics of the manhole cover, rain grate and other road accessories on the road, such as the local uneven area distributed around the manhole cover and rain grate. The specific calculation relationship is:

[0074] , where d1(j) represents the distance between the j-th supervoxel and the voxel set to which it belongs; v1(j) represents the local roughness of the j-th supervoxel; and J represents the number of supervoxels in the i-th voxel set.

[0075] The less the road pavement area corresponding to the i-th voxel set has the characteristics of local uneven areas distributed around manhole covers, rain grates and other road accessories on the road, the smaller the second eigenvalue is, and the more likely the road pavement area corresponding to the voxel set is to be a damaged pothole area in the road.

[0076] In other embodiments, the calculation relationship of the second eigenvalue of the i-th voxel set may also be: , which is set by the implementer.

[0077] S4, dividing the road point cloud data into multiple supervoxels, and forward fusing the first eigenvalue and the second eigenvalue in the voxel set to which each voxel in the supervoxel belongs, taking the average level of the result as the comprehensive eigenvalue of the supervoxel.

[0078] This embodiment uses a supervoxel cloud connectivity segmentation algorithm to segment the road surface point cloud data to obtain multiple supervoxels of the road surface point cloud data.

[0079] As a preferred implementation, the average level of the result of forward fusion of the first eigenvalue and the second eigenvalue in the voxel set to which each voxel in the supervoxel belongs is used as the comprehensive eigenvalue of the supervoxel.

[0080] For a voxel in the supervoxel that does not belong to any voxel set, the voxel set to which the voxel belongs is determined to be the voxel set corresponding to the maximum value in the result of forward fusion of the first eigenvalue and the second eigenvalue in the road surface point cloud data.

[0081] Specifically, in this embodiment, the mean of the first eigenvalue W1 and the second eigenvalue W2 of the i-th voxel set is recorded as the first comprehensive eigenvalue of the i-th voxel set, which is used to evaluate whether the road surface area corresponding to the voxel set is a road damaged pothole area. The less the road surface area has the distribution characteristics of road accessory areas such as manhole covers and rain gratings on the road, that is, the smaller the mean, the more likely the road surface area is to be a road damaged pothole area, that is, the smaller the first comprehensive eigenvalue.

[0082] The second comprehensive eigenvalue of each voxel in the road surface point cloud data is assigned to the first comprehensive eigenvalue of the voxel set to which the voxel belongs. The second comprehensive eigenvalue is used to evaluate whether the voxel is the voxel corresponding to the road damage pothole area in the road surface point cloud data. For the voxel in the super voxel that does not belong to any voxel set, the voxel set to which the voxel belongs is determined as the voxel set corresponding to the maximum value of the first comprehensive eigenvalue in the road surface point cloud data.

[0083] The mean of the second comprehensive eigenvalues ​​of all voxels in each supervoxel in the road surface point cloud data is taken as the third comprehensive eigenvalue corresponding to each supervoxel, which is also recorded as the summed eigenvalue corresponding to each supervoxel, and is used to evaluate whether the road surface area corresponding to the supervoxel is a road damage pothole area.

[0084] Step 3: Use the comprehensive characteristic values ​​of all supervoxels in the road point cloud data to distinguish supervoxels, and perform downsampling processing according to the distinguished supervoxels; perform plane fitting on the downsampled road point cloud data, and output the fitting plane and the outlier point set; take the average of the shortest distances from all outliers in the outlier point set to the fitting plane as the road surface flatness of the road to be tested.

[0085] As a preferred implementation, the present application uses the comprehensive feature values ​​of all supervoxels in the road point cloud data to distinguish supervoxels, and performs downsampling processing according to the distinguished supervoxels, which is specifically as follows in this embodiment:

[0086] The comprehensive eigenvalues ​​of all supervoxels in the road surface point cloud data are used as the input of the maximum inter-class variance algorithm, and the segmentation threshold w is output and recorded as the second threshold. All supervoxels in the road surface point cloud data whose comprehensive eigenvalues ​​are less than or equal to the second threshold are obtained, and they are formed into a first supervoxel set, which is used to characterize the set composed of all supervoxels corresponding to the road surface damaged pothole area in the road surface point cloud data, and all supervoxels remaining in the road surface point cloud data except those included in the first supervoxel set are obtained, and they are formed into a second supervoxel set.

[0087] In order to retain the detailed information of all road damaged pothole areas in the road surface point cloud data as much as possible, the geometric sampling method of the point cloud is used to downsample each supervoxel in the first supervoxel set. In order to suppress the detailed information in the remaining non-road damaged pothole areas in the road surface point cloud data as much as possible, the body centroid sampling method of the point cloud is used to downsample each supervoxel in the second supervoxel set to obtain the road surface point cloud data after downsampling. The geometric sampling method of the point cloud and the body centroid sampling method are both well-known technologies, and the specific process will not be repeated here.

[0088] Furthermore, as a preferred implementation, the present application performs plane fitting on the downsampled road point cloud data, outputs the fitting plane and the outlier point set; and takes the average of the shortest distances from all outliers in the outlier point set to the fitting plane as the road surface flatness of the road to be tested. In this embodiment, specifically:

[0089] The RANSAC random sampling consensus algorithm is used to perform plane fitting on the road point cloud data. The maximum number of iterations and the adjustment threshold in the algorithm take the default values ​​of 1000 and 0.4 in this embodiment, which can be set by the implementer. The fitting plane and outlier point set of the road point cloud data after downsampling are output. The method of obtaining the fitting plane and outlier point set by the RANSAC random sampling consensus algorithm is a well-known technology, and the specific process is not repeated here. In other embodiments, other plane fitting methods can also be used to perform plane fitting processing on the road point cloud data after downsampling.

[0090] All outliers in the outlier point set of the downsampled road point cloud data are traversed, and the average of the shortest distances from all outliers to their fitting planes is used as the road surface flatness of the road to be tested. In this embodiment, the shortest distance from the outliers to their fitting planes is calculated using the Euclidean distance calculation method.

[0091] Example 2

[0092] Another embodiment of the present application also provides a road surface flatness detection system based on binocular vision, which includes a point cloud data acquisition module, a point cloud data processing module, and a road surface flatness calculation module.

[0093] The point cloud data acquisition module includes at least six steps: camera calibration, road image acquisition, image preprocessing, image correction, stereo matching, and 3D reconstruction. It is used to collect road point cloud data in real time by installing a binocular camera on the test vehicle and input the road point cloud data into the point cloud data processing module. The block diagram of the point cloud data acquisition module is shown in the attached figure. Figure 3 As shown, specifically:

[0094] Step 1: Use Zhang Zhengyou's plane calibration method to calibrate the binocular camera to obtain the internal and external parameters and distortion parameters of the binocular camera for subsequent image correction and point cloud solution;

[0095] Step 2: Install a binocular camera on the test vehicle and use the binocular camera to collect road surface images of the road to be tested, including left and right view images of the road surface;

[0096] Step 3: Preprocess the left and right view images of the road surface, including grayscale, denoising, smoothing, and image enhancement, to improve image quality and reduce the impact of noise on subsequent processing;

[0097] Step 4: Use the camera calibration results to perform distortion correction and stereo correction on the pre-processed left and right view images of the road surface to reduce the impact of image distortion caused by the camera lens on the subsequently generated disparity map;

[0098] Step 5: Use the SGBM stereo matching algorithm to perform stereo matching on the corrected left and right view images of the road surface, and output a disparity map to determine the geometric relationship between the corresponding pixel points of the same spatial point on the left and right view images, and obtain the disparity map for subsequent point cloud solution;

[0099] Step 6: Use the camera calibration result to reconstruct the three-dimensional point cloud of the disparity map, where the forward direction of the test vehicle is the positive direction of the Y axis, the upward direction perpendicular to the binocular camera lens plane is the positive direction of the Z axis, and the direction perpendicular to the Y axis and the Z axis is the X axis direction. The positive direction of the X axis is determined by the right-hand rule. The road surface point cloud data of the road is input into the point cloud data processing module. Zhang Zhengyou's plane calibration method, image preprocessing, stereo correction, SGBM stereo matching algorithm, and three-dimensional point cloud reconstruction of the disparity map are all well-known technologies, and the specific process will not be repeated here.

[0100] The point cloud data processing module is used to process the data acquired by the point cloud data acquisition module. The processing method adopts step 2 of the road surface roughness detection method based on binocular vision, and the processed data is input into the road surface roughness calculation module;

[0101] The road surface flatness calculation module is used to analyze the road surface flatness of the data output by the point cloud data processing module, and the analysis method adopts step three of the road surface flatness detection method based on binocular vision.

[0102] The various embodiments in the present application are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0103] It should be noted that, unless otherwise specified and limited, terms such as "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such article or device. In the absence of further restrictions, an element defined by the sentence "including one..." does not exclude the existence of other identical elements in the article or device including the element. In addition, the term "and\or" used herein includes any and all combinations of one or more related listed items.

[0104] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not invented by the present application.

[0105] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A road surface roughness detection method based on binocular vision, which is implemented by a road surface roughness detection system based on binocular vision, and is characterized in that: The method comprises the following steps: Step 1: Use the point cloud data acquisition module of the system to obtain the road surface point cloud data of the road to be tested; wherein the positive direction of the Z axis of the road surface point cloud data is the direction perpendicular to the upward direction of the binocular camera lens plane; Step 2: Analyze the characteristics of the road point cloud data and construct the comprehensive characteristic value of each supervoxel in the road point cloud data; specifically: S1, using the difference between the Z coordinate value of each voxel in the road point cloud data and the mean Z coordinate value of all voxels, screen out abnormal voxels; perform clustering on all abnormal voxels, obtain the minimum bounding box of each cluster obtained by clustering, and form all voxels in the cluster into a voxel set; S2, obtaining a first eigenvalue of each voxel set based on the mean distance of the ESF histogram between each voxel set and all remaining voxel sets and the mean difference of the minimum bounding box volume; S3, dividing each voxel set into multiple supervoxels, forward fusion the standard deviation of the curvature of all voxels in each supervoxel in each voxel set and the distance between the corresponding supervoxel and the voxel set to which it belongs; and taking the average level of the forward fusion results of all supervoxels in each voxel set as the second eigenvalue of each voxel set; S4, dividing the road point cloud data into a plurality of supervoxels, and forward fusing the first eigenvalue and the second eigenvalue in the voxel set to which each voxel in the supervoxel belongs, taking the average level of the result as the comprehensive eigenvalue of the supervoxel; Step 3: Use the comprehensive characteristic values ​​of all supervoxels in the road point cloud data to distinguish supervoxels, and perform downsampling processing according to the distinguished supervoxels; perform plane fitting on the downsampled road point cloud data, and output the fitting plane and the outlier point set; take the average of the shortest distances from all outliers in the outlier point set to the fitting plane as the road surface flatness of the road to be tested.

2. The method for detecting road surface flatness based on binocular vision according to claim 1, characterized in that: In step S1, the method for screening abnormal voxels is: obtaining the segmentation threshold m of all the differences calculated from all voxels in the road surface point cloud data; The voxels in the road surface point cloud data whose differences are greater than or equal to the segmentation threshold m are recorded as abnormal voxels.

3. The method for detecting road surface flatness based on binocular vision as claimed in claim 2, characterized in that: In step S1, the clustering distance for clustering all abnormal voxels is the spatial coordinate distance between the abnormal voxels.

4. The method for detecting road surface flatness based on binocular vision according to claim 1, characterized in that: In step S2, the ESF histogram is extracted from each voxel set in the road surface point cloud data by using the point cloud global feature ESF algorithm.

5. The method for detecting road surface flatness based on binocular vision as claimed in claim 4, characterized in that: In step S2, the first eigenvalue of each voxel set is determined by the product of the distance mean of the ESF histogram calculated for the corresponding voxel set and the difference mean of the minimum bounding box volume.

6. The method for detecting road surface flatness based on binocular vision according to claim 1, characterized in that: In step S3, the distance between each supervoxel and the voxel set to which it belongs is: the distance between the center point coordinates of each supervoxel and the center point coordinates of the voxel set to which it belongs.

7. The method for detecting road surface flatness based on binocular vision according to claim 1, characterized in that: In step S4, for a voxel in the supervoxel that does not belong to any voxel set, the voxel set to which the voxel belongs is determined as the voxel set corresponding to the maximum value in the result of forward fusion of the first eigenvalue and the second eigenvalue in the road surface point cloud data.

8. The method for detecting road surface flatness based on binocular vision according to claim 1, characterized in that: The method of using the comprehensive characteristic values ​​of all supervoxels in the road surface point cloud data to distinguish supervoxels and performing downsampling processing respectively according to the distinguished supervoxels includes: Obtain a segmentation threshold w of the second comprehensive characteristic values ​​of all supervoxels in the road surface point cloud data; All supervoxels in the road point cloud data with comprehensive eigenvalues ​​less than or equal to the segmentation threshold w are downsampled using the point cloud geometric sampling method; all supervoxels in the road point cloud data with comprehensive eigenvalues ​​greater than the segmentation threshold w are downsampled using the point cloud body centroid sampling method.

9. A road surface roughness detection system based on binocular vision, which implements the road surface roughness detection method based on binocular vision as claimed in any one of claims 1 to 8, characterized in that: The system includes point cloud data acquisition module, point cloud data processing module and road surface smoothness calculation module: The point cloud data acquisition module is used to collect road point cloud data in real time by installing a binocular camera on the test vehicle, and input the road point cloud data into the point cloud data processing module; The point cloud data processing module is used to process the data acquired by the point cloud data acquisition module. The processing method adopts step 2 of the road surface roughness detection method based on binocular vision, and the processed data is input into the road surface roughness calculation module; The road surface flatness calculation module is used to analyze the road surface flatness of the data output by the point cloud data processing module, and the analysis method adopts step three of the road surface flatness detection method based on binocular vision.

Images