A method for processing multidimensional road damage data

Abstract

This invention relates to a multi-dimensional pavement damage data processing method, which includes the following steps: processing a acquired two-dimensional grayscale image using a two-dimensional image preprocessing algorithm to obtain a processed two-dimensional image; processing a acquired three-dimensional elevation image using a three-dimensional image preprocessing algorithm to obtain a three-dimensional image enhancement output result; and using the processed two-dimensional image and the three-dimensional image enhancement output result as input to a pavement damage identification model to locate and automatically and rapidly detect pavement damage areas. Compared with existing technologies, this invention overcomes the problems of missing image data, complex data features, and inconsistent data distribution, and effectively improves the accuracy and efficiency of damage identification.

Description

Technical Field

[0001] This invention belongs to the field of road information acquisition and detection technology, and relates to a method for processing multi-dimensional road damage data. Background Technology

[0002] Currently, road traffic is becoming increasingly congested and complex, leading to various types of road surface defects, such as cracks, potholes, ruts, crack sealing, block repairs, and loosening. To avoid disrupting daily traffic and ensuring driver safety, regular inspections of road surface damage are necessary for timely repair. Existing technologies generally employ unidirectional structured light scanning for intelligent road damage detection. However, this technology is limited by its underlying principles. Firstly, the optical characteristics of adjacent longitudinal sections differ. Secondly, the elevation of adjacent cross sections is superimposed with vehicle vibration displacement, resulting in complex data characteristics in the system's output of two-dimensional grayscale and three-dimensional elevation data. This irregular road surface data significantly increases the difficulty of intelligent road defect identification. Furthermore, a comprehensive evaluation system for road damage detection has not yet been established, and automatic defect classification methods do not yet achieve satisfactory results in terms of recognition effectiveness, category coverage, and real-time processing. Summary of the Invention

[0003] The purpose of this invention is to provide a multi-dimensional road damage data processing method to overcome the problems of missing image data and low accuracy and efficiency in identification.

[0004] The objective of this invention can be achieved through the following technical solutions:

[0005] A method for processing multidimensional road damage data, comprising the following steps:

[0006] The acquired two-dimensional grayscale image is processed using a two-dimensional image preprocessing algorithm to obtain a processed two-dimensional image.

[0007] The acquired 3D elevation images are processed using 3D image preprocessing algorithms to obtain 3D image enhancement output results;

[0008] The processed 2D and 3D image enhancement outputs are used as inputs to the road damage identification model to locate and automatically and rapidly detect road damage areas.

[0009] Furthermore, the two-dimensional image preprocessing algorithm includes:

[0010] Step S310: Calculate the initial preprocessing coefficients C0.

[0011]

[0012] in, It is the initial overall mean, and Benchmark is the set baseline grayscale value.

[0013] Step S320: Obtain a two-dimensional grayscale image I of the road surface. Based on a set threshold, determine whether the two-dimensional grayscale image of the road surface contains abnormal grayscale information. If so, remove the image; otherwise, add the image to the image column and obtain the grayscale mean Ave of the image column. current ;

[0014] Step S330: Based on preprocessing coefficients and grayscale mean Ave current The processed two-dimensional image I of the road surface grayscale image I is obtained. new ,

[0015] I new =I×C

[0016] Where C is the preprocessing coefficient. If I is the first image to be processed, C is C0; otherwise, C is updated using the following formula:

[0017]

[0018] Ave Back =αAve Back +(1-α)Ave current

[0019] α is the set update ratio, Ave Back It is the overall mean, and the initial value is

[0020] Step S340, return to step S320, until all image data processing is complete.

[0021] Furthermore, the initial overall mean The methods of obtaining it include:

[0022] A two-dimensional grayscale image of the road surface is acquired. Images containing abnormal grayscale information are removed based on a set threshold. N initial images are extracted that can be used to calculate the preprocessing coefficient C0. The grayscale mean of the N images is calculated as the initial overall mean.

[0023] Furthermore, the three-dimensional data preprocessing algorithm includes:

[0024] Step S410: Process the relative elevation data of the road surface obtained through data calibration using the pixel left-right adjacent interpolation method to obtain the set of elevation value points of the filtered three-dimensional image;

[0025] Step S420: Process the elevation point set of the original 3D image and the elevation point set of the filtered 3D image using a 3D image enhancement function to obtain the 3D image enhancement output result.

[0026] Furthermore, the data calibration utilizes the principle of line structured light to establish a pixel-to-spatial coordinate transformation relationship between two-dimensional grayscale images and three-dimensional elevation images, generating a calibration parameter file for obtaining road surface relative elevation data.

[0027] Furthermore, the pixel left-right adjacent interpolation method is as follows:

[0028]

[0029] Where X = {X1, X2, ..., X} w} is the pixel matrix of each row's cross section, X w =0 (w=1,2,...,W) is the set of outliers, X w ≠0 (w=1,2,...,W) is a normal point set, X′={X′1,X′2,...,X′ w} is the set of filtered elevation points, and W is the number of elevation points in the cross section.

[0030] Furthermore, the calculation formula for the three-dimensional image enhancement function is as follows:

[0031] A′(x, y) = A + MFF k |A|+MFF k |A(xI, yj)|

[0032]

[0033] Where A represents the matrix of elevation point sets in the original 3D image, A′ represents the matrix of elevation point sets in the filtered 3D image, MFF is an adaptive mean filtering technique, K is the filtering coefficient, and (2K+1)×(2K+1) is taken as the filtering template, A′ min Represents the minimum value of set A′, A′ max A' represents the maximum value of set A', and A″ represents the output result of 3D image enhancement, i, j∈[-K, K].

[0034] Furthermore, the method for constructing the pavement damage and distress identification model includes:

[0035] Step S510: Mark the defects in the high-quality two-dimensional and three-dimensional images containing pavement damage.

[0036] Step S520: Overlay the labeled two-dimensional and three-dimensional images to obtain a dual-channel training sample database;

[0037] Step S530: Train a deep learning network model using a dual-channel training sample database to obtain a road surface damage and defect identification model.

[0038] Furthermore, the road damage and defect identification model is superimposed with a conditional random field module to improve the accuracy of the road damage and defect identification model.

[0039] Furthermore, the aforementioned road surface damage includes cracks, potholes, ruts, crack sealant, block repairs, and loosening.

[0040] Compared with the prior art, the present invention has the following characteristics:

[0041] 1. This invention develops two-dimensional image preprocessing algorithms and three-dimensional image preprocessing algorithms for two-dimensional grayscale images and three-dimensional elevation data, respectively. These algorithms can effectively solve the problems of missing image data, complex data features, and inconsistent data distribution caused by factors such as vehicle bumps, mechanical vibrations, and uneven laser intensity distribution.

[0042] 2. This invention trains a deep learning network model using a dual-channel training sample database to obtain a pavement damage and distress identification model, which can quickly and accurately identify various pavement damage and distresses such as cracks, potholes, ruts, crack sealing, block repairs, and loosening over a wide range.

[0043] 3. This invention utilizes the principle of structured light to establish the conversion relationship between pixels and space for data calibration, thereby achieving high accuracy in the obtained road surface relative elevation data.

[0044] 4. This invention uses a conditional random field module superimposed on the road damage and defect identification model to improve the accuracy of the road damage and defect identification model. Attached Figure Description

[0045] Figure 1 This is a flowchart of a multi-dimensional road damage data processing method. Detailed Implementation

[0046] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0047] Example:

[0048] This embodiment provides a method for processing multi-dimensional road damage data, such as... Figure 1 As shown, it includes the following steps:

[0049] Step S1, Data Calibration: Based on the principle of line structured light, establish the transformation relationship between pixels and spatial coordinates for two-dimensional grayscale images and three-dimensional elevation images, and generate a calibration parameter file for obtaining road surface relative elevation data;

[0050] Step S2, Data Acquisition: After data calibration, multi-dimensional data of the asphalt pavement cross section are continuously acquired using vehicle-mounted measurement method, and the relative elevation data of the pavement is obtained by loading the calibration parameter file;

[0051] Step S3, Two-dimensional image preprocessing: The acquired two-dimensional grayscale image is processed using a two-dimensional image preprocessing algorithm to eliminate uneven grayscale values ​​and missing data, and to obtain the processed two-dimensional image.

[0052] Step S4, 3D image preprocessing: Performed simultaneously with step S3, the 3D image preprocessing algorithm is used to process the acquired 3D elevation image, perform anomaly correction and elevation calibration on the road surface relative elevation data, eliminate uneven data depth distribution and missing data, and obtain the 3D image enhancement output result.

[0053] Step S5: Use the processed two-dimensional and three-dimensional image enhancement outputs as input to the road damage identification model to locate and automatically and quickly detect the road damage area.

[0054] Step S2 above includes the following steps:

[0055] Step S210: Using the vehicle-mounted transverse line structured light imaging system, the camera is triggered by the wheel universal encoder, and the mileage and speed per unit time are calculated and used as the file name for storing multi-source data.

[0056] Step S220: The acquired multi-source data has two channels: Channel 1 is a two-dimensional grayscale image, and Channel 2 is a three-dimensional elevation image. Both data sources have the same size, M. H *N z By loading the calibration parameter file from step S210, the calibrated road surface relative elevation data is obtained.

[0057] The above-mentioned method uses the principle of structured light to establish the conversion relationship between pixels and space for data calibration, thereby achieving high accuracy in the obtained road surface relative elevation data.

[0058] Step S3 above includes the following steps:

[0059] Step S310: Calculate the initial preprocessing coefficients C0.

[0060]

[0061] in, It is the initial overall mean, and Benchmark is the set baseline grayscale value.

[0062] Step S320: Obtain a two-dimensional grayscale image I of the road surface. Based on a set threshold, determine whether the two-dimensional grayscale image of the road surface contains abnormal grayscale information. If so, remove the image; otherwise, add the image to the image column and obtain the grayscale mean Ave of the image column. current ;

[0063] Step S330: Based on preprocessing coefficients and grayscale mean Ave current The processed two-dimensional image I of the road surface grayscale image I is obtained. new ,

[0064] I new =I×C

[0065] Where C is the preprocessing coefficient. If I is the first image to be processed, C is C0; otherwise, C is updated using the following formula:

[0066]

[0067] Ave Back =αAve Back +(1-α)Ave current

[0068] α is the set update ratio, Ave Back It is the overall mean, and the initial value is

[0069] Step S340, return to step S320, until all image data processing is complete.

[0070] In one specific implementation, the initial overall mean is obtained. The methods include:

[0071] A two-dimensional grayscale image of the road surface is acquired. Images containing abnormal grayscale information are removed based on a set threshold. N initial images are extracted that can be used to calculate the preprocessing coefficient C0. The grayscale mean of the N images is calculated as the initial overall mean.

[0072] Step S4 above includes the following steps:

[0073] Step S410: Process the relative elevation data of the road surface obtained through data calibration using the pixel left-right adjacent interpolation method to obtain the set of elevation value points of the filtered three-dimensional image;

[0074] Step S420: Process the elevation point set of the original 3D image and the elevation point set of the filtered 3D image using a 3D image enhancement function to obtain the 3D image enhancement output result.

[0075] The pixel left-right adjacent interpolation method is as follows:

[0076]

[0077] Where X = {X1, X2, ..., X} w} is the pixel matrix of each row's cross section, X w =0 (w=1,2,...,W) is the set of outliers, X w ≠0 (w=1,2,...,W) is a normal point set, X′={X′1,X′2,...,X′ w} is the set of filtered elevation points, and W is the number of elevation points in the cross section.

[0078] The formula for calculating the 3D image enhancement function is:

[0079] A′(x, y) = A + MFF k |A|+MFF k |A(xI, yj)|

[0080]

[0081] Where A represents the matrix of elevation point sets in the original 3D image, A′ represents the matrix of elevation point sets in the filtered 3D image, MFF is an adaptive mean filtering technique, K is the filtering coefficient, and (2K+1)×(2K+1) is taken as the filtering template, A′ min Represents the minimum value of set A′, A′ max A' represents the maximum value of set A', and A″ represents the output result of 3D image enhancement, i, j∈[-K, K].

[0082] The aforementioned 3D image enhancement function is used to eliminate elevation irregularities.

[0083] The above data calibration establishes the conversion relationship between pixels and space using the principle of structured light, resulting in high measurement accuracy. It can assist vehicle-mounted structured light systems in acquiring two-dimensional grayscale images and three-dimensional high-precision data of the road surface.

[0084] The above steps S3 and S4 developed image preprocessing algorithms for two-dimensional grayscale images and three-dimensional elevation data, which can effectively solve the problems of missing image data, complex data features and inconsistent data distribution caused by factors such as vehicle bumps, mechanical vibrations and uneven laser distribution.

[0085] Step S5 above includes the following steps:

[0086] Step S510: Mark the defects in the high-quality two-dimensional and three-dimensional images containing pavement damage.

[0087] Step S520: Overlay the labeled two-dimensional and three-dimensional images to obtain a dual-channel training sample database;

[0088] Step S530: Train a deep learning network model using a dual-channel training sample database to obtain a road surface damage and defect identification model.

[0089] The aforementioned road surface damage includes types such as cracks, potholes, ruts, crack sealing, block repairs, and loosening.

[0090] Specifically, high-quality two-dimensional and three-dimensional images containing disease information are manually selected.

[0091] By training a deep learning network model using the aforementioned dual-channel training sample database, a pavement damage and distress identification model can be obtained, enabling rapid, accurate, and wide-ranging identification of various pavement damage and distresses such as cracks, potholes, ruts, crack sealing, block repairs, and loosening.

[0092] After training the deep learning network model using the dual-channel training sample database, a semantic segmentation network model is constructed using a fusion strategy of the Pyramid Scene Analysis Network (PSPNet) and the U-Net model. The fusion of the Conditional Random Field (CRF) module is also considered to obtain a road damage and disease identification model with high accuracy.

[0093] If the above methods are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] The above description of the embodiments is provided to enable those skilled in the art to understand and use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the invention should be within the protection scope of the present invention.

Claims

1. A method for processing multidimensional road surface damage data, characterized in that, The method includes the following steps: The acquired two-dimensional grayscale image is processed using a two-dimensional image preprocessing algorithm to obtain a processed two-dimensional image. The acquired 3D elevation images are processed using 3D image preprocessing algorithms to obtain 3D image enhancement output results; The processed two-dimensional and three-dimensional image enhancement outputs are used as inputs to the road damage and disease identification model to locate and automatically and quickly detect road damage areas. The two-dimensional image preprocessing algorithm includes: Step S310, calculating initial pre-processing coefficients , in, It is the initial overall mean. It is the set baseline grayscale value ; Step S320: Obtain a two-dimensional grayscale image I of the road surface. Based on a set threshold, determine whether the two-dimensional grayscale image of the road surface contains abnormal grayscale information. If so, remove the image; otherwise, add the image to the image column and obtain the grayscale mean of the image column. ; Step S330, obtaining a processed two-dimensional image of the two-dimensional gray-scale image I based on the preprocessing coefficient and the gray mean value , the processed two-dimensional image of the two-dimensional gray-scale image I , wherein C is the pre-processing coefficient, if I is the first image to be processed, C is the pre-processing coefficient, if I is the first image to be processed, , otherwise C is updated by the following equation: is a set update ratio, is the overall mean, with an initial value of , Step S340, return to step S320, until all image data processing is complete; The aforementioned 3D image preprocessing algorithm includes: Step S410: Establish the conversion relationship between pixels and spatial coordinates for the two-dimensional grayscale image and the three-dimensional elevation image through data calibration to obtain the road surface relative elevation data. Process the road surface relative elevation data using the pixel left-right adjacent interpolation method to obtain the filtered three-dimensional image elevation value point set. Step S420: Process the elevation point set of the original 3D image and the elevation point set of the filtered 3D image using a 3D image enhancement function to obtain the 3D image enhancement output result; The data calibration utilizes the principle of line structured light to establish the conversion relationship between pixels and spatial coordinates in two-dimensional grayscale images and three-dimensional elevation images, generating a calibration parameter file for obtaining road surface relative elevation data; The method for constructing the pavement damage and distress identification model includes: Step S510: Mark the defects in the high-quality two-dimensional and three-dimensional images containing pavement damage. Step S520: Overlay the labeled two-dimensional and three-dimensional images to obtain a dual-channel training sample database; Step S530: Train a deep learning network model using a dual-channel training sample database to obtain a road surface damage and defect identification model.

2. The multi-dimensional pavement distress data processing method of claim 1, wherein, The initial overall mean The method for obtaining includes: A two-dimensional grayscale image of the road surface is acquired. Images containing abnormal grayscale information are removed based on a set threshold, and N images are extracted for calculating preprocessing coefficients. The initial image is used to calculate the average grayscale value of N images as the initial overall average. .

3. The method of claim 1, wherein, The pixel left-right adjacent interpolation method is as follows: in, It is a pixel matrix of each row of cross sections. =0(w=1,2, , W ) is a set of outliers. 0(w=1,2, W) is a normal point set. It is the filtered set of elevation points. W This represents the number of elevation points on the cross-section.

4. The method of claim 1, wherein, The calculation formula for the three-dimensional image enhancement function is as follows: in, A A matrix representing the set of elevation points in the original 3D image. A matrix representing the set of elevation points in the filtered 3D image. MFF It is an adaptive mean filtering method. K Let be the filter coefficients, take... As a filter template, The minimum value of the set. represent The maximum value of the set. 3D image enhancement output results .

5. The method of claim 1, wherein, The road damage and defect identification model is superimposed with a conditional random field module to improve the accuracy of the road damage and defect identification model.

6. The method of claim 1, wherein, The road surface damage includes cracks, potholes, ruts, crack sealing, block repairs, and loosening.

Images