Pavement flatness measuring method and system

Abstract

The invention discloses a pavement flatness measuring method and system. The measuring method comprises the following steps: acquiring image data; acquiring a measuring point set according to the image data; attribute data of the measuring point set are extracted, and the attribute data comprise measuring point coordinate positions, acquisition speed and acquisition time; averaging the attribute data, converting the attribute data into frequency domain data, and calculating the pavement flatness of each measuring point; fallen leaves, gravels, weeds and other sundries on the road surface are recognized and filtered through an artificial intelligence recognition algorithm, a recognition result is fused into an original image, laser line point location prediction is carried out, interference of the sundries on laser line waveforms is filtered, and the average flatness of the road surface is calculated. And interference of the environmental factors on flatness calculation is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent image recognition, and particularly relates to a method and system for measuring road surface flatness. Background Art

[0002] Road surface flatness is one of the important indicators for measuring road quality, and it directly affects the smoothness and safety of vehicle driving. Specifically, the quality of road surface flatness can have the following impacts:

[0003] (1) Driving smoothness: Good road surface flatness can reduce the bumps of vehicles during driving, improve the comfort of passengers, and extend the service life of vehicles.

[0004] (2) Safety: Uneven roads are likely to cause vehicle out of control and increase the risk of traffic accidents.

[0005] (3) Navigation route planning: Modern navigation systems not only provide route planning but also consider the quality of roads. Especially in the transportation route planning of goods, road surface flatness is an important reference indicator. Smooth roads can improve the recommendation accuracy of navigation systems, optimize the travel experience, and reduce transportation losses caused by road surface bumps.

[0006] (4) Maintenance methods and timing: By regularly detecting road surface flatness, potential problems of roads can be discovered in time, providing data support for formulating reasonable maintenance plans, thereby extending the service life of roads and reducing maintenance costs.

[0007] The measurement methods of road surface flatness have undergone many changes with the development of technology, from the initial simple visual inspection to the modern high-precision instrument detection, and the technical means have been continuously improved. The following are some main measurement methods and their changes:

[0008] (1) Visual inspection method: In the early stage, road surface flatness mainly relied on manual visual inspection and experience judgment. This method is highly subjective and has low accuracy, making it difficult to meet the needs of modern road management.

[0009] (2) 3-meter straightedge method: This is a relatively traditional measurement method. By placing a 3-meter long straightedge on the road surface and measuring the maximum gap between the straightedge and the road surface to evaluate flatness. Although the operation is simple, the efficiency is low, and the measurement process is easily affected by the subjectivity of the measurer, resulting in affected accuracy.

[0010] (3) Continuous flatness meter: With the development of technology, the continuous flatness meter came into being. This instrument can continuously measure the flatness of the road during vehicle driving and has relatively high accuracy. Common continuous flatness meters include sectional detection devices and response detection devices.

[0011] Cross-section type detection device: Install a cross-section laser sensor and a rotary encoder on a motor vehicle. When the vehicle is moving, the rotary encoder triggers the laser sensor at equal distances to collect the elevation from the sensor to the road surface, which can cover a complete lane, and the evenness of the road surface can be deduced based on the change of the elevation value. This method features high speed and is currently the main detection method for measuring the evenness of high-grade highways in China.

[0012] Response type detection device: Install an acceleration sensor or a bump integrator on a motor vehicle to record the bump state of the vehicle when it is moving, and then calculate the evenness of the road surface inversely. This method has low cost and fast operation speed, and is currently widely used in the measurement of the evenness of low-grade highways in China.

[0013] (4) Laser point cloud scanning system: Modern vehicle-mounted laser scanning systems install high-precision laser point cloud sensors on motor vehicles. Using high-precision laser sensors and the Global Positioning System (GPS), they can collect three-dimensional point cloud data of the road surface in real time during the high-speed driving of motor vehicles, generate a high-precision road surface model, and can be used to calculate the longitudinal evenness condition of the road surface.

[0014] Existing automatic road evenness detection equipment mainly includes: continuous evenness meters (cross-section type detection devices, response type detection devices) and laser point cloud scanning systems. The existing equipment and technical solutions have the following disadvantages and deficiencies.

[0015] 1. Cross-section type detection device

[0016] (1) Dependence on laser sensors for elevation measurement: Cross-section type detection devices rely heavily on laser sensors to measure the road surface elevation. When there are sundries such as fallen leaves, gravel, and weeds on the road surface, these sundries will interfere with the normal operation of the laser sensor, resulting in inaccurate elevation measurement and ineffective correction through software algorithms. This makes this method have great limitations in the detection of medium and low-grade roads.

[0017] (2) Influence of wet road surface on measurement: When the road surface is wet, the laser beam may refract, resulting in inaccurate data collection. In this case, it is necessary to wait for the road surface to dry before re-measuring, which increases the detection time and cost.

[0018] (3) High vehicle modification cost: Installing a cross-section type detection device requires vehicle modification. After modification, the installed sensors include laser sensors, rotary encoders and other equipment. This not only increases the equipment cost, but may also affect the normal driving performance of the vehicle, increasing the complexity of maintenance and management, and is not conducive to detection in remote areas.

[0019] 2. Response type detection device

[0020] (1) Measurement of vehicle motion influence: The reaction-type detection device records the bump state of the vehicle during driving through an acceleration sensor or a bump integrator. When the vehicle accelerates, decelerates, or runs over gravel, these motion states will interfere with the detection results, resulting in inaccurate measurement data.

[0021] (2) Limited measurement range: Although the reaction-type detection device can use intelligent algorithms to eliminate the interference caused by vehicle bumps, its measurement range is limited. It can only measure the road surface area passed by the wheels, and the areas not passed by the wheels cannot be sensed. This makes the method have a certain degree of randomness when evaluating the flatness of the entire road surface and cannot provide comprehensive data support.

[0022] 3. Laser point cloud scanning system

[0023] (1) Large amount of data and slow data processing speed: The laser point cloud scanning system collects three-dimensional point cloud data of the road surface in real time during high-speed driving to generate a high-precision road surface model. However, the amount of point cloud data collected is extremely large, and powerful computing resources and efficient algorithms are required to process and analyze this data. The data processing speed is relatively slow, which may affect the detection efficiency, especially in large-scale road detection.

[0024] (2) High equipment cost: The laser point cloud scanning system usually includes high-precision laser sensors, global positioning systems (GPS) and other high-tech equipment, with a relatively high cost. This makes it difficult for the system to be widely popularized on medium and low-grade roads and is mainly used for the detection of high-grade highways and important sections. In addition, high-precision equipment needs to be regularly maintained and calibrated to ensure its long-term stable operation. The maintenance cost is relatively high, increasing the overall usage cost.

[0025] Existing automated road flatness detection methods have certain problems in terms of accuracy, adaptability, and cost. In order to improve the detection efficiency and accuracy while reducing the detection cost, there is an urgent need for a road flatness measurement method based on lightweight sensors and artificial intelligence recognition algorithms. Summary of the Invention

[0026] To solve the above technical problems, the present invention proposes a road flatness measurement method and system, which can improve the detection efficiency and accuracy while reducing the detection cost.

[0027] The present invention provides a road flatness measurement method, including:

[0028] Collect image data;

[0029] Obtain a measurement point set according to the image data;

[0030] Extract the attribute data of the measurement point set, where the attribute data includes the coordinate position of the measurement point, the acquisition speed, and the acquisition time;

[0031] Equalize the attribute data and convert it into frequency-domain data, and calculate the pavement evenness of each measuring point;

[0032] Summarize the pavement evenness of each measuring point and calculate the average evenness.

[0033] Optionally, the image data needs to be calibrated before being collected.

[0034] Optionally, obtaining the measuring point set according to the image data includes:

[0035] Input the image data into a neural network model for object recognition to obtain a mask image;

[0036] Fuse the mask image with the image data to obtain a fused image;

[0037] Input the fused image into a convolutional neural network model to obtain the laser line positions and the inference confidence of the laser line in the image;

[0038] Based on the laser line positions and the inference confidence of the laser line, obtain the measuring point set.

[0039] Optionally, the convolutional neural network model is trained by a training set, and the training set is the positions of the laser lines in the labeled images.

[0040] Optionally, before extracting the attribute data of the measuring point set, it also includes: determining whether the current cumulative distance meets the threshold distance;

[0041] If the threshold distance is met, extract the coordinate position data and speed data of the measuring point set;

[0042] Otherwise, re-obtain the measuring point set.

[0043] Optionally, equalizing the attribute data and converting it into frequency-domain data includes:

[0044] Equalize the coordinate position data and perform calibration mapping to obtain the processed coordinate position data;

[0045] Perform Fourier transform on the processed coordinate position data to obtain the power spectral density array in the corresponding frequency domain;

[0046] Filter and correct the power spectral density array in the corresponding frequency domain to obtain the corrected power spectral density array;

[0047] Calculate the root mean square value of the corrected power spectral density array to obtain the power spectral density array;

[0048] Based on the power spectral density array, obtain the frequency domain data.

[0049] Optionally, calculating the pavement evenness of each measurement point includes:

[0050] Input the power spectral density array and speed data into a multi-layer neural network model to obtain the pavement evenness data corresponding to the measurement point;

[0051] Calculate the average evenness of the pavement evenness data corresponding to all measurement points to obtain the pavement evenness.

[0052] The present invention also provides a pavement evenness measurement system, including: a data acquisition module, a data processing module, and a data storage module;

[0053] The data acquisition module is used to acquire image data and vehicle-related data;

[0054] The data processing module is used to process the image data and vehicle-related data to obtain the pavement evenness;

[0055] The data storage module is used to store data using a cloud server.

[0056] Compared with the prior art, the present invention has the following advantages and technical effects:

[0057] (1) Reduce interference from road surface debris: The present invention uses an artificial intelligence recognition algorithm to identify and filter out debris such as fallen leaves, gravel, and weeds on the road surface, and fuse the recognition results into the original image to predict the laser line position, filtering the interference of debris on the laser line waveform, thereby reducing the interference of these environmental factors on the evenness calculation.

[0058] (2) Reduce the influence of road surface wetness: Compared with traditional laser ranging recognition, the method and system mentioned in the present invention use an intelligent recognition model to extract the laser line positions in the image. Even if the road surface is a bit wet and some positions are missing, the neural network model can perform regression prediction on the missing positions, thereby reducing the influence of road surface wetness on subsequent calculations and improving the accuracy and reliability of measurement.

[0059] (3) Achieve lane coverage and improve the comprehensiveness and accuracy of data: Through multiple laser lines and high-resolution cameras, ensure that the acquisition covers the entire lane, reduce the influence of the vehicle driving path on the detection results, and improve the comprehensiveness and accuracy of measurement.

[0060] (4) Reduce equipment installation costs: Use lightweight sensors and general hardware devices to reduce the overall cost of the equipment. Coupled with intelligent algorithms, real-time analysis and processing of data can be achieved, avoiding a large amount of redundant data acquisition, making the system widely popular on medium and low-grade roads.

[0061] (5) Improve equipment versatility: The equipment is easy to install, has low requirements for vehicle modification, is applicable to various types of inspection vehicles, and improves the versatility and application scope of the equipment.

[0062] (6) Reduce equipment operation and maintenance costs: The equipment has a simple structure, low maintenance costs, and is easy to operate stably for a long time.

[0063] (7) Real-time data transmission: The methods and systems mentioned in the present invention, while using artificial intelligence recognition models to achieve intelligent data analysis, utilize 4G or 5G mobile communication technologies to achieve real-time data transmission, improving the timeliness and availability of data.

[0064] (8) Combined analysis of images and flatness: Compared with the traditional method of simply collecting road surface flatness, the methods and systems mentioned in the present invention combine image analysis and flatness analysis. The collected data can be used for further analysis and excavation, such as analyzing the disease types at bumpy road surfaces and measuring the height difference of manhole covers, providing data support for refined road surface management and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments and descriptions thereof of this application are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0066] Figure 1 is a schematic diagram of the equipment composition and data flow relationship of the embodiment of the present invention;

[0067] Figure 2 is a schematic diagram of the equipment and installation of the acquisition end of the embodiment of the present invention;

[0068] Figure 3 is a schematic diagram of image calibration of the embodiment of the present invention;

[0069] Figure 4 is a flowchart of a method for measuring road surface flatness of the embodiment of the present invention;

[0070] Figure 5 is a schematic diagram of image fusion of the embodiment of the present invention;

[0071] Figure 6 is a schematic diagram of the measuring point set of the laser line on the image of the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0072] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the drawings and combine the embodiments to detail this application.

[0073] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0074] The present invention provides a method for measuring pavement evenness, as Figure 4 shown, specifically including the following steps:

[0075] Collect the cumulative distance traveled by the vehicle currently;

[0076] Based on the cumulative distance traveled currently, collect image data;

[0077] According to the image data, obtain the set of measuring point data;

[0078] Extract the coordinate position data and speed data of the set of measuring point data;

[0079] Perform Fourier transform on the coordinate position data to obtain the power spectral density array;

[0080] According to the power spectral density array and the speed data, obtain the pavement evenness.

[0081] Furthermore, before collecting the cumulative distance traveled by the vehicle currently and collecting the image data, it is necessary to calibrate the cumulative distance traveled and the image data.

[0082] Specifically, 1. The present invention consists of a collection end and a cloud end. Among them, the devices at the collection end are installed on the inspection vehicle, and the installed devices include: a high-definition high-speed industrial camera, a rotary encoder, a combined navigation and positioning receiver, and an industrial control computer; the cloud devices include: a cloud server.

[0083] The composition of the devices and the data flow relationship, as Figure 1 shown. The devices at the collection end, during the vehicle driving process, collect data such as images, speeds, longitude and latitude, etc., and are identified and processed by the intelligent algorithm in the industrial control computer, and then the industrial control computer uploads the processed data to the cloud server for warehousing and storage.

[0084] 2. The hardware devices and performance of the collection end: The schematic diagram of the devices and installation at the collection end, as Figure 2 shown, generally includes 1 industrial control computer, 1 or more high-definition industrial cameras, 1 rotary encoder, 1 or more laser emitters, and 1 combined navigation and positioning receiver. Among them:

[0085] For the high-definition industrial camera in 2.1, the image acquisition frame rate ≥ 100fps, and the image resolution ≥ 1 million pixels,

[0086] Area array camera, with clear, distortion-free and delay-free images. The camera is installed behind the inspection vehicle and shoots towards the road surface behind the vehicle. The laser line on the ground can be clearly seen in the collected images, and the laser line is centered vertically in the image, and the length of the captured laser line ≥ 3.5m. The exposure time of the camera can be adjusted, and the maximum exposure time ≤ 10,000 microseconds. Place a calibration plate with a thickness ≥ 1cm on the ground. When the laser line is projected onto the ground and hits the calibration plate, the height difference between the calibration plate and the road surface can be clearly seen in the collected images. The industrial camera is powered by an industrial computer, triggered by the industrial computer to take pictures, and the collected images are transmitted back to the industrial computer, with a dust and waterproof rating ≥ IP65.

[0087] The 2.2 laser emitter has a power consumption ≥ 500mW. The emitting head is equipped with a Powell prism and can emit a uniform "one-line" laser in blue or green. The laser emitter is installed in the center at the bottom of the vehicle rear and emits a laser line towards the rear of the vehicle. The laser line projected onto the ground is more than 1m away from the vehicle body, the length of the laser line ≥ 3.5m, and it is perpendicular to the vehicle's forward direction. The laser emitter is powered by an industrial computer or vehicle power supply, with a dust and waterproof rating ≥ IP65.

[0088] The 2.3 rotary encoder is installed on the vehicle's rear wheel and sends signals as it rotates with the wheel. The resolution of the encoder ≥ 2048p / r, that is, it can send at least 2048 signals per revolution. The rotary encoder is powered by an industrial computer, and the collected signals are also sent to the industrial computer, with a dust and waterproof rating ≥ IP65.

[0089] The 2.4 combined navigation and positioning receiver has a positioning error ≤ 1m and is equipped with an inertial navigation system, which can collect the vehicle's real-time speed, and the acquisition frequency of the positioning data ≥ 20Hz.

[0090] The 2.5 industrial computer is installed inside the vehicle and is powered by the vehicle. It has the fourth-generation (4G) or fifth-generation (5G) mobile communication technology and is connected to a high-precision navigation and positioning receiver, a high-definition industrial camera, and a rotary encoder. It can analyze the signal data sent by the rotary encoder, send instructions to the high-definition camera to collect images, and continuously receive, process, and upload the collected data. In addition, the central industrial computer contains a micro graphic processing unit (GPU), and the deep neural network algorithm can use the GPU to accelerate image processing.

[0091] 3. Image and encoder calibration: After the devices at the acquisition end are installed, the devices need to be calibrated.

[0092] Image calibration refers to calibrating the collected images to establish a corresponding relationship between the pixel height in the collected images and the physical height in the real world. The schematic diagram of image calibration is as Figure 3 shown, and the specific calibration process is as follows:

[0093] 3.1 Park the vehicle on a horizontal road surface.

[0094] 3.2 Let the laser line project onto the ground and be horizontal and centered vertically in the captured image.

[0095] 3.3 Place a calibration block with a certain height at the laser line, record the true height Δh of the calibration block, and the calibrated height Δu in the image, where Δu refers to the pixel distance from the position where the laser line projects onto the calibration block to the horizontal laser line. With the acquisition end device unchanged, Δu will increase as Δh increases.

[0096] Here, set Δh = F(Δu), where F is a univariate N-th degree polynomial (N≥2). During calibration, perform multiple calibration records. Each time a calibration record is made, replace the calibration block, record the true height Δh of each calibration block and the corresponding image height Δu, and use the least squares method to solve for the corresponding coefficients of the polynomial F.

[0097] During calibration, the minimum height of the calibration block is 1 cm, the maximum height of the calibration block ≥ 40 cm, and at least 5 + N calibration blocks with different heights are replaced for calibration.

[0098] If there are two or more laser lines in the captured image, then set that each laser line has an independent Δh = F’(Δu), where F’ is a univariate N-th degree polynomial (N≥2). During calibration, record the height of each laser line separately and calibrate and calculate F’ separately.

[0099] Encoder calibration refers to establishing the relationship between the rotation of the rotary encoder by n turns and the corresponding forward distance of the vehicle. The goal is to accurately measure the forward distance of the vehicle.

[0100] Further, according to the image data, the acquired measurement point data set includes:

[0101] Input the image data into a neural network model for object recognition to obtain a mask image;

[0102] Fuse the mask image with the image data to obtain a fused image;

[0103] Input the fused image into a convolutional neural network model to obtain the laser line positions and the inference confidence of the laser lines in the image;

[0104] Based on the laser line positions and the inference confidence of the laser lines, obtain the measurement point data set.

[0105] Specifically, the confidence is mainly used to determine whether the current laser line is credible. In the implementation scheme, if the confidence of multiple consecutive laser lines inferred is lower than the corresponding value, the acquisition operator will be reminded on the acquisition system that there is a problem with the currently acquired data and it needs to be checked.

[0106] Further, the measurement point data set includes: laser line point position data, the corresponding vehicle speed, and the acquisition time.

[0107] Further, the convolutional neural network model is trained by a training set, and the training set is the position of the laser line in the labeled image.

[0108] Further, before extracting the coordinate position data and speed data of the measurement point data set, it also includes: determining whether the current cumulative distance meets the threshold distance;

[0109] If the threshold distance is met, extract the coordinate position data and speed data of the measurement point data set;

[0110] Otherwise, re-acquire the measurement point data set.

[0111] Further, performing a Fourier transform on the coordinate position data to obtain a power spectral density array includes:

[0112] Performing mean value processing on the coordinate position data and performing calibration mapping to obtain the processed coordinate position data;

[0113] Performing a Fourier transform on the processed coordinate position data to obtain a power spectral density array in the corresponding frequency domain;

[0114] Performing filtering and correction on the power spectral density array in the corresponding frequency domain to obtain a corrected power spectral density array;

[0115] Calculating the root mean square value of the corrected power spectral density array to obtain the power spectral density array.

[0116] Further, according to the power spectral density array and the speed data, obtaining the road surface flatness includes:

[0117] Inputting the power spectral density array and the speed data into a multi-layer neural network model to obtain the road surface flatness data corresponding to the measurement point;

[0118] Calculating the average flatness of the road surface flatness data corresponding to all measurement points to obtain the road surface flatness.

[0119] Specifically, after the device is calibrated, data acquisition and processing analysis can be performed, and the acquisition and analysis process is as Figure 4 shown. The specific acquisition and analysis process is as follows:

[0120] 4.1 The detection vehicle starts, the rotary encoder starts accumulating the distance the vehicle travels forward from 0, clears the buffer queue M, and records the current distance the vehicle travels forward as L.

[0121] 4.2 When the vehicle moves forward k meters, the system sends an instruction to the industrial control computer to collect an original image, which is recorded as mi. The value of k is determined according to the camera acquisition frame rate, the computing power performance of the industrial control computer, and the detected vehicle speed. Generally, k ≤ 0.2m. mi is the collected original image, and the size of the image is [hi, wi, ci], where hi is the height of the image, wi is the width of the image, and ci is the number of channels of the image. If it is an RGB camera, ci = 3; if it is a black-and-white camera, ci = 1.

[0122] 4.3 Input the collected image mi into the target detection neural network T1. T1 will identify target objects such as fallen leaves, spilled objects, manhole covers, curbstones, and markings in the image, record the positions of the target objects in the image, and generate a mask image mi' therefrom. The image size of mi' is [hi, wi, 1]. In the area without target objects, the pixel value is 0, and in the area with target objects, the pixel value is a value related to the target object. For example, if the target object recognized in this area is a fallen leaf, the pixel value is 200; if the target object in this area is a curbstone, the pixel value is 10, and so on.

[0123] 4.4 Stitch the original image mi and the mask image mi' along the image channel direction to obtain a fused image mi*. The image size of the fused image mi* is [hi, wi, ci + 1]. The fusion schematic is as Figure 5 shown.

[0124] 4.5 Input the fused image mi* into the convolutional neural network model T2 to identify the position points of the laser line in the image and give the inference confidence of the laser line.

[0125] The convolutional neural network model T2 is trained in a supervised manner. When the model is trained, the input data is the combined image after fusing the original image and the mask image, and the output label is the position of the laser line in the manually labeled image. When manually labeling the laser line, it is generally labeled according to the trend of the laser line. When the laser line fluctuates due to being projected onto weeds, spilled objects, fallen leaves, or gravel, the labeling is adjusted so that the laser line does not fluctuate due to these obstacles. The convolutional neural network model T2 trained in this way can infer the position of the laser line in the image and can filter the interference of road debris on the laser line waveform.

[0126] 4.6 Based on the recognition results of the neural network model T2, measurement points are equally spaced and extracted in the horizontal direction of the image to form a measurement point set P. The measurement point set P is as Figure 6 shown. The distance between the leftmost and rightmost measurement points ≥ 3.5m, and the horizontal distance between adjacent measurement points ≤ 0.3m.

[0127] 4.7 Store the position of the measurement point set P on the mi* image, as well as the corresponding acquisition speed v, acquisition time t, and other data in the buffer queue M.

[0128] 4.8 When the current forward distance L of the vehicle < the threshold distance L', repeat steps 4.2 - 4.7 to continuously collect images, analyze, and store them in the buffer queue M; when L ≥ L', extract all the information from the buffer queue M, that is, the vertical coordinate positions of the measurement point set P on the continuous sequence of images and the acquired speed information. Taking the j-th measurement point as an example, obtain the vertical coordinate position array [Vj1, Vj2, …, Vjn] of the measurement point j in the image mi*, and the corresponding speed array [v1, v2, …, vn]. The threshold distance L' is a fixed threshold, generally L' ≤ 10m.

[0129] 4.9 Further process the data extracted for each measurement point, and then calculate the corresponding IRI.

[0130] Taking the j-th measurement point as an example:

[0131] (1) Calculate the mean Vj_mean of [Vj1, Vj2, …, Vjn], and perform [Vj1, Vj2, …, Vjn] – Vj_mean to obtain the mean-centered array [Vj1’, Vj2’, …, Vjn’], and then perform F([Vj1’, Vj2’, …, Vjn’]) to calculate the calibrated and corrected array [Vj1*, Vj2*, …, Vjn*].

[0132] (2) Perform Fourier transform on the array [Vj1*, Vj2*, …, Vjn*] to calculate the power spectral density array PSDj in the corresponding frequency domain.

[0133] (3) Perform filtering correction on the power spectral density array PSDj for the corresponding frequencies to obtain a new power spectral density array PSDj*. The correction method is to multiply the spectral density values in a specific frequency range by a coefficient less than 1 for filtering. This coefficient is generally obtained based on a large number of experiments.

[0134] (4) Calculate the root mean square value PSD_RMSj of the power spectral density array PSDj.

[0135] (5) Input the value PSD_RMSj and the velocity array [v1, v2, …, vn] into a multi-layer neural network model T3, and use the T3 model to regress and infer the road surface roughness IRIj corresponding to the measuring point j in this data segment. The T3 model is a multi-layer neural network model. When the T3 model is trained, the model input is the array [PSD_RMSj, v1, v2, …, vn], which is the data collected and calculated by the equipment. The output of the model is the International Roughness Index IRI, which is the standard value of the road surface roughness measured by a level meter based on the actual location of the measuring point.

[0136] 4.10 Summarize the road surface roughness data of all measuring points, calculate an average roughness, and record the start time, end time, start longitude and latitude, end longitude and latitude of this data, summarize the relevant data, upload it to the cloud server, and then return to 4.1 for cyclic collection.

[0137] The present invention also provides a road surface flatness measurement system, comprising: a data acquisition module, a data processing module and a data storage module;

[0138] A data acquisition module, used to collect image data and vehicle-related data;

[0139] A data processing module is used to process image data and vehicle-related data to obtain road surface smoothness;

[0140] The data storage module is used to store data using a cloud server.

[0141] The present embodiment is described in detail below with reference to the accompanying drawings:

[0142] 1. Equipment Installation

[0143] Find an SUV vehicle that can normally power the outside world and install the equipment required for the system, including:

[0144] (1) Industrial computer installation

[0145] Install the industrial computer inside the inspection vehicle, ensure good ventilation and stay away from high temperature areas. The industrial computer is powered by the vehicle power supply to ensure stable power supply. The industrial computer is connected to the high-definition industrial camera, rotary encoder, integrated navigation positioning receiver and laser transmitter through data cables.

[0146] (2) Laser transmitter installation

[0147] The laser transmitter is installed at the bottom of the rear of the vehicle, facing the rear of the vehicle, and ensure that the laser line is projected to the ground and is perpendicular to the direction of the vehicle. Adjust the power and angle of the laser transmitter to ensure that the laser line length is ≥3.5m and is clearly visible.

[0148] (3) Installation of High-Definition Industrial Camera

[0149] The camera is installed at the rear of the inspection vehicle, facing the road surface at the rear of the vehicle, ensuring that the captured images are clear and distortion-free. Adjust the exposure time and focal length of the camera to ensure that the laser line is clearly visible and centered in the image, and the length of the laser line in the image is ≥ 3.5 m.

[0150] (4) Installation of Rotary Encoder

[0151] The rotary encoder is installed on the rear wheel of the vehicle to ensure that it rotates with the wheel.

[0152] (5) Installation of Integrated Navigation and Positioning Receiver

[0153] The receiver is installed on the top of the vehicle to ensure good signal reception.

[0154] 2. Equipment Calibration

[0155] (1) Image Calibration

[0156] Image calibration includes the following steps:

[0157] 1) Park the vehicle on a horizontal road surface.

[0158] Let the laser line project onto the ground, and the laser line is horizontal and centered vertically in the captured image, and the length of the laser line in the image exceeds 3.5 m.

[0159] Place a calibration block with a certain height at the laser line, record the true height Δh of the calibration block and the calibrated height Δu in the image. At least replace 5 + N calibration blocks with different heights for calibration. The minimum height of the calibration block is 1 cm, and the maximum height of the calibration block is ≥ 40 cm.

[0160] Input the data recorded during the calibration process into the acquisition software, and the background program of the acquisition software uses the least squares method to solve the corresponding coefficients of the polynomial F.

[0161] (2) Encoder Calibration

[0162] Encoder calibration includes the following steps:

[0163] 1) Park the vehicle on a horizontal road surface.

[0164] 2) Let the vehicle move forward a known distance of n meters and record the number of signals sent by the rotary encoder.

[0165] 3) Input the above data into the acquisition software, and the background program of the acquisition software establishes the corresponding relationship between the rotation of the rotary encoder by n circles and the forward distance of the vehicle by calculating the relationship between n meters and the number of signals.

[0166] In this embodiment, using a rotary encoder to record the forward distance of a vehicle is a commonly used method in the industry:

[0167] Generally, park the vehicle on a horizontal road surface, record the wheel where the encoder is installed, when the wheel rotates one circle, the forward distance L of the vehicle. When the wheel rotates one circle, the rotary encoder generally triggers information of a complete cycle. Assuming the number of signals is m, during the test, it is recorded that the rotary encoder has triggered a total of n signals, then the converted forward distance of the vehicle is n / m * L.

[0168] 3. Data acquisition and analysis

[0169] (1) Data acquisition

[0170] 1) The inspector clicks to start the "detection task" on the software, the vehicle starts, the rotary encoder accumulates the forward distance of the vehicle from 0, clears the system cache queue M, and records the current forward distance L of the vehicle.

[0171] 2) When the vehicle advances 0.1 meters each time, the system sends an instruction to the industrial control computer to collect an original image mi, and the size of mi is [hi, wi, ci].

[0172] (2) Data processing

[0173] 1) Input the collected image mi into the target detection neural network model T1 to identify target objects such as fallen leaves, litter, manhole covers, curbs, markings, etc. in the image, and generate a mask image mi' according to the position of the identified target objects in the image. The size of the mask image is [hi, wi, 1].

[0174] In this embodiment, the neural network T1 is a target detection neural network, including but not limited to yolov8. After training, it is specifically used to identify target objects such as fallen leaves, litter, manhole covers, curbs, markings, etc. in the image.

[0175] 2) Stitch the original image mi and the mask image mi' along the image channel direction to obtain a fused image mi*, and the size of the fused image is [hi, wi, ci + 1].

[0176] 3) Input the fused image mi* into the convolutional neural network model T2 to identify the positions of the laser lines in the image and give the inference confidence of the laser lines.

[0177] In this embodiment, the neural network T2 is an instance segmentation neural network, similar to lane line recognition. The network structure includes but not limited to lanenet, yolov8-seg. It is specifically used to identify the laser line area in the image, and then use morphological erosion operation on this area to continuously remove boundary pixels until no more removal can be continued, so as to retain a skeleton line as the identified laser line.

[0178] 4) Based on the recognition result of the neural network model T2, at the position of the laser line in the image, measurement points are extracted at equal intervals in the horizontal direction of the image to form a measurement point set P.

[0179] 5) The positions of the measurement point set P on the mi* image, as well as the corresponding acquisition speed v, acquisition time t and other data at this time are stored in the buffer queue M.

[0180] 6) When the currently advancing distance L of the vehicle is ≥ 10m, all buffered data is extracted from the buffer queue M. The buffered data corresponding to each measurement point is averaged and converted into frequency domain data, filtered in a specific frequency domain, and then input into the neural network model T3. The neural network model T3 infers and calculates the IRIj of each measurement point.

[0181] In this embodiment, the neural network T3 is a neural network that inputs sequence data and performs numerical regression, that is, it inputs the data buffered in the M queue and outputs a regressed IRI value. The network structure includes but is not limited to Transformer.

[0182] 7) The road surface flatness data of all measurement points is summarized, an average flatness is calculated, and information such as the start time, end time, start longitude and latitude, end longitude and latitude of this section of data is recorded in the local database and uploaded to the cloud. The system clears the buffer queue M, clears the currently accumulated distance L, and the system continues to collect data.

[0183] (3) End of acquisition

[0184] After the acquisition ends, the inspector clicks "End of acquisition", the system stops collecting, and continues to process the already collected data and upload the unuploaded completed data.

[0185] 4. Data upload and management

[0186] (1) Data upload

[0187] The industrial control computer uploads the processed data and the original collected data to the cloud server through 4G or 5G mobile communication technology.

[0188] (2) Data reception and management

[0189] The cloud server receives the uploaded data, performs warehousing and storage. Through the data management platform, the uploaded data is analyzed to generate a road surface flatness report.

[0190] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for measuring road surface flatness, characterized in that, Including: Collecting image data; Obtaining a set of measurement points according to the image data; Extracting the attribute data of the set of measurement points, where the attribute data includes the coordinate position of the measurement point, the acquisition speed, and the acquisition time; Averaging the attribute data and converting it into frequency-domain data, and calculating the pavement evenness of each measurement point; Summarizing the pavement evenness of each measurement point and calculating the average evenness.

2. The pavement evenness measurement method according to claim 1, wherein, Before collecting the image data, it is necessary to calibrate the image data.

3. A method for measuring the pavement evenness according to claim 1, characterized in that, Obtaining a set of measurement points according to the image data includes: Inputting the image data into a neural network model for object recognition to obtain a mask image; Fusing the mask image with the image data to obtain a fused image; Inputting the fused image into a convolutional neural network model to obtain the laser line positions and the inference confidence of the laser line in the image; Obtaining the set of measurement points based on the laser line positions and the inference confidence of the laser line.

4. A method for measuring road surface flatness according to claim 3, characterized in that, The convolutional neural network model is trained by a training set, and the training set is the position of the laser line in the labeled image.

5. A method for measuring the pavement evenness according to claim 1, characterized in that, Before extracting the attribute data of the set of measurement points, it also includes: judging whether the current cumulative distance meets the threshold distance; If the threshold distance is met, extracting the coordinate position data and speed data of the set of measurement points; Otherwise, re-obtaining the set of measurement points.

6. The method for measuring road surface flatness according to claim 5, characterized in that, Averaging the attribute data and converting it into frequency-domain data includes: Performing averaging processing on the coordinate position data and performing calibration mapping to obtain the processed coordinate position data; Performing Fourier transform on the processed coordinate position data to obtain a power spectral density array in the corresponding frequency domain; Performing filtering correction on the power spectral density array in the corresponding frequency domain to obtain a corrected power spectral density array; Calculating the root mean square value of the corrected power spectral density array to obtain a power spectral density array; Obtaining the frequency-domain data based on the power spectral density array.

7. A method for measuring pavement evenness according to claim 6, characterized in that, Calculating the pavement evenness of each measurement point includes: Inputting the power spectral density array and the speed data into a multi-layer neural network model to obtain the pavement evenness data corresponding to the measurement points; Calculating the average evenness of the pavement evenness data corresponding to all measurement points to obtain the pavement evenness.

8. A road surface flatness measurement system, characterized in that, Including: A data acquisition module, a data processing module, and a data storage module; The data acquisition module is used to collect image data and vehicle-related data; The data processing module is used to process the image data and vehicle-related data to obtain the pavement evenness; The data storage module is used to store data using a cloud server.

Images