A method and system for measuring road surface evenness

Abstract

The application discloses a kind of road surface flatness measurement method and system, measurement method includes: acquisition image data;According to the image data, obtain measurement point set;Extract the attribute data of measurement point set, wherein the attribute data includes measurement point coordinate position, acquisition speed and acquisition time;Meaning is carried out to the attribute data and is converted into frequency domain data, calculates the road surface flatness of each measurement point;The road surface flatness of each measurement point is summarized, and the average flatness is calculated.The application identifies and filters out fallen leaves, gravel, weeds and other sundries on the road surface through artificial intelligence recognition algorithm, and fuses the recognition result into the original image, predicts the laser line point position, filters the interference of sundries on the laser line waveform, and further reduces the interference of these environmental factors on the flatness calculation.

Description

Technical Field

[0001] This invention belongs to the field of intelligent image recognition technology, and in particular relates to a method and system for measuring road surface smoothness. Background Technology

[0002] Road smoothness is one of the important indicators for measuring road quality, directly affecting driving smoothness and safety. Specifically, the quality of road smoothness can have the following effects:

[0003] (1) Driving smoothness: Good road smoothness can reduce the bumps during vehicle driving, improve passenger comfort, and extend the service life of the vehicle.

[0004] (2) Safety: Uneven roads can easily cause vehicles to lose control, increasing the risk of traffic accidents.

[0005] (3) Navigation route planning: Modern navigation systems not only provide route planning but also consider road quality, especially in the planning of freight transportation routes, where road smoothness is an important reference indicator. Smooth roads can improve the accuracy of navigation system recommendations, optimize the travel experience, and reduce transportation losses caused by road bumps.

[0006] (4) Maintenance methods and timing: By regularly inspecting the road smoothness, potential problems of the road can be discovered in time, providing data support for the formulation of reasonable maintenance plans, thereby extending the service life of the road and reducing maintenance costs.

[0007] The methods for measuring road smoothness have undergone several changes with technological advancements, from simple visual inspection to modern high-precision instrument testing. The following are some of the main measurement methods and their evolution:

[0008] (1) Visual inspection: In the early days, road smoothness was mainly judged by manual visual inspection and experience. This method is highly subjective, has low accuracy, and is difficult to meet the needs of modern road management.

[0009] (2) 3-meter ruler method: This is a relatively traditional measurement method. The flatness is assessed by placing a 3-meter ruler on the road surface and measuring the maximum gap between the ruler and the road surface. Although it is simple to operate, it is inefficient and the measurement process is easily affected by the subjective influence of the surveyor, which affects the accuracy.

[0010] (3) Continuous Smoothness Meter: With the development of technology, continuous smoothness meters have emerged. This type of instrument can continuously measure the smoothness of the road while the vehicle is in motion, and has high accuracy. Common continuous smoothness meters include cross-sectional detection devices and reactive detection devices.

[0011] Cross-sectional inspection device: A cross-sectional laser sensor and rotary encoder are installed on the vehicle. As the vehicle moves, the rotary encoder sends out laser sensors at equal intervals to collect the elevation data from the sensors to the road surface. This can cover an entire lane, and the road surface smoothness is calculated based on the changes in elevation values. This method is characterized by its high speed and is currently the main inspection method for measuring the smoothness of high-grade highways in my country.

[0012] Reactive detection devices: Acceleration sensors or bump accelerators are installed on motor vehicles to record the vehicle's bumpiness during movement, and then the road surface smoothness is calculated. This method is low-cost and fast, and is currently widely used in the smoothness measurement of low-grade highways in my country.

[0013] (4) Laser point cloud scanning system: Modern vehicle-mounted laser scanning system is a high-precision laser point cloud sensor installed on a motor vehicle. Using the high-precision laser sensor and the Global Positioning System (GPS), it can collect three-dimensional point cloud data of the road surface in real time while the motor vehicle is traveling at high speed, generate a high-precision road surface model, and can be used to calculate the longitudinal smoothness of the road surface.

[0014] Existing automatic road smoothness detection equipment mainly consists of continuous smoothness meters (cross-section detection devices, reaction detection devices) and laser point cloud scanning systems. The existing equipment and technical solutions have the following shortcomings and deficiencies.

[0015] 1. Cross-section detection device

[0016] (1) Reliance on laser sensors for elevation measurement: Cross-section detection devices heavily rely on laser sensors to measure road surface elevation. When there are debris such as fallen leaves, gravel, and weeds on the road surface, these debris will interfere with the normal operation of the laser sensor, resulting in inaccurate elevation measurements that cannot be effectively corrected by software algorithms. This makes this method quite limited in the detection of low- and medium-grade roads.

[0017] (2) Impact of road surface moisture on measurement: When the road surface is wet, the laser beam may refract, resulting in inaccurate data. In this case, it is necessary to wait for the road surface to dry before re-measuring, which increases the detection time and cost.

[0018] (3) High vehicle modification costs: Installing cross-section detection devices requires vehicle modification, including the installation of sensors such as laser sensors and rotary encoders. This not only increases equipment costs but may also affect the normal driving performance of the vehicle, increase the complexity of maintenance and management, and is not conducive to detection in remote areas.

[0019] 2. Reaction-based detection devices

[0020] (1) Measurement of the impact of vehicle motion: Reactive detection devices record the bump state of a vehicle while it is moving by using an accelerometer or a bump accelerator. When the vehicle accelerates, decelerates, or runs over gravel, these motion states will interfere with the detection results, resulting in inaccurate measurement data.

[0021] (2) Limited measurement range: Although reactive detection devices can use intelligent algorithms to eliminate interference caused by vehicle bumps, their measurement range is limited. They can only measure the road surface area that has been rolled over by the wheels, while areas that have not been rolled over cannot be detected. This makes the method somewhat random when assessing the smoothness of the entire road surface and cannot provide comprehensive data support.

[0022] 3. Laser point cloud scanning system

[0023] (1) Large data volume and slow data processing speed: The laser point cloud scanning system collects three-dimensional point cloud data of the road surface in real time while driving at high speed to generate a high-precision road surface model. However, the amount of point cloud data collected is enormous, requiring powerful computing resources and efficient algorithms to process and analyze this data. The data processing speed is slow, which may affect the detection efficiency, especially in large-scale road detection.

[0024] (2) High equipment cost: Laser point cloud scanning systems typically include high-precision laser sensors, Global Positioning System (GPS), and other high-tech equipment, resulting in high costs. This makes it difficult to widely adopt this system on low- and medium-grade roads, and it is mainly used for the detection of high-grade highways and important road sections. In addition, high-precision equipment requires regular maintenance and calibration to ensure its long-term stable operation. The high maintenance costs increase the overall cost of use.

[0025] Existing automated road evenness detection methods have certain limitations in terms of accuracy, adaptability, and cost. To improve detection efficiency and accuracy while reducing costs, there is an urgent need for a road evenness measurement method based on lightweight sensors and artificial intelligence recognition algorithms. Summary of the Invention

[0026] To address the aforementioned technical problems, this invention proposes a method and system for measuring road surface smoothness, which can improve detection efficiency and accuracy while reducing detection costs.

[0027] This invention provides a method for measuring road surface smoothness, comprising:

[0028] Acquire image data;

[0029] Based on the image data, obtain the set of measurement points;

[0030] Extract the attribute data of the set of measuring points, wherein the attribute data includes the coordinate position of the measuring points, the acquisition speed, and the acquisition time;

[0031] The attribute data is averaged and converted into frequency domain data to calculate the road surface smoothness at each measuring point;

[0032] The road surface smoothness of each measuring point is summarized, and the average smoothness is calculated.

[0033] Optionally, the image data needs to be calibrated before it is acquired.

[0034] Optionally, obtaining the measurement point set based on the image data includes:

[0035] The image data is input into a neural network model for target recognition to obtain a mask image;

[0036] The mask image is fused with the image data to obtain a fused image;

[0037] The fused image is input into a convolutional neural network model to obtain the laser line locations and inference confidence of the laser lines in the image;

[0038] The set of measurement points is obtained based on the laser line locations and the inference confidence of the laser lines.

[0039] Optionally, the convolutional neural network model is trained using a training set, which consists of the positions of laser lines in the labeled image.

[0040] Optionally, before extracting the attribute data of the measurement point set, the method further includes: determining whether the current cumulative distance meets the threshold distance.

[0041] If the threshold distance is met, then the coordinate position data and velocity data of the measurement point set are extracted;

[0042] Otherwise, reacquire the set of measurement points.

[0043] Optionally, averaging and converting the attribute data into frequency domain data includes:

[0044] The coordinate position data is averaged and calibrated to obtain the processed coordinate position data.

[0045] The processed coordinate position data is subjected to Fourier transform to obtain the power spectral density array in the corresponding frequency domain;

[0046] The power spectral density array in the corresponding frequency domain is filtered and corrected to obtain the corrected power spectral density array;

[0047] Calculate the root mean square value of the corrected power spectral density array to obtain the power spectral density array;

[0048] The frequency domain data is obtained based on the power spectral density array.

[0049] Optionally, calculating the road surface smoothness at each measuring point includes:

[0050] The power spectral density array and velocity data are input into a multi-layer neural network model to obtain the road surface smoothness data corresponding to the measurement points;

[0051] Calculate the average smoothness of the road surface smoothness data corresponding to all measuring points to obtain the road surface smoothness.

[0052] The present invention also provides a road surface evenness measurement system, comprising: a data acquisition module, a data processing module, and a data storage module;

[0053] The data acquisition module is used to acquire image data and vehicle-related data;

[0054] The data processing module is used to process the image data and vehicle-related data to obtain road surface smoothness;

[0055] The data storage module is used to store data using a cloud server.

[0056] Compared with the prior art, the present invention has the following advantages and technical effects:

[0057] (1) Reduce interference from road surface debris: This invention uses artificial intelligence recognition algorithms to identify and filter out debris such as fallen leaves, gravel, and weeds on the road surface, and integrates the recognition results into the original image to predict the laser line position, filter out the interference of debris on the laser line waveform, and thus reduce the interference of these environmental factors on the smoothness calculation.

[0058] (2) Reduce the impact of road surface moisture: Compared with traditional laser ranging and recognition, the method and system mentioned in this invention use an intelligent recognition model to extract the laser line points in the image. Even if the road surface is somewhat wet and some points are missing, the neural network model can perform regression prediction on the missing points, thereby reducing the impact of road surface moisture on subsequent calculations and improving the accuracy and reliability of the measurement.

[0059] (3) Achieve lane coverage and improve the comprehensiveness and accuracy of data: By using multiple laser lines and high-resolution cameras, ensure that the data collection covers the entire lane, reduce the impact of vehicle travel paths on the detection results, and improve the comprehensiveness and accuracy of the measurement.

[0060] (4) Reduce equipment installation costs: Using lightweight sensors and general-purpose hardware devices reduces the overall cost of the equipment. Combined with intelligent algorithms, it enables real-time data analysis and processing, avoids a large amount of redundant data collection, and makes the system widely available on low- and medium-grade roads.

[0061] (5) Improve equipment versatility: The equipment is easy to install, requires little modification to vehicles, and is suitable for various types of inspection vehicles, thus improving the equipment's versatility and applicability.

[0062] (6) Reduce equipment operation and maintenance costs: The equipment has a simple structure, low maintenance costs, and is easy to operate stably for a long time.

[0063] (7) Real-time data transmission: The method and system mentioned in this invention, while using artificial intelligence recognition models to realize intelligent data analysis, utilize 4G or 5G mobile communication technology to realize real-time data transmission, thereby improving the timeliness and availability of data.

[0064] (8) Image and smoothness combined analysis: Compared with the traditional method of simply collecting road surface smoothness, the method and system mentioned in this invention combine image analysis and smoothness analysis. The collected data can be used for further analysis and mining, such as analyzing the types of defects in road surface bumps and measuring the height difference of manhole covers, providing data support for refined road maintenance. Attached Figure Description

[0065] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0066] Figure 1 This is a schematic diagram of the device composition and data flow relationship in an embodiment of the present invention;

[0067] Figure 2 This is a schematic diagram of the data acquisition device and its installation according to an embodiment of the present invention;

[0068] Figure 3 This is a schematic diagram of image calibration according to an embodiment of the present invention;

[0069] Figure 4 This is a flowchart of a road surface smoothness measurement method according to an embodiment of the present invention;

[0070] Figure 5 This is a schematic diagram of image fusion according to an embodiment of the present invention;

[0071] Figure 6 This is a schematic diagram of the measurement point set of laser lines on the image in an embodiment of the present invention. Detailed Implementation

[0072] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0073] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0074] This invention proposes a method for measuring road surface smoothness, such as... Figure 4 As shown, the specific steps include:

[0075] Collect the vehicle's current cumulative distance traveled;

[0076] Based on the current cumulative driving distance, collect image data;

[0077] Based on the image data, obtain the set of measurement point data;

[0078] Extract the coordinate position data and velocity data of the measurement point data set;

[0079] Perform a Fourier transform on the coordinate position data to obtain the power spectral density array;

[0080] The road surface smoothness is obtained based on the power spectral density array and velocity data.

[0081] Furthermore, both the cumulative distance traveled by the vehicle and the image data need to be calibrated before they are collected.

[0082] Specifically, 1. This invention consists of a data acquisition terminal and a cloud platform. The data acquisition terminal is installed on an inspection vehicle and includes: a high-definition high-speed industrial camera, a rotary encoder, a combined navigation and positioning receiver, and an industrial control computer; the cloud platform includes: a cloud server.

[0083] The composition of the equipment and the relationship of data flow, such as Figure 1 As shown, the data acquisition device collects data such as images, speed, latitude and longitude during vehicle operation. This data is then identified and processed by intelligent algorithms within the industrial control computer, which uploads the processed data to the cloud server for storage.

[0084] 2. Hardware and performance of the acquisition terminal: A diagram showing the equipment and installation of the acquisition terminal, such as... Figure 2 As shown, it typically includes one industrial control computer, one or more high-definition industrial cameras, one rotary encoder, one or more laser emitters, and one integrated navigation and positioning receiver. Among them:

[0085] A 2.1 high-definition industrial camera with an image acquisition frame rate ≥100fps and an image resolution ≥1 megapixel.

[0086] The area-scan camera produces clear, distortion-free, and delay-free images. Mounted behind the inspection vehicle, it captures images of the road surface behind the vehicle. The captured images clearly show the laser line on the ground, centered vertically, with a length ≥3.5m. The camera's exposure time is adjustable, with a maximum exposure time ≤10000 microseconds. When a calibration plate ≥1cm thick is placed on the ground, and the laser line is projected onto the ground and then onto the calibration plate, the captured image clearly shows the height difference between the calibration plate and the road surface. The industrial camera is powered by an industrial control computer, which triggers the image capture and transmits the captured image back to the computer. It has a dustproof and waterproof rating ≥IP65.

[0087] The laser emitter, model 2.2, has a power consumption ≥500mW and is equipped with a Powell prism in its emitter head. It emits a uniform blue or green "straight line" laser beam. The laser emitter is centrally mounted at the bottom of the rear of the vehicle and emits the laser line towards the rear of the vehicle. The laser line projected onto the ground is more than 1m away from the vehicle body, and the length of the laser line is ≥3.5m, perpendicular to the vehicle's direction of travel. The laser emitter is powered by an industrial control computer or vehicle-mounted power supply, and has a dustproof and waterproof rating ≥IP65.

[0088] The 2.3 rotary encoder is mounted on the rear wheel of the vehicle and rotates with the wheel to send signals. The encoder resolution is ≥2048p / r, meaning it can send at least 2048 signals per revolution. The rotary encoder is powered by an industrial computer, and the acquired signals are also sent to the industrial computer. Its dustproof and waterproof rating is ≥IP65.

[0089] The 2.4 integrated navigation and positioning receiver has a positioning error of ≤1m and is equipped with an inertial navigation system that can collect real-time vehicle speed and has a positioning data collection frequency of ≥20Hz.

[0090] The 2.5-inch industrial control computer is installed inside the vehicle, providing power to the vehicle. It features fourth-generation (4G) or fifth-generation (5G) mobile communication technology, connecting to a high-precision navigation and positioning receiver, a high-definition industrial camera, and a rotary encoder. It can analyze the signal data sent by the rotary encoder, send image acquisition commands to the high-definition camera, and continuously receive, process, and upload the acquired data. Furthermore, the central industrial control computer contains a miniature graphics processing unit (GPU), which can accelerate image processing using deep neural network algorithms.

[0091] 3. Image and encoder calibration: After the acquisition equipment is installed, it needs to be calibrated.

[0092] Image calibration refers to the process of calibrating acquired images to establish a correspondence between pixel heights in the acquired images and physical heights in the real world. A schematic diagram of image calibration is shown below. Figure 3 As shown, the specific calibration process is as follows:

[0093] 3.1 Park the vehicle on a level surface.

[0094] 3.2 Project the laser line onto the ground and center it horizontally and vertically in the acquired image.

[0095] 3.3 Place a calibration block of a fixed height at the laser line, and record the actual height Δh of the calibration block and the calibration height Δu in the image. Here, Δu refers to the pixel distance from the position where the laser line projects onto the calibration block to the horizontal laser line. With the acquisition device remaining unchanged, Δu will increase as Δh increases.

[0096] Here, we set Δh = F(Δu), where F is a univariate polynomial of degree N (N≥2). During calibration, multiple calibration records are made, with the calibration block being changed each time. The actual height Δh and the corresponding image height Δu of each calibration block are recorded, and the coefficients of the polynomial F are solved using the least squares method.

[0097] During calibration, the minimum calibration block height is 1cm, and the maximum calibration block height is ≥40cm. At least 5+N calibration blocks of different heights should be used for calibration.

[0098] If there are two or more laser lines in the acquired image, each laser line is set to have an independent Δh = F'(Δu), where F' is a univariate Nth degree polynomial (N≥2). During calibration, the height of each laser line is recorded and F' is calculated separately.

[0099] Encoder calibration refers to establishing a relationship between rotating a rotary encoder n times and the corresponding distance traveled by a vehicle. The goal is to accurately measure the distance traveled by the vehicle.

[0100] Furthermore, based on the image data, the set of measurement point data is obtained, including:

[0101] Image data is input into a neural network model for target recognition to obtain a mask image;

[0102] The mask image is fused with the image data to obtain the fused image;

[0103] The fused image is input into a convolutional neural network model to obtain the laser line locations and inference confidence of the laser lines in the image;

[0104] Based on the laser line location and the inference confidence of the laser line, a set of measurement point data is obtained.

[0105] Specifically, the confidence level is mainly used to determine whether the current laser line is reliable. In the implementation plan, if the confidence level of multiple consecutive laser lines in the inference is lower than the corresponding value, the acquisition system will remind the acquisition operator that there is a problem with the currently acquired data and it needs to be checked.

[0106] Furthermore, the measurement point data set includes: laser line point data, corresponding vehicle speed, and acquisition time.

[0107] Furthermore, the convolutional neural network model is trained using a training set, which consists of the positions of laser lines in the labeled images.

[0108] Furthermore, before extracting the coordinate position data and velocity data of the measurement point data set, it also includes: determining whether the current cumulative distance meets the threshold distance;

[0109] If the threshold distance is met, then the coordinate position data and velocity data of the measurement point data set are extracted;

[0110] Otherwise, reacquire the measurement point data set.

[0111] Furthermore, a Fourier transform is performed on the coordinate position data to obtain the power spectral density array, including:

[0112] The coordinate position data is averaged and then calibrated to obtain the processed coordinate position data.

[0113] The processed coordinate position data is subjected to Fourier transform to obtain the power spectral density array in the corresponding frequency domain;

[0114] The power spectral density array in the corresponding frequency domain is filtered and corrected to obtain the corrected power spectral density array;

[0115] Calculate the root mean square value of the corrected power spectral density array to obtain the power spectral density array.

[0116] Furthermore, based on the power spectral density array and velocity data, the road surface smoothness is obtained, including:

[0117] The power spectral density array and velocity data are input into a multi-layer neural network model to obtain the road surface smoothness data corresponding to the measurement points;

[0118] Calculate the average smoothness of the road surface smoothness data corresponding to all measuring points to obtain the road surface smoothness.

[0119] Specifically, after the equipment has been calibrated, data can be collected, processed, and analyzed. The collection and analysis process is as follows: Figure 4 As shown. The specific data collection and analysis process is as follows:

[0120] 4.1 Detect vehicle startup, rotate the encoder to accumulate the distance the vehicle has traveled starting from 0, clear the buffer queue M, and record the current distance the vehicle has traveled as L.

[0121] 4.2 Every k meters the vehicle travels, the system sends a command to the industrial control computer to acquire one original image, recorded as mi. The value of k is determined by the camera's frame rate, the industrial control computer's computing power, and the detected vehicle speed; generally, k ≤ 0.2m. mi is the acquired original image, with dimensions [hi, wi, ci], where hi is the image height, wi is the image width, and ci is the number of image channels. For an RGB camera, ci = 3; for a monochrome camera, ci = 1.

[0122] 4.3 The acquired image mi is input into the object detection neural network T1. T1 identifies objects such as fallen leaves, spilled debris, manhole covers, curbs, and road markings in the image, records the position of each object in the image, and generates a mask image mi'. The image size of mi' is [hi, wi, 1]. In areas without objects, the pixel value is 0; in areas with objects, the pixel value is a value related to the object. For example, if the identified object in a region is a fallen leaf, the pixel value is 200; if the object in a region is a curb, the pixel value is 10, and so on.

[0123] 4.4 The original image mi and the mask image mi' are stitched together along the image channel direction to obtain the fused image mi*. The image size of the fused image mi* is [hi, wi, ci+1]. The fusion diagram is shown below. Figure 5 As shown.

[0124] 4.5 The fused image mi* is input into the convolutional neural network model T2 to identify the location of the laser line in the image and give the inference confidence of the laser line.

[0125] The convolutional neural network model T2 is trained using a supervised method. During training, the input data is a combined image obtained by fusing the original image and the mask image. The output labels are the manually labeled positions of laser lines in the image. When manually labeling laser lines, the labels are generally made according to the direction of the laser line. However, if the laser line fluctuates due to weeds, debris, fallen leaves, or gravel, the labels are adjusted to prevent these obstacles from causing fluctuations. The convolutional neural network model T2, trained in this way, can infer the position of laser lines in an image and filter out interference from road debris on the laser line waveform.

[0126] 4.6 Based on the recognition results of neural network model T2, measurement points are extracted at equal intervals in the horizontal direction of the image, forming a measurement point set P, as shown in Figure 4.6. Figure 6 As shown. The distance between the measuring points from the leftmost to the rightmost point is ≥3.5m, and the lateral distance between adjacent measuring points is ≤0.3m.

[0127] 4.7 Store the position of the measurement point set P on the mi* image, as well as the corresponding acquisition speed v, acquisition time t, and other data in the buffer queue M.

[0128] 4.8 When the vehicle's current distance L is less than the threshold distance L', repeat steps 4.2-4.7 to continuously acquire images, analyze them, and store them in the buffer queue M. When L ≥ L', extract all information from the buffer queue M, namely the vertical coordinate positions of the measurement point set P on the continuous sequence of images and the acquired velocity information. Taking the j-th measurement point as an example, obtain the vertical coordinate position array [Vj1, Vj2, ..., Vjn] of measurement point j in mi* of the image, and the corresponding velocity array [v1, v2, ..., vn]. The threshold distance L' is a fixed threshold, generally L' ≤ 10m.

[0129] 4.9 The data extracted from each measurement point is further processed to calculate the corresponding IRI.

[0130] Taking the j-th measurement point as an example:

[0131] (1) Calculate the mean Vj_mean of [Vj1,Vj2,…,Vjn] and execute [Vj1,Vj2,…,Vjn]–Vj_mean to obtain the mean array [Vj1',Vj2',…,Vjn']. Then execute F([Vj1',Vj2',…,Vjn']) to calculate the calibrated array [Vj1*,Vj2*,…,Vjn*).

[0132] (2) Perform a Fourier transform on the array [Vj1*,Vj2*,…,Vjn*] to calculate the power spectral density array PSDj in the corresponding frequency domain.

[0133] (3) The power spectral density array PSDj is filtered and corrected according to the corresponding frequency to obtain a new power spectral density array PSDj*. The correction method is to multiply the spectral density value of a specific frequency range by a coefficient less than 1 and perform filtering. This coefficient is generally obtained based on a large number of experimental summaries.

[0134] (4) Calculate the root mean square value of the power spectral density array PSDj, PSD_RMSj.

[0135] (5) Input the numerical value PSD_RMSj and the velocity array [v1,v2,…,vn] into a multi-layer neural network model T3. The T3 model then uses regression inference to derive the road surface roughness IRIj corresponding to the measuring point j in this data. The T3 model is a multi-layer neural network model. During the training of the T3 model, the model input is the array [PSD_RMSj,v1,v2,…,vn], which is the data collected and calculated by the equipment. The model output is the International Roughness Index (IRI), which is the standard value of road surface roughness measured using a level instrument based on the actual location of the measuring point.

[0136] 4.10 Summarize the road surface smoothness data of all measuring points, calculate an average smoothness, and record the start time, end time, start latitude and longitude, end latitude and longitude of this data. Then, summarize the relevant data, upload it to the cloud server, and then return to 4.1 to collect data in a loop.

[0137] The present invention also provides a road surface evenness measurement system, comprising: a data acquisition module, a data processing module, and a data storage module;

[0138] The data acquisition module is used to collect image data and vehicle-related data;

[0139] The data processing module is used to process image data and vehicle-related data to obtain road surface smoothness.

[0140] The data storage module is used to store data using a cloud server.

[0141] The following is a detailed description of this embodiment with reference to the accompanying drawings:

[0142] 1. Equipment installation

[0143] Find an SUV vehicle capable of supplying power externally and install the necessary equipment for the system, including:

[0144] (1) Industrial control computer installation

[0145] The industrial PC is installed inside the inspection vehicle, ensuring good ventilation and away from high-temperature areas. It is powered by the vehicle's onboard power supply, ensuring a stable power supply. The industrial PC is connected to a high-definition industrial camera, rotary encoder, integrated navigation and positioning receiver, and laser emitter via data cables.

[0146] (2) Laser emitter installation

[0147] The laser emitter is mounted at the bottom of the rear of the vehicle, facing backward, ensuring the laser line is projected onto the ground and perpendicular to the vehicle's direction of travel. Adjust the power and angle of the laser emitter to ensure the laser line is at least 3.5m long and clearly visible.

[0148] (3) Installation of high-definition industrial cameras

[0149] The camera is mounted behind the inspection vehicle, facing the road behind it, ensuring that the captured image is clear and distortion-free. Adjust the camera's exposure time and focal length to ensure that the laser line is clearly visible and centered in the image, and that the length of the laser line in the image is ≥3.5m.

[0150] (4) Rotary encoder installation

[0151] The rotary encoder is mounted on the rear wheel of the vehicle to ensure it follows the wheel's rotation.

[0152] (5) Installation of integrated navigation and positioning receiver

[0153] The receiver is mounted on the top of the vehicle to ensure good signal reception.

[0154] 2. Equipment Calibration

[0155] (1) Image calibration

[0156] Image calibration includes the following steps:

[0157] 1) Park the vehicle on a level surface.

[0158] The laser line is projected onto the ground, and the laser line is horizontal and vertically centered in the acquired image, with a length exceeding 3.5m in the image.

[0159] Place a calibration block of a certain height at the laser line, record the actual height Δh of the calibration block and the calibration height Δu in the image, and perform calibration by replacing at least 5+N calibration blocks of different heights. The smallest calibration block height is 1cm and the largest calibration block height is ≥40cm.

[0160] The data recorded during the calibration process is input into the acquisition software, and the background program of the acquisition software uses the least squares method to solve for the corresponding coefficients of polynomial F.

[0161] (2) Encoder calibration

[0162] Encoder calibration includes the following steps:

[0163] 1) Park the vehicle on a level surface.

[0164] 2) Let the vehicle move forward a known distance n meters, and record the number of signals emitted by the rotary encoder.

[0165] 3) Input the above data into the acquisition software. The background program of the acquisition software calculates the relationship between n meters and the number of signals to establish the correspondence between the rotation of the rotary encoder n times and the distance the vehicle travels.

[0166] In this embodiment, a rotary encoder is used to record the distance the vehicle travels, which is a common method in the industry.

[0167] Generally, the vehicle is parked on a level surface. The distance L that the vehicle travels is recorded when the wheel with the encoder is installed rotates once. The rotary encoder typically triggers information for one complete cycle. Assuming the number of signals is m, and the total number of signals triggered by the rotary encoder during the test is recorded as n, then the distance the vehicle travels is calculated as n / m*L.

[0168] 3. Data Collection and Analysis

[0169] (1) Data Acquisition

[0170] 1) The inspector clicks "Start Inspection Task" on the software, the vehicle starts to be inspected, the rotary encoder starts to accumulate the distance the vehicle has traveled from 0, the system cache queue M is cleared, and the current distance L of the vehicle has traveled is recorded.

[0171] 2) When the vehicle moves forward 0.1 meters, the system sends a command to the industrial control computer to acquire an original image mi, where the size of mi is [hi, wi, ci].

[0172] (2) Data processing

[0173] 1) Input the acquired image mi into the target detection neural network model T1 to identify target objects such as fallen leaves, spilled objects, manhole covers, curb stones, and road markings in the image, and generate a mask image mi' based on the position of the identified target object in the image. The size of the mask image is [hi, wi, 1].

[0174] In this embodiment, the neural network T1 is a target detection neural network, including but not limited to YOLOv8. After training, it is specifically designed to identify targets such as fallen leaves, spilled objects, manhole covers, curb stones, and road markings in images.

[0175] 2) The original image mi and the mask image mi' are stitched together along the image channel direction to obtain the fused image mi*, and the size of the fused image is [hi, wi, ci+1].

[0176] 3) Input the fused image mi* into the convolutional neural network model T2 to identify the location of the laser line in the image and give the inference confidence of the laser line.

[0177] In this embodiment, the T2 neural network is an instance segmentation neural network, similar to lane line recognition. The network structure includes, but is not limited to, lanenet and yolov8-seg. It is specifically designed to identify laser line regions in an image, and then uses morphological erosion operations on these regions to continuously remove boundary pixels until no further removal is possible, thereby retaining a skeleton line as the laser line for recognition.

[0178] 4) Based on the recognition results of neural network model T2, measure points are extracted at equal intervals along the horizontal direction of the image at the laser line position, forming a measure point set P.

[0179] 5) Store the position of the measurement point set P on the mi* image, as well as the corresponding acquisition speed v, acquisition time t, and other data in the buffer queue M.

[0180] 6) When the current distance L of the vehicle is forward is greater than or equal to 10m, all cached data are extracted from the cache queue M, the cached data corresponding to each measurement point is averaged and converted into frequency domain data, filtered in a specific frequency domain, and then input into the neural network model T3. The neural network model T3 infers and calculates to obtain the IRIj of each measurement point.

[0181] In this embodiment, the T3 neural network is a neural network that takes input sequence data and performs numerical regression, that is, it takes input data in the M queue and outputs a regressed IRI value. The network structure includes, but is not limited to, Transformer.

[0182] 7) Summarize the road surface smoothness data from all measuring points, calculate an average smoothness, and record the start time, end time, start latitude and longitude, and end latitude and longitude of this data segment. Record these information in the local database and upload them to the cloud. The system clears the cache queue M and the current cumulative distance L, and then continues to collect data.

[0183] (3) Data collection ends

[0184] After the data collection is completed, the inspector clicks "End Collection". The system stops collecting data and continues to process the collected data and upload any data that has not yet been uploaded.

[0185] 4. Data Upload and Management

[0186] (1) Data Upload

[0187] The industrial control computer uploads the processed data and the original collected data to the cloud server via 4G or 5G mobile communication technology.

[0188] (2) Data Receipt and Management

[0189] The cloud server receives the uploaded data, processes it, and stores it. The data management platform then analyzes the uploaded data to generate a road smoothness report.

[0190] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for measuring road surface smoothness, characterized in that, include: Acquire image data; Based on the image data, obtain the set of measurement points; Based on the image data, the set of measurement points is obtained as follows: The image data is input into a neural network model for target recognition to obtain a mask image; The mask image is fused with the image data to obtain a fused image; The fused image is input into a convolutional neural network model to obtain the laser line locations and inference confidence of the laser lines in the image; Based on the laser line locations and the inference confidence of the laser lines, the set of measurement points is obtained; The convolutional neural network model is trained using a training set, which consists of the positions of laser lines in an annotated image. Extract the attribute data of the set of measuring points, wherein the attribute data includes the coordinate position of the measuring points, the acquisition speed, and the acquisition time; Before extracting the attribute data of the measurement point set, the method further includes: determining whether the current cumulative distance meets the threshold distance. If the threshold distance is met, then the coordinate position data and velocity data of the measurement point set are extracted; Otherwise, reacquire the set of measuring points; average the attribute data and convert it into frequency domain data, and calculate the road surface smoothness of each measuring point; The process of averaging and converting the attribute data into frequency domain data includes: The coordinate position data is averaged and calibrated to obtain the processed coordinate position data. The processed coordinate position data is subjected to Fourier transform to obtain the power spectral density array in the corresponding frequency domain; The power spectral density array in the corresponding frequency domain is filtered and corrected to obtain the corrected power spectral density array; Calculate the root mean square value of the corrected power spectral density array to obtain the power spectral density array; Based on the power spectral density array, the frequency domain data is obtained; The road surface smoothness of each measuring point is summarized, and the average smoothness is calculated.

2. The method for measuring road surface smoothness according to claim 1, characterized in that, The image data needs to be calibrated before it is acquired.

3. The method for measuring road surface smoothness according to claim 1, characterized in that, Calculating the road surface smoothness at each measuring point includes: The power spectral density array and velocity data are input into a multi-layer neural network model to obtain the road surface smoothness data corresponding to the measurement points; Calculate the average smoothness of the road surface smoothness data corresponding to all measuring points to obtain the road surface smoothness.

4. A road surface evenness measurement system, used to implement the method as described in any one of claims 1-3, characterized in that, include: Data acquisition module, data processing module, and data storage module; The data acquisition module is used to acquire image data and vehicle-related data; The data processing module is used to process the image data and vehicle-related data to obtain road surface smoothness; The data storage module is used to store data using a cloud server.

Images