A highway pavement disease detection method and system

Abstract

The application provides a highway pavement disease detection method and system, belonging to the field of image recognition, identifying an abnormal area of a road surface and determining a center point of the abnormal area, and acquiring data of vibration signals at multiple different sampling points on the road surface within a sampling time; according to the data of the vibration signals, a target feature vector of the center point of the abnormal area is calculated through a feature difference vector of the vibration signals; the target feature vector and the feature difference vectors of other sampling points are respectively calculated for similarity, the maximum value and the minimum value of the direction consistency are determined, and the difference between the two is calculated to obtain the mode difference degree of the abnormal area; according to the mode difference degree of the abnormal area, the existence of the pavement disease of the abnormal area is determined. Not only the calculation resource consumption of the traditional visual method is reduced, but also the recognition accuracy and the pre-preparedness of detection are significantly improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image recognition, and particularly relates to a highway pavement disease detection method and system. BACKGROUND

[0002] Highways are the core infrastructure of modern transportation systems, and their quality directly affects transportation efficiency and driving safety. However, due to long-term vehicle load, climate change and natural environmental impact, highways are prone to various diseases such as cracks, potholes, settlement and spalling. If these diseases are not discovered and treated in time, they will further expand and accelerate the aging of the highway, not only increasing maintenance costs, but also possibly leading to traffic accidents and threatening public safety. Therefore, timely detection and accurate positioning of highway pavement diseases are important links to ensure highway quality and safety.

[0003] Currently, highway pavement disease detection technologies mainly include manual detection, traditional sensor detection and computer vision-based automated detection.

[0004] Manual detection is usually used to identify and record diseases, which is inefficient, subjective and not suitable for large-scale or complex road sections.

[0005] Traditional sensor detection uses accelerometers, strain gauges and other sensors installed on detection vehicles to detect road vibrations during vehicle travel. It can cover a large detection range and is suitable for dynamic detection. However, its precision is limited and it is less sensitive to minor and early-stage diseases.

[0006] Image recognition algorithms such as convolutional neural networks are used to identify diseases from highway monitoring videos or images taken by drones. For example, the patent document with publication number CN102509291B describes a highway pavement disease detection and identification method based on wireless network video sensors, which has high automation and can classify disease types. However, it is greatly affected by environmental factors such as light and weather conditions, and has limited ability to identify hidden diseases.

[0007] Manual detection cannot efficiently cover a large range of highways and relies on labor costs and is highly subjective. Sensor detection relies on vehicles or fixed detection devices, making it difficult to cover complex areas such as bridges and tunnels.

[0008] Recognition accuracy is affected by the environment, and image recognition is sensitive to light and weather conditions, such as significantly reduced detection effectiveness at night or in fog. For example, the patent document with publication number CN109902668B describes a road surface detection system and method carried by a drone, which cannot effectively detect early-stage diseases and hidden structural problems such as micro-cracks or internal stress concentration of the road surface.

[0009] The pure visual-based detection method needs to continuously collect and process images of the entire road, and the operation of the deep learning model relies on high-performance computing devices, which is difficult to meet the real-time and economic demands. The pre-emptiveness of disease detection is insufficient, and the existing methods mainly focus on detecting the diseases that have already appeared, lacking the ability to early warn potential diseases, and failing to fully support active maintenance. SUMMARY

[0010] The purpose of the present application is to provide a highway pavement disease detection method and system to solve one or more technical problems existing in the prior art, at least to provide a beneficial choice or create conditions.

[0011] The present application provides a highway pavement disease detection method and system, which identifies an abnormal area of the road surface and determines the center point of the abnormal area, and obtains vibration signal data at multiple different sampling points on the road surface within the sampling time; according to the vibration signal data, the target feature vector of the center point of the abnormal area is calculated through the feature difference vector of the vibration signal; the similarity of the target feature vector and the feature difference vector of other sampling points is calculated respectively, the maximum and minimum values of the direction consistency are determined and the difference between the two is calculated, and the mode difference degree of the abnormal area is obtained; according to the mode difference degree of the abnormal area, the existence of the road surface disease of the abnormal area is determined.

[0012] In order to achieve the above purpose, according to one aspect of the present application, a highway pavement disease detection method is provided, the method comprising the following steps:

[0013] Based on the image recognition algorithm, an abnormal area of the road surface is identified and the center point of the abnormal area is determined, and vibration signal data is obtained at multiple different sampling points on the road surface within the sampling time;

[0014] According to the vibration signal data, the target feature vector of the center point of the abnormal area is calculated through the feature difference vector of the vibration signal;

[0015] The similarity of the target feature vector and the feature difference vector of other sampling points is calculated respectively, the maximum and minimum values of the direction consistency are determined and the difference between the two is calculated, and the mode difference degree of the abnormal area is obtained;

[0016] According to the mode difference degree of the abnormal area, the existence of the road surface disease of the abnormal area is determined.

[0017] If the center point coincides with the vibration signal sampling point position, the feature difference vector of the sampling point is directly taken as the target feature vector of the center point; if the center point does not coincide with the sampling point position, a distance weight is assigned according to the distance between the center point and each sampling point, and the feature difference vector of the sampling point is weighted to obtain the target feature vector of the center point.

[0018] Further, the image data used to identify the abnormal area of the road surface in the method comes from the unmanned aerial vehicle shooting and / or the camera monitoring of the highway.

[0019] Further, the method for obtaining vibration signal data at multiple different sampling points on the road surface: the vibration sensor includes an acceleration sensor and / or a geophone, and multiple vibration sensors are connected by a wireless sensor, and the vibration signal data obtained by the vibration sensor is acquired through multi-channel data acquisition.

[0020] Further, wherein the image recognition algorithm includes CNN and / or edge detection algorithm, for identifying the abnormal area of the road surface and determining the center point of the abnormal area.

[0021] Further, wherein the frequency domain feature matrix is extracted from the obtained vibration signal, and the feature difference vector of the vibration signal is calculated using the frequency domain feature matrix.

[0022] Further, wherein the method for extracting the frequency domain feature matrix from the obtained vibration signal is:

[0023] At each sampling point on the road surface within the sampling time, the vibration signal of each sampling point is sampled, and the data of the discrete time domain signal is obtained;

[0024] The time domain signal data is converted into frequency domain signal by Fourier transform, wherein each frequency component corresponds to a complex value, representing the amplitude and phase of the frequency;

[0025] The amplitude of each frequency component is obtained by calculating the modulus of the complex value, and the energy density of the corresponding frequency component is obtained by squaring the amplitude;

[0026] According to the amplitude and energy density, a frequency domain feature matrix is constructed, wherein the vibration signal is divided into multiple time windows to obtain the frequency components and their energy densities under different time windows, the rows of the frequency domain feature matrix correspond to each frequency component, the columns of the frequency domain feature matrix correspond to each time window, and the elements of each row and column position of the frequency domain feature matrix are the numerical values of the energy density of the corresponding frequency component in each time window.

[0027] The time domain signal is converted into frequency domain signal, the amplitude and energy density of the frequency component are extracted, which directly reflects the disease characteristics. Frequency domain analysis can filter the influence of non-disease signals (such as environmental vibration) on detection. Disease characteristics show significant changes in specific frequency components in the frequency domain, and frequency domain analysis can more effectively separate these characteristics. Decomposing the vibration signal into frequency components helps to identify regular characteristics in complex signals.

[0028] Further, wherein the method for calculating the feature difference vector of the vibration signal using the frequency domain feature matrix is:

[0029] respectively calculate each eigenvalue of the frequency domain feature matrix of each sampling point and the eigenvector corresponding to each eigenvalue;

[0030] According to the size of the eigenvalue, the average of the eigenvectors corresponding to the maximum eigenvalue and the second largest eigenvalue is calculated as the dominant mode vector, and the average of the eigenvectors corresponding to the minimum eigenvalue and the second smallest eigenvalue is calculated as the secondary mode vector;

[0031] Subtract the dominant mode vector from the secondary mode vector to obtain the feature difference vector of the sampling point.

[0032] Based on the frequency domain feature matrix, the feature difference vector is calculated. The difference between the dominant mode vector and the secondary mode vector reflects the characteristic change of the disease. Through eigenvalue and eigenvector analysis, the significant frequency characteristics of the disease can be accurately located. Eigenvalue and eigenvector analysis reveals the main change direction in the frequency domain feature matrix, which is highly related to the disease mode. The eigenvectors corresponding to the principal eigenvalue and the secondary eigenvalue represent the principal component and the secondary component of the data, and their difference can significantly distinguish abnormal modes.

[0033] Further, the similarity between the target eigenvector and the feature difference vector of each of the other sampling points is calculated, the maximum and minimum values of the direction consistency are determined, and the difference between the two is calculated to obtain the mode difference degree of the abnormal area, which specifically includes:

[0034] Take the maximum value of the cosine similarity between the target eigenvector and the feature difference vector of each of the other sampling points as the maximum direction consistency, and take the minimum value of the cosine similarity between the target eigenvector and the feature difference vector of each of the other sampling points as the minimum direction consistency. The difference between the maximum direction consistency and the minimum direction consistency is taken as the mode difference degree.

[0035] The difference between the maximum direction consistency and the minimum direction consistency reflects the significance of the disease characteristics. Using a statistical model, the threshold can be dynamically adjusted to adapt to different road conditions and disease types. The mode difference degree is a quantitative measure of the distribution difference between the target eigenvector and the surrounding feature difference vectors, which can clearly distinguish normal areas from abnormal areas.

[0036] Further, according to the mode difference degree of the abnormal area, the method for determining the existence of the road disease in the abnormal area is: if the mode difference degree is greater than the threshold, it is determined that the abnormal area exists.

[0037] Preferably, if the mode difference degree is greater than 1, it is determined that the abnormal area exists.

[0038] Preferably, the statistical model is used to learn the feature distribution of the target feature vector and the feature difference vectors of the other sampling points, and the specific value of the pattern difference degree is calculated. Generally, when the target feature vector is close to the normal sample, the feature difference vectors of the target feature vector and the other sampling points have high similarity, and the pattern difference degree is small. If the abnormal area has significant disease characteristics, the target feature vector and the feature distribution of the other sampling points have significant differences, and the pattern difference degree is large.

[0039] The application further provides a highway pavement disease detection system, which comprises a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the highway pavement disease detection method are implemented. The highway pavement disease detection system can run on desktop computers, notebook computers, mobile phones, palmtop computers, cloud data centers, and other computing devices. The executable system can include, but is not limited to, a processor, a memory, a server cluster. The processor executes the computer program and runs in the following system units:

[0040] An image recognition unit is configured to identify an abnormal area on the road surface and determine the center point of the abnormal area, and obtain data of vibration signals at multiple different sampling points on the road surface within a sampling time;

[0041] A data acquisition unit is configured to calculate a target feature vector of the center point of the abnormal area from the data of the vibration signals through a feature difference vector of the vibration signals.

[0042] A pattern calculation unit is configured to calculate the similarity of the target feature vector and the feature difference vectors of the other sampling points, determine the maximum and minimum values of the directional consistency and calculate the difference between the two values to obtain the pattern difference degree of the abnormal area.

[0043] A pattern judgment unit is configured to determine the existence of the road surface disease of the abnormal area according to the pattern difference degree of the abnormal area.

[0044] The application provides a highway pavement disease detection method and system, identifies an abnormal area of a pavement and determines a center point of the abnormal area, and obtains vibration signal data at a plurality of different sampling points on the pavement within a sampling time; according to the vibration signal data, a target feature vector of the center point of the abnormal area is calculated through a feature difference vector of the vibration signal; the target feature vector and the feature difference vectors of other sampling points are respectively calculated for similarity, the maximum and minimum values of the direction consistency are determined, and the difference between the two values is calculated to obtain the mode difference degree of the abnormal area; according to the mode difference degree of the abnormal area, the existence of the pavement disease of the abnormal area is determined. In combination with image recognition and vibration signal analysis, through feature extraction and mode difference degree analysis of the vibration signal, not only the calculation resource consumption of the traditional visual method is reduced, but also the recognition accuracy and the pre-preparedness of detection are significantly improved. This multi-modal combined method is particularly suitable for the disease monitoring demand of modern highways in a large range, low cost and high efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0045] The above and other features of the present application will become more apparent from the following detailed description of embodiments thereof, taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements in the several views, as best seen in the drawings. It is to be expressly understood that the drawings are included solely for purposes of illustration and that they are NOT intended to be used to limit the scope of the application to the specific embodiments presented in the drawings.

[0046] Figure 1 Fig. 1 shows a flowchart of a highway pavement disease detection method;

[0047] Figure 2 Fig. 2 shows a system structure diagram of a highway pavement disease detection system. DETAILED DESCRIPTION

[0048] The concept, specific structure and generated technical effects of the present application will be described clearly and completely in the following embodiments and drawings, so as to fully understand the purpose, scheme and effect of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0049] In the description of the present application, one or more means one or more, more than two, greater than, less than, more than, etc. are understood as not including the number, above, below, etc. are understood as including the number. If it is described as first, second, it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the sequence of indicated technical features.

[0050] As Figure 1A flow chart of a highway pavement disease detection method according to the present application is shown, and a highway pavement disease detection method and system according to an embodiment of the present application will be described below in conjunction with Figure 1

[0051] The present application provides a highway pavement disease detection method, which specifically comprises the following steps:

[0052] Identify the abnormal area of the road surface and determine the center point of the abnormal area, and obtain the data of the vibration signal at a plurality of different sampling points on the road surface within the sampling time;

[0053] According to the data of the vibration signal, the target feature vector of the center point of the abnormal area is calculated through the feature difference vector of the vibration signal;

[0054] Calculate the similarity of the target feature vector and the feature difference vector of the other sampling points respectively, determine the maximum and minimum values of the direction consistency and calculate the difference value between the two, and obtain the pattern difference degree of the abnormal area;

[0055] According to the pattern difference degree of the abnormal area, it is determined whether the road surface disease of the abnormal area exists.

[0056] If the center point coincides with the vibration signal sampling point position, the feature difference vector of the sampling point is directly taken as the target feature vector of the center point; if the center point does not coincide with the sampling point position, the distance weight is assigned according to the distance between the center point and each sampling point, and the feature difference vector of the sampling point is weighted to obtain the target feature vector of the center point.

[0057] Further, the image data used to identify the abnormal area of the road surface in the method comes from the unmanned aerial vehicle shooting and / or the camera monitoring of the highway.

[0058] Further, the method for obtaining vibration signal data at a plurality of different sampling points on the road surface: the vibration sensor includes an acceleration sensor and / or a geophone, and a plurality of vibration sensors are connected to the wireless sensor at the same time, and the vibration signal data obtained by the vibration sensor is collected through multi-channel data acquisition.

[0059] Further, the image recognition algorithm includes CNN and / or edge detection algorithm, which is used to identify the abnormal area of the road surface and determine the center point of the abnormal area.

[0060] Further, the frequency domain feature matrix is extracted from the obtained vibration signal, and the feature difference vector of the vibration signal is calculated using the frequency domain feature matrix.

[0061] Further, the method for extracting the frequency domain feature matrix from the obtained vibration signal is:

[0062] ​sampling the vibration signal of each sampling point to obtain discrete time-domain signal data;

[0063] converting the time-domain signal data into a frequency-domain signal by Fourier transform, wherein each frequency component corresponds to a complex value representing the amplitude and phase of the frequency;

[0064] obtaining the amplitude of each frequency component by calculating the modulus of the complex value, and obtaining the energy density of the corresponding frequency component by squaring the amplitude;

[0065] constructing a frequency-domain feature matrix according to the amplitude and energy density, wherein the frequency components and their energy densities under different time windows are obtained by dividing the vibration signal into multiple time windows, the rows of the frequency-domain feature matrix correspond to the frequency components, the columns of the frequency-domain feature matrix correspond to the time windows, and the elements of the rows and columns of the frequency-domain feature matrix are the values of the energy densities of the corresponding frequency components in the time windows.

[0066] Further, the method for calculating the feature difference vector of the vibration signal using the frequency-domain feature matrix is:

[0067] calculating the eigenvalues of the frequency-domain feature matrix of each sampling point and the eigenvectors corresponding to the eigenvalues, respectively;

[0068] calculating the average of the eigenvectors corresponding to the maximum eigenvalue and the second maximum eigenvalue as the dominant mode vector, and the average of the eigenvectors corresponding to the minimum eigenvalue and the second minimum eigenvalue as the secondary mode vector according to the eigenvalue size;

[0069] subtracting the dominant mode vector from the secondary mode vector to obtain the feature difference vector of the sampling point.

[0070] Further, the similarity between the target eigenvector and the feature difference vectors of other sampling points is calculated, the maximum and minimum values of the directional consistency are determined and the difference between the two is calculated to obtain the mode difference degree of the abnormal area, which specifically includes:

[0071] taking the maximum value of the cosine similarity between the target eigenvector and the feature difference vectors of the remaining sampling points as the maximum directional consistency, and taking the minimum value of the cosine similarity between the target eigenvector and the feature difference vectors of the remaining sampling points as the minimum directional consistency, and taking the difference between the maximum directional consistency and the minimum directional consistency as the mode difference degree.

[0072] Further, according to the mode difference degree of the abnormal area, the method for determining the existence of the pavement disease of the abnormal area is: if the mode difference degree is greater than a threshold value, it is determined that the abnormal area exists pavement disease.

[0073] Preferably, if the pattern difference is greater than 1, it is determined that the abnormal area has road diseases.

[0074] Preferably, the specific value of the pattern difference is calculated by using a statistical model to learn the feature distribution of the target feature vector and the feature difference vectors of the other sampling points. Generally, when the target feature vector is close to the normal sample, the feature difference vectors of the target feature vector and the other sampling points have high similarity, and the pattern difference is small. If the abnormal area has significant disease characteristics, the target feature vector and the feature distribution of the other sampling points have significant differences, and the pattern difference is large.

[0075] In some embodiments provided, the vibration sensor type includes an optional acceleration sensor for measuring the intensity and frequency characteristics of the vibration in the vertical direction. While the geophone is more suitable for detecting low-frequency vibration signals and is suitable for large-area road disease detection. The frequency response range covers at least 0.1 Hz to 100 Hz. The dynamic range adapts from weak vibration (such as micro-cracks) to strong vibration (such as impact caused by large potholes). High-sensitivity equipment is selected to capture weak signals.

[0076] The data acquisition module supports simultaneous connection of multiple vibration sensors through multi-channel data acquisition. A high sampling frequency (such as 1 kHz or higher) is required to ensure that high-frequency details are not lost. Noise filtering function is also added to avoid environmental noise interference with the signal. The device can include a multi-channel data acquisition card (DAQ) and an embedded data acquisition device (portable or fixed).

[0077] The communication module transmits the vibration data collected in real time to the server or local storage device, supporting wired and wireless transmission methods. Wireless transmission requires a Wi-Fi module, Zigbee module, or LoRa communication module. Wired transmission is fiber network or Ethernet connection.

[0078] The data processing and control system is responsible for controlling the sampling frequency and synchronization timing of the sensor, receiving data in real time and performing preprocessing (such as denoising and filtering). An industrial control computer or a single-chip microcomputer can be used. Edge computing devices are used for data preprocessing and preliminary analysis.

[0079] The number of sampling points is determined according to the road surface area and the disease detection accuracy requirement. It is recommended to arrange one sensor per 1 to 2 square meters.

[0080] The sampling point distribution can be uniformly distributed, suitable for large-scale road detection, and a grid-shaped sensor array is arranged; or it can be non-uniformly distributed, with denser sensors arranged in key areas (such as bridge joints and areas with more potholes). The sampling point positioning uses GPS or high-precision range finders to accurately calibrate the position of each sensor.

[0081] The sensor installation can be fixedly installed by ground drilling, burying the sensor under the road surface to reduce environmental noise interference; or can be installed on the surface by adhesion, using high-strength glue or special clamps to fix the sensor on the road surface. The installation requirements ensure that the sensor is in full contact with the road surface and avoids loosening. After installation, calibration is performed to verify the accuracy of the signal.

[0082] All sensors need to be sampled synchronously, and it is recommended to use a GPS time synchronization module or clock synchronization technology to ensure consistency of data at different sampling points. All sensors are connected to the central data acquisition system by wired mode.

[0083] Or use a wireless communication module to transmit the data collected by each sensor to a unified server.

[0084] Wired power supply is suitable for fixed installation system, powered by main power or solar power. Battery-powered portable or temporary installed sensors can use lithium batteries, equipped with backup power.

[0085] Example one:

[0086] Record a vibration signal as x(t), where t is the independent variable representing time. For example, in one specific data, x(t) is composed of two sine waves: one with a frequency of 5Hz and an amplitude of 2; one with a frequency of 15Hz and an amplitude of 1.

[0087] The sampling frequency of the signal is 50Hz, collecting 50 data points per second, and sampling for 1 second, resulting in a total of 50 points.

[0088] Then, the time domain signal is collected. Through sampling, the discrete signal in the time domain can be obtained, including the following:

[0089] x[0]=0, x[1]=2.37, x[2]=1.85, x[3]=-0.62,…

[0090] This signal contains two main frequency components (5Hz and 15Hz), and in the time domain it looks like a complex waveform.

[0091] Then, the time domain signal is subjected to fast Fourier transform (FFT). In the transformation, the fast Fourier transform (FFT) is used to convert the time domain signal to the frequency domain signal. Each frequency component corresponds to a complex value, representing the amplitude and phase of that frequency.

[0092] Calculate the amplitude by calculating the modulus of the complex number to obtain the amplitude of each frequency component. For example: for frequency 5Hz, the amplitude is 2; for frequency 15Hz, the amplitude is 1; the amplitude of other frequencies is close to 0.

[0093] Recalculate the energy density, the energy density is the square of the amplitude, for example: the energy density of 5Hz is 2^2=4; the energy density of 15Hz is 1^2=1.

[0094] Then, the frequency component and the corresponding amplitude / energy are generated, and the frequency component and the corresponding amplitude / energy density are obtained, for example: the frequency is 1Hz, the amplitude is 0.1, and the energy density is 0.01; the frequency is 5Hz, the amplitude is 2, and the energy density is 4; the frequency is 15Hz, the amplitude is 1, and the energy density is 1.

[0095] Next, the frequency domain feature matrix is constructed. In some embodiments, data of multiple time windows are continuously collected, for example, 1 second each time, and the FFT of the signals of these time windows is performed respectively to obtain the respective frequency distribution characteristics. The final result can be arranged into a matrix:

[0096] The row represents the frequency component, for example, f1=1Hz, f2=2Hz, f3=5Hz, and so on;

[0097] The column represents different time windows, for example, the first column is the frequency distribution of window 1, and the second column is the frequency distribution of window 2.

[0098]

[0099]

[0100] The data therein is sampled to form a matrix, and the sampling result is as follows:

[0101]

[0102] Among them, the first row corresponds to the energy density of frequency 1Hz, the second row corresponds to the energy density of frequency 5Hz, and the third row corresponds to the energy density of frequency 15Hz. Each column represents the data of different time windows. The values of the energy density of three frequencies corresponding to three windows are sampled to form a 3x3 matrix. The matrix formed by the sampling data should be a square matrix, and the number of rows and columns should be no less than 3.

[0103] In some embodiments, the matrix can be analyzed, the main frequency can be extracted to find the frequency with the maximum energy density in each column, and the main frequency component can be determined. For example, in window 1, the main frequency is 5Hz. Frequency band energy analysis can also be performed, the frequencies are divided into low frequency band (0-10Hz) and high frequency band (10-20Hz), and the total energy of these frequency bands is calculated respectively for analyzing different types of road diseases. Feature extraction can also be performed, the feature vector is generated by statistical analysis (such as mean, maximum, variance, etc.) of the matrix, and is used as the input of the subsequent machine learning model.

[0104] The values of each row and column in the above matrix are taken to calculate the eigenvalue of the matrix, the eigenvector corresponding to each eigenvalue is calculated, and each eigenvector corresponding to each eigenvalue is spliced in order of the numerical value of each eigenvalue to obtain a long vector.

[0105] The eigenvalues are sorted in descending order as: λ1=5.0839502, λ2=0.05309408, λ3=-0.02704428;

[0106] The eigenvector corresponding to each eigenvalue is:

[0107] Corresponding to λ1=5.0839502: v1=[-0.0043, -0.9619, -0.2735];

[0108] Corresponding to λ2=0.05309408: v2=[-0.1317, -0.6556, 0.7436];

[0109] Corresponding to λ3=-0.02704428: v3=[-0.3131, 0.8197, -0.4797];

[0110] Example two point 1:

[0111] The eigenvectors corresponding to the eigenvalues are spliced, wherein each eigenvector is spliced into a long vector according to the order of the size of the eigenvalues:

[0112] V=[-0.0043, -0.9619, -0.2735, -0.1317, -0.6556, 0.7436, -0.3131, 0.8197, -0.4797].

[0113] The largest eigenvalue λ1=5.0839502 represents the energy intensity of the principal component of the matrix. Here it indicates that most of the energy is concentrated in the corresponding eigenvector (principal component direction).

[0114] The second largest eigenvalue λ2=0.05309408 represents a secondary dynamic component, which may be noise or a secondary mode of the signal.

[0115] The negative eigenvalue λ3=-0.02704428, if there is a negative eigenvalue, may reflect the instability of the matrix or the existence of a reverse dynamic effect in the signal.

[0116] The principal eigenvector (v1) represents the dominant pattern of variation in the signal in the principal direction (the principal frequency band or the principal feature of the defect). It corresponds to the most prominent pattern in the signal, such as the energy contribution at the principal frequency 5 Hz. The secondary eigenvectors (v2, v3, etc.) represent the secondary patterns of variation, which can be related to background noise, random perturbations, or secondary frequency characteristics. For example, high-frequency defect characteristics (such as micro-cracks) or low-frequency noise characteristics.

[0117] The positive and negative values in the eigenvectors represent the inter-relationship between different components in the matrix. For example, a positive value and a negative value can represent mutually weakening signal components.

[0118] Example two point 2:

[0119] Instead of concatenating the eigenvectors corresponding to each eigenvalue, the eigenvector corresponding to the first largest eigenvalue and the eigenvector corresponding to the second largest eigenvalue are averaged point by point in each dimension to obtain a vector V12, and the eigenvector corresponding to the first smallest eigenvalue and the eigenvector corresponding to the second smallest eigenvalue are averaged point by point in each dimension to obtain a vector V23. The vector obtained by subtracting V23 from V12 in each dimension is taken as Vgap.

[0120] In one embodiment, the average of the eigenvectors corresponding to the largest and second largest eigenvalues V12 is: V12 = 12 × (v1 + v2) = [-0.0680, -0.80875, 0.23505];

[0121] The average of the eigenvectors corresponding to the second largest and smallest eigenvalues V23 is:

[0122] V23 = 12 × (v2 + v3) = [-0.2224, 0.08205, 0.13195];

[0123] Vgap (the resulting vector of V12 minus V23) is:

[0124] Vgap = V12 - V23 = [0.1544, -0.8908, 0.1031];

[0125] V12 represents the intermediate pattern of the dominant feature, and V12 reflects the comprehensive performance of the vibration patterns corresponding to the largest and second largest eigenvalues. It can be understood as the core feature of the principal component of the signal.

[0126] V23 represents the intermediate pattern of the secondary feature, and V23 reflects the comprehensive pattern of the smallest and second smallest eigenvalues, which can contain noise or secondary features in the signal.

[0127] Vgap represents the difference between the main component and the secondary component, Vgap represents the significant difference between the main mode and the secondary mode, which can be used to distinguish the key disease mode from the background mode.

[0128] In some embodiments, the method for detecting road surface diseases on a highway can specifically include:

[0129] Using an image recognition algorithm, an abnormal area of the road surface is identified;

[0130] A center point of the abnormal area is selected;

[0131] The Euclidean distance between the center point of the abnormal area and each vector sampling point is calculated, and a vector corresponding to the center point of the abnormal area is calculated.

[0132] For example, if the center point of the abnormal area coincides with the coordinates of a vector sampling point, the Vgap vector corresponding to the coinciding vector sampling point is taken as the feature vector corresponding to the center point of the abnormal area, which is called Vtarget;

[0133] Each vector sampling point can also be sorted according to its Euclidean distance from the center point of the abnormal area from small to large, and the distance between them is weighted, the closer the distance, the greater the weight, the weight is a floating point number between 0 and 1, and the weight can be calculated by a normalization algorithm according to the Euclidean distance, and the mean vector composed of the weighted sum and average value of Vgap corresponding to each sampling point according to the weight is taken as the feature vector Vtarget corresponding to the center point of the abnormal area;

[0134] The cosine similarity of Vtarget and Vgap corresponding to each sampling point except itself is calculated, the maximum value of the cosine similarity is taken as maxDir, the minimum value of the cosine similarity is taken as minDir, the difference between maxDir and minDir is calculated, and if gapDir is greater than 1, the abnormal area is detected as having a road surface disease.

[0135] Alternatively, for example, each vector sampling point is a sampling point for calculating the Vgap vector by acquiring a vibration signal on the road surface, the distance between each sampling point is equal, each vector sampling point is sorted according to its Euclidean distance from the center point of the abnormal area from small to large, the three sampling points with the smallest distance are obtained, and the mean vector composed of the average value of Vgap corresponding to the three sampling points according to the dimension is taken as the feature vector Vtarget corresponding to the center point of the abnormal area.

[0136] The Vtarget and the Vgap corresponding to the three sampling points are calculated respectively cosine similarity, take the maximum value of the cosine similarity as maxDir, take the minimum value of the cosine similarity as minDir, calculate the difference value gapDir between maxDir and minDir, if gapDir is greater than 1, the abnormal area is detected as existing road disease.

[0137] Vtarget is a target feature vector, Vgap is a feature difference vector, V12 is a dominant mode vector, V23 is a secondary mode vector, gapDir is a mode difference degree, maxDir is a maximum direction consistency, minDir is a minimum direction consistency, and wi is a distance weight.

[0138] Using a pure computer vision method, the image data needs to be continuously collected and processed for the entire road area, and the calculation overhead of image processing (especially deep learning models such as CNN) is huge. Due to the need for long-time running of the vision model, the demand for hardware resources is high, such as long-time occupation of GPU. The local visual detection of the present application only locates the suspected disease area through the image recognition algorithm, rather than continuously analyzing the image of the entire road. After locating the area, the disease characteristics are analyzed through the feature difference vector of the vibration signal, reducing the dependence on image processing. The calculation resources required for vibration signal analysis are much lower than continuous image analysis, so the overall calculation resources are significantly reduced.

[0139] Image recognition algorithms are susceptible to environmental lighting, weather conditions (such as rainy or foggy days), and changes in road surface materials, which may lead to misjudgment. Visual algorithms usually require higher resolution images when identifying minor diseases (such as micro-cracks), increasing the difficulty of data processing.

[0140] The improved vibration signal analysis of the present application supplements the deficiencies of image recognition, and through vibration signal features (such as frequency domain feature matrix and feature difference vector), it can accurately detect road diseases, even in the case of minor diseases or surface visual features are not obvious, it can still be effectively identified. Vibration signals are not affected by environmental lighting and can be operated stably at night or in adverse weather conditions. By combining visual and vibration features and fusing multi-modal information, the accuracy of the detection results is enhanced.

[0141] Due to the lag of visual methods, image recognition can only detect diseases when they have already become apparent, such as obvious cracks, pits, etc. For early structural problems that have not yet manifested as surface diseases, visual methods can be completely unable to detect. The vibration signal of the present application can capture changes in the pavement structure that have not yet manifested as obvious surface diseases, such as internal stress concentration points or slight uneven settlement. Through pattern difference analysis, possible problem areas can be predicted at the early stage of disease occurrence, i.e. when the disease impact has not yet appeared on the surface. The present application not only identifies existing diseases, but also provides early warning, thereby supporting more proactive pavement maintenance strategies.

[0142] In actual application scenarios, it is not necessary to continuously shoot and process images of the entire highway. Only key areas (such as bridges, sharp turns, etc.) or suspected disease areas need to be detected using image recognition combined with vibration signals. This reduces the deployment pressure of hardware devices and is more suitable for large-scale highway network monitoring. It saves computing resources and hardware costs, and the vibration sensor is relatively low in cost, making it easy to arrange in a large area. The combination of image and vibration sensors can flexibly expand the detection range while avoiding excessive resource consumption. Through early detection and accurate identification of diseases, the pavement diseases can be timely maintained and repaired, avoiding further expansion of the diseases and prolonging the service life of the highway.

[0143] In tests, the method based on image recognition combined with vibration signal analysis showed obvious advantages compared to the pure image recognition method: the comprehensive method reached 95% in accuracy (compared to 85% for image recognition), and was particularly outstanding in early detection (i.e. when the disease has not fully manifested), with a 30% increase in advance rate and a 40% reduction in computing resource usage, fully demonstrating the superiority of the present application in efficiency and effectiveness.

[0144] The highway pavement disease detection system runs in any computing device of a desktop computer, a notebook computer, a mobile phone, a palm computer, or a cloud data center, and the computing device includes a processor, a memory, and a computer program stored in the memory and running on the processor. The processor implements the steps in the highway pavement disease detection method when executing the computer program. The executable system can include, but is not limited to, a processor, a memory, and a server cluster.

[0145] An embodiment of the present application provides a highway pavement disease detection system, as shown in Figure 2 The highway pavement disease detection system of the embodiment includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor implements the steps in the highway pavement disease detection method embodiment when executing the computer program to control the system of the present application. The processor executes the computer program and runs in the following system units:

[0146] An image recognition unit is configured to recognize a road surface abnormal area and determine a center point of the abnormal area, and acquire data of vibration signals at a plurality of different sampling points on the road surface within a sampling time;

[0147] A data acquisition unit is configured to calculate a target feature vector of the center point of the abnormal area according to the data of the vibration signals through a feature difference vector of the vibration signals;

[0148] A mode calculation unit is configured to calculate a similarity between the target feature vector and a feature difference vector of other sampling points, determine a maximum value and a minimum value of a direction consistency and calculate a difference value between the two, and obtain a mode difference degree of the abnormal area;

[0149] A mode judgment unit is configured to determine the existence of a road surface disease of the abnormal area according to the mode difference degree of the abnormal area.

[0150] Preferably, all undefined variables in the present application can be threshold values set by humans if not defined.

[0151] Preferably, dimensionless values are used to calculate physical quantities of different units.

[0152] The highway road surface disease detection system can be run on desktop computers, notebook computers, mobile phones, palmtop computers, cloud data centers and other computing devices. The highway road surface disease detection system includes, but is not limited to, a processor and a memory. Those skilled in the art can understand that the example is only an example of the highway road surface disease detection method and system, and does not constitute a limitation on the highway road surface disease detection method and system, and can include more or fewer components, or combine certain components, or different components, for example, the highway road surface disease detection system can also include input and output devices, network access devices, buses, etc.

[0153] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The processor is a control center of the road surface disease detection system, and is connected with each sub-area of the road surface disease detection system through various interfaces and lines.

[0154] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the road surface disease detection method and system by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0155] Although the description of the present application has been quite detailed and particularly described with respect to several embodiments, it is not intended to be limited to any of these details or embodiments or any special embodiment, so as to effectively cover the intended scope of the present application. In addition, the present application is described above in the embodiments that the inventors can foresee, and the purpose is to provide a useful description, and non-essential modifications to the present application that have not yet been foreseen can still represent equivalent modifications of the present application.

Claims

1. A method for detecting pavement defects on highways, characterized in that, The method includes the following steps: Based on image recognition algorithms, abnormal areas on the road surface are identified and the center point of the abnormal areas is determined. Vibration signal data were acquired at multiple different sampling points on the road surface within the sampling time. Based on the vibration signal data, the target feature vector of the center point of the abnormal area is calculated through the feature difference vector of the vibration signal. This includes: if the center point coincides with the sampling point of the vibration signal, the feature difference vector of the sampling point is directly used as the target feature vector of the center point; if the center point does not coincide with the sampling point, distance weights are assigned according to the distance between the center point and each sampling point, and the feature difference vectors of the sampling points are weighted to obtain the target feature vector of the center point. The similarity between the target feature vector and the feature difference vectors of other sampling points is calculated, the maximum and minimum values ​​of directional consistency are determined, and the difference between the two is calculated to obtain the pattern difference degree of the abnormal region. Based on the pattern difference degree of the abnormal area, the existence of pavement distress in the abnormal area is determined; Specifically, a frequency domain feature matrix is ​​extracted from the obtained vibration signal, and the feature difference vector of the vibration signal is calculated using the frequency domain feature matrix. Calculate the eigenvalues ​​of the frequency domain feature matrix and the corresponding eigenvectors for each sampling point. Based on the magnitude of the eigenvalues, the average value of the eigenvectors corresponding to the largest and second largest eigenvalues ​​is calculated as the dominant mode vector, and the average value of the eigenvectors corresponding to the smallest and second smallest eigenvalues ​​is calculated as the secondary mode vector. Subtracting the dominant mode vector from the secondary mode vector yields the feature difference vector of the sampling points.

2. The method for detecting pavement defects according to claim 1, characterized in that, The image data used to identify abnormal areas on the road surface in the method comes from drone footage and / or highway camera monitoring.

3. The method for detecting pavement defects according to claim 1, characterized in that, A method for acquiring vibration signal data from multiple different sampling points on a road surface: The vibration sensor includes an accelerometer and / or a seismograph, and multiple vibration sensors are connected simultaneously via wireless sensors to acquire vibration signal data from the vibration sensors through multi-channel data acquisition.

4. The method for detecting pavement defects according to claim 1, characterized in that, in, The image recognition algorithm includes CNN and / or edge detection algorithms, used to identify abnormal areas on the road surface and determine the center point of the abnormal area.

5. The method for detecting pavement defects according to claim 1, characterized in that, in, The method for extracting the frequency domain feature matrix from the obtained vibration signal is as follows: During the sampling time, vibration signals are sampled at each sampling point on the road surface, and discrete time-domain signal data are obtained. The data of the time-domain signal is converted into the frequency-domain signal through Fourier transform, where each frequency component corresponds to a complex value, representing the amplitude and phase of that frequency; The amplitude of each frequency component is obtained by calculating the modulus of the complex value, and the energy density of the corresponding frequency component is obtained by squaring the amplitude. A frequency domain feature matrix is ​​constructed based on the amplitude and energy density. The vibration signal is divided into multiple time windows to obtain the frequency components and their energy densities under different time windows. The rows of the frequency domain feature matrix correspond to each frequency component, and the columns of the frequency domain feature matrix correspond to each time window. The elements at each row and column position of the frequency domain feature matrix are the values ​​of the energy density of the corresponding frequency component in each time window.

6. The method for detecting pavement defects according to claim 1, characterized in that, The similarity between the target feature vector and the feature difference vectors of other sampling points is calculated. The maximum and minimum values ​​of directional consistency are determined, and the difference between the two is calculated to obtain the pattern difference degree of the abnormal region. Specifically, this includes: The maximum value of the cosine similarity between the target feature vector and the feature difference vectors of the other sampling points is taken as the maximum directional consistency, and the minimum value of the cosine similarity between the target feature vector and the feature difference vectors of the other sampling points is taken as the minimum directional consistency. The difference between the maximum directional consistency and the minimum directional consistency is taken as the pattern difference degree.

7. A method for detecting pavement defects according to any one of claims 1, 5, or 6, characterized in that, The method for determining the existence of pavement distress in an abnormal area based on the pattern difference is as follows: if the pattern difference is greater than a threshold, then the abnormal area is determined to have pavement distress.

8. A highway pavement defect detection system, characterized in that, The aforementioned highway pavement distress detection system operates on any computing device, such as a desktop computer, a laptop computer, or a cloud data center. The computing device includes a processor, a memory, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the highway pavement distress detection method as described in any one of claims 1 to 6.

Images