Highway pavement disease detection method and system

Abstract

The invention provides a highway pavement disease detection method and system, and belongs to the field of image recognition, and the method comprises the steps: recognizing an abnormal region of a pavement, determining a central point of the abnormal region, and obtaining the data of vibration signals at a plurality of different sampling points on the pavement within the sampling time; according to the data of the vibration signal, calculating a target feature vector of the center point of the abnormal region through the feature difference vector of the vibration signal; respectively calculating similarities of the target feature vector and feature difference vectors of other sampling points, determining a maximum value and a minimum value of direction consistency, and calculating a difference value between the maximum value and the minimum value to obtain a mode difference degree of the abnormal region; and according to the mode difference degree of the abnormal area, determining the existence of the pavement disease in the abnormal area. The computing resource consumption of a traditional visual method is reduced, and the recognition accuracy and the detection pre-performance are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition, and particularly relates to a method and system for detecting highway pavement diseases. Background Art

[0002] Highways are the core infrastructure of the modern transportation system, and their quality directly affects transportation efficiency and driving safety. However, due to long-term vehicle loads, climate change, and natural environmental impacts, various diseases are likely to occur on highway pavements, such as cracks, potholes, settlements, and spalling. If these diseases are not discovered and treated in a timely manner, they will further expand, accelerating highway aging, not only increasing maintenance costs but also potentially leading to traffic accidents and threatening public safety. Therefore, the timely detection and accurate positioning of highway pavement diseases are important links to ensure highway quality and safety.

[0003] Currently, highway pavement disease detection technologies mainly include manual detection, traditional sensor detection, and computer vision-based automated detection.

[0004] Diseases are usually identified and recorded through manual inspections, which are inefficient, subjective, and not suitable for large-scale or complex sections.

[0005] Traditional sensor detection uses sensors such as accelerometers and strain gauges installed on detection vehicles to detect through the vibration signals of the road surface during vehicle driving. It can cover a large detection range and is suitable for dynamic detection. However, the accuracy is limited, and the sensitivity to minor diseases and early diseases is relatively low.

[0006] Image recognition algorithms, such as convolutional neural networks, are used to identify diseases from highway surveillance videos or images taken by drones. For example, the highway pavement disease detection and recognition method based on wireless network video sensors described in the patent document with the publication number CN102509291B has a high degree of automation and can achieve disease type classification. However, it is greatly affected by environmental factors such as lighting and weather conditions, and the ability to identify hidden diseases is limited.

[0007] Manual detection cannot efficiently cover a large range of highways, relies on labor costs, and is highly subjective. Sensor detection relies on vehicles or fixed detection devices and is difficult to cover complex areas such as bridges and tunnels.

[0008] The recognition accuracy is affected by the environment. Image recognition is sensitive to lighting and weather conditions. For example, the detection effect significantly decreases at night or in foggy weather. The unmanned aerial vehicle-mounted pavement detection system and detection method described in the patent document with the publication number CN109902668B cannot effectively detect early diseases and hidden structural problems, such as micro-cracks or internal stress concentration in the pavement.

[0009] Pure vision-based detection methods require continuous image acquisition and processing of the entire road. The operation of deep learning models depends on high-performance computing devices, making it difficult to meet the requirements of real-time performance and economy. The predictability of disease detection is insufficient. Existing methods mainly focus on detecting obvious diseases and lack the ability to early warn of potential diseases, failing to fully support proactive maintenance. Summary of the Invention

[0010] The purpose of the present invention is to provide a method and system for detecting road surface diseases, so as to solve one or more technical problems existing in the prior art, and at least provide a beneficial choice or create conditions.

[0011] The present invention provides a method and system for detecting road surface diseases, which identify abnormal areas on the road surface, determine the center points of the abnormal areas, and obtain data of vibration signals at multiple different sampling points on the road surface within the sampling time; according to the data of the vibration signals, through the characteristic difference vectors of the vibration signals, calculate the target characteristic vectors of the center points of the abnormal areas; calculate the similarity between the target characteristic vectors and the characteristic difference vectors of other sampling points respectively, determine the maximum and minimum values of the direction consistency and calculate the difference between the two to obtain the pattern difference degree of the abnormal area; according to the pattern difference degree of the abnormal area, determine the existence of road surface diseases in the abnormal area.

[0012] To achieve the above purpose, according to one aspect of the present invention, a method for detecting road surface diseases is provided, and the method includes the following steps:

[0013] Based on an image recognition algorithm, identify abnormal areas on the road surface, determine the center points of the abnormal areas, and obtain data of vibration signals at multiple different sampling points on the road surface within the sampling time;

[0014] According to the data of the vibration signals, through the characteristic difference vectors of the vibration signals, calculate the target characteristic vectors of the center points of the abnormal areas;

[0015] Calculate the similarity between the target characteristic vectors and the characteristic difference vectors of other sampling points respectively, determine the maximum and minimum values of the direction consistency and calculate the difference between the two to obtain the pattern difference degree of the abnormal area;

[0016] According to the pattern difference degree of the abnormal area, determine the existence of road surface diseases in the abnormal area.

[0017] Among them, it includes: if the center point coincides with the position of the vibration signal sampling point, directly use the characteristic difference vector of this sampling point as the target characteristic vector of the center point; if the center point does not coincide with the sampling point position, distribute distance weights according to the distance between the center point and each sampling point, and weight the characteristic difference vectors of the sampling points to obtain the target characteristic vector of the center point.

[0018] Further, the image data for identifying the abnormal road surface area in the method comes from the shooting of a drone and / or the camera monitoring of the road.

[0019] Further, the method for obtaining vibration signal data at multiple different sampling points on the road surface: The vibration sensors include acceleration sensors and / or geophones, and multiple vibration sensors are connected simultaneously by a wireless sensor, and the vibration signal data obtained by the vibration sensors is acquired through multi-channel data collection.

[0020] Further, the image recognition algorithm includes CNN and / or edge detection algorithm, which is used to identify the abnormal road surface area and determine the center point of the abnormal area.

[0021] Further, a frequency domain feature matrix is extracted from the obtained vibration signal, and a feature difference vector of the vibration signal is calculated using the frequency domain feature matrix.

[0022] Further, the method for extracting the frequency domain feature matrix from the obtained vibration signal is as follows:

[0023] During the sampling time, vibration signals are sampled at each sampling point on the road surface, and discrete time-domain signal data is obtained by sampling.

[0024] The data of the time-domain signal is converted into a frequency-domain signal through Fourier transform, where each frequency component corresponds to a complex value, representing the amplitude and phase of that frequency.

[0025] The amplitude of each frequency component is obtained by calculating the modulus of the complex value, and the energy density of the corresponding frequency component is obtained by squaring the amplitude.

[0026] According to the amplitude and energy density, a frequency domain feature matrix is formed. The vibration signal is divided into multiple time windows, and the frequency components and their energy densities under different time windows are obtained respectively. The rows of the frequency domain feature matrix correspond to the respective frequency components, the columns of the frequency domain feature matrix correspond to the respective time windows, and the elements at the respective row and column positions of the frequency domain feature matrix are the numerical values of the energy density of the corresponding frequency component in each time window.

[0027] Converting the time-domain signal into a frequency-domain signal and extracting the amplitude and energy density of the frequency components can directly reflect the disease characteristics. Frequency-domain analysis can filter the influence of non-disease signals (such as environmental vibrations) on detection. Disease characteristics are manifested as significant changes in specific frequency components in the frequency domain, and frequency-domain analysis can more effectively isolate these characteristics. Decomposing the vibration signal into frequency components helps to identify the regular characteristics in complex signals.

[0028] Further, the method for calculating the feature difference vector of the vibration signal using the frequency domain feature matrix is as follows:

[0029] Calculate the eigenvalues of the frequency-domain feature matrix at each sampling point and the eigenvectors corresponding to each eigenvalue respectively;

[0030] According to the magnitudes of the eigenvalues, calculate the average of the eigenvectors corresponding to the largest eigenvalue and the second-largest eigenvalue as the dominant mode vector, and calculate the average of the eigenvectors corresponding to the smallest eigenvalue and the second-smallest eigenvalue as the secondary mode vector;

[0031] Subtract the secondary mode vector from the dominant mode vector to obtain the feature difference vector of the sampling point.

[0032] Based on the frequency-domain feature matrix, calculate the feature difference vector. The difference between the dominant mode vector and the secondary mode vector reflects the characteristic changes of the disease. Through eigenvalue and eigenvector analysis, the significant frequency characteristics of the disease can be accurately located. Eigenvalue and eigenvector analysis reveals the main change directions in the frequency-domain feature matrix, which are highly correlated with the disease pattern. The eigenvectors corresponding to the main eigenvalue and the secondary eigenvalue represent the main components and secondary components of the data, and their difference can significantly distinguish the abnormal pattern.

[0033] Furthermore, calculate the similarity between the target feature vector and the feature difference vectors of other sampling points respectively, determine the maximum and minimum values of the direction consistency and calculate the difference between the two to obtain the pattern difference degree of this abnormal area, specifically including:

[0034] Take the maximum value of the cosine similarities between the target feature vector and the feature difference vectors of the remaining sampling points as the maximum direction consistency, take the minimum value of the cosine similarities between the target feature vector and the feature difference vectors of the remaining sampling points as the minimum direction consistency, and use the difference between the maximum direction consistency and the minimum direction consistency as the pattern difference degree.

[0035] The difference between the maximum direction consistency and the minimum direction consistency reflects the significance of the disease characteristics. A statistical model can be used to dynamically adjust the threshold to adapt to different road conditions and disease types. The pattern difference degree is a quantification of the distribution difference between the target feature vector and the surrounding feature difference vectors, and can clearly distinguish the normal area from the abnormal area.

[0036] Furthermore, according to the pattern difference degree of this abnormal area, the method for determining the existence of pavement diseases in this abnormal area is: if the pattern difference degree is greater than the threshold, it is determined that there are pavement diseases in this abnormal area.

[0037] Preferably, if the pattern difference degree is greater than 1, it is determined that there are pavement diseases in this abnormal area.

[0038] Preferably, a statistical model is used to learn the feature distribution of the targeted feature vector and the feature difference vectors of the remaining sampling points, and the specific value of the pattern difference degree is calculated. Usually, when the targeted feature vector is close to the normal sample, its feature difference vector from other sampling points has a high similarity, and the pattern difference degree is small; if the abnormal area has significant disease characteristics, the feature distribution of its targeted feature vector is significantly different from that of other sampling points, and the pattern difference degree is large.

[0039] The present invention also provides a highway pavement disease detection system, which includes: a processor, a memory, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the steps in the highway pavement disease detection method are implemented. The highway pavement disease detection system can run on computing devices such as desktop computers, laptop computers, mobile phones, palm computers, and cloud data centers. The operable system may include, but is not limited to, a processor, a memory, and a server cluster. The processor executes the computer program and runs in the following units of the system:

[0040] An image recognition unit, configured to recognize the abnormal area of the road surface and determine the center point of the abnormal area, and obtain the data of the vibration signal at multiple different sampling points on the road surface at the sampling time;

[0041] A data acquisition unit, configured to calculate the targeted feature vector of the center point of the abnormal area through the feature difference vector of the vibration signal according to the data of the vibration signal;

[0042] A pattern calculation unit, configured to calculate the similarity between the targeted feature vector and the feature difference vectors of other sampling points respectively, determine the maximum and minimum values of the direction consistency and calculate the difference between the two to obtain the pattern difference degree of the abnormal area;

[0043] A pattern judgment unit, configured to determine the existence of the road surface disease in the abnormal area according to the pattern difference degree of the abnormal area.

[0044] The beneficial effects of the present invention are as follows: The present invention provides a method and system for detecting highway pavement diseases, which identify abnormal areas on the pavement and determine the central points of the abnormal areas, and obtain data of vibration signals at multiple different sampling points on the pavement within the sampling time; according to the data of the vibration signals, through the characteristic difference vectors of the vibration signals, calculate the target characteristic vectors of the central points of the abnormal areas; calculate the similarity between the target characteristic vectors and the characteristic difference vectors of other sampling points respectively, determine the maximum and minimum values of the direction consistency and calculate the difference between the two to obtain the pattern difference degree of the abnormal area; according to the pattern difference degree of the abnormal area, determine the existence of pavement diseases in the abnormal area. By combining image recognition and vibration signal analysis, through the feature extraction of vibration signals and the analysis of pattern difference degree, not only the consumption of computing resources of traditional visual methods is reduced, but also the recognition accuracy and the predictability of detection are significantly improved. This multi-modal combination method is particularly suitable for the disease monitoring requirements of modern highways with large scale, low cost and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] By elaborating on the embodiments shown in the accompanying drawings in detail, the above and other features of the present invention will become more obvious. The same reference numerals in the drawings of the present invention represent the same or similar elements. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings. In the drawings:

[0046] Figure 1 Shown is a flowchart of a method for detecting highway pavement diseases;

[0047] Figure 2 Shown is a system structure diagram of a system for detecting highway pavement diseases. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The following will clearly and completely describe the concept, specific structure and technical effects generated by the present invention in combination with the embodiments and the drawings, so as to fully understand the purpose, solution and effects of the present invention. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0049] In the description of the present invention, the meaning of several is one or more, the meaning of multiple is two or more, and understandings such as greater than, less than, exceeding, etc. do not include the present number, and understandings such as above, below, within, etc. include the present number. If there is a description of first and second, it is only for the purpose of distinguishing technical features and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence relationship of the indicated technical features.

[0050] Such as Figure 1The following is a flowchart of a method for detecting highway pavement diseases according to the present invention. The following will be combined with Figure 1 to elaborate a method and system for detecting highway pavement diseases according to an embodiment of the present invention.

[0051] The present invention proposes a method for detecting highway pavement diseases, and the method specifically includes the following steps:

[0052] Identify the abnormal area of the road surface and determine the center point of the abnormal area, and obtain the data of the vibration signal at multiple different sampling points on the road surface within the sampling time;

[0053] According to the data of the vibration signal, calculate the target feature vector of the center point of the abnormal area through the feature difference vector of the vibration signal.

[0054] Calculate the similarity between the target feature vector and the feature difference vectors of other sampling points respectively, determine the maximum and minimum values of the direction consistency and calculate the difference between the two to obtain the pattern difference degree of the abnormal area.

[0055] Judge the existence of pavement diseases in the abnormal area according to the pattern difference degree of the abnormal area.

[0056] Among them, it may include: if the center point coincides with the position of the vibration signal sampling point, directly use the feature difference vector of this sampling point as the target feature vector of the center point; if the center point does not coincide with the sampling point position, allocate distance weights according to the distance between the center point and each sampling point, and weight the feature difference vectors of the sampling points to obtain the target feature vector of the center point.

[0057] Further, the image data used to identify the abnormal area of the road surface in the method comes from the shooting of an unmanned aerial vehicle and / or the camera monitoring of the highway.

[0058] Further, the method for obtaining vibration signal data at multiple different sampling points on the road surface: the vibration sensor includes an acceleration sensor and / or a geophone, and multiple vibration sensors are connected by a wireless sensor, and the vibration signal data obtained by the vibration sensors is collected through multi-channel data.

[0059] Further, among them, the image recognition algorithm includes CNN and / or edge detection algorithm, which is used to identify the abnormal area of the road surface and determine the center point of the abnormal area.

[0060] Further, among them, a frequency domain feature matrix is extracted from the obtained vibration signal, and the feature difference vector of the vibration signal is calculated using the frequency domain feature matrix.

[0061] Further, among them, the method for extracting the frequency domain feature matrix from the obtained vibration signal is:

[0062] During the sampling time, at each sampling point on the road surface, vibration signals are sampled at each sampling point, and discrete time-domain signal data is obtained through sampling;

[0063] The data of the time-domain signal is converted into a frequency-domain signal through Fourier transform. Among them, each frequency component corresponds to a complex value, representing the amplitude and phase of that frequency;

[0064] The amplitude of each frequency component is obtained by calculating the modulus value of the complex value, and the energy density of the corresponding frequency component is obtained by squaring the amplitude;

[0065] According to the amplitude and energy density, a frequency-domain feature matrix is constructed. Among them, the vibration signal is divided into multiple time windows, and the frequency components and their energy densities under different time windows are respectively obtained. The rows of the frequency-domain feature matrix respectively correspond to each frequency component, the columns of the frequency-domain feature matrix respectively correspond to each time window, and the element at each row and column position of the frequency-domain feature matrix is the value of the energy density of the corresponding frequency component in each time window.

[0066] Furthermore, the method for calculating the feature difference vector of the vibration signal using the frequency-domain feature matrix is as follows:

[0067] Calculate the eigenvalues and eigenvectors corresponding to the eigenvalues of the frequency-domain feature matrix of each sampling point respectively;

[0068] According to the magnitude of the eigenvalues, calculate the average value of the eigenvectors corresponding to the largest eigenvalue and the second largest eigenvalue as the dominant mode vector, and calculate the average value of the eigenvectors corresponding to the smallest eigenvalue and the second smallest eigenvalue as the secondary mode vector;

[0069] Subtract the secondary mode vector from the dominant mode vector to obtain the feature difference vector of the sampling point.

[0070] Furthermore, calculate the similarity between the target feature vector and the feature difference vectors of other sampling points respectively, determine the maximum and minimum values of the direction consistency and calculate the difference between the two to obtain the mode difference degree of this abnormal area, specifically including:

[0071] Take the maximum value of the cosine similarities between the target feature vector and the feature difference vectors of the remaining sampling points as the maximum direction consistency, take the minimum value of the cosine similarities between the target feature vector and the feature difference vectors of the remaining sampling points as the minimum direction consistency, and use the difference between the maximum direction consistency and the minimum direction consistency as the mode difference degree.

[0072] Furthermore, according to the mode difference degree of this abnormal area, the method for determining the existence of road surface diseases in this abnormal area is: if the mode difference degree is greater than the threshold, it is determined that there are road surface diseases in this abnormal area.

[0073] Preferably, if the pattern difference degree is greater than 1, it is determined that there is a road surface disease in the abnormal area.

[0074] Preferably, a statistical model is used to learn the feature distribution of the target feature vector and the feature difference vectors of the remaining sampling points, and the specific value of the pattern difference degree is calculated. Usually, when the target feature vector is close to the normal sample, its feature difference vector from other sampling points has a high similarity, and the pattern difference degree is small; if the abnormal area has significant disease characteristics, the feature distribution of its target feature vector is significantly different from that of other sampling points, and the pattern difference degree is large.

[0075] In some provided embodiments, the types of vibration sensors include, optionally, acceleration sensors for measuring the vibration intensity and frequency characteristics in the vertical direction. While geophones are more suitable for detecting low-frequency vibration signals and are suitable for large-area road surface disease detection. The frequency response range covers at least 0.1 Hz to 100 Hz. The dynamic range adapts to vibrations from weak (such as microcracks) to strong (such as impacts caused by large potholes). Selecting high-sensitivity devices can capture weak signals.

[0076] The data acquisition module supports connecting multiple vibration sensors simultaneously through multi-channel data acquisition. It requires a high sampling frequency (such as above 1 kHz) to ensure that high-frequency details are not lost. Additionally, a noise filtering function should be added to avoid environmental noise interfering with the signal. The device can include a multi-channel data acquisition card (DAQ) and an embedded data acquisition device (portable or fixed).

[0077] The vibration data collected by the communication module is transmitted to the server or local storage device in real time, supporting both wired and wireless transmission methods. For wireless transmission, there should be a Wi-Fi module, Zigbee module, or LoRa communication module. Wired transmission is through a fiber optic network or Ethernet connection.

[0078] The data processing and control system is responsible for controlling the sampling frequency and synchronization timing of the sensors, receiving data in real time and performing preprocessing (such as denoising and filtering). An industrial control computer or a single-chip microcomputer can be used. Edge computing devices are used for data preprocessing and preliminary analysis.

[0079] The number of sampling points is determined according to the road surface area and the requirements for disease detection accuracy. It is recommended to arrange one sensor per 1 to 2 square meters.

[0080] The sampling point distribution can be evenly distributed, which is suitable for large-scale road surface detection, and a grid-like sensor array is arranged; it can also be unevenly distributed, with denser sensors arranged in key areas (such as bridge joints and areas with more potholes). For sampling point positioning, a GPS or high-precision rangefinder is used to accurately calibrate the position of each sensor.

[0081] The sensor can be installed by drilling a hole in the ground in a fixed manner and burying the sensor below the road surface to reduce environmental noise interference; or it can be installed by surface adhesion, using high-strength glue or special fixtures to fix the sensor on the road surface. The installation requirements ensure that the sensor is in full contact with the road surface to avoid loosening. After installation, calibration is carried out to verify the signal accuracy.

[0082] All sensors need to sample synchronously. It is recommended to use a GPS time synchronization module or clock synchronization technology to ensure the consistency of data at different sampling points. Connect all sensors to the central data acquisition system by wired means.

[0083] Or use a wireless communication module to transmit the data collected by each sensor to a unified server.

[0084] Wired power supply is suitable for fixed-installation systems, powered by the main power supply or solar energy. Battery power supply Portable or temporarily installed sensors can use lithium batteries and are equipped with a backup power supply.

[0085] Example 1:

[0086] Record a vibration signal as x(t), where t is the independent variable representing time. For example, in one specific data, x(t) is composed of the superposition of two sine waves: one with a frequency of 5Hz and an amplitude of 2; the other with a frequency of 15Hz and an amplitude of 1.

[0087] The sampling frequency of the signal is 50Hz, 50 data points are collected per second, and sampling is carried out for 1 second, so a total of 50 points are collected.

[0088] Then, collect the time-domain signal. Through sampling, a discrete signal in the time domain can be obtained, including the following:

[0089] x[0]=0, x[1]=2.37, x[2]=1.85, x[3]=-0.62, …

[0090] This signal contains two main frequency components (5Hz and 15Hz) and appears as a complex waveform in the time domain.

[0091] Then perform a fast Fourier transform (FFT) on the time-domain signal. During the transformation, use the fast Fourier transform (FFT) to convert the time-domain signal into a frequency-domain signal. Each frequency component corresponds to a complex value, representing the amplitude and phase of that frequency.

[0092] Calculate the amplitude. By calculating the modulus of the complex number, the amplitude of each frequency component is obtained. For example: for the frequency of 5Hz, the amplitude is 2; for the frequency of 15Hz, the amplitude is 1; the amplitudes of other frequencies are close to 0.

[0093] Then calculate the energy density, which is the square of the amplitude. For example, the energy density at 5 Hz is 2^2 = 4; the energy density at 15 Hz is 1^2 = 1.

[0094] Then, generate the frequency components and the corresponding amplitudes / energies, obtaining the frequency components and the corresponding amplitude / energy densities. For example: at a frequency of 1 Hz, the amplitude is 0.1 and the energy density is 0.01; at a frequency of 5 Hz, the amplitude is 2 and the energy density is 4; at a frequency of 15 Hz, the amplitude is 1 and the energy density is 1.

[0095] Next, construct the frequency-domain feature matrix. In some embodiments, data of multiple time windows are continuously collected. For example, each collection is for 1 second, and the signals of these time windows are respectively subjected to FFT to obtain their respective frequency distribution characteristics. The final result can be organized into a matrix:

[0096] The rows represent the frequency components. For example, f1 = 1 Hz, f2 = 2 Hz, f3 = 5 Hz, and so on;

[0097] The columns represent different time windows. For example, the first column is the frequency distribution of window 1, and the second column is the frequency distribution of window 2.

[0098]

[0099]

[0100] Sample the data therein to form a matrix. Example of the sampling result:

[0101]

[0102] Among them, the first row corresponds to the energy density at frequency 1 Hz, the second row corresponds to the energy density at frequency 5 Hz, the third row corresponds to the energy density at frequency 15 Hz, and each column represents the data of different time windows. The matrix formed by sampling takes the numerical values of the energy densities corresponding to three frequencies for three windows respectively to form a 3×3 matrix. The matrix formed by the sampling data should be a square matrix, and the number of rows and columns should be no less than 3.

[0103] In some embodiments, the matrix can be used for analysis. The main frequency can be extracted to find the frequency with the largest energy density in each column to determine the main frequency component. For example, in window 1, the main frequency is 5 Hz. Frequency band energy analysis can also be performed, dividing the frequency into a low-frequency band (0 - 10 Hz) and a high-frequency band (10 - 20 Hz), and respectively calculating the total energy of these frequency bands for analyzing different road surface disease types. Feature extraction can also be performed. By performing statistical analysis (such as mean, maximum value, variance, etc.) on the matrix, a feature vector is generated for input to subsequent machine learning models.

[0104] Take the values of each row and column in the above matrix to calculate the eigenvalues of the matrix, calculate the eigenvectors corresponding to each eigenvalue, and concatenate the eigenvectors corresponding to each eigenvalue in the order of the numerical values of the eigenvalues to obtain a long vector.

[0105] The eigenvalues are sorted from largest to smallest as: λ1 = 5.0839502, λ2 = 0.05309408, λ3 = -0.02704428;

[0106] The eigenvectors corresponding to each eigenvalue are:

[0107] Corresponding to λ1 = 5.0839502: v1 = [-0.0043, -0.9619, -0.2735];

[0108] Corresponding to λ2 = 0.05309408: v2 = [-0.1317, -0.6556, 0.7436];

[0109] Corresponding to λ3 = -0.02704428: v3 = [-0.3131, 0.8197, -0.4797];

[0110] Example 2.1:

[0111] Concatenate the eigenvectors corresponding to the eigenvalues. Among them, in the order of the magnitudes of the eigenvalues, each eigenvector is concatenated by columns into a long vector:

[0112] V = [-0.0043, -0.9619, -0.2735, -0.1317, -0.6556, 0.7436, -0.3131, 0.8197, -0.4797].

[0113] The largest eigenvalue λ1 = 5.0839502 represents the energy intensity of the main components of the matrix. Here it indicates that most of the energy is concentrated on the corresponding eigenvector (the main component direction).

[0114] The second largest eigenvalue λ2 = 0.05309408 represents the secondary dynamic components, which may be noise or secondary patterns of the signal.

[0115] The negative eigenvalue λ3 = -0.02704428, if it exists, may reflect the instability of the matrix or the existence of reverse dynamic effects in the signal.

[0116] The main eigenvector (v1) represents the variation pattern of the signal in the main direction (main frequency band or main disease feature). It corresponds to the most significant pattern of the disease in the signal, such as the energy contribution at the main frequency of 5 Hz. The secondary eigenvectors (v2, v3, etc.) represent secondary variation patterns, which may be related to background noise, random perturbations, or secondary frequency characteristics. For example, high-frequency diseases (such as microcracks) or low-frequency noise characteristics.

[0117] Regarding the positive and negative values of the eigenvector, the positive and negative values in the eigenvector represent the mutual relationship between different components in the matrix. For example, a positive value and a negative value may represent signal components that weaken each other.

[0118] Example 2.2:

[0119] Instead of concatenating the eigenvectors corresponding to each eigenvalue, the vector obtained by taking the point-by-point numerical average of the eigenvector corresponding to the numerically largest (maximum) eigenvalue and the eigenvector corresponding to the numerically second-largest (second-largest) eigenvalue in each dimension is denoted as V12. The vector obtained by taking the point-by-point numerical average of the eigenvector corresponding to the numerically smallest (minimum) eigenvalue and the eigenvector corresponding to the numerically second-smallest (second-smallest) eigenvalue in each dimension is denoted as V23. The numerical vector obtained by subtracting V23 from V12 in each dimension is used as Vgap.

[0120] In one embodiment, the average V12 of the eigenvectors corresponding to the largest and second-largest eigenvalues is: V12 = 1 / 2×(v1 + v2) = [-0.0680, -0.80875, 0.23505];

[0121] The average V23 of the eigenvectors corresponding to the second-largest and smallest eigenvalues:

[0122] V23 = 1 / 2×(v2 + v3) = [-0.2224, 0.08205, 0.13195];

[0123] Vgap (the resulting vector of V12 minus V23):

[0124] Vgap = V12 - V23 = [0.1544, -0.8908, 0.1031];

[0125] V12 represents the intermediate pattern of the dominant feature. V12 reflects the comprehensive performance of the vibration patterns corresponding to the largest and second-largest eigenvalues, and can be understood as the core feature of the main components of the signal.

[0126] V23 represents the intermediate pattern of the secondary feature. V23 reflects the comprehensive pattern of the smallest and second-smallest eigenvalues, and may contain noise or secondary features in the signal.

[0127] Vgap represents the difference between the main component and the secondary component. Vgap represents the significant difference between the main mode and the secondary mode, and can be used to distinguish the key disease mode from the background mode.

[0128] In some embodiments, the method for detecting highway pavement diseases may specifically include:

[0129] Using an image recognition algorithm to identify the abnormal areas on the road surface;

[0130] Select the center point of the abnormal area;

[0131] Calculate the Euclidean distance between the center point of the abnormal area and each vector sampling point, and calculate the vector corresponding to the center point of the abnormal area.

[0132] For example, if the coordinates of the center point of the abnormal area coincide with those of a vector sampling point, then the Vgap vector corresponding to the coincident vector sampling point is used as the feature vector corresponding to the center point of the abnormal area, which is called Vtarget;

[0133] Alternatively, sort each vector sampling point in ascending order of its Euclidean distance from the center point of the abnormal area. Using the distance as the weight, the closer the distance, the greater the weight. The weight is a floating point number between 0 and 1, and the weight can be calculated through a normalization algorithm based on the Euclidean distance. Calculate the weighted sum of the Vgap corresponding to each sampling point by dimension according to the weight and then take the average value to form the mean vector as the feature vector Vtarget corresponding to the center point of the abnormal area;

[0134] Calculate the cosine similarity between Vtarget and the Vgap corresponding to each sampling point except itself respectively. Take the maximum value of the cosine similarity as maxDir, take the minimum value of the cosine similarity as minDir, calculate the difference gapDir between maxDir and minDir. If gapDir is greater than 1, then the abnormal area is detected as having a pavement disease.

[0135] Or for another example, each vector sampling point is the sampling point for calculating the Vgap vector by respectively obtaining vibration signals on the road surface. The distances between the sampling points are equal. Sort each vector sampling point in ascending order of its Euclidean distance from the center point of the abnormal area, and obtain the three sampling points with the smallest distances among the top three. Use the mean vector formed by taking the average value of the Vgap corresponding to these three sampling points by dimension as the feature vector Vtarget corresponding to the center point of the abnormal area;

[0136] Calculate the cosine similarity between Vtarget and the Vgap corresponding to these three sampling points respectively. Take the maximum value of the cosine similarity as maxDir, and take the minimum value of the cosine similarity as minDir. Calculate the difference gapDir between maxDir and minDir. If gapDir is greater than 1, then this abnormal area is detected as having road surface diseases.

[0137] Vtarget is the target feature vector, Vgap is the feature difference vector, V12 is the dominant mode vector, V23 is the secondary mode vector, gapDir is the mode difference degree, maxDir is the maximum direction consistency, minDir is the minimum direction consistency, and wi is the distance weight.

[0138] Using pure computer vision methods, it is necessary to continuously collect and process image data for the entire highway area. The computational overhead of image processing (especially deep learning models such as CNN) is huge. Since the visual model needs to run for a long time, the demand for hardware resources is relatively high, such as the long-term occupation of the GPU. The local visual detection of the present invention only locates the suspected disease area through the image recognition algorithm, rather than continuously analyzing the images of the entire highway. After locating the area, analyze the disease characteristics through the feature difference vector of the vibration signal, reducing the dependence on image processing. The computational resources required for vibration signal analysis are much lower than those for continuous image analysis, so the overall computational resources are significantly reduced.

[0139] Image recognition algorithms are easily affected by environmental lighting, weather conditions (such as rainy days or foggy days), and changes in road surface materials, which may lead to misjudgments. When visual algorithms identify minor diseases (such as microcracks), usually higher-resolution images are required, increasing the difficulty of data processing.

[0140] The improved vibration signal analysis of the present invention compensates for the deficiencies of image recognition. Through vibration signal features (such as frequency domain feature matrices and feature difference vectors), road surface diseases can be accurately detected. Even in the case of minor diseases or unclear surface visual features, they can still be effectively identified. Vibration signals are not affected by environmental lighting and can operate stably at night or under bad weather conditions. Combining visual and vibration features and fusing multi-modal information enhance the accuracy of the detection results.

[0141] Due to the lag of visual methods, image recognition can usually only detect diseases when they have already appeared significantly, such as obvious cracks, potholes, etc. For early structural problems that have not yet manifested as surface diseases, visual methods may be completely unable to detect them. The vibration signal of the present invention can capture changes in the road surface structure that have not yet manifested as obvious surface diseases, such as internal stress concentration points or slight uneven settlements. Through pattern difference analysis, possible problem areas can be predicted in the early stage of the disease, that is, when the impact of the disease has not yet appeared on the surface. The present invention can not only identify existing diseases but also provide early warnings, thus supporting a more proactive road maintenance strategy.

[0142] In actual application scenarios, there is no need to continuously capture and process images of the entire highway. Only key areas (such as bridges, sharp turns, etc.) or suspected disease areas need to be detected using image recognition combined with vibration signals. This reduces the deployment pressure of hardware devices and is more suitable for large-scale highway network monitoring. It saves computing resources and hardware costs. Vibration sensors are relatively inexpensive and are convenient for large-scale deployment. The combined use of image and vibration sensors can flexibly expand the detection range while avoiding excessive resource consumption. By detecting and accurately identifying diseases at an early stage, road surface diseases can be maintained and repaired in a timely manner, avoiding the further expansion of diseases and extending the service life of the highway.

[0143] In the test, the method combining image recognition and vibration signal analysis showed obvious advantages compared with the method of simply using image recognition: the comprehensive method achieved an accuracy rate of 95% (compared with 85% of image recognition), was particularly outstanding in the ability of early detection (that is, when the disease has not fully manifested), with a 30% increase in the advance rate, and at the same time, the consumption of computing resources was reduced by about 40%, fully demonstrating the superiority of the present invention in terms of efficiency and effect.

[0144] The described highway pavement disease detection system runs on any computing device such as a desktop computer, laptop, mobile phone, personal digital assistant, or cloud data center. The computing device includes: a processor, a memory, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps in the described highway pavement disease detection method. The operable system may include, but is not limited to, a processor, a memory, and a server cluster.

[0145] An embodiment of the present invention provides a highway pavement disease detection system, as Figure 2 shown. The highway pavement disease detection system of this embodiment includes: a processor, a memory, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, it implements the steps in the above-mentioned embodiment of the highway pavement disease detection method to control the system of the present invention. The processor executes the computer program and runs in the following units of the system:

[0146] An image recognition unit, configured to recognize an abnormal area on the road surface, determine the center point of the abnormal area, and obtain data of vibration signals at multiple different sampling points on the road surface within the sampling time;

[0147] A data acquisition unit, configured to calculate a target feature vector of the center point of the abnormal area according to the data of the vibration signals through a feature difference vector of the vibration signals;

[0148] A pattern calculation unit, configured to calculate similarities between the target feature vector and feature difference vectors of other sampling points respectively, determine the maximum value and the minimum value of the direction consistency and calculate the difference between the two, so as to obtain the pattern difference degree of the abnormal area;

[0149] A pattern judgment unit, configured to judge the existence of road surface diseases in the abnormal area according to the pattern difference degree of the abnormal area.

[0150] Preferably, for all undefined variables in the present invention, if there is no clear definition, they can all be manually set thresholds.

[0151] Among them, dimensionless numerical calculations are adopted between physical quantities of different units.

[0152] The described highway road surface disease detection system can run on computing devices such as desktop computers, laptop computers, mobile phones, palm computers, and cloud data centers. The described highway road surface disease detection system includes, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above examples are only examples of a highway road surface disease detection method and system, and do not constitute a limitation on a highway road surface disease detection method and system. It may include more or fewer components than the examples, or combine some components, or different components. For example, the described highway road surface disease detection system may further include input / output devices, network access devices, buses, etc.

[0153] The so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete component gate circuits, or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the highway pavement disease detection system, and connects each sub-region of the entire highway pavement disease detection system through various interfaces and lines.

[0154] The memory can be used to store the computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory, and by calling the data stored in the memory, the processor realizes various functions of the highway pavement disease detection method and system. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0155] Although the description of the present invention has been quite detailed and several embodiments have been particularly described, it is not intended to be limited to any of these details or embodiments or any particular embodiment, so as to effectively cover the intended scope of the present invention. In addition, the present invention has been described above with embodiments foreseeable by the inventor for the purpose of providing a useful description, and non-substantive changes to the present invention that are not currently foreseeable may still represent equivalent changes to the present invention.

Claims

1. A method for detecting road pavement defects, characterized in that: The method comprises the following steps: Based on the image recognition algorithm, the abnormal area of ​​the road surface is identified and the center point of the abnormal area is determined; and obtaining vibration signal data at multiple different sampling points on the road surface during the sampling time; According to the data of the vibration signal, the targeted feature vector of the center point of the abnormal area is calculated through the feature difference vector of the vibration signal, including: if the center point coincides with the position of the vibration signal sampling point, the feature difference vector of the sampling point is directly used as the targeted feature vector of the center point; if the center point and the sampling point do not coincide with each other, a distance weight is allocated according to the distance between the center point and each sampling point, and the feature difference vector of the sampling point is weighted to obtain the targeted feature vector of the center point; The similarity between the targeted feature vector and the feature difference vectors of other sampling points is calculated respectively, the maximum and minimum values ​​of the directional consistency are determined and the difference between the two is calculated to obtain the pattern difference of the abnormal area; The presence of pavement damage in the abnormal area is determined based on the pattern difference of the abnormal area.

2. A method for detecting road pavement defects according to claim 1, characterized in that: The image data used to identify abnormal road surface areas in the method comes from drone photography and / or camera monitoring of the highway.

3. A method for detecting road pavement defects according to claim 1, characterized in that: A method for obtaining vibration signal data at multiple different sampling points on a road surface: the vibration sensor includes an acceleration sensor and / or a seismic detector, multiple vibration sensors are simultaneously connected with a wireless sensor, and the vibration signal data obtained by the vibration sensor is collected through multi-channel data.

4. A method for detecting road pavement defects according to claim 1, characterized in that: in, The image recognition algorithm includes a CNN and / or an edge detection algorithm, which is used to identify abnormal road surface areas and determine the center point of the abnormal area.

5. A method for detecting road pavement defects according to claim 1, characterized in that: in, A frequency domain feature matrix is ​​extracted from the obtained vibration signal, and a feature difference vector of the vibration signal is calculated using the frequency domain feature matrix.

6. A method for detecting road pavement defects according to claim 5, characterized in that: in, The method to extract the frequency domain feature matrix from the obtained vibration signal is: At each sampling point on the road surface during the sampling time, the vibration signal is sampled at each sampling point to obtain discrete time domain signal data; The data of the time domain signal is converted into a frequency domain signal through Fourier transform, where each frequency component corresponds to a complex value representing the amplitude and phase of the frequency; The amplitude of each frequency component is obtained by calculating the modulus of the complex value, and the energy density of the corresponding frequency component is obtained by squaring the amplitude; A frequency domain feature matrix is ​​constructed according to the amplitude and energy density, wherein the vibration signal is divided into multiple time windows and the frequency components and their energy densities in different time windows are obtained respectively. The rows of the frequency domain feature matrix correspond to the frequency components respectively, and the columns of the frequency domain feature matrix correspond to the time windows respectively. The elements at each row and column position of the frequency domain feature matrix are the values ​​of the energy density of the corresponding frequency components in each time window.

7. A method for detecting road pavement defects according to claim 5, characterized in that: in, The method of using the frequency domain feature matrix to calculate the characteristic difference vector of the vibration signal is: Calculate the eigenvalues ​​of the frequency domain feature matrix of each sampling point and the eigenvectors corresponding to the eigenvalues ​​respectively; According to the size of the eigenvalue, the average values ​​of the eigenvectors corresponding to the largest eigenvalue and the second largest eigenvalue are calculated as the dominant mode vector, and the average values ​​of the eigenvectors corresponding to the smallest eigenvalue and the second smallest eigenvalue are calculated as the secondary mode vector; Subtract the dominant mode vector from the secondary mode vector to obtain the characteristic difference vector of the sampling point.

8. A method for detecting road pavement defects according to claim 5, characterized in that: The similarity between the target feature vector and the feature difference vectors of other sampling points is calculated respectively, the maximum and minimum values ​​of the directional consistency are determined and the difference between the two is calculated to obtain the pattern difference of the abnormal area, which includes: The maximum value of the cosine similarity between the targeted feature vector and the feature difference vectors of the remaining sampling points is taken as the maximum directional consistency, the minimum value of the cosine similarity between the targeted feature vector and the feature difference vectors of the remaining sampling points is taken as the minimum directional consistency, and the difference between the maximum directional consistency and the minimum directional consistency is taken as the pattern difference.

9. A method for detecting road pavement defects according to any one of claims 5 to 8, characterized in that: According to the pattern difference of the abnormal area, the method of determining the existence of the road surface disease in the abnormal area is as follows: if the pattern difference is greater than a threshold value, it is determined that the road surface disease exists in the abnormal area.

10. A highway pavement disease detection system, characterized in that: The highway pavement defect detection system runs in any computing device such as a desktop computer, a laptop computer or a cloud data center. The computing device includes: a processor, a memory and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps in the highway pavement defect detection method as described in any one of claims 1 to 8 are implemented.

Images