Intelligent identification method for hidden pavement diseases based on deep learning

Abstract

The present application relates to the technical field of pavement hidden disease intelligent identification, and specifically discloses a pavement hidden disease intelligent identification method based on deep learning, which comprises the following steps: generating an abnormal vibration trigger instruction, directionally collecting multi-modal data, generating a structural response characteristic spectrum, generating an enhanced feature map, generating a multi-modal joint diagnosis map, intelligently identifying hidden diseases, and adopting a closed-loop optimization and active verification mechanism; the abnormal vibration trigger instruction is generated by using multi-source vibration sensing data, the multi-modal data is directionally collected, the structural response characteristic spectrum and the enhanced feature map are generated, and the multi-modal joint diagnosis map is generated by fusion, so that the intelligent identification of hidden diseases is realized; meanwhile, the closed-loop optimization and active verification mechanism is adopted to dynamically adjust the collection parameters, the deep coupling of information at the feature level is realized, the single sensor misjudgment is effectively eliminated, different types of hidden diseases are accurately distinguished, and the two-stage detection strategy is adopted to greatly reduce the amount of original data and improve the detection efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent identification of hidden pavement diseases, in particular to an intelligent identification method of hidden pavement diseases based on deep learning. BACKGROUND

[0002] As the lifeline of national economic development, the safety and smoothness of road infrastructure is of great importance. Under the joint action of long-term traffic load and natural environmental factors, various diseases will inevitably occur in the pavement structure. Compared with surface diseases such as cracks and potholes, hidden diseases in the internal structure of the pavement, such as the void between the base and the surface, the void formed in the roadbed or the base, and the internal looseness of the structural material, have stronger potential harmfulness. These diseases usually do not leave obvious traces on the surface of the pavement at the early stage, and are difficult to be found by conventional visual inspection methods. However, they will continuously weaken the bearing capacity and overall stability of the pavement structure, and with the passage of time, under the repeated rolling of heavy vehicles, they may suddenly evolve into serious pavement subsidence, collapse and other malignant accidents, posing a great threat to traffic safety, and the repair cost in the later stage is also extremely high. Therefore, to realize the early, accurate and efficient detection of hidden pavement diseases is a key technical problem to be solved in the modern road maintenance and management system.

[0003] The common drawback of the prior art is the dependence on a single physical information source. Whether it is electromagnetic wave, sound wave or thermal radiation, the information expression ability of a single modal data is limited when facing complex and diverse pavement structures and disease forms, which directly leads to the multi-solution and uncertainty of the detection result. At the same time, most of the existing technologies lack an intelligent data processing and decision-making framework, and the data collection and analysis process is relatively independent. Once the sensor parameters are set, they are fixed throughout the process, and cannot be dynamically adjusted according to the real-time findings in the detection process. This "open-loop" detection mode not only causes a large amount of redundant data collection, but also makes it impossible to automatically conduct detailed investigation when encountering suspected disease areas, thereby affecting the reliability of the final diagnosis. SUMMARY

[0004] In view of this, in order to solve the problems raised in the background art, an intelligent identification method of hidden pavement diseases based on deep learning is proposed.

[0005] The purpose of the present application can be achieved by the following technical solutions: The present application provides an intelligent identification method of hidden pavement diseases based on deep learning, comprising the following steps: S1, generating an abnormal vibration trigger instruction: acquiring multi-source vibration sensing data in the driving process of a pavement detection vehicle, and generating an abnormal vibration trigger instruction based on the multi-source vibration sensing data.

[0006] S2, multi-modal data directional collection: based on the abnormal vibration trigger instruction, directional collection of multi-modal data of the road surface is performed to generate a time-space aligned original data set.

[0007] S3, structure response characteristic spectrum generation: the sound wave reflection signal in the original data set is analyzed to generate a structure response characteristic spectrum.

[0008] S4, enhanced feature map generation: the high-definition texture image in the original data set is processed, and the vibration feature in the abnormal vibration trigger instruction is combined to generate a vibration-related enhanced feature map.

[0009] S5, multi-modal joint diagnosis map generation: the structure response characteristic spectrum and the vibration-related enhanced feature map are fused to generate a multi-modal joint diagnosis map.

[0010] S6, hidden disease intelligent identification: hidden disease classification results are output by identifying the disease feature mode in the multi-modal joint diagnosis map, wherein the hidden disease types include a cavity mode, a void mode and a loose mode.

[0011] S7, closed-loop optimization and active verification: based on the matching distance value in the hidden disease classification results, an active verification instruction is generated to close-loop optimize the collection parameters.

[0012] Compared with the prior art, the embodiments of the present application have at least the following advantages or beneficial effects: (1) the present application creatively proposes a multi-modal data deep fusion diagnosis mechanism, through a series of ingenious algorithm design, realizes the deep coupling of information at the feature level, and further dynamically adjusts the expression of the visual feature map by taking the sound wave response feature representing the internal structure density as a weight coefficient, so that the small changes of the internal structure can enhance or suppress the saliency of the surface texture. This cross-modal information complementary and feature enhancement mechanism builds a more rich and robust diagnosis basis than a single information source, which can effectively eliminate the misjudgment of a single sensor, so as to more accurately distinguish different types of hidden diseases such as cavities and voids under complex road conditions, and significantly reduce the probability of false negatives and false positives.

[0013] (2) The present application adopts a two-stage detection strategy of "vibration preliminary screening and directional detailed investigation". At the normal driving speed of the vehicle, a high-sensitivity vibration sensor array is used for continuous and rapid preliminary scanning, and once an abnormal vibration signal is found, the system generates a trigger instruction to accurately guide the high-definition camera and ultrasonic equipment to conduct high-resolution multi-modal data acquisition on the specific area. This strategy avoids indiscriminate high-precision data acquisition on the entire road section, greatly reduces the amount of raw data to be processed, concentrates valuable computing resources and time on the most valuable areas, thereby realizing revolutionary improvement of detection efficiency on the premise of ensuring detection accuracy, greatly reducing the dependence on manual operation and back-end analysis, and promoting the development of pavement detection towards high efficiency and automation.

[0014] (3) After identifying the disease from the joint diagnosis graph, the present application evaluates the reliability of the current diagnosis according to the matching distance value of the matching result. For the low confidence area, the system does not simply mark it as uncertain, but actively identifies it as a target that needs further verification. Subsequently, based on the preliminary diagnosis information of the area, the acquisition parameters of the front-end sensor are intelligently and automatically adjusted, and after the adjustment, the system starts the secondary acquisition and analysis process of the area. This closed-loop design of active verification and parameter self-optimization ensures that the detection system can dynamically adapt to the field conditions, continuously optimize the data quality, and continuously improve the diagnosis ability of difficult diseases, forming an intelligent evolution system that gets more and more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0016] Figure 1 The present application is a method step flowchart.

[0017] Figure 2 The present application is a schematic diagram of the sound wave signal reflected from the inside of the road surface.

[0018] Figure 3 The present application is an energy attenuation gradient curve.

[0019] Figure 4 The present application is a schematic diagram of the cavity mode in the hidden disease type.

[0020] Figure 5 The present application is a schematic diagram of the void mode in the hidden disease type.

[0021] Figure 6A schematic diagram of a loose mode in a concealed disease type of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0023] Please refer to Figure 1 The present application provides a pavement concealed disease intelligent identification method based on deep learning, which comprises the following steps: S1, generating an abnormal vibration trigger instruction: acquiring multi-source vibration sensing data in the driving process of a pavement detection vehicle, and generating an abnormal vibration trigger instruction based on the multi-source vibration sensing data.

[0024] In specific embodiments of the present application, the specific process of generating an abnormal vibration trigger instruction based on multi-source vibration sensing data is as follows: through a vibration sensor array arranged on the chassis of the detection vehicle, real-time collection of three-axis vibration waveform signals of each measuring point on the pavement.

[0025] It should be noted that each measuring point measures acceleration data in the x-axis, y-axis and z-axis directions through multiple sensors to form a time series of three-axis vibration waveform signals.

[0026] The energy distribution characteristics of the three-axis vibration waveform signals in a preset specific frequency band are analyzed, and when the energy peak value continuously exceeds the preset dynamic threshold value, the corresponding measuring point is marked as an abnormal measuring point.

[0027] It should be noted that for the three-axis data of each measuring point, the vibration energy value of the preset specific frequency band is calculated respectively. After the time domain signal is converted into a frequency domain signal by fast Fourier transform, the amplitude square sum and average value of all frequency points in the frequency band are calculated, and then the maximum energy value in the frequency band is identified as the energy peak value. When the energy peak value of a certain measuring point continuously exceeds the preset dynamic threshold value, the measuring point is marked as an abnormal measuring point.

[0028] It should be further noted that the specific way of calculating the energy peak value of the preset specific frequency band is as follows: the vibration waveform signal is defined as an acceleration sequence , and the Fourier transform thereof is . For a frequency band , the energy peak value , wherein is the average energy value of the reference frequency band, which can be taken as by experiment.

[0029] In one specific embodiment of the present application, the preset dynamic threshold refers to a variable comparison value, and a specific value of the variable comparison value is dynamically adjusted according to a road surface material type, wherein a dynamic threshold corresponding to each road surface material type is obtained through experimental analysis based on a plurality of groups of industrial sensor measured data.

[0030] The position coordinates of the abnormal measurement points are fused with the impact response spectrum to generate an abnormal vibration trigger instruction containing spatial positioning and vibration characteristics.

[0031] It should be noted that the position coordinates of all abnormal measurement points and the corresponding impact response spectrum, i.e., the frequency-amplitude spectrum extracted from the vibration waveform signal, are fused to generate an abnormal vibration trigger instruction file containing spatial positioning information and vibration characteristics, wherein the vibration characteristics include energy peak value and spectrum shape.

[0032] S2, multi-modal data directional collection: based on the abnormal vibration trigger instruction, directional collection of road multi-modal data is performed to generate a spatio-temporal aligned original data set.

[0033] In specific embodiments of the present application, the specific process of generating the spatio-temporal aligned original data set is as follows: according to the positioning coordinates in the abnormal vibration trigger instruction, a high-speed camera is controlled to capture a high-definition texture image of the target area.

[0034] It should be noted that after receiving the abnormal vibration trigger instruction generated in step S1, the instruction contains the positioning coordinates of the abnormal measurement points and the vibration characteristics, and according to the positioning coordinates in the trigger instruction, a high-speed camera installed on a road detection vehicle is controlled to align the camera lens to the target area to capture a high-definition texture image of the road surface of the area.

[0035] The ultrasonic transducer is synchronously activated to emit a sweep frequency sound wave to the target area and receive a reflected sound wave signal.

[0036] Please refer to Figure 2 It should be noted that the ultrasonic transducer device installed on the detection vehicle is synchronously activated to emit a sweep frequency sound wave to the same target area and receive a reflected sound wave signal returned from the inside of the road surface.

[0037] The high-definition texture image, the reflected sound wave signal, and the vibration characteristics in the abnormal vibration trigger instruction are bound with the same time stamp and spatial coordinates to generate a spatio-temporal aligned original data set.

[0038] It should be noted that the captured high-definition texture image, the received reflected sound wave signal, and the vibration characteristics in the abnormal vibration trigger instruction are added with the same system time stamp and spatial coordinates through a data binding module to generate a spatio-temporal aligned original data set.

[0039] S3, structure response characteristic spectrum generation: analyzing the reflected acoustic wave signal in the original data set to generate a structure response characteristic spectrum.

[0040] In specific embodiments of the present application, the specific process of generating the structure response characteristic spectrum is: frame processing of the reflected acoustic wave signal, and extracting the frequency domain energy distribution of each frame.

[0041] It should be noted that the reflected acoustic wave signal is extracted from the original data set, which is a time-domain amplitude data sequence. First, the reflected acoustic wave signal is frame processed: the signal is divided into multiple time frames, and the frequency domain energy distribution of each frame is extracted: the fast Fourier transform is performed on each frame signal, and the modulus square value of the transform result is calculated to obtain the energy value distribution of each frame on the frequency axis, i.e. the energy amplitude of each frequency point.

[0042] Please refer to Figure 3 The energy attenuation gradient curve of the acoustic wave in the road surface medium is calculated.

[0043] In specific embodiments of the present application, the specific way of calculating the energy attenuation gradient curve of the reflected acoustic wave in the road surface medium is: based on the propagation speed of the reflected acoustic wave in the road surface medium, converting the time axis into the propagation distance axis, calculating the attenuation coefficient value of each frequency point by fitting the exponential decay model, forming the gradient curve of the attenuation coefficient with the frequency change, and thus obtaining the energy attenuation gradient curve.

[0044] It should be further noted that the specific way of calculating the energy attenuation gradient is: let the energy value in the frequency domain energy distribution be , the time variable be , and the acoustic wave propagation speed be . The attenuation coefficient is calculated by fitting the data points: , wherein is the change amount of the energy logarithm, is the time interval, and the value is obtained by using the linear regression method to minimize the residual error.

[0045] Fusing the energy attenuation gradient curve and the frequency domain energy distribution generates a structure response characteristic spectrum representing the internal structure density.

[0046] It should be noted that the process of fusing the energy attenuation gradient curve and the frequency domain energy distribution is: combining the attenuation coefficient value of the energy attenuation gradient curve with the energy value of the corresponding frequency point to generate a two-dimensional graph, the horizontal axis of which is the frequency, the vertical axis of which is the energy value, and the attenuation coefficient is color-mapped, thereby representing the change of the road internal structure density, and obtaining the structure response characteristic spectrum.

[0047] S4, enhanced feature map generation: process the high-definition texture image in the original data set, and combine the vibration features in the abnormal vibration triggering instruction to generate a vibration-related enhanced feature map.

[0048] In specific embodiments of the application, the specific process of generating the vibration-related enhanced feature map is: detecting the crack orientation distribution and local deformation features in the high-definition texture image.

[0049] It should be noted that the specific way of detecting the crack orientation distribution and local deformation features in the high-definition texture image is: using an edge detection algorithm to identify the linear structure in the image, calculating the main direction angle of each crack to form an orientation distribution map; and calculating the deformation vector of adjacent pixel blocks through local area pixel displacement analysis to obtain a local deformation feature map.

[0050] It should be further noted that the specific way of calculating the local deformation features is: letting the reference block pixel coordinates be , the current block be , and the deformation vector be , wherein, represents the block pixel number, , represents the number of block pixels.

[0051] The parameters of the edge detection algorithm are adaptively adjusted in combination with the vibration wave propagation direction in the abnormal vibration triggering instruction, so that the crack detection direction is aligned with the vibration wave propagation direction.

[0052] It should be noted that the parameters of the edge detection algorithm are adaptively adjusted in combination with the vibration wave propagation direction in the abnormal vibration triggering instruction, i.e. the main propagation axis direction in the vibration features generated by S1, so that the crack detection sensitive direction is aligned with the main propagation axis direction according to the vibration wave propagation direction angle, and the convolution kernel direction of the edge detection operator is rotated to make the crack detection sensitive direction consistent with the main propagation axis direction of the vibration wave.

[0053] It should be further noted that the rotation of the convolution kernel direction of the edge detection operator means that the convolution kernel used for edge detection is angle-rotated according to the vibration wave propagation direction angle. Through such rotation adjustment, the sensitive direction of crack detection can be made consistent with the main propagation axis direction of the vibration wave, so that the edge detection algorithm can better capture the crack feature information related to the main propagation direction of the vibration wave during crack detection, thereby improving the accuracy and pertinence of crack detection.

[0054] The aligned crack orientation distribution and local deformation features are superimposed to generate a vibration-related enhanced feature map.

[0055] It should be noted that the crack orientation distribution and local deformation feature map after direction alignment are pixel-level superimposed to generate an enhanced feature map that fuses crack direction information and deformation data.

[0056] For example, the original data set texture image shows a longitudinal crack, and the vibration propagation direction triggered by the instruction is 90 degrees. The initial crack direction distribution shows that the main direction is 85 degrees, and the local deformation feature shows that the maximum displacement is 0.8 pixels. The edge detection parameters are adjusted, the detection kernel direction is constrained to be in the range of 75-105 degrees, and the crack direction distribution is extracted again to be 89 degrees. After alignment, the enhanced feature map is generated by superposition, and it is shown that the crack extends along the 90-degree direction.

[0057] S5, multi-modal joint diagnosis map generation: fusion of structure response feature spectrum and vibration-related enhanced feature map to generate a multi-modal joint diagnosis map.

[0058] In specific embodiments of the present application, the specific process of generating a multi-modal joint diagnosis map is to map the structure response feature spectrum to the spatial coordinate system of the vibration-related enhanced feature map.

[0059] It should be noted that the specific way of mapping the structure response feature spectrum to the spatial coordinate system of the vibration-related enhanced feature map is to align each data point of the structure response feature spectrum to the corresponding pixel coordinate of the image according to the geographical coordinates and pixel positions of the enhanced feature map, and to achieve spatial consistency through a coordinate conversion algorithm, wherein the coordinate conversion algorithm uses bilinear interpolation to achieve accurate alignment.

[0060] It should be further noted that the bilinear interpolation used is that, based on the correspondence between the geographical coordinates and pixel positions of the enhanced feature map, for each data point to be mapped in the structure response feature spectrum, the four adjacent known pixel points in the pixel coordinate system of the enhanced feature map are first determined, and then the accurate value of the to-be-mapped point at the corresponding pixel coordinate of the enhanced feature map is calculated through linear weighting according to the values of the four pixel points and the distances of the to-be-mapped point from them in the horizontal and vertical directions, so as to achieve accurate alignment of the structure response feature spectrum data points and the image pixel coordinates and achieve spatial consistency.

[0061] The medium density value in the structure response feature spectrum is used as a weight coefficient to dynamically adjust the gray value of the corresponding pixel of the vibration-related enhanced feature map.

[0062] It should be noted that the medium density value specifically refers to the normalized attenuation coefficient value.

[0063] In specific embodiments of the present application, the specific way of dynamically adjusting the gray value of the corresponding pixel of the vibration-related enhanced feature map is to extract the normalized attenuation coefficient value of each pixel position, multiply it by the original gray value of the enhanced feature map, and obtain the adjusted gray value.

[0064] It should be further noted that the original gray value of the enhanced feature map is , the medium density value of the structural response feature spectrum is a normalized weight coefficient , the adjusted gray value . Wherein The normalized attenuation coefficient is obtained as follows: , is an attenuation coefficient, and The minimum and maximum attenuation coefficient values are calibrated based on multiple sets of road surface sound wave experimental data.

[0065] A multi-modal joint diagnostic map that fuses the sound wave structural features and visual deformation features is generated by weighted superposition.

[0066] It should be noted that by weighted superposition operation, the adjusted gray value is combined with the original deformation feature to generate a multi-modal joint diagnostic map that fuses the sound wave structural features and visual deformation features that can reflect the internal density. The map is in a composite image format, wherein the visual deformation features include surface cracks and displacement.

[0067] It should be further noted that the weighted superposition operation specifically refers to: according to the pre-set weight coefficient, the gray value corresponding to the adjusted sound wave structural feature reflecting the internal density and the numerical value of the visual deformation features such as surface cracks and displacement are weighted, and then the weighted sound wave structural feature gray value and visual deformation feature numerical value are superimposed and fused at the corresponding pixel position, thereby generating a multi-modal joint diagnostic map that can simultaneously reflect internal density information and surface crack and displacement information. The pre-set weight coefficient needs to consider multiple factors, such as the importance of sound wave structural features and visual deformation features in overall diagnosis, the needs of different application scenarios, experimental data and experience, etc. The appropriate weight value needs to be determined through a large number of experiments, data analysis and optimization according to the specific detection object, detection environment and expected diagnostic effect.

[0068] For example, if the structural response feature spectrum shows an attenuation coefficient of 0.05 per meter at a coordinate of 30.5 degrees north latitude and 120.3 degrees east longitude, the normalized weight coefficient (hypothetical =0.1)). The enhanced feature map shows an original gray value of 150 at the same coordinate. The adjusted gray value The weighted superposition generates a joint diagnostic map, which shows that the gray value at this point is reduced, emphasizing the influence of acoustic density features on crack appearance.

[0069] Please refer to Figure 4 , Figure 5 and Figure 6 , S6, concealed disease intelligent identification: identifying disease feature patterns in the multi-modal joint diagnostic map, outputting concealed disease classification results, wherein the concealed disease types include hollow pattern, empty pattern and loose pattern.

[0070] In specific embodiments of the present application, the specific process of outputting the hidden disease classification result is: extracting the topological structure features and energy distribution features of the abnormal area in the multi-modal joint diagnosis graph to generate a to-be-identified feature vector.

[0071] It should be noted that the specific way of identifying the abnormal area from the multi-modal joint diagnosis graph is: segmenting the pixel group with a gray value higher than a set gray threshold in the multi-modal joint diagnosis graph, and recording the area composed of the pixel group as the abnormal area.

[0072] In one specific embodiment of the present application, the set gray threshold value can be 120, which is set according to the optimization experiment based on multiple sets of diagnosis graph test images.

[0073] It should be further noted that the continuous skeleton length and the average value of the main bending angle of the crack in each abnormal area are obtained as the topological structure features, and the average value of the gray value and the standard deviation of the gray value of all pixels in each abnormal area are obtained as the energy distribution features, thereby obtaining the to-be-identified feature vector , wherein is the normalized continuous skeleton length of the crack, is the normalized average value of the main bending angle, is the normalized average value of the gray value, is the normalized standard deviation of the gray value.

[0074] It should be further noted that the continuous skeleton length of the crack is obtained by first extracting the skeleton of the crack, i.e. performing thinning processing on the crack to obtain a single-pixel-width line representation, then calculating the number of pixel points of the line, and converting it in combination with the actual physical size of the image to finally obtain the normalized continuous skeleton length; the average value of the main bending angle is first selected on the crack skeleton, the tangent direction angle at each key point is calculated, and then the angles are statistically averaged to obtain the average value of the main bending angle; the standard deviation of the gray value is first calculated by calculating the average value of the gray value of all pixels in the abnormal area, then for the gray value of each pixel in the area, the square of the difference between the average value and the gray value is calculated, the sum of these square values is divided by the total number of pixels, and finally the square root of the result is taken. The final result is the standard deviation of the gray value of the pixels in the abnormal area.

[0075] The to-be-identified feature vector is matched with the hollow, empty, and loose patterns in the preset disease feature library.

[0076] It should be noted that the to-be-identified feature vector is matched with the hollow, empty, and loose patterns in the preset disease feature library: the preset disease feature library stores the hollow pattern, and the feature vector is: short skeleton length, high average gray value, the empty pattern, the feature vector is: long skeleton length, low average gray value, the loose pattern, the feature vector is: medium skeleton length, medium average gray value, and the matching process is realized by calculating the Euclidean distance between the to-be-identified feature vector and the preset disease feature library vector. The mode with the smallest distance value and lower than the set threshold is determined as the matching result.

[0077] It should be further pointed out that the preset disease feature library vector is , wherein , , and are the normalized continuous skeleton length, the average value of the main bending angle, the average value of the gray value, and the gray value standard deviation of the crack in the standard sample in the preset disease feature library, respectively, and the matching distance .

[0078] In one specific embodiment of the present application, the set threshold value can be 0.4. Based on a plurality of groups of road disease sample test data experiments, the specific calibration process is as follows: 1) data collection: first, collect sample data of a plurality of groups of different types of hidden road diseases, including their feature vectors and matching distance values identified by the algorithm, and the following is a few groups of test data:

[0079]

[0080] 2) Set an initial threshold range: according to experience or preliminary experiments, set an initial threshold range, for example, between 0.3 and 0.5, 3) Classification experiment: use the set initial threshold range to classify the sample data, and observe the classification accuracy under different thresholds, 4) Find a threshold value that makes the classification accuracy relatively balanced by repeatedly adjusting the threshold and performing classification experiments, 5) In this example, after multiple experiments and adjustments, it is found that when the threshold is set to 0.4, the classification effect is best, therefore, 0.4 is determined as the final set threshold.

[0081] According to the matching degree, the disease type and the position coordinates are output, and the hidden disease classification result is formed.

[0082] It should be noted that based on the matching result, the disease type and the position coordinates of the corresponding abnormal area are output, and the coordinates are inherited from the spatial coordinate data of the joint diagnosis graph.

[0083] S7, closed loop optimization and active verification: based on the matching distance value in the hidden disease classification result, an active verification instruction is generated to optimize the acquisition parameters in a closed loop.

[0084] In specific embodiments of the present application, the specific process of generating the active verification instruction to optimize the acquisition parameters in a closed loop is as follows: based on the matching distance value in the hidden disease classification result, the low-confidence region coordinates are screened out.

[0085] It should be noted that the matching distance value specifically refers to the matching distance calculated above .

[0086] It should also be noted that the specific way of screening out the low-confidence region coordinates is to compare the matching distance value with the set distance threshold value, and when the matching distance value is greater than the set distance threshold value, it is considered as low-confidence, thereby obtaining the low-confidence region, so as to screen out the position coordinates corresponding to the disease region whose matching distance value is greater than the set distance threshold value.

[0087] In one specific embodiment of the present application, the set distance threshold value can be 0.3, which is set according to statistical analysis of classification error rate.

[0088] According to the disease type of the low-confidence region, the vibration sensor trigger threshold of the next round of acquisition is automatically adjusted, and the exposure parameter of the camera is adjusted according to the average gray value of the low-confidence region, and a secondary directional acquisition verification process is started.

[0089] It should be noted that for the vibration sensor array, according to the disease type of the low-confidence region, such as cavity, void or looseness, the dynamic trigger threshold is proportionally adjusted downward; for the high-speed camera, the exposure parameter is adjusted according to the average gray value of the low-confidence region in the joint diagnosis graph. Finally, the adjusted parameters and low-confidence region coordinate instructions are sent to the vehicle-mounted control system, and the whole process from vibration data acquisition to disease identification is restarted, i.e. from steps S1 to S6, to complete the closed loop verification.

[0090] It should be further noted that the matching distance value is marked as low-confidence when The vibration sensor trigger threshold adjustment formula is , wherein represents the original threshold value of the vibration sensor, represents the trigger threshold adjustment coefficient corresponding to the disease type stored in the database. The camera exposure time adjustment formula is , wherein represents the original value of the camera exposure time, represents the exposure time adjustment coefficient corresponding to the disease type stored in the database.

[0091] The above merely illustrates and describes the concept of the present application, and those skilled in the art can make various modifications or supplements to the described specific embodiments or adopt similar ways to replace, as long as the modifications or supplements do not deviate from the concept of the present application or exceed the defined scope of the present application, and should belong to the protection scope of the present application.

Claims

1. A method for intelligent identification of hidden pavement diseases based on deep learning, characterized in that, The method comprises the following steps: S1, abnormal vibration trigger instruction generation: acquiring multi-source vibration sensing data in the driving process of the road detection vehicle, and generating an abnormal vibration trigger instruction based on the multi-source vibration sensing data; S2, multi-modal data directional collection: directional collection of road multi-modal data based on the abnormal vibration trigger instruction, to generate a time-space aligned original data set; S3, structure response feature spectrum generation: analyzing the sound wave reflection signal in the original data set to generate a structure response feature spectrum; S4, enhanced feature map generation: processing the high-definition texture image in the original data set, and combining the vibration features in the abnormal vibration trigger instruction to generate a vibration-related enhanced feature map; S5, multi-modal joint diagnosis map generation: fusing the structure response feature spectrum and the vibration-related enhanced feature map to generate a multi-modal joint diagnosis map; S6, hidden disease intelligent identification: identifying the disease feature mode in the multi-modal joint diagnosis map, and outputting a hidden disease classification result, wherein the hidden disease types include a cavity mode, a void mode and a loose mode; S7, closed-loop optimization and active verification: generating an active verification instruction based on the matching distance value in the hidden disease classification result to optimize the collection parameters in a closed loop.

2. The deep learning-based intelligent pavement hidden disease identification method according to claim 1, characterized in that: The specific process of generating the abnormal vibration trigger instruction based on the multi-source vibration sensing data is as follows: Through the vibration sensor array arranged on the detection vehicle chassis, the triaxial vibration waveform signals of each measuring point on the road are collected in real time; The energy distribution characteristics of the triaxial vibration waveform signals in the preset frequency band are analyzed, and when the energy peak value continuously exceeds the preset dynamic threshold, the corresponding measuring point is marked as an abnormal measuring point; The position coordinates and impact response spectrum of the abnormal measuring point are fused to generate an abnormal vibration trigger instruction containing spatial positioning and vibration characteristics.

3. The deep learning-based intelligent pavement hidden disease identification method according to claim 2, characterized in that: The specific process of generating the time-space aligned original data set is as follows: According to the spatial positioning coordinates in the abnormal vibration trigger instruction, a high-speed camera is controlled to capture the high-definition texture image of the target area; The ultrasonic transducer is synchronously activated to emit sweep frequency sound waves to the target area and receive reflected sound wave signals; The high-definition texture image, the reflected sound wave signal and the vibration features in the abnormal vibration trigger instruction are bound to the same time stamp and spatial coordinates to generate a time-space aligned original data set.

4. The deep learning-based intelligent pavement hidden disease identification method according to claim 1, characterized in that: The specific process of generating the structure response feature spectrum is as follows: The reflected sound wave signal is frame-processed to extract the frequency energy distribution of each frame; The energy attenuation gradient curve of the reflected sound wave in the road medium is calculated; The energy attenuation gradient curve and the frequency energy distribution are fused to generate a structure response feature spectrum representing the internal structure density.

5. The deep learning-based intelligent pavement hidden disease identification method according to claim 4, characterized in that: The specific way of calculating the energy attenuation gradient curve of the reflected sound wave in the road medium is as follows: based on the propagation speed of the reflected sound wave in the road medium, the time axis is converted into a propagation distance axis, the attenuation coefficient value of each frequency point is calculated by fitting an exponential decay model, a gradient curve of the attenuation coefficient varying with the frequency is formed, and thus the energy attenuation gradient curve is obtained.

6. The deep learning-based intelligent pavement hidden disease identification method according to claim 1, characterized in that: The specific process of generating the vibration-related enhanced feature map is as follows: The crack trend distribution and local deformation features in the high-definition texture image are detected; The vibration wave propagation direction in the abnormal vibration triggering instruction is combined, and the parameters of the edge detection algorithm are adaptively adjusted, so that the crack detection direction is aligned with the vibration wave propagation direction; The aligned crack trend distribution and local deformation features are superimposed to generate vibration-related enhanced feature maps.

7. The deep learning-based intelligent pavement hidden disease identification method according to claim 5, characterized in that: The specific process of generating the multi-modal joint diagnosis map is as follows: Map the structural response feature spectrum to the spatial coordinate system of the vibration-related enhanced feature map; Taking the medium density value in the structural response feature spectrum as the weight coefficient, the gray value of the corresponding pixel of the vibration-related enhanced feature map is dynamically adjusted. The multi-modal joint diagnosis map is generated by weighting and superimposing the fusion of acoustic wave structure features and visual deformation features.

8. The deep learning-based intelligent pavement hidden disease identification method according to claim 7, characterized in that: The specific way of dynamically adjusting the gray value of the corresponding pixel of the vibration-related enhanced feature map is as follows: for each pixel position, the normalized attenuation coefficient value of the pixel position is extracted, multiplied by the original gray value of the enhanced feature map, and the adjusted gray value is obtained. 9.The deep learning-based intelligent pavement hidden disease identification method according to claim 1, characterized in that: The specific process of outputting the hidden disease classification result is as follows: Extract the topological structure features and energy distribution features of the abnormal area in the multi-modal joint diagnosis map to generate a feature vector to be identified; Match the feature vector to be identified with the hollow, empty, and loose patterns in the preset disease feature library; According to the matching degree, output the disease type and position coordinates to form the hidden disease classification result.

10. The deep learning-based intelligent pavement hidden disease identification method according to claim 1, characterized in that: The specific process of generating the active verification instruction to optimize the acquisition parameters in a closed loop is as follows: Based on the matching distance value in the hidden disease classification result, the low confidence area coordinates are selected; According to the disease type of the low confidence area, the vibration sensor triggering threshold of the next round of acquisition is automatically adjusted, the exposure parameters of the camera are adjusted according to the average gray value of the low confidence area, and the secondary directional acquisition verification process is started.

Images