Pavement defect detection method and system based on infrared thermal imaging

Through the combination of infrared thermal imaging technology and thermal conduction model, asphalt pavement defects are identified and evaluated, the accuracy of initial detection is solved, high-precision defect identification and maintenance priority planning is achieved, and the efficiency and quality of road maintenance are improved.

CN118967573APending Publication Date: 2024-11-15XIAN MUNICIPAL CONSTR GRP CO LTD
View PDF 0 Cites 15 Cited by

Patent Information

Application Number
CN202410952775.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The prior art is difficult to accurately identify and locate the initial defects of asphalt pavement, which are greatly disturbed by environmental factors, reducing the reliability and accuracy of detection.

Method used

Using the road surface defect detection method based on infrared thermal imaging, the thermal imaging data and environmental parameters are obtained, and the thermal conduction model is used for correction, combining adaptive threshold segmentation and morphological operation, the thermal abnormality area is identified, and the pre-trained defect classification model is used for calibration. Classification and severity assessment, a three-dimensional thermal distribution map was constructed based on GPS positioning, and a maintenance priority list was generated.

Benefits of technology

It realizes high-precision early identification and classification of asphalt pavement defects, automated assessment of damage severity, intelligent planning and maintenance priorities, significantly improves the reliability and accuracy of detection, and provides advanced solutions for road maintenance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967573A_ABST
    Figure CN118967573A_ABST
Patent Text Reader

Abstract

The invention relates to a pavement defect detection method and system based on infrared thermal imaging, and belongs to the technical field of road maintenance, the detection method comprises the following steps: obtaining thermal imaging data and environmental parameter information of a target asphalt pavement; performing data preprocessing on the thermal imaging data; based on the environmental parameter information, correcting the preprocessed thermal imaging data according to a heat conduction model; identifying a thermal anomaly region in the corrected thermal imaging data; refining the thermal anomaly region based on morphological operation to obtain corresponding defect features, inputting the defect features into a pre-trained defect classification model, and obtaining a defect feature classification result and a severity quantitative score; constructing a three-dimensional thermal distribution diagram to obtain a defect distribution position of the target asphalt pavement; and according to the severity quantitative score, performing priority ranking on each defect distribution position, and according to a maintenance priority list, sequentially sending corresponding maintenance instructions to a maintenance terminal. The method can improve the reliability and accuracy of initial detection of pavement defects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of road maintenance, and in particular to a road surface defect detection method and system based on infrared thermal imaging. Background Art

[0002] In the field of road engineering and maintenance management, the construction quality and subsequent maintenance management of asphalt pavement are directly related to the long-term stability and safety of transportation infrastructure. In recent years, the research on uniformity control of asphalt pavement construction using infrared thermal imaging technology has become the focus of industry attention, aiming to improve the precision of the construction process and the durability of the finished pavement through high-tech means. Although infrared thermal imaging technology has achieved remarkable results in monitoring the temperature uniformity of asphalt paving and ensuring the material performance during the construction process, its application in the initial detection and maintenance strategy of pavement defects still faces many challenges.

[0003] At present, defects in asphalt pavement, such as cracks, potholes, delamination, etc., are often difficult to detect in the early stages. However, with repeated rolling of vehicles and the influence of environmental factors, these tiny defects will rapidly deteriorate, leading to large-scale road damage, increasing the difficulty and cost of repair. Therefore, how to accurately and quickly identify and locate defects in the early stages has become a key issue that needs to be solved urgently.

[0004] However, environmental factors, such as temperature differences caused by seasonal changes and moisture from rainy and snowy weather, can interfere with pavement thermal imaging data, making it extremely complicated to accurately distinguish between pavement defects and environmental influences from thermal images, greatly reducing the reliability and accuracy of initial detection of pavement defects. Summary of the invention

[0005] In order to improve the reliability and accuracy of initial detection of road surface defects, the present application provides a road surface defect detection method based on infrared thermal imaging.

[0006] In the first aspect, the present application provides a road surface defect detection method based on infrared thermal imaging, which adopts the following technical solution: A road surface defect detection method based on infrared thermal imaging, the detection method comprising: Obtain thermal imaging data and environmental parameter information of the target asphalt road surface; Performing data preprocessing on the thermal imaging data; Based on the environmental parameter information, correcting the preprocessed thermal imaging data according to a heat conduction model; Based on an adaptive threshold segmentation algorithm, identifying thermal anomaly areas in the corrected thermal imaging data; Refining the thermal anomaly region based on morphological operations to obtain corresponding defect features; Inputting the defect features into a pre-trained defect classification model to obtain defect feature classification results and severity quantitative scores; Combining the defect feature classification result with GPS positioning information to construct a three-dimensional thermal distribution map to obtain the defect distribution position of the target asphalt pavement; Prioritize each defect distribution location according to the quantitative severity score to obtain a maintenance priority list; According to the maintenance priority list, corresponding maintenance instructions are sent to the maintenance terminal in sequence.

[0007] By adopting the above-mentioned technical solution, integrating infrared thermal imaging technology with environmental parameter correction, combined with intelligent image processing and deep learning analysis, the interference of environmental factors is effectively overcome, and high-precision early identification and classification of asphalt pavement defects are achieved. At the same time, the severity of damage is automatically assessed and maintenance priorities are intelligently planned, which significantly improves the reliability and accuracy of early detection of pavement defects and provides an advanced and practical solution for road maintenance.

[0008] Optionally, based on the environmental parameter information, the step of correcting the preprocessed thermal imaging data according to the heat conduction model includes: Acquiring geometric structure data and physical properties of the target asphalt pavement; Establishing a heat conduction model of the target asphalt pavement according to the geometric structure data and physical properties; Inputting the environmental parameter information into the heat conduction model to calculate a theoretical heat distribution diagram under current environmental conditions; Comparing the theoretical thermal distribution map with the preprocessed thermal imaging data to extract a difference image; Analyze the difference image and calculate the correction factor; Each pixel value of the preprocessed thermal imaging data is corrected pixel by pixel based on the correction factor to obtain corrected thermal imaging data.

[0009] By adopting the above technical solution, constructing a heat conduction model and combining it with actual environmental parameters, a refined correction of the pre-processed thermal imaging data is achieved, effectively improving the accuracy and reliability of the test results. Through a systematic comparative analysis of the theoretical thermal distribution and the measured data, the interference of environmental factors is determined and eliminated, so that the corrected thermal imaging data finally obtained can more realistically reflect the thermal characteristics and potential defects of the asphalt pavement, providing a scientific basis for pavement maintenance and management, and greatly improving the efficiency and quality of road maintenance work. In addition, the solution of this application has strong adaptability and versatility, and can be widely used in thermal state monitoring of pavements under different environmental conditions and of various types.

[0010] Optionally, the detection method further includes a defect classification model training step, and the training step includes: Acquire a historical sample data set; wherein the historical sample data set includes historical thermal imaging data and pre-annotated defect characteristics and severity; Performing data preprocessing on the historical sample data set; Divide the preprocessed historical sample data set into a training set, a validation set, and a test set; The pre-built convolutional neural network model is initially trained based on the training set, and the initially trained convolutional neural network model is verified and model parameters are optimized and adjusted based on the verification set and the test set to obtain the defect classification model.

[0011] By adopting the above technical solutions, a CNN model was constructed that can efficiently identify and classify defects in thermal imaging images of asphalt pavement. At the same time, the generalization ability and practicality of the model were ensured, and automatic defect classification and rating were realized, which improved the intelligent level of detection and made maintenance decisions more scientific and reasonable.

[0012] Optionally, the steps of performing initial training on a pre-built convolutional neural network model based on the training set, and performing verification and model parameter optimization adjustment on the initially trained convolutional neural network model based on the verification set and the test set include: Input the training set into a pre-built convolutional neural network model for training, optimize the model parameters and calculate the combined loss function of the model until the combined loss function meets the preset conditions, thereby obtaining the trained defect classification model; wherein the combined loss function includes a classification loss function and a regression loss function; Validating the defect classification model according to the validation set, evaluating the performance of the defect classification model and adjusting the hyperparameters of the model; The prediction accuracy of the adjusted defect classification model is tested based on the test set.

[0013] By adopting the above technical solution, an efficient classification and severity assessment model for asphalt pavement defects is constructed, which can not only automatically identify various types of pavement defects, but also accurately quantify their severity, providing a scientific basis for pavement maintenance and greatly improving the efficiency and quality of road maintenance. By comprehensively applying advanced deep learning technologies, this application has demonstrated significant technological innovation and practical value in the field of road maintenance.

[0014] Optionally, the pre-constructed convolutional neural network includes a convolution layer, an activation layer, a pooling layer, a fully connected layer one and a fully connected layer two, the fully connected layer one is used for defect feature classification, the fully connected layer two is used for severity quantitative scoring, and the combined loss function includes a classification loss function and a regression loss function.

[0015] Optionally, after obtaining the defect distribution position of the target asphalt pavement, the method further includes: Acquiring the corrected thermal imaging data of the defect distribution position; Analyzing the thermal imaging data of the corrected defect distribution position based on the heat conduction model to obtain a corresponding temperature distribution map; Acquiring traffic flow information of the target asphalt road surface; the traffic flow information includes vehicle flow, vehicle type and vehicle path distribution information; Calculating the expected load effects of the defect distribution locations according to the traffic flow information to generate a load effect matrix; Inputting the temperature distribution map and the load effect matrix into a pre-built pavement thermal response model to obtain pavement damage prediction indicators for multiple areas on a target asphalt pavement; Determine the area where the pavement damage prediction index is higher than a preset threshold as a damage prediction area; A corresponding maintenance strategy is generated according to the pavement damage prediction index of the damage prediction area.

[0016] By adopting the above technical solutions, integrating traffic data analysis, thermal imaging technology, heat conduction model and machine learning algorithm, a comprehensive pavement damage prediction and maintenance management system has been constructed. The system can proactively identify and evaluate the potential damage risks of asphalt pavements, and formulate efficient and reasonable maintenance strategies based on precise data analysis, effectively extending the service life of the pavement, reducing traffic interruptions caused by sudden damage, and optimizing road maintenance costs. It has important application value and promotion significance for improving the level of urban infrastructure management and ensuring traffic safety.

[0017] In the second aspect, the present application provides a road surface defect detection system based on infrared thermal imaging, which adopts the following technical solution: A road surface defect detection system based on infrared thermal imaging, the detection system comprising: The first acquisition module is used to acquire thermal imaging data and environmental parameter information of the target asphalt road surface; A preprocessing module, used for performing data preprocessing on the thermal imaging data; A correction module, used for correcting the pre-processed thermal imaging data according to the heat conduction model based on the environmental parameter information; A thermal anomaly recognition module, used to identify thermal anomaly areas in the corrected thermal imaging data based on an adaptive threshold segmentation algorithm; A defect feature extraction module, used to refine the thermal anomaly area based on morphological operations to obtain corresponding defect features; A defect classification module is used to input the defect features into a pre-trained defect classification model to obtain defect feature classification results and severity quantitative scores; A defect distribution location generation module is used to combine the defect feature classification result with the GPS positioning information to construct a three-dimensional thermal distribution map to obtain the defect distribution location of the target asphalt pavement; A priority sorting module, used to prioritize each defect distribution location according to the severity quantitative score to obtain a maintenance priority list; The maintenance instruction sending module is used to send corresponding maintenance instructions to the maintenance terminal in sequence according to the maintenance priority list.

[0018] Optionally, the road surface defect detection system further includes: A second acquisition module, used to acquire the corrected thermal imaging data of the defect distribution position; A temperature distribution map generating module, used for analyzing the thermal imaging data of the corrected defect distribution position based on the heat conduction model to obtain a corresponding temperature distribution map; A third acquisition module is used to acquire traffic flow information of the target asphalt road surface; the traffic flow information includes vehicle flow, vehicle type and vehicle path distribution information; A load effect matrix generation module, used to calculate the expected load effect of the defect distribution position according to the traffic flow information, and generate a load effect matrix; A pavement damage prediction module, used to input the temperature distribution map and the load effect matrix into a pre-built pavement thermal response model to obtain pavement damage prediction indicators for multiple areas on a target asphalt pavement; A damage prediction area determination module determines an area where the road surface damage prediction index is higher than a preset threshold as a damage prediction area; The maintenance strategy generation module is used to generate a corresponding maintenance strategy according to the pavement damage prediction index of the damage prediction area.

[0019] In a third aspect, the present application provides a computer device, which adopts the following technical solution: A computer device comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium stores a computer program that can be loaded by a processor and execute any one of the methods in the first aspect.

[0021] In summary, the present application includes at least one of the following beneficial technical effects: infrared thermal imaging technology with integrated environmental parameter correction, combined with intelligent image processing and deep learning analysis, effectively overcomes the interference of environmental factors, and realizes high-precision early identification and classification of asphalt pavement defects. At the same time, it automatically assesses the severity of damage and intelligently plans maintenance priorities, significantly improving the reliability and accuracy of early detection of pavement defects, and providing an advanced and practical solution for road maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a first flow chart of a road surface defect detection method according to one embodiment of the present application.

[0023] Figure 2 It is a second flow chart of a road surface defect detection method according to one embodiment of the present application.

[0024] Figure 3 It is a third flow chart of a road surface defect detection method according to one embodiment of the present application.

[0025] Figure 4 It is a fourth flow chart of a road surface defect detection method according to one embodiment of the present application.

[0026] Figure 5 It is a fifth flow chart of a road surface defect detection method according to one embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-5 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] The embodiment of the present application discloses a road surface defect detection method based on infrared thermal imaging.

[0029] Reference Figure 1 , a road surface defect detection method based on infrared thermal imaging, the detection method includes: Step S101, obtaining thermal imaging data and environmental parameter information of a target asphalt road surface; In some embodiments, a high-precision infrared thermal imager carried by a drone can be used to scan the asphalt pavement at an appropriate time (such as at night) to capture thermal imaging data of its surface temperature distribution.

[0030] At the same time, record the environmental parameters such as temperature, humidity, wind speed, etc. at that time, because these factors may affect the reading of thermal imaging data and need to be considered to ensure the comprehensiveness and accuracy of the data and provide a basis for subsequent analysis.

[0031] Step S102, performing data preprocessing on the thermal imaging data; Among them, preprocessing steps such as median filtering or Gaussian filtering can improve image quality by removing random noise in the image; and techniques such as contrast enhancement and edge sharpening can improve the sensitivity and accuracy of detection by making thermal anomaly areas easier to identify.

[0032] Step S103, based on the environmental parameter information, correct the pre-processed thermal imaging data according to the heat conduction model; Among them, the recorded environmental parameter information is combined with the heat conduction model in physics to correct the thermoforming data to eliminate the influence of non-road factors on heat distribution in the external environment, such as temperature deviation caused by atmospheric convection or sunlight.

[0033] It can be understood that by correcting the thermal imaging data and eliminating environmental interference, the thermoforming data can more truly reflect the thermal state of the road surface itself, thereby improving the objectivity and accuracy of the test results.

[0034] Step S104, identifying thermal anomaly areas in the corrected thermal imaging data based on an adaptive threshold segmentation algorithm; Specifically, the thermal anomaly area may be caused by abnormal heat conduction due to structural damage under the road surface. Through the adaptive threshold method, the threshold is dynamically set according to the local characteristics of the image, and the image is segmented into the foreground (thermal anomaly area) and the background to effectively identify high temperature or low temperature areas that are significantly different from the surrounding environment; Step S105, refining the thermal anomaly area based on morphological operations to obtain corresponding defect features; In some embodiments, morphological operations such as opening and closing, corrosion or expansion can be applied to refine the shape and clarify the boundaries of the initially identified thermal anomaly area, and further distinguish different types of defects (such as cracks and potholes), thereby helping subsequent classification models to more accurately identify defect types and provide a basis for refined management and maintenance.

[0035] Step S106, inputting the defect features into a pre-trained defect classification model to obtain defect feature classification results and severity quantitative scores; Among them, deep learning technology is used to automatically classify the refined defect features and quantitatively evaluate the severity of each type of defect, which improves the intelligence level of detection and makes maintenance decisions more scientific and reasonable.

[0036] Step S107, combining the defect feature classification result with the GPS positioning information to construct a three-dimensional thermal distribution map to obtain the defect distribution position of the target asphalt pavement; Specifically, the classification results are integrated with the GPS positioning information at the time of shooting, and the spatial distribution of road defects is displayed through three-dimensional reconstruction technology. The defect locations are intuitively presented. By providing an intuitive visual display, it is easy for managers to understand the defect distribution and efficiently plan maintenance actions.

[0037] Step S108, prioritizing each defect distribution location according to the severity quantitative score to obtain a maintenance priority list; Among them, the defect locations are ranked according to severity based on quantitative scoring standards, and the priority repair areas are determined to ensure the effective allocation of resources. By giving priority to maintaining the most urgent defects, maintenance efficiency and road safety are improved.

[0038] Step S109: sending corresponding maintenance instructions to the maintenance terminal in sequence according to the maintenance priority list.

[0039] Among them, according to the maintenance priority list, the system automatically sends instructions to the terminal responsible for maintenance, prompting the maintenance personnel to start the maintenance work in accordance with the maintenance order to improve the response speed and work efficiency.

[0040] In the above implementation, infrared thermal imaging technology with integrated environmental parameter correction, combined with intelligent image processing and deep learning analysis, effectively overcomes the interference of environmental factors and realizes high-precision early identification and classification of asphalt pavement defects. At the same time, it automatically assesses the severity of damage and intelligently plans maintenance priorities, significantly improving the reliability and accuracy of early detection of pavement defects, and providing an advanced and practical solution for road maintenance.

[0041] Reference Figure 2 As an implementation of step S103, the step of correcting the pre-processed thermal imaging data according to the heat conduction model based on the environmental parameter information includes: Step S201, obtaining geometric structure data and physical properties of a target asphalt pavement; The geometric structure data can be obtained through field measurement methods such as laser scanning, GPS measurement or photogrammetry to ensure that the established heat conduction model has high accuracy and reliability; the physical characteristics include thermal conductivity, heat capacity, density, etc. The thermal conductivity determines the thermal conductivity of the material, while the heat capacity affects the heat capacity of the material and the response to temperature changes. In some embodiments, the physical property data that best meets the actual situation can be selected according to the type of target asphalt pavement; for example, for common road asphalt, the corresponding standard data can be selected.

[0042] Step S202, establishing a heat conduction model of the target asphalt pavement according to the geometric structure data and physical properties; Among them, the heat conduction model is used to describe the way and speed of heat transfer in the material. Thermodynamic principles and mathematical modeling methods can be used to establish a heat conduction model that describes the process of heat transfer inside and on the surface of the road surface. The model will consider heat transfer mechanisms such as thermal radiation, convection heat transfer and heat conduction to ensure that the model can fully reflect the actual situation.

[0043] Specifically, the physical parameters of asphalt materials are combined with geometric structure data to establish a heat conduction model; the heat conduction model is based on the heat conduction equation and is solved by numerical methods (such as finite difference method or finite element method) to simulate the heat transfer process in the road surface under given initial conditions and boundary conditions. The initial conditions of the model include the initial temperature distribution of the road surface, and the boundary conditions are based on factors such as ambient temperature, solar radiation and wind speed.

[0044] Step S203, inputting the environmental parameter information into the heat conduction model to calculate and obtain a theoretical heat distribution map under the current environmental conditions; Among them, by inputting environmental parameter information into the established heat conduction model for simulation, a theoretical heat distribution map under specific environmental conditions can be generated. The theoretical heat distribution map provides the temperature distribution of the asphalt pavement under ideal conditions and is the reference standard for subsequent correction of thermal imaging data. Through this process, the heat conduction model can accurately reflect the heat conduction behavior of the actual road surface, laying a solid foundation for achieving high-precision road defect detection.

[0045] Step S204, comparing the theoretical thermal distribution map with the pre-processed thermal imaging data to extract a difference image; Specifically, the theoretical thermal distribution map is compared with the actual thermal imaging data after preprocessing at the pixel level, and the difference between the two is calculated to obtain a difference image, which reflects the temperature deviation caused by the external environment and other non-road factors, and provides a data basis for calculating the correction factor.

[0046] Step S205, analyzing the difference image and calculating the correction factor; The correction factor can be a global correction coefficient for the entire image or a local correction coefficient for a specific area or a specific type of pixel, which is intended to reflect and correct systematic errors caused by environmental conditions.

[0047] In the embodiment of the present application, the correction factor is used to quantify the influence of the environmental parameters on the thermal imaging data. The correction factor can be obtained by analyzing the temperature deviation in the difference image and determining the degree to which each pixel in the thermal imaging data needs to be adjusted; It can be understood that the calculation of the correction factor makes the subsequent pixel correction more accurate and can effectively eliminate the interference of environmental factors on thermal imaging data.

[0048] Step S206 , performing pixel-by-pixel correction on each pixel value of the preprocessed thermal imaging data based on the correction factor to obtain corrected thermal imaging data.

[0049] Among them, the correction factor is applied to the preprocessed thermal imaging data, and each pixel value is corrected pixel by pixel. The correction method may be to directly add or subtract the correction value, or to apply a more complex function transformation to ensure that the corrected data can better meet the unity of theoretical expectations and actual observations, so as to eliminate environmental interference factors, make the thermal imaging data more accurate and reliable, and provide a high-quality data foundation for subsequent thermal anomaly area identification and defect feature extraction.

[0050] In one embodiment of the present application, the correction formula is: T corrected (x, y) = T preprocessed (x, y)-ΔT(x, y), where, T corrected (x, y) is the corrected thermal imaging data, T preprocessed (x, y) is the preprocessed thermal imaging data, and ΔT(x, y) is the correction factor.

[0051] In the above implementation, a heat conduction model is constructed and combined with actual environmental parameters to achieve refined correction of pre-processed thermal imaging data, effectively improving the accuracy and reliability of the test results. Through systematic comparative analysis of theoretical thermal distribution and measured data, the interference of environmental factors is determined and eliminated, so that the corrected thermal imaging data finally obtained can more realistically reflect the thermal characteristics and potential defects of the asphalt pavement, providing a scientific basis for pavement maintenance and management, and greatly improving the efficiency and quality of road maintenance work. In addition, the solution of the present application has strong adaptability and versatility, and can be widely used in thermal state monitoring of pavements under different environmental conditions and of various types.

[0052] Reference Figure 3 As a further implementation of the detection method, the detection method further includes a defect classification model training step, the training step includes: Step S301, obtaining a historical sample data set; wherein the historical sample data set includes historical thermal imaging data and pre-annotated defect features and severity; Specifically, a large amount of corrected historical thermal imaging data needs to be collected as a training data set. The data set should contain samples of various known defects and samples of normal road surfaces. Each picture must be clearly labeled to indicate the type of defect (such as cracks, potholes, delamination, etc.) and the severity. The severity can be pre-defined based on experience (such as mild, moderate, severe, etc.), and then manually pre-labeled according to the standard.

[0053] It should be noted that we should pay attention to checking and balancing the number of samples under the severity labels to avoid skewed learning caused by too many or too few samples at certain levels. Balance can be achieved by downsampling the over-class or upsampling the under-class.

[0054] Step S302, performing data preprocessing on the historical sample data set; Among them, the data preprocessing steps include data enhancement and normalization; data enhancement such as rotation, flipping, scaling, brightness adjustment, etc. is used to generate more diverse training samples; normalization is used to standardize the pixel values ​​of all images, usually converted into floating-point numbers in the range of 0-1, to facilitate model training.

[0055] Step S303, dividing the preprocessed historical sample data set into a training set, a validation set and a test set; The division ratios are usually 70%, 15%, and 15%, respectively, to ensure that the model can learn on the training data and generalize effectively on unseen data.

[0056] Step S304: Perform initial training on the pre-built convolutional neural network model based on the training set, and verify and optimize the model parameters of the initially trained convolutional neural network model based on the verification set and the test set to obtain a defect classification model.

[0057] Among them, the Convolutional Neural Network (CNN) is used as the basic algorithm of the defect classification model. This model is particularly suitable for processing image data and can automatically learn and extract local features in the image for defect detection and classification tasks.

[0058] In the embodiment of the present application, a multi-layer CNN model can be designed, including a convolution layer, an activation layer, a pooling layer, a fully connected layer 1 and a fully connected layer 2. Among them, the convolution layer is used to automatically extract local features in the image, such as edges, textures, etc.; the activation layer (such as ReLU) introduces nonlinearity; the pooling layer reduces the spatial dimension, reduces the computational complexity while maintaining important features; there are two fully connected layers, the fully connected layer 1 is used to classify the extracted defect features, and the fully connected layer 2 is used for regression scoring (quantitative severity scoring).

[0059] It should be noted that during the training process, the model needs to learn how to classify defects and assess their severity at the same time. Therefore, hyperparameters such as learning rate and batch size need to be adjusted more carefully to adapt to more complex training objectives.

[0060] In the above implementation, a CNN model is constructed that can efficiently identify and classify defects in thermal imaging images of asphalt pavement, while ensuring the generalization ability and practicality of the model, realizing automatic defect classification and rating, improving the intelligence level of detection, and making maintenance decisions more scientific and reasonable.

[0061] Reference Figure 4 As an implementation method of step S304, the steps of performing initial training on the pre-built convolutional neural network model based on the training set, and verifying the initially trained convolutional neural network model and optimizing the model parameters based on the verification set and the test set include: Step S401, input the training set into a pre-built convolutional neural network model for training, optimize the model parameters and calculate the combined loss function of the model until the combined loss function meets the preset conditions, thereby obtaining a trained defect classification model; Among them, the combined loss function includes classification loss function and regression loss function; In some embodiments, a classification loss function (such as cross entropy loss) and a regression loss function (such as mean square error or Huber loss) may be used; the classification loss function ensures that the model can correctly identify the defect type, while the regression loss is used to optimize the accuracy of the severity score. The two work together to optimize the model in both dimensions; It can be understood that the use of the combined loss function ensures the multi-dimensionality of the model training objectives, which can not only accurately classify but also accurately quantify the severity of defects, thus improving the practicality and comprehensiveness of the model.

[0062] Specifically, the training set containing rich features is input into the pre-designed convolutional neural network (CNN) model. CNN automatically learns and extracts features in the image through multi-layer convolution, pooling, full connection and other operations to identify and classify defects in asphalt pavement. During the training process, model parameters (such as filter weights, bias terms, etc.) are iteratively adjusted through back-propagation algorithms and optimization algorithms (such as gradient descent, Adam, etc.) to minimize the model's loss function, which is a combination of the classification loss function and the regression loss function.

[0063] Step S402, verifying the defect classification model according to the verification set, evaluating the performance of the defect classification model and adjusting the hyperparameters of the model; The validation set is used to evaluate the performance of the trained model. This is a data set independent of the training set and is used to test the generalization ability of the model. By observing the performance of the model on the validation set, such as accuracy, recall, F1 score and other indicators, as well as the error indicators of the regression task, it can be determined whether the model is overfitting or underfitting, and the hyperparameters of the model (such as learning rate, regularization coefficient, etc.) can be adjusted accordingly.

[0064] Step S403, testing the prediction accuracy of the adjusted defect classification model according to the test set.

[0065] The test set is the last round of testing for the model after all parameters and structures are optimized. It is completely independent of the training and validation process and is used to ultimately evaluate the performance of the model on new data to verify whether the generalization ability of the model meets expectations.

[0066] In the above implementation, an efficient classification and severity assessment model for asphalt pavement defects is constructed, which can not only automatically identify various types of pavement defects, but also accurately quantify their severity, provide a scientific basis for pavement maintenance, and greatly improve the efficiency and quality of road maintenance; through the comprehensive use of advanced deep learning technology, this application has demonstrated significant technological innovation and practical value in the field of road maintenance.

[0067] Reference Figure 5 As a further implementation of the road surface defect detection method, after obtaining the defect distribution position of the target asphalt road surface in step S107, the method further includes: Step S501, obtaining corrected thermal imaging data of defect distribution positions; Specifically, based on the correction of the thermal imaging data and the determination of the defect distribution location in the previous steps, the corrected thermal imaging data is directly acquired for the defect distribution location, avoiding the high cost and time consumption of full road coverage, making data collection more targeted and efficient.

[0068] Step S502, analyzing the corrected thermal imaging data of the defect distribution position based on the heat conduction model to obtain a corresponding temperature distribution map; Among them, based on the heat conduction model that has been constructed in the previous steps, the corresponding temperature distribution map can be analyzed using the heat conduction model; the temperature distribution map is used to reveal the thermal state of the road surface, provides an intuitive basis for evaluating the changes in the road surface structure caused by temperature, and helps to identify potential damage areas caused by thermal stress.

[0069] Step S503, obtaining traffic flow information of the target asphalt road surface; Among them, traffic flow information includes vehicle flow, vehicle type and vehicle path distribution information; In some embodiments, real-time or historical traffic monitoring data on the target asphalt pavement is collected, including vehicle flow counts, vehicle types (such as cars, trucks, buses, etc.), and vehicle path distribution information. Since different types and numbers of vehicles exert different pressures and wear on the pavement, this can be used as one of the key factors affecting the pavement condition. Accurate vehicle path distribution information can help understand the vehicle's driving route data, which is also particularly critical for predicting future damage to different areas of the pavement.

[0070] It is understandable that obtaining accurate traffic flow information provides basic data for evaluating and predicting pavement load effects, which helps to understand the actual bearing capacity of the pavement at different time and space points.

[0071] Step S504, calculating the expected load effect of the defect distribution location according to the traffic flow information, and generating a load effect matrix; This step takes into account the combined impact of vehicle type, number and driving path, and uses a mechanical model to calculate the expected load effect at each defect distribution location, that is, the pressure impact of vehicle activities on the pavement structure in the area. These calculation results are presented in a matrix form to facilitate subsequent analysis and processing.

[0072] In some specific embodiments, for each defect location, a finite element analysis (FEA) method may be used to calculate the stress distribution under the vehicle load by considering the pavement material properties and structural characteristics; the cumulative damage degree of the location may be evaluated by calculating the cumulative load effects of all vehicles passing through the defect location within a certain time period and considering the fatigue life theory; Next, a two-dimensional matrix can be constructed, with the horizontal axis being the time series or specific time period and the vertical axis being the defect location coordinate. Each element in the matrix represents the expected load effect value at the corresponding time and location. The cumulative load effect value of each defect location at each time point is filled into the corresponding position of the matrix to form a load effect matrix.

[0073] It can be understood that the load effect matrix is ​​a bridge connecting traffic activities and pavement damage prediction. It quantifies the direct impact of vehicle activities on the pavement and provides an important basis for formulating preventive maintenance measures.

[0074] Step S505, inputting the temperature distribution map and the load effect matrix into a pre-built pavement thermal response model to obtain pavement damage prediction indicators for multiple areas on the target asphalt pavement; In one of the embodiments of the present application, a random forest algorithm can be used as a modeling tool. By integrating historical thermal imaging data and corresponding load effect information, combined with the actual pavement damage situation, a comprehensive data set is formed. The model is trained using the comprehensive data set, so that the model continuously learns the complex nonlinear relationship between temperature distribution, load effect and pavement structure changes. During this period, model parameters are tuned to optimize prediction performance. Finally, the optimized model can be applied to actual scenarios to guide efficient and targeted pavement maintenance decisions.

[0075] Specifically, the pavement thermal response model learns the interaction pattern between complex thermal response and load effect. When the current temperature distribution map and load effect matrix are input, the model can output the pavement damage prediction index of each defective area, so as to identify the damaged areas that may deteriorate earlier and provide strong support for maintenance decisions.

[0076] Step S506, determining the area where the road damage prediction index is higher than a preset threshold as a damage prediction area; The preset threshold can be pre-set or adjusted based on experience or in combination with historical data. By comparing the pavement damage prediction index with the preset threshold, the area exceeding the threshold is regarded as a damage prediction area that requires attention.

[0077] Step S507: generating a corresponding maintenance strategy according to the pavement damage prediction index of the damage prediction area.

[0078] Among them, according to the road damage prediction indicators in the damage prediction area, the road use strategy and maintenance plan are formulated or adjusted, such as arranging maintenance time, selecting appropriate repair technology and materials, etc. By reinforcing or repairing vulnerable areas, closed-loop management from data collection to strategy generation is achieved, ensuring the efficient allocation of maintenance resources, improving the overall durability and adaptability of the road system, and ensuring traffic safety and smoothness.

[0079] In the above implementation, traffic data analysis, thermal imaging technology, heat conduction model and machine learning algorithm are integrated to construct a comprehensive road damage prediction and maintenance management system. The system can proactively identify and evaluate the potential damage risks of asphalt pavements, and formulate efficient and reasonable maintenance strategies based on precise data analysis, effectively extending the service life of the pavement, reducing traffic interruptions caused by sudden damage, and optimizing road maintenance costs. It has important application value and promotion significance for improving the level of urban infrastructure management and ensuring traffic safety.

[0080] It is understandable that since the defects of asphalt pavement are not easy to detect in the early stages, and with the repeated rolling of vehicles, these tiny defects will rapidly deteriorate, resulting in large-scale road damage, increasing the difficulty and cost of repair. Therefore, this application uses thermal imaging data of defect distribution locations combined with traffic flow information to focus on the core area of ​​the problem, and uses highly relevant data for in-depth analysis to improve the accuracy and efficiency of predictions. Not only can it discover and understand the potential evolution trend of road damage earlier, but also through intelligent prediction models, it provides strong technical support for the realization of scientific and refined road maintenance management, thereby showing significant advantages in maintenance cost control, traffic flow assurance, and road safety improvement.

[0081] The embodiment of the present application also discloses a road surface defect detection system based on infrared thermal imaging.

[0082] A road surface defect detection system based on infrared thermal imaging, the detection system includes: The first acquisition module is used to acquire thermal imaging data and environmental parameter information of the target asphalt road surface; A preprocessing module, used for preprocessing thermal imaging data; A correction module, used for correcting the pre-processed thermal imaging data according to the heat conduction model based on the environmental parameter information; A thermal anomaly recognition module, used to identify thermal anomaly areas in the corrected thermal imaging data based on an adaptive threshold segmentation algorithm; Defect feature extraction module, used to refine the thermal anomaly area based on morphological operations to obtain the corresponding defect features; The defect classification module is used to input defect features into a pre-trained defect classification model to obtain defect feature classification results and severity quantitative scores; The defect distribution location generation module is used to combine the defect feature classification results with the GPS positioning information to construct a three-dimensional thermal distribution map and obtain the defect distribution location of the target asphalt pavement; The priority sorting module is used to prioritize the distribution location of each defect according to the quantitative severity score to obtain a maintenance priority list; The maintenance instruction sending module is used to send corresponding maintenance instructions to the maintenance terminal in sequence according to the maintenance priority list.

[0083] As a further implementation of the road surface defect detection system, the road surface defect detection system further includes: A second acquisition module is used to acquire the corrected thermal imaging data of the defect distribution position; A temperature distribution map generation module is used to analyze the thermal imaging data of the corrected defect distribution position based on the heat conduction model to obtain the corresponding temperature distribution map; The third acquisition module is used to acquire the traffic flow information of the target asphalt road surface; the traffic flow information includes vehicle flow, vehicle type and vehicle path distribution information; A load effect matrix generation module is used to calculate the expected load effect of defect distribution locations based on traffic flow information and generate a load effect matrix; A pavement damage prediction module is used to input the temperature distribution map and the load effect matrix into a pre-built pavement thermal response model to obtain pavement damage prediction indicators for multiple areas on the target asphalt pavement; A damage prediction area determination module determines an area where the road surface damage prediction index is higher than a preset threshold as a damage prediction area; The maintenance strategy generation module is used to generate a corresponding maintenance strategy according to the pavement damage prediction index of the damage prediction area.

[0084] The pavement defect detection system of the embodiment of the present application can implement any of the above-mentioned pavement defect detection methods, and the specific working process of each module in the pavement defect detection system can refer to the corresponding process in the above-mentioned method embodiment.

[0085] In the several embodiments provided in this application, it should be understood that the provided methods and systems can be implemented in other ways. For example, the system embodiments described above are only illustrative; for example, the division of a certain module is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0086] The embodiment of the present application also discloses a computer device.

[0087] The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the road surface defect detection method as described above is implemented.

[0088] The embodiment of the present application also discloses a computer-readable storage medium.

[0089] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute any of the above-mentioned road surface defect detection methods.

[0090] Among them, computer-readable storage media can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0091] It should be noted that in the above embodiments, the description of each embodiment has different emphases, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A road surface defect detection method based on infrared thermal imaging, characterized in that: The detection method comprises: Obtain thermal imaging data and environmental parameter information of the target asphalt road surface; Performing data preprocessing on the thermal imaging data; Based on the environmental parameter information, correcting the preprocessed thermal imaging data according to a heat conduction model; Based on an adaptive threshold segmentation algorithm, identifying thermal anomaly areas in the corrected thermal imaging data; Refining the thermal anomaly region based on morphological operations to obtain corresponding defect features; Inputting the defect features into a pre-trained defect classification model to obtain defect feature classification results and severity quantitative scores; Combining the defect feature classification result with GPS positioning information to construct a three-dimensional thermal distribution map to obtain the defect distribution position of the target asphalt pavement; Prioritize each defect distribution location according to the quantitative severity score to obtain a maintenance priority list; According to the maintenance priority list, corresponding maintenance instructions are sent to the maintenance terminal in sequence.

2. A road surface defect detection method based on infrared thermal imaging according to claim 1, characterized in that: Based on the environmental parameter information, the step of correcting the preprocessed thermal imaging data according to the heat conduction model includes: Acquiring geometric structure data and physical properties of the target asphalt pavement; Establishing a heat conduction model of the target asphalt pavement according to the geometric structure data and physical properties; Inputting the environmental parameter information into the heat conduction model to calculate a theoretical heat distribution diagram under current environmental conditions; Comparing the theoretical thermal distribution map with the preprocessed thermal imaging data to extract a difference image; Analyze the difference image and calculate the correction factor; Each pixel value of the preprocessed thermal imaging data is corrected pixel by pixel based on the correction factor to obtain corrected thermal imaging data.

3. The road surface defect detection method based on infrared thermal imaging according to claim 1 is characterized in that: The detection method further comprises a defect classification model training step, wherein the training step comprises: Acquire a historical sample data set; wherein the historical sample data set includes historical thermal imaging data and pre-annotated defect characteristics and severity; Performing data preprocessing on the historical sample data set; Divide the preprocessed historical sample data set into a training set, a validation set, and a test set; The pre-built convolutional neural network model is initially trained based on the training set, and the initially trained convolutional neural network model is verified and model parameters are optimized and adjusted based on the verification set and the test set to obtain the defect classification model.

4. The road surface defect detection method based on infrared thermal imaging according to claim 3 is characterized in that: The steps of performing initial training on the pre-built convolutional neural network model based on the training set, and performing verification and model parameter optimization adjustment on the initially trained convolutional neural network model based on the verification set and the test set include: Inputting the training set into a pre-built convolutional neural network model for training, optimizing the model parameters and calculating the combined loss function of the model until the combined loss function meets the preset conditions, thereby obtaining the trained defect classification model; Validating the defect classification model according to the validation set, evaluating the performance of the defect classification model and adjusting the hyperparameters of the model; The prediction accuracy of the adjusted defect classification model is tested based on the test set.

5. The method for detecting road defects based on infrared thermal imaging according to claim 4, characterized in that: The pre-constructed convolutional neural network includes a convolution layer, an activation layer, a pooling layer, a fully connected layer 1 and a fully connected layer 2. The fully connected layer 1 is used for defect feature classification, the fully connected layer 2 is used for severity quantitative scoring, and the combined loss function includes a classification loss function and a regression loss function.

6. A road surface defect detection method based on infrared thermal imaging according to any one of claims 1 to 5, characterized in that: After the step of obtaining the defect distribution position of the target asphalt pavement, the method further includes: Acquiring the corrected thermal imaging data of the defect distribution position; Analyzing the thermal imaging data of the corrected defect distribution position based on the heat conduction model to obtain a corresponding temperature distribution map; Acquiring traffic flow information of the target asphalt road surface; the traffic flow information includes vehicle flow, vehicle type and vehicle path distribution information; Calculating the expected load effects of the defect distribution locations according to the traffic flow information to generate a load effect matrix; Inputting the temperature distribution map and the load effect matrix into a pre-built pavement thermal response model to obtain pavement damage prediction indicators for multiple areas on a target asphalt pavement; Determine the area where the pavement damage prediction index is higher than a preset threshold as a damage prediction area; A corresponding maintenance strategy is generated according to the pavement damage prediction index of the damage prediction area.

7. A road surface defect detection system based on infrared thermal imaging, characterized in that: The detection system comprises: The first acquisition module is used to acquire thermal imaging data and environmental parameter information of the target asphalt road surface; A preprocessing module, used for performing data preprocessing on the thermal imaging data; A correction module, used for correcting the pre-processed thermal imaging data according to the heat conduction model based on the environmental parameter information; A thermal anomaly recognition module, used to identify thermal anomaly areas in the corrected thermal imaging data based on an adaptive threshold segmentation algorithm; A defect feature extraction module, used to refine the thermal anomaly area based on morphological operations to obtain corresponding defect features; A defect classification module is used to input the defect features into a pre-trained defect classification model to obtain defect feature classification results and severity quantitative scores; A defect distribution location generation module is used to combine the defect feature classification result with the GPS positioning information to construct a three-dimensional thermal distribution map to obtain the defect distribution location of the target asphalt pavement; A priority sorting module, used to prioritize each defect distribution location according to the severity quantitative score to obtain a maintenance priority list; The maintenance instruction sending module is used to send corresponding maintenance instructions to the maintenance terminal in sequence according to the maintenance priority list.

8. The road surface defect detection system based on infrared thermal imaging according to claim 7 is characterized in that: The road surface defect detection system further comprises: A second acquisition module, used to acquire the corrected thermal imaging data of the defect distribution position; A temperature distribution map generating module, used for analyzing the thermal imaging data of the corrected defect distribution position based on the heat conduction model to obtain a corresponding temperature distribution map; A third acquisition module is used to acquire traffic flow information of the target asphalt road surface; the traffic flow information includes vehicle flow, vehicle type and vehicle path distribution information; A load effect matrix generation module, used to calculate the expected load effect of the defect distribution position according to the traffic flow information, and generate a load effect matrix; A pavement damage prediction module, used to input the temperature distribution map and the load effect matrix into a pre-built pavement thermal response model to obtain pavement damage prediction indicators for multiple areas on a target asphalt pavement; A damage prediction area determination module determines an area where the road surface damage prediction index is higher than a preset threshold as a damage prediction area; The maintenance strategy generation module is used to generate a corresponding maintenance strategy according to the pavement damage prediction index of the damage prediction area.

9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the method according to any one of claims 1 to 6 is implemented when the processor executes the program.

10. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 6.

Citation Information

Cited By

  • Road change detection system

    CN119177598A

  • A road change detection system

    CN119177598B

  • Product detection method and system based on machine vision

    CN119323659A

  • Intelligent detection method for asphalt pavement segregation based on multi-source data analysis

    CN119478642A

  • Asphalt performance detection method and asphalt performance detection system

    CN119619155A