Pavement disease intelligent diagnosis method based on image recognition

Abstract

The invention discloses an intelligent pavement disease diagnosis method based on image recognition, and relates to the technical field of pavement disease diagnosis, and the method comprises the steps: deploying an image collection device and a pavement monitoring sensor in a target pavement region, so as to obtain multi-source pavement data; preprocessing the multi-source road surface data, and performing feature extraction to obtain a road surface feature sequence; and based on a deep learning algorithm and in combination with the pavement feature sequence, learning features of different disease types, constructing a disease identification classification model, and identifying different types of pavement diseases. Through the high-definition camera and the image processing technology, various disease types such as cracks, pit slots and ruts can be quickly and accurately identified, and in combination with a deep learning algorithm, disease features can be automatically extracted, high-precision identification of road diseases is realized, the disease detection efficiency is remarkably improved, manual intervention is reduced, and the detection efficiency is improved. And timely and accurate data support is provided for road maintenance.

Description

Technical Field

[0001] The present invention relates to the technical field of pavement disease diagnosis, and particularly relates to an intelligent pavement disease diagnosis method based on image recognition. Background Art

[0002] With the rapid development of highway traffic, the problem of pavement diseases has become increasingly prominent. Traditional manual detection methods have disadvantages such as low efficiency, low accuracy, affecting traffic, and being unsafe, and it is difficult to meet the current demand for large-scale and wide-range pavement disease detection. Therefore, a more efficient, accurate, and safe detection method is needed to cope with the growing demand for highway maintenance. In recent years, image processing and machine learning technologies have made remarkable progress, especially in the fields of computer vision and deep learning. These technologies have made it possible for intelligent diagnosis of pavement diseases based on image recognition.

[0003] In the prior art, there are many types of pavement diseases, and sometimes they present a complex mixed state, resulting in difficult accurate distinction of different types of diseases, affecting the diagnosis effect. Moreover, on a large-scale pavement, it is difficult to accurately locate the disease position, affecting the subsequent repair and treatment of the disease. Therefore, how to combine the high-precision positioning of the Internet of Things technology and image recognition technology to accurately distinguish the disease types and accurately calibrate the disease position is the problem we need to solve. For this purpose, an intelligent pavement disease diagnosis method based on image recognition is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent pavement disease diagnosis method based on image recognition to solve the problems raised in the above background art.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is:

[0006] An intelligent pavement disease diagnosis method based on image recognition, comprising the following steps:

[0007] Step 1: Deploy image acquisition devices and pavement monitoring sensors in the target pavement area to obtain multi-source pavement data;

[0008] Step 2: Preprocess the multi-source pavement data and perform feature extraction to obtain a pavement feature sequence;

[0009] Step 3: Based on a deep learning algorithm and combined with the pavement feature sequence, learn the features of different disease types, construct a disease recognition and classification model, and identify different types of pavement diseases;

[0010] Step 4: Combine the data of the pavement monitoring sensors to analyze the environmental condition information of the pavement diseases to analyze the expansion trend of the diseases;

[0011] Step 5: Combine the identified diseases with the high-precision positioning data to obtain the coordinate information of the diseases, and mark the positions of the diseases in the road surface planning system;

[0012] Step 6: Visualize the results of image recognition and the positioning information of the diseases to generate a disease distribution map, and mark the types, positions, and expansion trends of the diseases.

[0013] A further improvement of the technical solution of the present invention is that in the above Step 1, the process of obtaining multi-source road surface data includes:

[0014] According to the road maintenance requirements, determine the target road surface area to be monitored, clarify the monitoring objectives, determine the types of diseases to be monitored and the requirements for disease positioning accuracy, and then select image acquisition devices and road surface monitoring sensors according to the monitoring objectives, and determine the deployment positions, quantities, and data acquisition frequencies of the image acquisition devices and road surface monitoring sensors. Among them, the types of diseases include cracks, potholes, ruts, and crazing. The image acquisition device is a high-definition camera, and the road surface monitoring sensors are lidar and temperature and humidity sensors;

[0015] Start the image acquisition device, collect road surface images at a preset frequency, start the road surface monitoring sensors, collect the physical state data of the road surface, and use Internet of Things technologies (such as 5G, NB-IoT, 4G, etc.) to transmit the image data and sensor data to the data processing center;

[0016] In the data processing center, synchronize the image data and sensor data according to the time stamp and device identifier to form a road surface monitoring data set with multi-source data fusion.

[0017] A further improvement of the technical solution of the present invention is that in the above Step 2, the process of obtaining the road surface feature sequence includes:

[0018] Arrange the data of each acquisition point in chronological order, compare the adjacent data, find the data with the same geographical location information, only retain the data recorded for the first time, and delete the other data;

[0019] Perform unified preprocessing and analysis on the multi-source road surface data in the road surface monitoring data set. Among them, perform grayscale conversion, normalization, denoising and enhancement, and binarization processing on the image data, perform preprocessing operations such as point cloud denoising, point cloud segmentation, and coordinate conversion on the lidar point cloud data, and perform preprocessing operations such as data cleaning and data synchronization on the data of the temperature and humidity sensors;

[0020] Extract features from the preprocessed multi-source road surface data, extract image features and physical state features therefrom, fuse the image features and physical state features, and arrange the extracted features in spatial positions to form a road surface feature sequence.

[0021] A further improvement of the technical solution of the present invention lies in that: the process of obtaining the road surface feature sequence includes:

[0022] For the extraction of image features, including edge, corner and texture features. Among them, the edge detection algorithm (Canny edge detection) is used to extract the edge information in the road surface image. Edges often correspond to the boundaries of diseases such as cracks and potholes. The corner detection algorithm (Harris corner detection) is used to identify the corners in the image. Corners represent the significant feature points of road surface diseases. The texture features in the image are analyzed, and the gray-level co-occurrence matrix (GLCM) is used to extract texture feature parameters, which helps to identify different materials or disease types of the road surface;

[0023] For the extraction of physical state features, including lidar point cloud features and temperature and humidity sensor features. Among them, the curvature, normal vector and height change geometric features of the point cloud are calculated to detect the change of road surface flatness. The clustering algorithm (K-means) is used to segment the point cloud to extract the local features of the disease area, which are used to describe the shape and size of the disease, so as to obtain the point cloud feature vector of the disease area. The average value, variance and extreme value statistical features of the temperature and humidity data are calculated to analyze the influence of temperature and humidity changes on the development of diseases, so as to obtain the temperature and humidity feature vector;

[0024] According to the spatial position information, the image features and physical state features are aligned to ensure that each feature point corresponds to the correct road surface position. The image features and physical state features are directly spliced into a feature vector, and different weights are assigned to each feature according to the importance of the features. Then, the fused feature vector is normalized to make each feature value have a unified scale range, which is convenient for subsequent analysis and processing;

[0025] The fused features are arranged according to the spatial position to form a road surface feature sequence. Each position corresponds to a feature vector, which contains the road surface image features and physical state features of that position.

[0026] A further improvement of the technical solution of the present invention lies in that: in the third step, the construction process of the disease recognition and classification model includes:

[0027] Obtain a historical data set of the target road surface area containing road surface disease images and corresponding annotation information, covering various disease types (cracks, potholes, ruts and crazing), and set the reference values of each feature according to the road maintenance requirements. Then, the abnormal image features and physical state features are marked, which are the road surface disease features, and a marked historical road surface feature sequence is formed to train the disease recognition and classification model;

[0028] Divide the historical road surface feature sequence into a training set, a validation set, and a test set, and select a convolutional neural network as the training architecture for the disease recognition and classification model. Design the network structure of the model, including the input layer, hidden layer, output layer, etc., and determine the parameters of the model, such as the learning rate, batch size, number of iterations, etc.;

[0029] Use the training set to train the convolutional neural network, construct a disease recognition and classification model, adjust the parameters of the model to minimize the loss function, and use the backpropagation algorithm to update the weights and biases of the model to improve the accuracy of the model. During the training process, use the validation set to evaluate the model, adjust the parameters and structure of the model according to the evaluation results, and then use the test set to evaluate the trained model, calculate indicators such as the accuracy rate, recall rate, and F1 score of the model, analyze the performance of the model, and identify the advantages and disadvantages of the model;

[0030] Deploy the trained disease recognition and classification model in the road surface planning system, use the feature vectors in the road surface feature sequence as the model input, and identify and classify road surface diseases to output the disease types of the road surface.

[0031] A further improvement of the technical solution of the present invention lies in that: the process of identifying and classifying road surface diseases includes:

[0032] After the disease recognition and classification model is deployed, the feature vectors in the road surface feature sequence are input into the disease recognition and classification model one by one;

[0033] After receiving the feature vector, the disease recognition and classification model performs forward propagation calculation. The forward propagation process includes passing through network structures such as convolutional layers, activation functions, and pooling layers to calculate the probabilities of different types of diseases, and finally outputs the probability distribution of disease types;

[0034] According to the probability distribution output by the disease recognition and classification model, determine the disease type corresponding to each feature vector, and set a disease classification threshold to determine the category with the highest output probability as the disease type.

[0035] A further improvement of the technical solution of the present invention lies in that: in the fourth step, the process of analyzing the expansion trend of diseases includes:

[0036] Integrate the data of road surface monitoring sensors with the disease type data output by the road surface disease recognition and classification model to ensure the consistency of sensor data and disease data in time and space, and use statistical analysis methods to establish a relationship model between environmental condition characteristics and disease types to identify environmental condition characteristics that have a significant impact on the expansion trend of diseases;

[0037] Extract environmental condition features related to pavement diseases from pavement monitoring sensor data, including temperature features and humidity features. Among them, for temperature features, analyze the daily average temperature, maximum temperature, and minimum temperature of the target pavement area. For humidity features, analyze the relative humidity and absolute humidity of the target pavement area, and then analyze the impact of temperature changes and humidity changes on diseases;

[0038] Using the analysis results of regression analysis and the relationship model between environmental condition features and disease types, determine the characteristic reference values of the temperature features and humidity features that affect the spread of diseases. Among them, the characteristic reference value represents the critical state of disease spread under specific environmental conditions;

[0039] Combining the extracted temperature features and humidity features, as well as the characteristic reference values that affect the spread of diseases, calculate the disease spread tendency index, analyze the change trend of diseases under different environmental conditions, and generate a disease spread trend chart to visualize the change law of diseases over time;

[0040] Preset a disease risk threshold, evaluate the disease risk of the pavement in combination with the disease spread tendency index, identify high-risk disease areas, and formulate targeted maintenance plans and measures for high-risk areas, including repair priority ranking, resource allocation, and maintenance strategies, to reduce the impact of diseases on road performance.

[0041] A further improvement of the technical solution of the present invention is that: the identification process of the high-risk disease area includes:

[0042] According to the disease type and road use requirements, set a disease risk threshold D t , refer to the relationship model between environmental condition features and disease types, and determine the characteristic reference values that affect the spread of diseases;

[0043] Combining the identification and classification results of current pavement diseases and the data of pavement monitoring sensors, calculate the disease spread tendency index of each monitoring point, and compare the calculated disease spread tendency index with the preset disease risk threshold D t for comparison to identify high-risk disease areas;

[0044] Rank the high-risk areas according to the magnitude of the disease spread tendency index D, formulate a repair priority list, mark the monitoring points classified as high-risk areas and their disease types, and give priority to repairing the areas with higher disease spread tendency index values;

[0045] Allocate maintenance resources according to the number of high-risk areas and the severity of diseases, prioritize the repair work in high-risk areas to ensure the effective utilization of resources, implement maintenance strategies, repair diseases in high-risk areas, such as crack filling and pothole repair, and monitor the repaired areas after repair, evaluate the repair effect, and regularly check the disease development in the repaired areas to ensure the effectiveness of the repair measures.

[0046] A further improvement of the technical solution of the present invention lies in: in step five, the process of calibrating the position of the disease includes:

[0047] During the process of collecting images to identify and classify road diseases, use the GPS positioning system to obtain the coordinate information of the target road area, including longitude, latitude, and elevation, and ensure that the GPS positioning accuracy reaches centimeter level or higher to meet the requirements of precise disease positioning;

[0048] Associate the disease recognition results with the corresponding coordinate information to form a dataset containing disease types, position coordinates (longitude and latitude), and risk information, and ensure that each disease record contains disease types, coordinate information (longitude and latitude), and risk information;

[0049] Select a road surface planning system based on the GIS (Geographic Information System) platform, convert the disease dataset into a format supported by the system, and then import the dataset containing disease types, coordinate information (longitude and latitude), and risk information into the road surface planning system. The road surface planning system accurately calibrates the position of the disease in the road surface map according to the coordinate information of the disease.

[0050] A further improvement of the technical solution of the present invention lies in: in step six, the process of generating the disease distribution map includes:

[0051] Based on the position calibration results of the diseases, generate a disease distribution map on the road surface planning system, and mark the disease types, coordinate information (longitude and latitude), and expansion trends on the disease distribution map to visually display the positions and types of the diseases;

[0052] On the disease distribution map, use arrows and heat maps to represent the expansion directions and intensities of the diseases, and adjust the zoom level, layer order, and annotation style parameters of the map to make the map display clearer and more intuitive;

[0053] Use the report automatic generation module to generate a detailed list in Excel format for the disease information and generate a road disease analysis report, including the disease distribution map, detailed list, and repair suggestions.

[0054] Due to the adoption of the above technical solution, the technical progress achieved by the present invention compared with the prior art is:

[0055] 1. The present invention provides an intelligent diagnosis method for road surface diseases based on image recognition. Through high-definition cameras and image processing technologies, it can quickly and accurately identify various disease types such as cracks, potholes, and ruts. Moreover, by combining deep learning algorithms, it can automatically extract disease features, achieve high-precision identification of road diseases, significantly improve the disease detection efficiency, reduce manual intervention, and provide more timely and accurate data support for road maintenance.

[0056] 2. The present invention provides an intelligent diagnosis method for road surface diseases based on image recognition. By combining the disease recognition results with coordinate information and importing them into a GIS-based road surface planning system, it can visually mark the type, location, and expansion trend of diseases on the disease distribution map. The visual display method not only facilitates maintenance personnel to quickly understand the disease distribution situation but also provides accurate geographical information for disease repair and road maintenance, optimizes maintenance decisions, and enhances the pertinence and effectiveness of maintenance work.

[0057] 3. The present invention provides an intelligent diagnosis method for road surface diseases based on image recognition. By presetting disease risk thresholds, it can identify high-risk disease areas, early warning of the potential development of diseases, which helps to formulate targeted maintenance plans and measures, including repair priority ranking, resource allocation, and preventive maintenance strategies, thereby reducing the impact of diseases on road performance, extending the service life of roads, and reducing the threat of sudden diseases to traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0059] Figure 1 It is a schematic flow diagram of the present invention;

[0060] Figure 2 It is a schematic flow diagram of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0062] Example 1, as Figure 1 、 Figure 2As shown in the figure, the present invention provides an intelligent diagnosis method for road surface diseases based on image recognition, including the following steps:

[0063] Step 1: Deploy image acquisition devices and road surface monitoring sensors in the target road surface area to obtain multi-source road surface data. According to road maintenance requirements, determine the target road surface area to be monitored, clarify the monitoring objectives, determine the disease types to be monitored and the requirements for disease location accuracy, and then select image acquisition devices and road surface monitoring sensors according to the monitoring objectives. Determine the deployment positions, quantities, and data acquisition frequencies of the image acquisition devices and road surface monitoring sensors. Among them, the disease types include cracks, potholes, ruts, and crazing. The image acquisition device is a high-definition camera. Install a high-definition camera in the target road surface area to ensure that the camera can clearly capture the road surface condition. The installation position is selected in an area with an open view and no obstruction to obtain the best shooting effect. The road surface monitoring sensors are lidar and temperature-humidity sensors. Install the road surface monitoring sensors at appropriate positions near the road surface to ensure that the road surface environmental conditions can be accurately measured. Start the image acquisition device and collect road surface images at a preset frequency. Start the road surface monitoring sensors and collect the physical state data of the road surface. Use Internet of Things technologies (such as 5G, NB-IoT, 4G, etc.) to transmit the image data and sensor data to the data processing center. In the data processing center, synchronize and process the image data and sensor data according to the time stamp and device identifier to form a road surface monitoring data set with multi-source data fusion;

[0064] Step 2: Preprocess the multi-source pavement data and perform feature extraction to obtain the pavement feature sequence. Arrange the data of each collection point in chronological order, compare adjacent data, find the data with the same geographical location information, only retain the data recorded for the first time, and delete the other data. Perform unified preprocessing and analysis on the multi-source pavement data in the pavement monitoring dataset. Among them, perform grayscale conversion, normalization, denoising, enhancement, and binarization on the image data. Convert the color image to a grayscale image through grayscale conversion and normalization to reduce data redundancy, and perform normalization processing on the image to unify the pixel value range. Use filtering algorithms (such as Gaussian filtering, median filtering) to remove noise in the image. At the same time, enhance the image contrast through methods such as histogram equalization, and convert the image to a binary image for subsequent feature extraction. Perform preprocessing operations of point cloud denoising, point cloud segmentation, and coordinate transformation on the lidar point cloud data. Remove noise points in the point cloud through statistical filtering or voxel-based methods, and segment the point cloud data into different regions to extract local features. Convert the point cloud data from the sensor coordinate system to a unified global coordinate system to ensure spatial consistency with the image data. Perform preprocessing operations of data cleaning and data synchronization on the data of the temperature and humidity sensors. Remove outliers and missing values in the sensor data through data cleaning, and align the timestamps collected by different sensors through data synchronization to ensure the temporal consistency of the image data, point cloud data, temperature and humidity data, etc. Perform feature extraction on the preprocessed multi-source pavement data, extract image features and physical state features from it, and fuse the image features and physical state features. Arrange the extracted features in spatial position to form a pavement feature sequence;

[0065] In addition, the process of obtaining the pavement feature sequence includes:

[0066] For the extraction of image features, including edge, corner, and texture features. Among them, the edge detection algorithm (Canny edge detection) is used to extract edge information in the road surface image. Edges often correspond to the boundaries of diseases such as cracks and potholes. The corner detection algorithm (Harris corner detection) is used to identify corners in the image. Corners represent significant feature points of road surface diseases. The texture features in the image are analyzed, and the gray-level co-occurrence matrix (GLCM) is used to extract texture feature parameters, which helps to identify different road surface materials or disease types. For the extraction of physical state features, including lidar point cloud features and temperature and humidity sensor features. Among them, the curvature, normal vector, and height change geometric features of the point cloud are calculated to detect the flatness change of the road surface. The clustering algorithm (K-means) is used to segment the point cloud and extract the local features of the disease area to describe the shape and size of the disease, so as to obtain the point cloud feature vector of the disease area. The average value, variance, and extreme value statistical features of the temperature and humidity data are calculated, and the influence of temperature and humidity changes on the development of diseases is analyzed to obtain the temperature and humidity feature vector. According to the spatial position information, the image features and physical state features are aligned to ensure that each feature point corresponds to the correct road surface position. The image features and physical state features are directly spliced into a feature vector, and different weights are assigned to each feature according to the importance of the feature. Then, the fused feature vector is normalized so that each feature value has a unified scale range for subsequent analysis and processing. The fused features are arranged according to the spatial position to form a road surface feature sequence, and each position corresponds to a feature vector, which contains the road surface image features and physical state features of that position

[0067] Step 3: Based on the deep learning algorithm, combine with the pavement feature sequence to learn the features of different disease types, construct a disease recognition and classification model, identify different types of pavement diseases, obtain a historical dataset of the target pavement area containing pavement disease images and corresponding annotation information, covering multiple disease types (cracks, potholes, ruts, and crocodile cracks), and set the benchmark values of each feature according to the road maintenance requirements. Then, annotate the abnormal image features and physical state features among them as pavement disease features, forming a marked historical pavement feature sequence for training the disease recognition and classification model. Divide the historical pavement feature sequence into a training set, a validation set, and a test set, and select a convolutional neural network as the training architecture of the disease recognition and classification model. Design the network structure of the model, including the input layer, hidden layer, output layer, etc., and determine the parameters of the model, such as the learning rate, batch size, number of iterations, etc. Use the training set to train the convolutional neural network to construct the disease recognition and classification model, adjust the parameters of the model to minimize the loss function, and use the backpropagation algorithm to update the weights and biases of the model to improve the accuracy of the model. During the training process, use the validation set to evaluate the model, adjust the parameters and structure of the model according to the evaluation results, and use techniques such as data augmentation, regularization, and dropout to prevent the model from overfitting. Use techniques such as transfer learning to accelerate the convergence speed of the model and improve the generalization ability of the model. Then, use the test set to evaluate the trained model, calculate indicators such as the accuracy, recall rate, and F1 score of the model, analyze the performance of the model, identify the advantages and disadvantages of the model, and deploy the trained disease recognition and classification model in the pavement planning system. Use the feature vectors in the pavement feature sequence as the model input to identify and classify pavement diseases to output the disease types of the pavement;

[0068] In addition, the process of identifying and classifying pavement diseases includes:

[0069] After the disease recognition and classification model is deployed, input the feature vectors in the pavement feature sequence into the disease recognition and classification model one by one;

[0070] After receiving the feature vectors, the disease recognition and classification model performs forward propagation calculations. The forward propagation process includes passing through network structures such as convolutional layers, activation functions, and pooling layers to calculate the probabilities of different types of diseases, and finally outputs the probability distribution of the disease types;

[0071] According to the probability distribution output by the disease recognition and classification model, determine the disease type corresponding to each feature vector, and set a disease classification threshold to determine the category with the highest output probability as the disease type;

[0072] The calculation expression for the probabilities of different types of diseases is:

[0073]

[0074]

[0075] Wherein, P i is the probability of the i-th type of disease, C is the total number of disease types, Z is the normalization constant used to ensure that the sum of all probabilities is 1, and f j is the j-th eigenvalue in the feature vector, and b j is the reference value of the j-th feature, which is used to measure the difference between the feature and the reference. F is the dimension of the feature vector, that is, the total number of features, and σ i is the feature standard deviation of the i-th type of disease, which is used to adjust the width of the probability distribution. exp(·) is the exponential function used to convert the feature difference into a probability. The value range of P i is [0, 1]. When the difference between the feature and the reference value is small, the probability value is close to 1. When the difference is large, the probability value is close to 0. By calculating the difference between the feature vector and the reference value and combining the exponential function and the normalization constant, the probability distribution of each disease type is generated. This method can effectively reflect the correlation between the feature and the disease type, and at the same time ensure the rationality of the probability distribution through normalization;

[0076] Step Four: Combine the data of the road surface monitoring sensors to analyze the environmental condition information of the road surface diseases so as to analyze the development trend of the diseases;

[0077] Step Five: Combine the identified diseases with the high-precision positioning data to obtain the coordinate information of the diseases and mark the positions of the diseases in the road surface planning system;

[0078] Step Six: Visualize the results of the image recognition and the positioning information of the diseases to generate a disease distribution map, and mark the types, positions and development trends of the diseases.

[0079] Example 2, as Figure 1 、 Figure 2 shown. On the basis of Example 1, the present invention provides a technical solution: Preferably, in Step Four, the process of analyzing the development trend of the diseases includes:

[0080] Integrate the data from pavement monitoring sensors with the disease type data output by the pavement disease identification and classification model to ensure the consistency of sensor data and disease data in terms of time and space. Use statistical analysis methods to establish a relationship model between environmental condition characteristics and disease types, identify environmental condition characteristics that have a significant impact on the disease development trend, and extract environmental condition characteristics related to pavement diseases from the pavement monitoring sensor data, including temperature characteristics and humidity characteristics. Among them, for temperature characteristics, analyze the daily average temperature, maximum temperature, and minimum temperature of the target pavement area. For humidity characteristics, analyze the relative humidity and absolute humidity of the target pavement area. Then, analyze the impact of temperature changes and humidity changes on diseases. Use the analysis results of regression analysis and the relationship model between environmental condition characteristics and disease types to determine the characteristic benchmark values of temperature characteristics and humidity characteristics that affect disease development. The characteristic benchmark value represents the critical state of disease development under specific environmental conditions. Combine the extracted temperature characteristics and humidity characteristics, as well as the characteristic benchmark values that affect disease development, calculate the disease development trend index, analyze the change trend of diseases under different environmental conditions, and generate a disease development trend chart to visualize the change law of diseases over time. Preset a disease risk threshold, combine the disease development trend index to evaluate the disease risk of the pavement, identify high-risk disease areas, and formulate targeted maintenance plans and measures for high-risk areas, including repair priority ranking, resource allocation, and maintenance strategies, to reduce the impact of diseases on road performance;

[0081] In addition, the process of identifying high-risk disease areas includes:

[0082] Set the disease risk threshold D according to the disease type and road usage requirements t , refer to the relationship model between environmental condition characteristics and disease types to determine the characteristic benchmark values that affect disease development. Combine the identification and classification results of current pavement diseases and the data from pavement monitoring sensors, calculate the disease development trend index for each monitoring point, and compare the calculated disease development trend index with the preset disease risk threshold D to identify high-risk disease areas. Sort the high-risk areas according to the magnitude of the disease development trend index D, formulate a repair priority list, mark the monitoring points classified as high-risk areas and their disease types, and give priority to repairing areas with higher disease development trend index values. Allocate maintenance resources according to the number and severity of high-risk areas, give priority to ensuring the repair work in high-risk areas to ensure the effective use of resources, implement maintenance strategies, repair diseases in high-risk areas, such as crack filling and pothole patching, and monitor the repaired areas after repair to evaluate the repair effect, and regularly check the disease development in the repaired areas to ensure the effectiveness of the repair measures;

[0083] The calculation expression of the disease development trend index is:

[0084]

[0085] In the formula, D is the disease spread tendency index, and T avg , T max and T min represent the daily average temperature, the highest temperature, and the lowest temperature respectively. RH and AH represent the relative humidity and the absolute humidity respectively. T avg，b is the reference value of the daily average temperature, T max，b is the reference value of the highest temperature, T min，b is the reference value of the lowest temperature, RH b is the reference value of the relative humidity, AH b is the reference value of the absolute humidity, used to measure the difference between the current environmental conditions and the reference conditions. The value range of D is [0, 1]. When D is close to 1, it indicates a strong disease spread trend, and the environmental conditions are more favorable for the development of the disease. When D is close to 0, it indicates a weak disease spread trend, and the environmental conditions are less favorable for the development of the disease. The greater the difference between the temperature and humidity and the reference values, the stronger the disease spread trend. The increase in relative humidity and absolute humidity will enhance the disease spread trend. The logarithmic function log(RH + RH b ) and the square root function are used to adjust the weights of the humidity characteristics, making the impact of humidity changes on the disease spread tendency more significant. The reference values of relative humidity and absolute humidity represent normal or standard environmental conditions. By adding the reference values, the current humidity conditions can be compared with the standard conditions, thus better reflecting the impact of humidity changes on the disease;

[0086] In step five, the process of calibrating the location of the disease includes:

[0087] During the process of collecting images to identify and classify road surface diseases, use the GPS positioning system to obtain the coordinate information of the target road surface area, including longitude, latitude, and elevation, ensuring that the GPS positioning accuracy reaches centimeter level or higher to meet the needs of precise disease positioning. Associate the disease recognition results with the corresponding coordinate information to form a data set containing disease types, location coordinates (longitude and latitude), and risk information, ensuring that each disease record contains disease types, coordinate information (longitude and latitude), and risk information. Select a road surface planning system based on the GIS (Geographic Information System) platform, convert the disease data set into a format supported by the system, and then import the data set containing disease types, coordinate information (longitude and latitude), and risk information into the road surface planning system. The road surface planning system accurately calibrates the location of the disease in the road surface map according to the coordinate information of the disease;

[0088] In step six, the process of generating the disease distribution map includes:

[0089] Based on the calibration results of the disease location, a disease distribution map is generated on the road surface planning system, and the type, coordinate information (longitude, latitude) and expansion trend of the disease are marked on the disease distribution map to visually display the location and type of the disease. On the disease distribution map, arrows and heat maps are used to represent the expansion direction and intensity of the disease, and the zoom level, layer order and annotation style parameters of the map are adjusted to make the map display clearer and more intuitive. Using the report automatic generation module, the disease information is generated into a detailed list in Excel format, and a road disease analysis report is generated, including the disease distribution map, the detailed list and repair suggestions.

[0090] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An intelligent diagnosis method for road surface diseases based on image recognition, characterized in that, It includes the following steps: Step 1: Deploy image acquisition devices and road surface monitoring sensors in the target road surface area to obtain multi-source road surface data; Step 2: Preprocess the multi-source road surface data and perform feature extraction to obtain the road surface feature sequence; Step 3: Based on the deep learning algorithm, combine the road surface feature sequence to learn the features of different disease types, construct a disease recognition and classification model, and identify different types of road surface diseases; Step 4: Combine the data of the road surface monitoring sensors to analyze the environmental condition information of the road surface diseases to analyze the expansion trend of the diseases; Step 5: Combine the identified diseases with the high-precision positioning data to obtain the coordinate information of the diseases, and mark the positions of the diseases in the road surface planning system; Step 6: Visualize the results of image recognition and the positioning information of the diseases to generate a disease distribution map, and mark the types, positions and expansion trends of the diseases.

2. The intelligent diagnosis method for road surface diseases based on image recognition according to claim 1, wherein: In the said Step 1, the process of obtaining the multi-source road surface data includes: According to the road maintenance requirements, determine the target road surface area to be monitored, clarify the monitoring objectives, determine the disease types to be monitored and the disease positioning accuracy requirements, and then select image acquisition devices and road surface monitoring sensors according to the monitoring objectives, determine the deployment positions, quantities and data acquisition frequencies of the image acquisition devices and road surface monitoring sensors. Among them, the disease types include cracks, potholes, ruts and crazing. The image acquisition device is a high-definition camera, and the road surface monitoring sensors are lidar and temperature and humidity sensors; Start the image acquisition device, collect road surface images at a preset frequency, start the road surface monitoring sensors, collect the physical state data of the road surface, and use the Internet of Things technology to transmit the image data and sensor data to the data processing center; In the data processing center, synchronize the image data and sensor data according to the time stamp and device identifier to form a road surface monitoring data set with multi-source data fusion.

3. The intelligent diagnosis method for road surface diseases based on image recognition according to claim 2, characterized in that: In the said Step 2, the process of obtaining the road surface feature sequence includes: Arrange the data of each acquisition point in chronological order, compare the adjacent data, find the data with the same geographical location information, only retain the data recorded for the first time, and delete the other data; Perform unified preprocessing and analysis on the multi-source road surface data in the road surface monitoring data set. Among them, perform grayscale conversion and normalization, denoising and enhancement, and binarization processing on the image data, perform preprocessing operations such as point cloud denoising, point cloud segmentation and coordinate conversion on the lidar point cloud data, and perform preprocessing operations such as data cleaning and data synchronization on the data of the temperature and humidity sensors; Perform feature extraction on the preprocessed multi-source road surface data, extract image features and physical state features from it, fuse the image features and physical state features, and arrange the extracted features in spatial positions to form a road surface feature sequence.

4. The intelligent diagnosis method for road surface diseases based on image recognition according to claim 3, characterized in that: The process of obtaining the said road surface feature sequence includes: For the extraction of image features, it includes edge, corner and texture features. Among them, use the edge detection algorithm to extract the edge information in the road surface image, use the corner detection algorithm to identify the corners in the image, analyze the texture features in the image, and use the gray level co-occurrence matrix to extract the texture feature parameters; For the extraction of physical state features, including lidar point cloud features and temperature and humidity sensor features. Among them, calculate the curvature, normal vector, and height change geometric features of the point cloud to detect the change in road surface flatness. Use the clustering algorithm to segment the point cloud and extract the local features of the disease area to describe the shape and size of the disease, so as to obtain the point cloud feature vector of the disease area. Calculate the average value, variance, and extreme value statistical features of the temperature and humidity data, and analyze the impact of temperature and humidity changes on the development of the disease to obtain the temperature and humidity feature vector; According to the spatial position information, align the image features and physical state features, directly splice the image features and physical state features into a feature vector, and assign different weights to each feature according to the importance of the feature, and then perform normalization processing on the fused feature vector; Arrange the fused features according to the spatial position to form a road surface feature sequence, and each position corresponds to a feature vector, which contains the road surface image features and physical state features of that position.

5. The intelligent diagnosis method for road surface diseases based on image recognition according to claim 4, characterized in that: In step three, the construction process of the disease recognition and classification model includes: Obtain a historical dataset of the target road surface area containing road surface disease images and corresponding annotation information, covering various disease types, and set the benchmark values of each feature according to the road maintenance requirements. Then annotate the abnormal image features and physical state features among them as road surface disease features to form a marked historical road surface feature sequence; Divide the historical road surface feature sequence into a training set, a validation set, and a test set, and select a convolutional neural network as the training architecture of the disease recognition and classification model, and design the network structure of the model; Use the training set to train the convolutional neural network to construct a disease recognition and classification model, adjust the parameters of the model to minimize the loss function, and use the backpropagation algorithm to update the weights and biases of the model. During the training process, use the validation set to evaluate the model, adjust the parameters and structure of the model according to the evaluation results, and then use the test set to evaluate the trained model to analyze the performance of the model and identify the advantages and disadvantages of the model; Deploy the trained disease recognition and classification model in the road surface planning system, use the feature vector in the road surface feature sequence as the model input, and identify and classify the road surface diseases to output the disease types of the road surface.

6. The intelligent diagnosis method for road surface diseases based on image recognition according to claim 5, characterized in that: The process of identifying and classifying road surface diseases includes: After the disease recognition and classification model is deployed, input the feature vectors in the road surface feature sequence into the disease recognition and classification model one by one; After receiving the feature vector, the disease recognition and classification model performs forward propagation calculation. The forward propagation process includes passing through the network structure of the convolutional layer, activation function, and pooling layer to calculate the probabilities of different types of diseases, and finally output the probability distribution of the disease types; According to the probability distribution output by the disease recognition and classification model, determine the disease type corresponding to each feature vector, set the disease classification threshold, and determine the disease type with the highest output probability as the disease type.

7. The intelligent diagnosis method for road surface diseases based on image recognition according to claim 6, characterized in that: In step four, the process of analyzing the expansion trend of the disease includes: Integrate the data from road surface monitoring sensors with the disease type data output by the road surface disease identification and classification model, and use statistical analysis methods to establish a relationship model between environmental condition characteristics and disease types, and identify the environmental condition characteristics that have a significant impact on the disease expansion trend; Extract environmental condition characteristics related to road surface diseases from the road surface monitoring sensor data, including temperature characteristics and humidity characteristics. Among them, for temperature characteristics, analyze the daily average temperature, maximum temperature, and minimum temperature of the target road surface area. For humidity characteristics, analyze the relative humidity and absolute humidity of the target road surface area, and then analyze the impact of temperature changes and humidity changes on diseases; Use regression analysis and the analysis results of the relationship model between environmental condition characteristics and disease types to determine the characteristic reference values of temperature characteristics and humidity characteristics that affect disease expansion; Combine the extracted temperature characteristics and humidity characteristics, as well as the characteristic reference values that affect disease expansion, calculate the disease expansion trend index, analyze the change trend of diseases under different environmental conditions, and generate a disease expansion trend chart; Preset a disease risk threshold, combine the disease expansion trend index to evaluate the disease risk of the road surface, identify high-risk disease areas, and formulate targeted maintenance plans and measures for high-risk areas, including repair priority ranking, resource allocation, and maintenance strategies.

8. An intelligent diagnosis method for road surface diseases based on image recognition according to claim 7, characterized in that: The identification process of the high-risk disease area described above includes: Set a disease risk threshold according to the disease type and road use requirements, refer to the relationship model between environmental condition characteristics and disease types, and determine the characteristic reference values that affect disease expansion; Combine the identification and classification results of current road surface diseases and the data of road surface monitoring sensors, calculate the disease expansion trend index of each monitoring point, and compare the calculated disease expansion trend index with the preset disease risk threshold to identify high-risk disease areas; Sort the high-risk areas according to the magnitude of the disease expansion trend index, formulate a repair priority list, mark the monitoring points classified as high-risk areas and their disease types, and give priority to repairing the areas with higher disease expansion trend index values; Allocate maintenance resources according to the number and disease severity of high-risk areas, give priority to ensuring the repair work of high-risk areas, implement maintenance strategies, repair diseases in high-risk areas, and monitor the repair area after repair to evaluate the repair effect, and regularly check the disease development of the repaired area.

9. The intelligent diagnosis method for road surface diseases based on image recognition according to claim 8, characterized in that: In the process described in step five above, the process of calibrating the location of the disease includes: During the process of collecting images to identify and classify road surface diseases, use the GPS positioning system to obtain the coordinate information of the target road surface area, including longitude, latitude, and elevation; Associate the disease identification result with the corresponding coordinate information to form a data set containing disease type, location coordinates, and risk information, ensuring that each disease record contains disease type, coordinate information, and risk information; Select a road surface planning system based on the GIS platform, convert the disease data set into a format supported by the system, and then import the data set containing disease type, coordinate information (longitude and latitude), and risk information into the road surface planning system. The road surface planning system calibrates the location of the disease on the road surface map according to the coordinate information of the disease.

10. The intelligent diagnosis method for road surface diseases based on image recognition according to claim 9, characterized in that: In the sixth step, the generation process of the disease distribution map includes: Based on the location calibration results of the diseases, generate a disease distribution map on the road surface planning system, and mark the type, coordinate information, and expansion trend of the diseases on the disease distribution map to visually display the location and type of the diseases; On the disease distribution map, use arrows and heat maps to represent the expansion direction and intensity of the diseases, and adjust the zoom level, layer order, and annotation style parameters of the map; Utilize the report automatic generation module to generate a detailed list in Excel format with the disease information, and generate a road disease analysis report, including the disease distribution map, the detailed list, and repair suggestions.

Images