A deep learning-based pavement disease automatic detection method and system

Abstract

The application relates to the technical field of automatic road disease detection, and particularly discloses a road disease automatic detection method and system based on deep learning, which comprises a municipal road data acquisition module, a road image quality analysis module, a road image reacquisition module, a road disease evaluation feedback module and a database; the application dynamically adjusts flight parameters (speed and height) through an environmental quality index (EQI), ensures image acquisition quality, improves detection precision in complex environments, constructs a multi-disease coupling risk model, comprehensively considers regional weight and traffic volume dynamic weight, outputs disease risk grades, finally predicts disease development trends through a once exponential smoothing method, realizes double functions of 'current status evaluation+trend early warning', predicts disease evolution trends based on historical data, significantly improves the scientificity and timeliness of municipal road risk management, reduces maintenance cost and prevents safety accidents.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of automatic detection of road surface diseases, and relates to an automatic detection method and system for road surface diseases based on deep learning. BACKGROUND

[0002] With the acceleration of urbanization, the maintenance demand of municipal roads is increasing. At present, road surface disease detection mainly relies on manual inspection or traditional image processing technology. Manual inspection is low in efficiency and high in cost, and is limited by the recognition accuracy of the human eye, making it difficult to find small diseases. Traditional image processing methods are usually based on fixed thresholds or simple feature extraction, which are easily disturbed by environmental factors such as light changes, weather conditions (such as rain and fog), resulting in high false detection and missed detection rates. Therefore, in order to realize accurate disease risk assessment and trend prediction, it is necessary to develop an automatic detection method for road surface diseases based on deep learning.

[0003] For example, Chinese patent CN117671495A discloses a real-time automatic detection method and system for road surface diseases based on edge computing technology, which includes receiving road surface original image data and automatic identification request, decoding and preprocessing the road surface original image data to generate road surface image data, performing deep neural network calculation on the road surface image data to obtain the detection type of the disease and the detection frame of the disease location, using a non-maximum suppression algorithm to retain the detection frame with the highest confidence, optimizing the data of the detection frame with the highest confidence based on the connectivity principle, determining the optimal detection frame position, generating response data for the disease type and location identification result, and returning to the road condition collection device to improve the edge data processing efficiency and realize real-time identification and processing of road surface disease image data.

[0004] The above prior art has the following problems: 1. Only relying on fixed road condition collection devices to receive original images, without considering the influence of environmental changes on image quality, which cannot ensure the image quality in harsh environments, resulting in decreased detection accuracy.

[0005] 2. Only relying on deep neural network detection frame output, lacking quantitative analysis of disease geometric parameters (such as crack width and depression depth), and the detection result is not linked to traffic load and regional function, which limits its practicality.

[0006] 3. Only returning the disease type and location, without providing risk level assessment and road surface disease development trend prediction, without predicting the disease evolution law based on historical data, and unable to timely feedback the development trend of the disease, which cannot support the development of priority maintenance decision of municipal roads. SUMMARY

[0007] In view of the above problems in the background art, an automatic detection method and system for road surface diseases based on deep learning are proposed.

[0008] The purpose of the application can be achieved by the following technical solutions: the first aspect of the application provides a road disease automatic detection method based on deep learning, comprising: S1, the unmanned aerial vehicle collects natural environment data and road image data of municipal roads at position time through the sensor and camera.

[0009] S2, the natural environment data is coupled with the historical illumination data to analyze the environment quality index of the municipal road at the current time, and the flight parameter adjustment mechanism is triggered when the environment quality index is within the set environment quality index range.

[0010] S3, the image reacquisition flight parameters of the unmanned aerial vehicle are adjusted based on the flight parameter adjustment mechanism, and the reacquisition image with the image quality index meeting the standard is used as the final road image.

[0011] S4, the road disease categories and the corresponding characteristic parameter sets of various road diseases are identified from the collected road images, the disease risk assessment index and the development trend assessment index of various road diseases of the municipal road are constructed, and the disease risk grade and the influence trend grade of various road diseases are output and fed back correspondingly.

[0012] The second aspect of the application provides a road disease automatic detection system based on deep learning, comprising: a municipal road data acquisition module, which is used for unmanned aerial vehicle to collect natural environment data and road image data of municipal roads at position time through the sensor and camera.

[0013] The road image quality analysis module is used for coupling the natural environment data with the historical illumination data to analyze the environment quality index of the municipal road at the current time, and the flight parameter adjustment mechanism is triggered when the environment quality index is within the set environment quality index range.

[0014] The road image reacquisition module is used for adjusting the image reacquisition flight parameters of the unmanned aerial vehicle based on the flight parameter adjustment mechanism, and the reacquisition image with the image quality index meeting the standard is used as the final road image.

[0015] The road disease assessment feedback module is used for identifying the road disease categories and the corresponding characteristic parameter sets of various road diseases from the collected road images, constructing the disease risk assessment index and the development trend assessment index of various road diseases of the municipal road, and outputting and feeding back the disease risk grade and the influence trend grade of various road diseases correspondingly.

[0016] Compared with the prior art, the embodiments of the present application have at least the following advantages or beneficial effects: (1) The present application synchronously collects natural environment data and road surface images by combining the sensors and cameras carried by the unmanned aerial vehicle, dynamically adjusts the flight parameters (speed, height) through the environment quality index, ensures the image collection quality, and improves the detection accuracy in complex environments.

[0017] (2) The present application breaks through the traditional single disease assessment mode by constructing a multi-disease coupling risk model, comprehensively considering the regional weight, traffic volume dynamic weight, and outputting the disease risk grade. It realizes multi-dimensional risk quantification and dynamic adaptability optimization, can capture the synergistic effect of diseases such as cracks and depressions, can real-time adjust the regional weight according to the road function grade and surrounding environment, and can dynamically correct the risk coefficient combined with traffic flow, load, etc., to form a precise risk portrait.

[0018] (3) The present application realizes the dual functions of "current status assessment + trend warning" by predicting the disease development trend through the exponential smoothing method, predicts the disease evolution trend based on historical data, significantly improves the scientificity and timeliness of municipal road risk management, reduces the maintenance cost and prevents safety accidents. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 The present application is a method step flowchart.

[0021] Figure 2 The present application is a system structure connection schematic diagram.

[0022] Figure 3 The present application is a trigger flight parameter adjustment mechanism judgment flowchart. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0024] Please refer to Figure 1As shown, the first aspect of the present application provides a deep learning-based automatic detection method for road surface diseases, comprising: S1, a UAV collects natural environment data and road surface image data of municipal roads at a position and time sequence by carrying sensors and cameras.

[0025] S2, coupling the natural environment data with historical illumination data to analyze the environmental quality index of municipal roads at the current time, and triggering the flight parameter adjustment mechanism when the environmental quality index is within the set environmental quality index range.

[0026] In an embodiment of the present application, the specific process of coupling the natural environment data with historical illumination data to analyze the environmental quality index of municipal roads at the current time is: coupling the current illumination intensity, environmental humidity and visibility in the natural environment data with the maximum illumination intensity, environmental humidity critical threshold and minimum visibility for image acquisition in the historical illumination time stored in the database to obtain the environmental quality index of municipal roads at the current time.

[0027] It should be noted that the process of obtaining the environmental quality index of municipal roads at the current time is: first, the current illumination intensity is compared with the maximum illumination intensity in the historical illumination time, the ratio is squared and multiplied by the weight coefficient of illumination intensity, second, the current environmental humidity is compared with the humidity critical threshold, the ratio is used as the power of the exponential function with e as the base, and then the function result is multiplied by the weight coefficient of environmental humidity, third, the minimum visibility for image acquisition is compared with the current visibility, the ratio is squared and multiplied by the weight coefficient of visibility, fourth, the results of the above three steps are added to obtain the final environmental quality index.

[0028] It should also be noted that visibility directly determines image clarity and recognizability, illumination intensity affects image brightness and contrast, and high environmental humidity may cause road surface reflection, water accumulation or fog, which indirectly affects image quality, therefore, in environmental quality assessment, the importance of visibility is greater than that of illumination intensity and environmental humidity, therefore, in an embodiment of the present application, the weight coefficients of illumination intensity, environmental humidity and visibility corresponding to environmental quality assessment can be respectively taken as 0.5, 0.3 and 0.2.

[0029] Among them, the current illumination intensity, environmental humidity and visibility in the natural environment data are detected by photodiodes, humidity sensors and transmission type visibility sensors respectively, and the detection principles and processes of the above three are all prior art, which will not be repeated here.

[0030] Please refer to Figure 3As shown, the environmental quality index of the municipal road at the current time is compared with the range of the environmental quality index corresponding to the flight parameter adjustment mechanism triggered in the database, and if the environmental quality index of the municipal road at the current time is located in the range of the environmental quality index corresponding to the flight parameter adjustment mechanism triggered, it indicates that the quality of the collected road image reaches the flight parameter adjustment mechanism triggered, otherwise, it indicates that the quality of the collected road image does not reach the flight parameter adjustment mechanism triggered.

[0031] S3, based on the image reacquisition flight parameter adjustment mechanism, the image reacquisition flight parameter of the unmanned aerial vehicle is adjusted, and the reacquired image with the image quality index reaching the standard is taken as the final road image.

[0032] In the embodiment of the application, the specific process of adjusting the image reacquisition flight parameter of the unmanned aerial vehicle is: the current flight speed in the current flight parameter of the unmanned aerial vehicle is coupled with the minimum flight speed of the unmanned aerial vehicle preset in the database to obtain the image reacquisition flight speed. The current flight parameter of the unmanned aerial vehicle is directly extracted from the unmanned aerial vehicle control background.

[0033] It should be noted that the image reacquisition flight speed is obtained by the formula , wherein, represents the environmental quality index, represents the minimum flight speed of the unmanned aerial vehicle preset in the database, represents the current flight speed in the current flight parameter of the unmanned aerial vehicle, and respectively represent the speed adjustment proportion coefficient and the environmental quality index threshold.

[0034] In one embodiment of the application, the set value of is 0.2, the set value of is 1.2, as the speed adjustment proportion coefficient, if the value is too large, the speed attenuation amplitude is too high, which may cause the unmanned aerial vehicle flight attitude to lose control, and if the value is too small, the speed cannot be effectively reduced in response to the demand for environmental improvement, therefore, through a large number of debugging and verification, the proportion of 0.2 can not only reduce the flight speed to obtain clearer images when the environmental quality is improved, but also avoid excessive speed fluctuation to ensure the stability of the unmanned aerial vehicle flight, is the environmental quality index threshold, and the threshold of 1.2 is obtained by analyzing the correlation between historical environmental data and image quality, which can avoid frequent speed adjustment due to no significant improvement of environmental quality, and can timely trigger the speed reduction mechanism when the environment is suitable to improve the image acquisition effect.

[0035] The current flight height in the current flight parameter of the unmanned aerial vehicle is comprehensively analyzed with the maximum flight height of the unmanned aerial vehicle preset in the database to obtain the image reacquisition flight height.

[0036] It should be noted that, through the formula The image reacquisition flight altitude was obtained, where, This indicates the preset maximum flight altitude of the drone stored in the database. This indicates the current flight altitude in the drone's current flight parameters. and These represent the height-adjusted baseline coefficient and the moderating factor for the impact of environmental quality on height, respectively.

[0037] In one specific embodiment of the present invention, The value is set to 0.8. The setting value is 0.1, and 0.8 is the base coefficient for height adjustment. If the value is too large, the height adjustment will rely too much on the base coefficient, weakening the effect. The altitude adjustment value, if too small, may lead to excessive altitude adjustment, affecting the flight safety of the drone. 0.8, after testing and verification, achieves a balance between a stable baseline and dynamic adjustment, ensuring that the altitude adjustment meets actual engineering requirements. 0.1 is the adjustment... Regarding the balancing factor for the impact of altitude, if the value is too large, the altitude change will be too drastic; if it is too small, the adjustment effect will be insignificant. Testing has shown that a value of 0.1 ensures that altitude changes proportionally. Reasonable changes: When the environmental quality is high, the altitude is reduced to obtain a clearer image; when the environmental quality is low, the altitude adjustment is slowed down to avoid altitude anomalies caused by environmental interference.

[0038] This invention combines sensors and cameras mounted on a drone to simultaneously collect natural environmental data and road images. By dynamically adjusting flight parameters (speed, altitude) using the Environmental Quality Index (EQI), the quality of image acquisition is ensured, and the detection accuracy in complex environments is improved.

[0039] In a specific embodiment of the present invention, the specific analysis process of using the re-acquired image that meets the image quality index as the final road surface image is as follows: obtain the pixel value of each pixel point from each re-acquired image, calculate the average value of the pixel gradient value of each re-acquired image, and use it as the clarity of each re-acquired image.

[0040] It should be noted that the average pixel gradient value of each re-acquired image is calculated using the following formula: ,in, Indicates the first image in the re-acquired image The horizontal and vertical coordinates of each pixel Indicates the first Pixel values ​​at the coordinates of each pixel. To represent a second-order differential operator, Indicates the pixel number. , represents the number of pixel points.

[0041] According to the pixel brightness of each pixel point obtained in each recaptured image, the brightness variance of each recaptured image is obtained, and the brightness variance is taken as the illumination uniformity of each recaptured image.

[0042] It should be noted that the formula for calculating the brightness variance of each recaptured image is: , wherein, represents the pixel brightness at the pixel point coordinate, represents .

[0043] The recaptured image with the highest definition and the largest illumination uniformity is selected as the final road surface image.

[0044] S4, identifying the road disease type and the characteristic parameter set corresponding to each type of road disease from the collected road surface image, constructing the disease risk assessment index of the municipal road and the development trend assessment index of each type of road disease, and outputting the disease risk grade and the influence trend grade of each type of road disease and corresponding feedback.

[0045] In specific embodiments of the present application, the specific way of identifying the road disease type from the collected road surface image is: based on the historical road disease atlas, constructing the disease characteristic set corresponding to each type of road disease, obtaining the image features of the collected road surface image through preprocessing, comparing the image features with the disease characteristic set corresponding to each type of road disease, if an image feature is located in the disease characteristic set corresponding to a type of road disease, then the type of road disease is taken as the road disease type of the municipal road, thereby obtaining each road disease type of the municipal road.

[0046] In specific embodiments of the present application, the specific process of constructing the disease risk assessment index of the municipal road is: according to the characteristic parameter set corresponding to each type of road disease, obtaining the risk index corresponding to each type of road disease, and bringing it into the composite disease superposition formula to obtain the multi-disease coupling risk index of the market road.

[0047] In one specific embodiment of the present application, the road disease type includes but is not limited to cracks, depressions and potholes.

[0048] It should be noted that the specific process of obtaining the risk index corresponding to each type of road disease according to the characteristic parameter set corresponding to each type of road disease is: extracting the total length of extension and the maximum opening width corresponding to the crack disease, the projection area and the maximum depth corresponding to the depression disease, and the volume corresponding to the pothole disease from the characteristic parameter set corresponding to each type of road disease.

[0049] ​It should be noted that the collection method of the total length of extension and the maximum opening width corresponding to the crack disease is that the total length of extension is automatically calculated and counted by using image recognition technology, the actual maximum opening width is converted by analyzing the pixel ratio through software, the collection method of the projection area and the maximum depth corresponding to the depression disease is that the multi-angle images of the depression area are shot by a drone, a three-dimensional model is constructed, and the projection area and the maximum depth of the depression disease are extracted from the three-dimensional model, and the collection method of the volume corresponding to the pit disease is that multi-angle images of the pit are shot by a drone, a three-dimensional model is generated, and the accurate volume is automatically calculated by software.

[0050] The total length of extension and the maximum opening width corresponding to the crack disease are respectively subjected to dynamic difference calculation with corresponding set values, and the sum of the normalized processing of each dynamic difference value is obtained as the crack risk index of the market road surface.

[0051] It should be noted that the dynamic difference calculation is that the total length of extension is subtracted from the crack warning length set by the municipal road safety criterion to obtain the crack length difference value, and the maximum opening width is subtracted from the safety width set by the municipal road safety criterion to obtain the crack width difference value. The normalized processing is that the crack length difference value is divided by the crack warning length to obtain the crack length difference value normalized processing, and the crack width difference value is divided by the safety width to obtain the crack width difference value normalized processing.

[0052] The projection area and the maximum depth corresponding to the depression disease are respectively subjected to ratio calculation with corresponding set values and summation, and the summation result is taken as the power into the exponential function with e as the base to obtain the depression risk index of the market road surface.

[0053] When the volume corresponding to the pit disease is less than or equal to the pit volume set by the municipal road safety criterion, the ratio of the volume corresponding to the pit disease to the pit volume set by the municipal road safety criterion is taken as the pit risk index of the market road surface, and when the volume corresponding to the pit disease is greater than the pit volume set by the municipal road safety criterion, the result of taking the square of the ratio of the volume corresponding to the pit disease to the pit volume set by the municipal road safety criterion is taken as the pit risk index of the market road surface.

[0054] It should be further noted that the composite disease superposition formula is: wherein, respectively represent the crack risk index, the depression risk index and the pit risk index of the market road surface, The second high index represents the second high value in multiple disease indexes (CSI, DSI, PRI) except the maximum value. By introducing the second high index, the formula not only focuses on the main disease risk, but also considers the secondary risk, avoids the evaluation deviation caused by ignoring other diseases, and at the same time in the engineering evaluation model, 0.3 is a verified compromise parameter. If the coefficient is too high (such as 0.5 or more), the second high index has too much influence on the result, which may cover up the main risk. If it is too low (such as 0.1 or less), it cannot reflect the design intention of 'considering secondary risk'. 0.3 is adjusted repeatedly, so that the model is more in line with the actual damage degree of disease superposition in the primary and secondary risk balance.

[0055] The municipal road region type to which the municipal pavement belongs is extracted from the database, and the municipal pavement corresponding region weight is obtained by comparing the region weight corresponding to each road region type.

[0056] In one specific embodiment of the application, the road region type includes but is not limited to trunk road, secondary trunk road, residential area road and industrial area road. The region weight corresponding to each road region type is scored by organizing traffic engineering experts and municipal maintenance personnel based on road function positioning, traffic flow bearing capacity, surrounding facility importance and other dimensions, combined with industry maintenance specifications, to form an initial weight framework and store it in the municipal road management system.

[0057] The daily traffic volume of the municipal road is extracted, and the traffic volume dynamic weight corresponding to the municipal road is obtained by comparing the reference traffic volume of the municipal road.

[0058] It should be noted that the daily traffic volume of the municipal road can be extracted from the municipal traffic bureau.

[0059] The disease risk evaluation index of the municipal road is obtained by multiplying the multi-disease coupling risk index of the municipal road, the region weight and the traffic volume dynamic weight.

[0060] In specific embodiments of the application, the specific way of outputting the disease risk level is: comparing the disease risk evaluation index of the municipal road with the disease risk evaluation index range corresponding to each disease risk level stored in the database. If the disease risk evaluation index of the municipal road is located in the disease risk evaluation index range corresponding to a certain disease risk level, the disease risk level is taken as the disease risk level of the municipal road.

[0061] This invention constructs a multi-disease coupled risk model, which integrates regional weights and dynamic traffic volume weights to output disease risk levels. It breaks through the traditional single-disease assessment model and achieves multi-dimensional risk quantification and dynamic adaptive optimization. It can capture the synergistic effects of diseases such as cracks and depressions, and can adjust regional weights in real time according to road function level and surrounding environment. It can also dynamically correct risk coefficients by combining traffic flow, load, etc., to form an accurate risk profile.

[0062] In a specific embodiment of the present invention, the specific process of constructing the evaluation index of the development trend of various pavement defects of municipal pavement is as follows: extract the pavement images corresponding to each historical collection from the pavement image data of the municipal pavement in the location and time sequence, and similarly analyze the risk index corresponding to each historical collection of various pavement defects according to the risk index analysis method corresponding to various pavement defects.

[0063] Based on the predicted and actual values ​​of the development trend assessment index corresponding to a single historical data collection for various types of pavement defects, a development trend assessment index for various types of pavement defects in municipal pavements is constructed using a single exponential smoothing method.

[0064] It should be noted that the assessment index for the development trend of various pavement defects in municipal roads is constructed using the first exponential smoothing method: ,in, The first one representing the municipal road surface Road surface distress development trend assessment index. It is the first The predicted value of the development trend assessment index corresponding to the first historical data collection for each type of pavement distress. It is the first The actual value of the development trend assessment index corresponding to the first historical data collection for each type of pavement distress. It is a smoothing coefficient, and its value range is... , This indicates the type number of road surface defects. .

[0065] In one specific embodiment of the present invention, the and The following table 1 provides an example to illustrate this. Here, we assume that the risk index data for a certain type of pavement distress on a municipal road surface was collected three consecutive times, as shown in Table 1 below:

[0066] Table 1 Risk Index Data

[0067]

[0068] In a single-stage exponential smoothing method, we need to set the smoothing coefficient. Here we assume According to the data in the table, the actual value of the risk index corresponding to the third historical collection of the pavement disease is 8, the predicted value of the risk index corresponding to the third historical collection of the pavement disease is , and the predicted value of the risk index corresponding to the fourth historical collection of the pavement disease is .

[0069] It should be noted that, The smaller the value is, the stronger the "memory" of the historical data is, and the greater the influence of the past value on the prediction result is, The larger the value is, the more the latest data is emphasized, and 0.3 is a commonly used compromise value, which can retain a certain historical trend and moderately respond to new data, so the smoothing coefficient is taken as 0.3.

[0070] It should be further noted that the once exponential smoothing method constructs a disease development trend evaluation index, has the characteristics of simple calculation and easy implementation, efficiently processes time series data through the formula of integrating historical actual values and predicted values, can filter random noise to restore the real evolution trend of the disease, can rely on newly collected data to dynamically update the evaluation result, can support disease development prediction, and at the same time, has a low demand for the amount of historical data, so that even limited data can be used to mine trends, and provides a practical and dynamic analysis tool for municipal pavement disease evaluation and maintenance decision-making.

[0071] In specific embodiments of the present application, the specific way of outputting the influence trend grade of each type of pavement disease is: the specific way of outputting the influence trend grade of each type of pavement disease is: extracting the pavement disease development trend evaluation index intervals corresponding to the low-risk grade, the medium-risk grade and the high-risk grade from the database respectively, if the pavement disease development trend evaluation index of a certain type of pavement disease is located in the low-risk grade interval, the influence trend grade of the pavement disease is the low-risk grade, if the pavement disease development trend evaluation index of a certain type of pavement disease is located in the medium-risk grade interval, the influence trend grade of the pavement disease is the medium-risk grade, and when the pavement disease development trend evaluation index of a certain type of pavement disease is located in the high-risk grade interval, the influence trend grade of the pavement disease is the high-risk grade.

[0072] In one specific embodiment of the present application, the pavement disease development trend evaluation index intervals corresponding to the low-risk grade, the medium-risk grade and the high-risk grade are , and , as the low-risk and medium-risk boundary, indicating that when the disease impact trend index is less than 0.4, the disease is in a relatively stable state, and there is no significant deterioration, and when it exceeds 0.4, it begins to enter the slow development stage and needs to be included in the monitoring category, and this threshold is determined by analyzing the turning point of the historical disease data "stable to development", matching the initial evolution characteristics of the disease, 0.7 is the division of medium-risk and high-risk, and when it exceeds 0.7, it means that the disease is deteriorating at a faster rate and needs to be treated urgently, and this value is based on engineering experience, corresponding to the critical state of the disease from "slow development" to "rapid deterioration", ensuring that the risk warning matches the actual hazard escalation node, and 1 is the upper limit of the index, representing the extreme state of the disease impact trend, and clearly defining the boundary of the risk level to ensure the closure of the grading logic.

[0073] The embodiment of the present application predicts the disease development trend by the one exponential smoothing method, realizes the dual functions of "current situation evaluation + trend warning", predicts the disease evolution trend based on historical data, significantly improves the scientificity and timeliness of municipal road risk management, reduces the maintenance cost and prevents safety accidents.

[0074] Referring to Figure 2 The second aspect of the present application provides a deep learning-based automatic detection system for road surface diseases, comprising: a municipal road surface data acquisition module, a road surface image quality analysis module, a road surface image reacquisition module and a road surface disease evaluation feedback module.

[0075] It should be noted that the present application also includes a database for storing the maximum illumination intensity in the historical illumination moment, the environmental humidity critical threshold and the minimum visibility of image acquisition, storing the environmental quality index range corresponding to the trigger flight parameter adjustment mechanism, storing the preset minimum flight speed and maximum flight height of the unmanned aerial vehicle, storing the road area type to which the municipal road belongs, and storing the road surface disease development trend evaluation index interval corresponding to the low-risk level, the medium-risk level and the high-risk level respectively.

[0076] The municipal road surface data acquisition module and the road surface image quality analysis module are connected, the road surface image reacquisition module and the road surface disease evaluation feedback module are both connected with the road surface image quality analysis module, the road surface image reacquisition module and the road surface disease evaluation feedback module are connected, and the road surface image quality analysis module, the road surface image reacquisition module and the road surface disease evaluation feedback module are all connected with the database.

[0077] The municipal road surface data acquisition module is used for unmanned aerial vehicles to collect natural environment data and road surface image data of municipal roads in position time sequence by carrying sensors and cameras.

[0078] The road surface image quality analysis module is configured to couple natural environment data with historical illumination data to analyze an environment quality index of the municipal road surface at the current time, and trigger a flight parameter adjustment mechanism when the environment quality index is within a set environment quality index range.

[0079] The road surface image reacquisition module is configured to adjust image reacquisition flight parameters of the unmanned aerial vehicle based on the flight parameter adjustment mechanism, and take the reacquired image meeting the image quality index as the final road surface image.

[0080] The road surface disease evaluation feedback module is configured to identify road surface disease categories and corresponding characteristic parameter sets of various road surface diseases from the acquired road surface images, construct a disease risk evaluation index and a development trend evaluation index of various road surface diseases of the municipal road surface, and output and correspondingly feedback a disease risk grade and an influence trend grade of various road surface diseases.

[0081] The above content is merely an example and a description of the concept of the present application. Those skilled in the art can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, as long as they do not deviate from the concept of the present application or exceed the scope defined by the present application, and all of them shall fall within the protection scope of the present application.

Claims

1. A method for automatic detection of pavement defects based on deep learning, characterized in that, include: S1. Drones are equipped with sensors and cameras to collect natural environmental data and road image data of municipal roads in a location and time sequence. S2. Couple natural environment data with historical illumination data to analyze the environmental quality index of municipal roads at the current moment. When the environmental quality index is within the set environmental quality index range, trigger the flight parameter adjustment mechanism. S3. Adjust the UAV's image re-acquisition flight parameters based on the flight parameter adjustment mechanism, and use the re-acquisitioned image that meets the image quality index as the final road surface image; S4. Identify the types of pavement defects and the corresponding feature parameter sets from the collected final pavement images, construct the defect risk assessment index and the development trend assessment index of various pavement defects, output the defect risk level and the impact trend level of various pavement defects and provide corresponding feedback. The specific process of constructing the risk assessment index for municipal pavement defects is as follows: based on the set of characteristic parameters of various types of pavement defects obtained from the final pavement image, the risk index corresponding to each type of pavement defect is obtained, and then the index is substituted into the composite defect superposition formula to obtain the multi-defect coupling risk index of municipal pavement. The formula for superimposed compound diseases is: ,in, These represent the risk indices for cracks, depressions, and potholes on municipal road surfaces, respectively. The index represents the second highest level. The road area type of the municipal road surface is extracted from the database and compared with the area weight corresponding to each road area type to obtain the area weight corresponding to the municipal road surface. The average daily traffic volume of the municipal road surface is extracted and compared with the reference traffic volume of the municipal road surface to obtain the dynamic traffic volume weight corresponding to the municipal road surface. The multiplication of the multi-disease coupling risk index, area weight and traffic volume dynamic weight of the municipal road surface is performed to obtain the disease risk assessment index of the municipal road surface. The specific process for constructing the development trend assessment index for various types of pavement defects in municipal roads is as follows: Extract pavement images from historical data collected at different times from the pavement image data at different locations; similarly analyze the risk index corresponding to each type of pavement defect in each historical data collection, following the same risk index analysis method; based on the predicted and actual values ​​of the development trend assessment index corresponding to each historical data collection for each type of pavement defect, construct the development trend assessment index for various types of pavement defects in municipal roads using a single exponential smoothing method. ,in, The first one representing the municipal road surface Road surface distress development trend assessment index. It is the first The predicted value of the development trend assessment index corresponding to the first historical data collection for each type of pavement distress. It is the first The actual value of the development trend assessment index corresponding to the first historical data collection for each type of pavement distress. It is the smoothing coefficient.

2. The method for automatic detection of pavement defects based on deep learning according to claim 1, characterized in that: The specific process for analyzing the environmental quality index of municipal road surfaces at the current moment is as follows: The environmental quality index of the municipal road surface at the current moment is obtained by coupling the current light intensity, ambient humidity and visibility in the natural environment data with the maximum light intensity, ambient humidity threshold and minimum visibility of the historical light time stored in the database. The environmental quality index of the municipal road surface at the current moment is compared with the environmental quality index range corresponding to the trigger flight parameter adjustment mechanism stored in the database. If the environmental quality index of the municipal road surface at the current moment is within the range of the environmental quality index corresponding to the trigger flight parameter adjustment mechanism, it indicates that the quality of the collected road surface image has reached the trigger flight parameter adjustment mechanism; otherwise, it indicates that the quality of the collected road surface image has not reached the trigger flight parameter adjustment mechanism.

3. The method for automatic detection of pavement defects based on deep learning according to claim 2, characterized in that: The specific process for adjusting the image reacquisition flight parameters of the UAV is as follows: The image reacquisition flight speed is obtained by coupling the current flight speed in the current flight parameters of the UAV with the preset minimum flight speed of the UAV stored in the database. The image re-acquisition flight altitude is obtained by comprehensively analyzing the current flight altitude of the drone in the current flight parameters and the preset maximum flight altitude of the drone stored in the database.

4. The method for automatic detection of pavement defects based on deep learning according to claim 3, characterized in that: The specific analysis process for using re-acquired images that meet the image quality standards as the final road surface images is as follows: Based on the pixel values ​​of each pixel obtained in each re-acquired image, the average value of the pixel gradient values ​​of each re-acquired image is calculated and used as the sharpness of each re-acquired image. Based on the pixel brightness of each pixel obtained in each re-acquisition image, the brightness variance of each re-acquisition image is obtained, and it is used as the illumination uniformity of each re-acquisition image. The re-acquired image with the highest resolution and the greatest uniformity of illumination was selected as the final road surface image.

5. The method for automatic detection of pavement defects based on deep learning according to claim 1, characterized in that: The specific method for identifying road surface defects from the collected road surface images is as follows: Based on historical road surface defect atlases, a defect feature set corresponding to each type of road surface defect is constructed. The collected road surface images are preprocessed to obtain each image feature. Each image feature is compared with the defect feature set corresponding to each type of road surface defect. If a certain image feature is located in the defect feature set corresponding to a certain type of road surface defect, then that type of road surface defect is taken as the type of road surface defect for municipal roads, thereby obtaining the types of road surface defects for municipal roads.

6. The method for automatic detection of pavement defects based on deep learning according to claim 5, characterized in that: The specific method for outputting the disease risk level is as follows: compare the disease risk assessment index of the municipal road surface with the disease risk assessment index range corresponding to each disease risk level stored in the database. If the disease risk assessment index of the municipal road surface is within the disease risk assessment index range corresponding to a certain disease risk level, then the disease risk level is taken as the disease risk level of the municipal road surface.

7. The method for automatic detection of pavement defects based on deep learning according to claim 6, characterized in that: The specific method for outputting the impact trend levels of various pavement defects is as follows: extract the pavement defect development trend assessment index ranges corresponding to low-risk, medium-risk, and high-risk levels from the database respectively. If the development trend assessment index of a certain type of pavement defect is within the low-risk level range, then the impact trend level of that type of pavement defect is low-risk. If the development trend assessment index of a certain type of pavement defect is within the medium-risk level range, then the impact trend level of that type of pavement defect is medium-risk. When the development trend assessment index of a certain type of pavement defect is within the high-risk level range, then the impact trend level of that type of pavement defect is high-risk.

8. A deep learning-based automatic pavement distress detection system, used to perform the steps of the deep learning-based automatic pavement distress detection method as described in any one of claims 1-7, characterized in that, include: The municipal road surface data acquisition module is used by drones to collect natural environmental data and road surface image data of municipal roads in a location and time sequence by carrying sensors and cameras. The road surface image quality analysis module is used to couple natural environment data with historical illumination data to analyze the environmental quality index of municipal roads at the current moment. When the environmental quality index is within the set environmental quality index range, the flight parameter adjustment mechanism is triggered. The road surface image reacquisition module is used to adjust the UAV's image reacquisition flight parameters based on the flight parameter adjustment mechanism, and to use the reacquisitioned image that meets the image quality index as the final road surface image. The pavement distress assessment and feedback module is used to identify the types of pavement distress and the corresponding feature parameter sets from the collected pavement images, construct the distress risk assessment index of municipal pavement and the development trend assessment index of various types of pavement distress, output the distress risk level and the impact trend level of various types of pavement distress and provide corresponding feedback.

Images