Road defect evaluation system and method based on machine vision

Abstract

The invention relates to the technical field of machine vision, and discloses a road defect evaluation system and method based on machine vision, and the method comprises the steps: collecting road surface pictures and road surface gradient data, analyzing road surface defects based on the road surface pictures and the road surface gradient data, collecting slope soil data and vegetation pictures, and evaluating the road surface defects. The method comprises the steps of analyzing soil loosening based on a soil picture, analyzing a vegetation disease risk based on a vegetation picture, analyzing a slope risk based on the soil loosening and the vegetation disease risk, collecting future weather data, predicting the influence of weather on a landslide based on the weather data and the slope risk, and collecting historical traffic flow data in the same period. The influence of traffic flow on landslide is predicted based on the traffic flow data and the slope risk, the safety of the road under the influence of the landslide is predicted by integrating the influence of weather on the landslide, the influence of the traffic flow on the landslide and pavement defects, and whether the road needs traffic restriction is analyzed based on the safety of the road, so that the accuracy of road risk prediction is improved, and the traffic safety is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a road defect assessment system and method based on machine vision. Background Technology

[0002] Mountain roads are winding roads built in mountainous and hilly areas, following the contours of the terrain. Their core characteristics include complex geological conditions and significant environmental impacts, which can easily lead to geological disaster risks and driving safety risks. Therefore, it is extremely important to conduct safety analysis on mountain roads.

[0003] Existing technologies for assessing road defects and road safety rely primarily on road baseline data, traffic accidents, and environmental analysis. They lack consideration for slope anomalies caused by abnormal weather and traffic flow, thus failing to combine the interaction between road defects and landslide risks in the analysis of road safety, resulting in inaccurate assessments of road risks.

[0004] To address the aforementioned problems, this invention provides a road defect assessment system and method based on machine vision. Summary of the Invention

[0005] This invention provides a road defect assessment system and method based on machine vision. By assessing road safety, it identifies roads with landslide risks and restricts traffic on them, which helps improve the accuracy of road risk prediction and ensures traffic safety.

[0006] In a first aspect, the present invention provides a road defect assessment method based on machine vision, the road defect assessment method based on machine vision comprising: Step 1: Collect road surface images and pavement slope data, and analyze pavement defects based on the road surface images and pavement slope data; Step 2: Collect slope soil data and vegetation images. Analyze soil looseness based on soil images, analyze vegetation disease risk based on vegetation images, and analyze slope risk based on soil looseness and vegetation disease risk. Step 3: Collect future weather data and predict the impact of weather on landslides based on weather data and slope risk. Step 4: Collect historical traffic flow data for the same period, and predict the impact of traffic flow on landslides based on traffic flow data and slope risk. Step 5: Based on the combined effects of weather, traffic flow, and road surface defects, predict the road safety under the influence of landslides, and analyze whether road restrictions are necessary.

[0007] Specifically, step 1 includes the following steps: Step 11: Collect images of the road surface and road slope data; Step 12: Obtain crack data and pothole data based on road surface images. The crack data includes crack length, crack width, and crack density. The pothole data includes pothole area and pothole density. The road surface slope data is the road surface transverse slope. Step 13: Collect several sets of data on cracks, potholes, road surface slope, and traffic accident probability of the road to be repaired. Divide the crack, pothole, and road surface slope data into 80% training dataset and 20% test dataset. Input the 80% training dataset into the neural network analysis model for traffic accident probability to train and obtain the initial neural network analysis model for traffic accident probability. Then input the 20% test dataset into the initial neural network analysis model for traffic accident probability to test and obtain the neural network analysis model for traffic accident probability with the highest accuracy in analyzing traffic accident probability. Step 14: Input crack data, pothole data, and road slope data, and obtain the probability of traffic accidents based on the neural network analysis model for the probability of traffic accidents. Step 15: Obtain the road surface defect assessment value by dividing the probability of traffic accidents by the safe traffic accident probability threshold.

[0008] Specifically, step 2 includes the following steps: Step 21: Collect slope soil data and vegetation images. The slope soil data includes soil moisture and rock weathering degree. Based on the vegetation images, obtain the vegetation health and the degree of abnormality in vegetation morphology. Step 22: Obtain soil moisture anomalies based on the ratio of soil moisture to soil moisture threshold; obtain rock weathering anomalies based on the ratio of rock weathering degree to rock weathering threshold; and obtain soil looseness anomaly values ​​based on the product of soil moisture anomalies and rock weathering anomalies. Step 23: Collect the standard vegetation health and standard vegetation morphological change abnormality of vegetation that meet the preset growth standards. Obtain the vegetation health abnormality value based on the ratio of standard vegetation health to vegetation health. Obtain the vegetation morphological change abnormality value based on the ratio of vegetation morphological change abnormality to standard vegetation morphological change abnormality. Obtain the vegetation disease abnormality value based on the product of vegetation health abnormality value and vegetation morphological change abnormality value. Step 24: Obtain slope risk anomaly values ​​by weighted summation of soil looseness anomalies and vegetation disease anomalies.

[0009] Specifically, step 3 includes the following steps: Step 31: Collect future weather data, including rainfall and wind speed; Step 32: Obtain abnormal rainfall values ​​by dividing the rainfall amount by the safe rainfall threshold, obtain abnormal wind speed values ​​by dividing the wind speed by the safe wind speed threshold, and obtain abnormal weather values ​​by weighted summation of the abnormal rainfall values ​​and abnormal wind speed values. Step 33: Obtain the impact value of weather on landslides by multiplying the slope risk anomaly value by the power of the weather anomaly value of the natural constant e.

[0010] Specifically, step 4 includes the following steps: Step 41: Collect historical traffic flow data from the same period, including average vehicle speed, vehicle vibration amplitude, vehicle vibration frequency, traffic volume, and average vehicle mass. Step 42: Obtain abnormal vibration amplitude values ​​by dividing vehicle vibration amplitude by a safe vibration amplitude threshold; obtain abnormal vibration frequency values ​​by dividing vehicle vibration frequency by a safe vibration frequency threshold; obtain abnormal vehicle vibration values ​​by multiplying abnormal vibration amplitude values ​​and abnormal vibration frequency values; obtain abnormal traffic flow values ​​by dividing traffic flow by a safe traffic flow threshold; obtain abnormal vehicle mass values ​​by dividing average vehicle mass by the maximum design load of the road; and obtain abnormal traffic flow values ​​by weighted summation of abnormal vehicle vibration values, abnormal traffic flow values, and abnormal vehicle mass values. Step 43: Obtain the impact value of traffic flow on landslide by multiplying the weighted sum of the slope risk anomaly value, the traffic flow anomaly value, and the road surface defect assessment value by the power of the weighted sum of the natural constant e.

[0011] Specifically, step 5 includes the following steps: Step 51: Obtain the landslide anomaly value by weighted summation of the impact values ​​of weather and traffic flow on the landslide; Step 52: Obtain abnormal vehicle speed values ​​by dividing the average vehicle speed by the maximum speed limit on the road, and obtain abnormal vehicle driving values ​​by combining abnormal vehicle speed values ​​with abnormal vehicle mass values. Step 53: Obtain the vehicle traffic impact coefficient by weighted summation of the vehicle traffic anomaly value and the road surface defect assessment value, and obtain the road safety value under the influence of landslide based on the product of the vehicle traffic impact coefficient and the landslide anomaly value. Step 54: Filter out roads with safety values ​​below the preset road safety threshold and restrict traffic on the filtered roads.

[0012] Secondly, the present invention provides a road defect assessment system based on machine vision, the road defect assessment system based on machine vision comprising: The road surface defect analysis module is used to collect road surface images and road surface slope data, and analyze road surface defects based on the road surface images and road surface slope data; The slope risk analysis module is used to collect slope soil data and vegetation pictures, analyze soil looseness based on soil pictures, analyze vegetation disease risk based on vegetation pictures, and analyze slope risk based on soil looseness and vegetation disease risk. The weather impact prediction module is used to collect future weather data and predict the impact of weather on landslides based on the weather data and slope risk. The traffic flow impact prediction module is used to collect historical traffic flow data for the same period and predict the impact of traffic flow on landslides based on the traffic flow data and slope risk. The road safety prediction module is used to predict road safety under the influence of landslides by comprehensively considering the impact of weather on landslides, the impact of traffic flow on landslides, and road surface defects. The restricted road screening module is used to analyze whether a road needs to be restricted based on its safety.

[0013] Thirdly, the present invention provides an electronic device, comprising: a memory and at least one processor, wherein the memory stores instructions, and the at least one processor invokes the instructions in the memory to cause the electronic device to execute the above-described machine vision-based road defect assessment method.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the aforementioned machine vision-based road defect assessment method.

[0015] The technical solution provided by this invention involves collecting road surface images and road surface slope data, analyzing road surface defects based on these images and data, collecting slope soil data and vegetation images, analyzing soil looseness based on soil images, analyzing vegetation disease risks based on vegetation images, analyzing slope risks based on soil looseness and vegetation disease risks, collecting future weather data, predicting the impact of weather on landslides based on weather data and slope risks, collecting historical traffic flow data for the same period, predicting the impact of traffic flow on landslides based on traffic flow data and slope risks, and comprehensively considering the impact of weather on landslides, the impact of traffic flow on landslides, and road surface defects to predict road safety under the influence of landslides. Based on road safety, the invention analyzes whether road restrictions are necessary. By assessing road safety and selecting roads with landslide risks for traffic restrictions, this invention helps improve the accuracy of road risk prediction and ensures traffic safety. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the road defect assessment method based on machine vision in an embodiment of the present invention; Figure 2 This is a schematic diagram of step 1 of the road defect assessment method based on machine vision in an embodiment of the present invention; Figure 3 This is a schematic diagram of step 2 of the road defect assessment method based on machine vision in an embodiment of the present invention; Figure 4 This is a schematic diagram of the road defect assessment system based on machine vision in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0019] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the present invention provides a road defect assessment method based on machine vision, comprising: Step 1: Collect road surface images and pavement slope data, and analyze pavement defects based on the road surface images and pavement slope data; Please see Figure 2 In this embodiment, step 1 includes the following specific steps: Step 11: Collect images of the road surface and road slope data; In this embodiment, the detection vehicle equipped with a high-resolution industrial camera, a dedicated linear LED lighting array, a laser contour scanner, and an inertial measurement unit is controlled to move at a constant speed. The encoder pulse triggers all sensors, the industrial camera captures images of the road surface, the laser contour scanner acquires the 3D contour of the laser line, the inertial measurement unit acquires the three-axis acceleration and angular velocity of the detection vehicle, and the GPS acquires the location information. The Net model performs pixel-level segmentation on the road surface image, generates a binary mask of crack pixels, and performs post-processing on the mask. The crack length is the sum of the pixels of the crack skeleton, which can be converted into the actual length by multiplying by the image resolution. The crack width is the average width of the crack region, which can be calculated by the cross section perpendicular to the skeleton line. The crack density is the proportion of crack pixels to the total number of road surface pixels. The YOLO model locates pothole regions in road surface images and outputs pothole bounding boxes. The pothole area is the number of pixels within the bounding box, which can be multiplied by the image resolution to get the actual area. The pothole density is the proportion of pothole pixels to the total number of road surface pixels. The laser profilometer emits a single-frame laser beam to scan the cross-section of the road surface, obtaining a set of height data. The height data is then fitted with a straight line, and the slope of the fitted line is the cross slope. Step 12: Obtain crack data and pothole data based on road surface images. Crack data includes crack length, crack width, and crack density. Pothole data includes pothole area and pothole density. Road surface slope data is the road surface transverse slope. In this embodiment, the network cracks are caused by the repeated action of wheel loads, which cause the material to reach its life limit and lose its toughness, resulting in large-area cracking that reflects the aging and fatigue of the asphalt layer and seriously reduces the overall strength of the pavement layer. Potholes cause the road surface to lose support, which can damage vehicle tires and suspension systems, leading to traffic accidents. The cross slope of the road is used to quickly drain water to the roadside ditch. If the slope is insufficient, flat, or even reversed, it will lead to water accumulation. Water accumulation will accelerate the peeling of asphalt mixture, seep into the base layer, and cause damage. Step 13: Collect several sets of data on cracks, potholes, road surface slope, and traffic accident probability of the road to be repaired. Divide the crack, pothole, and road surface slope data into 80% training dataset and 20% test dataset. Input the 80% training dataset into the neural network analysis model for traffic accident probability to train and obtain the initial neural network analysis model for traffic accident probability. Then input the 20% test dataset into the initial neural network analysis model for traffic accident probability to test and obtain the neural network analysis model for traffic accident probability with the highest accuracy in analyzing traffic accident probability. Step 14: Input crack data, pothole data, and road slope data, and obtain the probability of traffic accidents based on the neural network analysis model for the probability of traffic accidents. Step 15: Obtain the road surface defect assessment value by dividing the probability of traffic accidents by the safe traffic accident probability threshold.

[0020] In this embodiment, the neural network analysis model for the probability of traffic accidents includes calculation formulas for specific neurons. The calculation formulas for specific neurons are as follows: In the formula, This represents the Sigmoid activation function. This represents the output of the m-th neuron in the (n+1)th layer of the neural network analysis model for the probability of traffic accidents. This represents the number of neurons in the nth layer. The connection weights between neuron p in layer n and neuron m in layer n+1 of the neural network analysis model representing the probability of traffic accidents are given. This represents the output of neuron p in the nth layer of the neural network analysis model for the probability of traffic accidents. The bias represents the linear relationship between the nth layer neuron p and the m-term neurons in the (n+1)th layer of the neural network analysis model for the probability of traffic accidents. In this embodiment, the neural network analysis model for the probability of traffic accidents combines the unique terrain characteristics of mountain roads (many curves and frequent changes in cross slope) and the defect formation mechanism (road wear and water damage) to achieve a comprehensive assessment.

[0021] Step 2: Collect slope soil data and vegetation images. Analyze soil looseness based on soil images, analyze vegetation disease risk based on vegetation images, and analyze slope risk based on soil looseness and vegetation disease risk. Please see Figure 3 In this embodiment, step 2 includes the following specific steps: Step 21: Collect slope soil data and vegetation images. Slope soil data includes soil moisture and rock weathering degree. Based on vegetation images, obtain vegetation health and the degree of abnormality in vegetation morphology. In this embodiment, soil moisture sensors are deployed on high-risk slopes along the mountain road. The sensors transmit real-time moisture data back to the monitoring center via IoT technologies such as cellular networks or LoRa. Select naturally exposed rock surfaces and set up ten test points at the top, middle, and bottom of the slope. Use a rebound hammer to tap each test point three times and calculate the average rebound value. The degree of rock weathering is judged based on the rebound value. Rocks with hard texture and no weathering traces have an average rebound value greater than 60. Rocks with slight discoloration, a few cracks, and high hardness have an average rebound value between 40 and 60. Rocks with obvious discoloration, developed cracks, and medium hardness have an average rebound value between 20 and 40. Rocks with severe discoloration, extensive peeling, and extremely low hardness have an average rebound value less than 20. Multispectral cameras are used to collect vegetation images, and NDVI (Normalized Difference Vegetation Index) values ​​are calculated based on the vegetation images. The higher the NDVI value, the higher the vegetation health. The lower the NDVI value, the lower the vegetation health, indicating sparse, withered, or bare vegetation. A large area of ​​vegetation on the slope showing a decline in health may be an early indication of internal slope sliding, water shortage, or disease. Ground points and other low-lying vegetation were filtered out from the LiDAR point cloud, retaining only the tree trunk point cloud. The DBSCAN clustering algorithm was used to separate the point cloud of individual tree trunks. Principal Component Analysis (PCA) was performed on the point cloud of each tree trunk, yielding three principal component directions (PC1, PC2, PC3). PC1 represents the primary growth direction of the tree trunk point cloud (the long axis of the trunk extension), PC2 represents the secondary direction (the width direction of the trunk), and PC3 represents the vertical direction (the height direction of the trunk). In the local coordinate system, the z-axis was set as the vertical direction (the direction of gravity), and the tilt angle was calculated. The formula for calculating the tilt angle is: , The unit vector representing the z-axis, for example [0,0,1]. This represents the dot product of the first principal component and the z-axis. The magnitude of the vector is used to obtain the rate of change of the two tilt angles, which is the degree of abnormality in vegetation morphology. The tilt angle directly reflects the degree to which the tree trunk deviates from the vertical direction. By observing the change in the tilt angle, signs of slope slippage can be detected early. Step 22: Obtain soil moisture anomalies based on the ratio of soil moisture to soil moisture threshold; obtain rock weathering anomalies based on the ratio of rock weathering degree to rock weathering threshold; and obtain soil looseness anomaly values ​​based on the product of soil moisture anomalies and rock weathering anomalies. Step 23: Collect the standard vegetation health and standard vegetation morphological change abnormality of vegetation that meet the preset growth standards. Obtain the vegetation health abnormality value based on the ratio of standard vegetation health to vegetation health. Obtain the vegetation morphological change abnormality value based on the ratio of vegetation morphological change abnormality to standard vegetation morphological change abnormality. Obtain the vegetation disease abnormality value based on the product of vegetation health abnormality value and vegetation morphological change abnormality value. In this embodiment, excessive soil moisture will increase the weight of the soil and reduce the friction between soil particles, which can easily trigger landslides or debris flows. If the soil bearing capacity is insufficient and cannot withstand the pressure of the soil above, settlement or shear failure will occur. For example, if the roadbed of a mountain road is built on fill with low bearing capacity, uneven settlement is likely to occur, which can induce slope instability. The deep roots of trees can penetrate the soil layer, binding the shallow soil to the deep bedrock and enhancing the overall stability of the slope. The fibrous root network of shrubs and herbs can fill the soil pores and improve the soil's shear strength. The vegetation canopy can intercept some of the rainfall and reduce the erosion of the slope by surface runoff. For example, the natural forest land next to the mountain road has small fluctuations in soil moisture and dense root system, and the slope stability is much higher than that of bare mountain slopes. Step 24: Obtain slope risk anomaly values ​​by weighted summation of soil looseness anomalies and vegetation disease anomalies.

[0022] Step 3: Collect future weather data and predict the impact of weather on landslides based on weather data and slope risk. In this embodiment, step 3 includes the following specific steps: Step 31: Collect future weather data, including rainfall and wind speed; Step 32: Obtain abnormal rainfall values ​​by dividing the rainfall amount by the safe rainfall threshold, obtain abnormal wind speed values ​​by dividing the wind speed by the safe wind speed threshold, and obtain abnormal weather values ​​by weighted summation of the abnormal rainfall values ​​and abnormal wind speed values. Step 33: Obtain the impact value of weather on landslides by multiplying the slope risk anomaly value by the power of the weather anomaly value of the natural constant e.

[0023] In this embodiment, rainfall increases soil gravity and reduces soil shear strength, directly inducing landslides, while wind speed indirectly induces landslides by damaging the soil stabilization function of vegetation and exacerbating surface erosion.

[0024] Step 4: Collect historical traffic flow data for the same period, and predict the impact of traffic flow on landslides based on traffic flow data and slope risk. In this embodiment, step 4 includes the following specific steps: Step 41: Collect historical traffic flow data for the same period. The historical traffic flow data for the same period includes average vehicle speed, vehicle vibration amplitude, vehicle vibration frequency, traffic volume, and average vehicle mass. In this embodiment, the vibration generated by the vehicle will repeatedly impact the slope, which will accelerate soil fatigue over a long period of time. The greater the vehicle mass and the higher the traffic volume, the more serious the impact on the slope. Step 42: Obtain abnormal vibration amplitude values ​​by dividing vehicle vibration amplitude by a safe vibration amplitude threshold; obtain abnormal vibration frequency values ​​by dividing vehicle vibration frequency by a safe vibration frequency threshold; obtain abnormal vehicle vibration values ​​by multiplying abnormal vibration amplitude values ​​and abnormal vibration frequency values; obtain abnormal traffic flow values ​​by dividing traffic flow by a safe traffic flow threshold; obtain abnormal vehicle mass values ​​by dividing average vehicle mass by the maximum design load of the road; and obtain abnormal traffic flow values ​​by weighted summation of abnormal vehicle vibration values, abnormal traffic flow values, and abnormal vehicle mass values. Step 43: Obtain the impact value of traffic flow on landslide by multiplying the weighted sum of the slope risk anomaly value, the traffic flow anomaly value, and the road surface defect assessment value by the power of the weighted sum of the natural constant e.

[0025] Step 5: Based on the combined effects of weather, traffic flow, and road surface defects, predict the road safety under the influence of landslides, and analyze whether road restrictions are necessary.

[0026] In this embodiment, step 5 includes the following specific steps: Step 51: Obtain the landslide anomaly value by weighted summation of the impact values ​​of weather and traffic flow on the landslide; Step 52: Obtain abnormal vehicle speed values ​​by dividing the average vehicle speed by the maximum speed limit on the road, and obtain abnormal vehicle driving values ​​by combining abnormal vehicle speed values ​​with abnormal vehicle mass values. Step 53: Obtain the vehicle traffic impact coefficient by weighted summation of the vehicle traffic anomaly value and the road surface defect assessment value, and obtain the road safety value under the influence of landslide based on the product of the vehicle traffic impact coefficient and the landslide anomaly value. Step 54: Filter out roads with safety values ​​below the preset road safety threshold and restrict traffic on the filtered roads.

[0027] In this embodiment, the steps for obtaining weights and thresholds are as follows: collect 500 sets of road surface images, road slope data, slope soil data, vegetation images, weather data, traffic flow data, and whether the road has caused traffic accidents due to landslides. Substitute the above data into each step of this embodiment to obtain the restricted roads. Simultaneously import the roads that actually caused traffic accidents and the restricted roads selected in this embodiment into the fitting software to obtain the set of weight and threshold values ​​with the highest screening accuracy. Road defects can directly or indirectly damage slope stability through hydrological and mechanical processes. For example, during heavy rain, large amounts of rainwater enter the slope soil through road cracks or potholes, causing an increase in pore water pressure and a sharp decrease in interparticle friction in saturated soil, making the slope prone to shallow landslides. Changes in road cross slope or potholes can lead to poor drainage, causing rainwater to accumulate on the road surface or slope, softening the underlying soil. Road defects can also cause uneven distribution of road loads, with some loads being transferred to the slope through the foundation, increasing additional stress on the slope. Especially in sections with soft soil foundations, this additional stress may induce deep slope sliding, resulting in a larger area of ​​damage. Slope instability can also destroy or degrade roads, hindering vehicle traffic and even causing traffic accidents. By assessing road safety and identifying roads with landslide risks for traffic restrictions, the accuracy of road risk prediction can be improved, ensuring traffic safety.

[0028] The above describes the road defect assessment method based on machine vision in the embodiments of the present invention. The following describes the road defect assessment system based on machine vision in the embodiments of the present invention. Please refer to [link / reference]. Figure 4 Machine vision-based road defect assessment systems include: The road surface defect analysis module is used to collect road surface images and road surface slope data, and analyze road surface defects based on the road surface images and road surface slope data; The slope risk analysis module is used to collect slope soil data and vegetation pictures, analyze soil looseness based on soil pictures, analyze vegetation disease risk based on vegetation pictures, and analyze slope risk based on soil looseness and vegetation disease risk. The weather impact prediction module is used to collect future weather data and predict the impact of weather on landslides based on the weather data and slope risk. The traffic flow impact prediction module is used to collect historical traffic flow data for the same period and predict the impact of traffic flow on landslides based on the traffic flow data and slope risk. The road safety prediction module is used to predict road safety under the influence of landslides by comprehensively considering the impact of weather on landslides, the impact of traffic flow on landslides, and road surface defects. The restricted road screening module is used to analyze whether a road needs to be restricted based on its safety.

[0029] This invention also provides an electronic device, including: a memory and at least one processor, wherein the memory stores instructions, and the at least one processor invokes the instructions in the memory to cause the electronic device to perform the steps of the above-described machine vision-based road defect assessment method.

[0030] The electronic device can vary considerably depending on its configuration or performance. It may include one or more Central Processing Units (CPUs) and one or more memories, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the machine vision-based road defect assessment method provided in the above-described method embodiments. The electronic device may also include other components for implementing its functions. For example, it may have wired or wireless network interfaces and input / output interfaces for data input and output, which will not be elaborated upon in this embodiment.

[0031] This invention also proposes a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the above-described machine vision-based road defect assessment method.

[0032] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0033] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0034] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this invention.

[0035] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only one, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms.

[0036] In the description of this specification, references to the terms "first," "second," "third," "fourth," etc. (if applicable) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data used can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A road defect assessment method based on machine vision, characterized in that, The specific steps include the following: Step 1: Collect road surface images and pavement slope data, and analyze pavement defects based on the road surface images and pavement slope data; Step 2: Collect slope soil data and vegetation images. Analyze soil looseness based on soil images, analyze vegetation disease risk based on vegetation images, and analyze slope risk based on soil looseness and vegetation disease risk. Step 3: Collect future weather data and predict the impact of weather on landslides based on weather data and slope risk. Step 4: Collect historical traffic flow data for the same period, and predict the impact of traffic flow on landslides based on traffic flow data and slope risk. Step 5: Based on the combined effects of weather, traffic flow, and road surface defects, predict the road safety under the influence of landslides, and analyze whether road restrictions are necessary.

2. The road defect assessment method based on machine vision according to claim 1, characterized in that, Step 1 includes the following specific steps: Step 11: Collect images of the road surface and road slope data; Step 12: Obtain crack data and pothole data based on road surface images. The crack data includes crack length, crack width, and crack density. The pothole data includes pothole area and pothole density. The road surface slope data is the road surface transverse slope. Step 13: Collect several sets of data on cracks, potholes, road surface slope, and traffic accident probability of the road to be repaired. Divide the crack, pothole, and road surface slope data into 80% training dataset and 20% test dataset. Input the 80% training dataset into the neural network analysis model for traffic accident probability to train and obtain the initial neural network analysis model for traffic accident probability. Then input the 20% test dataset into the initial neural network analysis model for traffic accident probability to test and obtain the neural network analysis model for traffic accident probability with the highest accuracy in analyzing traffic accident probability. Step 14: Input crack data, pothole data, and road slope data, and obtain the probability of traffic accidents based on the neural network analysis model for the probability of traffic accidents. Step 15: Obtain the road surface defect assessment value by dividing the probability of traffic accidents by the safe traffic accident probability threshold.

3. The road defect assessment method based on machine vision according to claim 2, characterized in that, Step 2 includes the following specific steps: Step 21: Collect slope soil data and vegetation images. The slope soil data includes soil moisture and rock weathering degree. Based on the vegetation images, obtain the vegetation health and the degree of abnormality in vegetation morphology. Step 22: Obtain soil moisture anomalies based on the ratio of soil moisture to soil moisture threshold; obtain rock weathering anomalies based on the ratio of rock weathering degree to rock weathering threshold; and obtain soil looseness anomaly values ​​based on the product of soil moisture anomalies and rock weathering anomalies. Step 23: Collect the standard vegetation health and standard vegetation morphological change abnormality of vegetation that meet the preset growth standards. Obtain the vegetation health abnormality value based on the ratio of standard vegetation health to vegetation health. Obtain the vegetation morphological change abnormality value based on the ratio of vegetation morphological change abnormality to standard vegetation morphological change abnormality. Obtain the vegetation disease abnormality value based on the product of vegetation health abnormality value and vegetation morphological change abnormality value. Step 24: Obtain slope risk anomaly values ​​by weighted summation of soil looseness anomalies and vegetation disease anomalies.

4. The road defect assessment method based on machine vision according to claim 3, characterized in that, Step 3 includes the following specific steps: Step 31: Collect future weather data, including rainfall and wind speed; Step 32: Obtain abnormal rainfall values ​​by dividing the rainfall amount by the safe rainfall threshold, obtain abnormal wind speed values ​​by dividing the wind speed by the safe wind speed threshold, and obtain abnormal weather values ​​by weighted summation of the abnormal rainfall values ​​and abnormal wind speed values. Step 33: Obtain the impact value of weather on landslides by multiplying the slope risk anomaly value by the power of the weather anomaly value of the natural constant e.

5. The road defect assessment method based on machine vision according to claim 4, characterized in that, Step 4 includes the following specific steps: Step 41: Collect historical traffic flow data from the same period, including average vehicle speed, vehicle vibration amplitude, vehicle vibration frequency, traffic volume, and average vehicle mass. Step 42: Obtain abnormal vibration amplitude values ​​by dividing vehicle vibration amplitude by a safe vibration amplitude threshold; obtain abnormal vibration frequency values ​​by dividing vehicle vibration frequency by a safe vibration frequency threshold; obtain abnormal vehicle vibration values ​​by multiplying abnormal vibration amplitude values ​​and abnormal vibration frequency values; obtain abnormal traffic flow values ​​by dividing traffic flow by a safe traffic flow threshold; obtain abnormal vehicle mass values ​​by dividing average vehicle mass by the maximum design load of the road; and obtain abnormal traffic flow values ​​by weighted summation of abnormal vehicle vibration values, abnormal traffic flow values, and abnormal vehicle mass values. Step 43: Obtain the impact value of traffic flow on landslide by multiplying the weighted sum of the slope risk anomaly value, the traffic flow anomaly value, and the road surface defect assessment value by the power of the weighted sum of the natural constant e.

6. The road defect assessment method based on machine vision according to claim 5, characterized in that, Step 5 includes the following specific steps: Step 51: Obtain the landslide anomaly value by weighted summation of the impact values ​​of weather and traffic flow on the landslide; Step 52: Obtain abnormal vehicle speed values ​​by dividing the average vehicle speed by the maximum speed limit on the road, and obtain abnormal vehicle driving values ​​by combining abnormal vehicle speed values ​​with abnormal vehicle mass values. Step 53: Obtain the vehicle traffic impact coefficient by weighted summation of the vehicle traffic anomaly value and the road surface defect assessment value, and obtain the road safety value under the influence of landslide based on the product of the vehicle traffic impact coefficient and the landslide anomaly value. Step 54: Filter out roads with safety values ​​below the preset road safety threshold and restrict traffic on the filtered roads.

7. A machine vision-based road defect assessment system, used to implement the machine vision-based road defect assessment method as described in any one of claims 1 to 6, characterized in that, The machine vision-based road defect assessment system includes: The road surface defect analysis module is used to collect road surface images and road surface slope data, and analyze road surface defects based on the road surface images and road surface slope data; The slope risk analysis module is used to collect slope soil data and vegetation pictures, analyze soil looseness based on soil pictures, analyze vegetation disease risk based on vegetation pictures, and analyze slope risk based on soil looseness and vegetation disease risk. The weather impact prediction module is used to collect future weather data and predict the impact of weather on landslides based on the weather data and slope risk. The traffic flow impact prediction module is used to collect historical traffic flow data for the same period and predict the impact of traffic flow on landslides based on the traffic flow data and slope risk. The road safety prediction module is used to predict road safety under the influence of landslides by comprehensively considering the impact of weather on landslides, the impact of traffic flow on landslides, and road surface defects. The restricted road screening module is used to analyze whether a road needs to be restricted based on its safety.

8. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the machine vision-based road defect assessment method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by a processor, it causes the processor to perform the machine vision-based road defect assessment method as described in any one of claims 1 to 6.

Images