A visual intelligence-based road surface pit slot intelligent detection method and system

Abstract

The application relates to the technical field of road maintenance, and discloses a road surface pit slot intelligent detection method and system based on visual intelligence, the method comprising the following steps: acquiring multi-angle images and performing illumination compensation, and extracting pit slot potential edges; fusing multiple views to obtain a depth mapping diagram, combining the edges to complete three-dimensional modeling and calculating volume shape parameters, and generating a repair priority sequence; tracking the depth evolution and edge degradation of the repair priority sequence, and outputting a dynamic monitoring report and a joint risk verification result; updating a database according to the risk and optimizing sensor parameters, and forming a next cycle collection strategy. The method can solve the problem of low detection precision in the prior art.

Description

Technical Field

[0001] This invention relates to the field of road maintenance technology, and in particular to a visual intelligence-based intelligent detection method and system for road potholes. Background Technology

[0002] Currently, potholes are a common problem on urban roads. Their depth, area, and rate of expansion directly affect driving safety and road life. Inspection operations require the rapid acquisition of the three-dimensional morphology of potholes under open traffic conditions and the assessment of their deterioration trend in order to develop targeted maintenance plans. This task falls under the category of image detection.

[0003] In one existing technology, a detection scheme based on a monocular camera and fixed exposure parameters is widely used: the system acquires two-dimensional images under the same lighting conditions, extracts suspected areas using edge operators, and then converts pixel area into the severity of the damage using empirical formulas; simultaneously, it uses periodic manual verification to record depth changes, forming the basis for the next inspection cycle. For example, when a vehicle enters a tunnel or in rainy weather, the overall brightness drops sharply, and the fixed exposure results in a near-black image, causing the edge operators to misjudge shadows as pothole boundaries; subsequently, the empirical formulas rely only on the longest path and cannot reflect the true volume, causing subsequent maintenance sequencing to be inconsistent with the degree of danger.

[0004] Existing technologies fail to simultaneously process changes in illumination and depth information, and lack continuous tracking of the spatiotemporal degradation of pit edges. This leads to decreased reliability of detection results in complex environments, and a disconnect between inspection data and the actual rate of deterioration. Therefore, existing technologies suffer from low detection accuracy. Summary of the Invention

[0005] This invention provides a visual intelligence-based intelligent detection method and system for road potholes, in order to solve the problem of low detection accuracy in existing technologies.

[0006] In a first aspect, to address the aforementioned technical problems, this invention provides a visual intelligence-based intelligent method for detecting road potholes, comprising:

[0007] Acquire multi-angle image data and perform illumination compensation processing on the multi-angle image data to obtain a preliminary road surface image set;

[0008] Based on the preliminary road surface image set, environmental variable analysis and image gradient feature extraction are performed to obtain the potential edge locations of potholes;

[0009] Based on the multi-angle image data, multi-view fusion is performed to obtain a pit depth mapping map;

[0010] Based on the pothole depth mapping map and the potential edge position of the pothole, a three-dimensional model is performed to obtain the three-dimensional model of the pothole;

[0011] Based on the three-dimensional model of the pothole, the volume and shape parameters of the pothole are calculated to obtain the corresponding severity index. Then, a sorting operation is performed according to the corresponding severity index to obtain a repair priority sequence.

[0012] Based on the pit depth mapping map, the repair priority sequence is dynamically monitored to obtain a dynamic monitoring report. Based on the dynamic monitoring report, joint risk verification is performed to obtain the joint risk verification result.

[0013] Based on the joint risk verification results and the dynamic monitoring report, the road surface database is updated and sensor parameters are optimized to obtain the next cycle's data acquisition strategy.

[0014] Preferably, multi-angle image data is acquired, and illumination compensation processing is performed on the multi-angle image data to obtain a preliminary road surface image set, including:

[0015] Acquire multi-angle image data;

[0016] Based on the multi-angle image data, illumination compensation processing is performed to obtain an illumination-compensated image set;

[0017] Based on the illumination-compensated image set, smoothing processing is performed to obtain a clear road surface feature image set;

[0018] Based on the clear road surface feature image set, multi-angle image fusion is performed to obtain the preliminary road surface image set.

[0019] Preferably, based on the preliminary road surface image set, environmental variable analysis and image gradient feature extraction are performed to obtain the potential edge locations of potholes, including:

[0020] Based on the preliminary road surface image set, a ray standardization operation is performed to obtain a ray-standardized image set;

[0021] Based on the light-normalized image set, it is determined whether there are weather interference features. If there are interference features, the interference area is filtered to obtain a weather-corrected image set.

[0022] Based on the weather-corrected image set, the spatial change rate of image pixels is calculated and gradient features are extracted to obtain a gradient feature image set.

[0023] Based on the gradient feature image set, the potential edge position of the pit is determined, resulting in a pit edge image set.

[0024] Preferably, based on the multi-angle image data, multi-view fusion is performed to obtain a pit depth mapping map, including:

[0025] Based on the multi-angle image data, feature points in the road surface image are extracted and the spatial correspondence between feature points is calculated to obtain a set of registered multi-view images.

[0026] Based on the registered multi-view image set, the disparity values ​​of feature points are calculated to obtain a disparity image set;

[0027] Based on the set of disparity images, the disparity values ​​are mapped to depth information and combined with scene complexity parameters to generate a preliminary pit depth mapping map.

[0028] Based on the preliminary pit depth mapping map, a 3D point cloud transformation is performed to obtain the pit depth mapping map.

[0029] Preferably, a three-dimensional model of the pit is obtained by performing three-dimensional modeling based on the pit depth mapping map and the potential edge positions of the pit, including:

[0030] Based on the potential edge positions of the pits, the pit edge feature points are identified and the spatial correspondence of the feature points is calculated to obtain an edge feature view set;

[0031] Based on the edge feature view set, the edge features are converted into three-dimensional point cloud data, and combined with the pothole depth mapping map, a preliminary three-dimensional model of the pothole is generated.

[0032] If the point cloud density of the preliminary 3D model of the pit is lower than the preset density threshold, then point cloud data supplementation and edge optimization are performed to obtain the 3D model of the pit.

[0033] Preferably, based on the three-dimensional model of the pothole, the volume and shape parameters of the pothole are calculated to obtain the corresponding severity index, and a sorting operation is performed according to the corresponding severity index to obtain a repair priority sequence, including:

[0034] Based on the three-dimensional model of the pit, the volume and shape parameters of the pit are calculated to obtain a first pit parameter set. If the volume parameter in the first pit parameter set is lower than a preset volume threshold, the volume parameter is smoothed to obtain a second pit parameter set.

[0035] Based on the second pothole parameter set, calculate the pothole severity score to obtain the corresponding severity index;

[0036] Based on the corresponding severity index and combined with the geographical location information of the potholes, they are sorted in descending order to generate a repair priority sequence.

[0037] Preferably, the repair priority sequence is dynamically monitored based on the pit depth mapping map to obtain a dynamic monitoring report. Based on the dynamic monitoring report, joint risk verification is performed to obtain joint risk verification results, including:

[0038] If there are high-risk pits in the repair priority sequence, the pit area and the background area are separated according to the pit depth mapping map and the geographical location distribution information, and environmental noise is filtered to obtain a depth feature set.

[0039] Based on the depth feature set, spatiotemporal sequence analysis is performed and the rate of change of pit depth is calculated to obtain the evolution trend set;

[0040] Based on the set of evolution trends and combined with preset risk level assessment standards, a dynamic monitoring report is generated.

[0041] Based on the dynamic monitoring report, the evolution trend of high-risk potholes in the repair priority sequence is extracted to obtain evolution trend data;

[0042] Based on the evolution trend data and the blurred display details of the potential edge positions of the pits, the edge contours are extracted and noise is filtered to obtain the edge blur distribution;

[0043] Based on the evolution trend data and the edge ambiguity distribution, the edge ambiguity and evolution trend are fused and the joint risk value is calculated to obtain the joint risk verification result.

[0044] Preferably, based on the joint risk verification results and the dynamic monitoring report, the road surface database is updated and sensor parameters are optimized to obtain the next cycle's data acquisition strategy, including:

[0045] Obtain the road surface database;

[0046] Based on the dynamic monitoring report and the joint risk verification results, the road surface data and risk values ​​are integrated and the road surface database is updated to obtain the updated road surface database.

[0047] Based on the updated road surface database, the sensor sensitivity and sampling frequency are adjusted to obtain optimized sensor parameters;

[0048] Based on the joint risk verification results and the dynamic monitoring report, the road surface database is updated and sensor parameters are optimized to obtain the next cycle's data acquisition strategy.

[0049] Secondly, the present invention provides a visual intelligence-based intelligent detection system for road potholes, comprising:

[0050] The data acquisition and preprocessing module is used to acquire multi-angle image data and perform illumination compensation processing on the multi-angle image data to obtain a preliminary road surface image set.

[0051] The edge detection module is used to perform environmental variable analysis and image gradient feature extraction based on the preliminary road surface image set to obtain the potential edge locations of potholes;

[0052] The depth mapping module is used to perform multi-view fusion based on the multi-angle image data to obtain a pit depth mapping map;

[0053] The 3D modeling module is used to perform 3D modeling based on the pit depth mapping map and the potential edge position of the pit to obtain a 3D model of the pit.

[0054] The priority sequence generation module is used to calculate the volume and shape parameters of the potholes based on the three-dimensional model of the potholes, obtain the corresponding severity index, and perform a sorting operation based on the corresponding severity index to obtain a repair priority sequence.

[0055] The risk verification generation module is used to dynamically monitor the repair priority sequence according to the pit depth mapping map, obtain a dynamic monitoring report, perform joint risk verification according to the dynamic monitoring report, and obtain joint risk verification results.

[0056] The data acquisition strategy adjustment module is used to update the road surface database and optimize sensor parameters based on the joint risk verification results and the dynamic monitoring report to obtain the data acquisition strategy for the next cycle.

[0057] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the visual intelligence-based intelligent detection method for road potholes described in any one of the above.

[0058] Fourthly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute any one of the above-described methods for intelligent detection of road potholes based on visual intelligence.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] (1) The present invention performs illumination compensation, weather interference filtering and gradient normalization operations on multi-angle image data. All preliminary road surface image sets are subjected to brightness standardization and edge reliability screening, so that the pothole edge information remains analyzable in environments such as tunnels and water accumulation. Through this normalization process, the system can perform subsequent three-dimensional reconstruction and risk scoring more stably, and finally realize the continuous availability of pothole detection data.

[0061] (2) The present invention performs joint three-dimensional modeling based on the pit depth mapping map and the potential edge position of the pit. After merging the edge feature point cloud and the depth pixel point cloud, the complete pit three-dimensional model is obtained by adaptive interpolation and weighted filtering based on the density threshold. This process reduces the dependence on high-density original point cloud, so that weak texture areas can also obtain continuous curved surfaces and improve the integrity of the model.

[0062] (3) The present invention calculates the volume, aspect ratio and rectangularity of the pit based on the three-dimensional model of the pit, and introduces spatial proximity weighted smoothing when the volume is below the threshold to form a second pit parameter set; this strategy reduces the impact of single frame noise on the severity index, making the scoring results closer to the actual damage range, and providing robust input for maintenance decisions.

[0063] (4) This invention performs dynamic monitoring of high-risk pits and troughs. It obtains the depth change rate by fitting the depth time sequence of the area of ​​interest. Combined with the edge ambiguity distribution, it weights and fuses the temporal risk score and the spatial ambiguity score to obtain a joint risk verification result that can be updated with the inspection frame. This method can still maintain risk sensitivity and reduce the probability of missed alarms in scenarios where the edge degrades quickly but the depth change is difficult to distinguish with the naked eye.

[0064] (5) The present invention updates the road database in reverse according to the joint risk verification results and adaptively adjusts the camera gain, laser power and sampling frequency to form the next cycle acquisition strategy. This closed loop enables the sensor parameters to be dynamically matched with the on-site risk level, avoids over-sampling of stable road sections, and increases the data density of rapidly deteriorating areas to improve the utilization rate of inspection resources. Attached Figure Description

[0065] Figure 1 This is a schematic diagram of a visual intelligence-based intelligent detection method for road potholes provided in the first embodiment of the present invention;

[0066] Figure 2 This is a schematic diagram of a road surface pothole intelligent detection system based on visual intelligence provided in the second embodiment of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Reference Figure 1 The first embodiment of the present invention provides a visual intelligence-based intelligent detection method for road potholes, comprising the following steps:

[0069] S11, acquire multi-angle image data, and perform illumination compensation processing on the multi-angle image data to obtain a preliminary road surface image set;

[0070] S12, Based on the preliminary road surface image set, perform environmental variable analysis and image gradient feature extraction to obtain the potential edge location of potholes;

[0071] S13, perform multi-view fusion based on the multi-angle image data to obtain a pit depth mapping map;

[0072] S14. Based on the pit depth mapping map and the potential edge position of the pit, perform three-dimensional modeling to obtain a three-dimensional model of the pit.

[0073] S15. Based on the three-dimensional model of the pothole, calculate the volume and shape parameters of the pothole to obtain the corresponding severity index, and sort the potholes according to the corresponding severity index to obtain a repair priority sequence.

[0074] S16, Based on the pit depth mapping map, the repair priority sequence is dynamically monitored to obtain a dynamic monitoring report. Based on the dynamic monitoring report, joint risk verification is performed to obtain the joint risk verification result.

[0075] S17. Based on the joint risk verification results and the dynamic monitoring report, update the road surface database and optimize the sensor parameters to obtain the next cycle's acquisition strategy.

[0076] In step S11, multi-angle image data is acquired, and illumination compensation processing is performed on the multi-angle image data to obtain a preliminary road surface image set, including:

[0077] Acquire multi-angle image data;

[0078] Based on the multi-angle image data, illumination compensation processing is performed to obtain an illumination-compensated image set;

[0079] Based on the illumination-compensated image set, smoothing processing is performed to obtain a clear road surface feature image set;

[0080] Based on the clear road surface feature image set, multi-angle image fusion is performed to obtain the preliminary road surface image set.

[0081] This step acquires multi-angle image data and performs illumination compensation processing on the multi-angle image data to obtain a preliminary road surface image set. First, multi-angle image data is acquired by installing an industrial camera on the front bumper, left and right rearview mirrors, and rear of the inspection vehicle. The angle between the optical axis and the road surface is maintained between 30° and 45°. This angle range can balance field of view coverage and texture gradient. Below 30°, near-end image compression is severe; above 45°, far-end resolution decreases. The overlap rate of adjacent camera fields of view is no less than 30%. The overlapping area is used for subsequent feature matching and pixel-level fusion. Insufficient overlap reduces the number of matching points, easily causing splicing breaks. While the vehicle is moving, all cameras are synchronously exposed at a frequency of 10Hz via a central trigger module to obtain time-aligned multi-angle image data. The exposure time is uniformly controlled by GPS timing pulses, with a time deviation of less than 1ms, thereby eliminating perspective misalignment caused by vehicle speed.

[0082] The average grayscale value of the entire image is calculated. If the average grayscale value is lower than a preset brightness threshold, the overall brightness is increased using a linear mapping method until the average grayscale value equals the threshold. If the average grayscale value is higher than the threshold, the original image is maintained. The brightness threshold is set to 70% of the average grayscale value of historical sunny noon images of the same road segment, thereby bringing the brightness of low-light images such as those from cloudy days and tunnels to a repeatable range, forming a set of illumination-compensated images. A 3×3 median filter is applied to the illumination-compensated images, replacing the center pixel with the median grayscale value of its neighbors to remove salt-and-pepper noise while preserving the gradient of crack edges, resulting in a clear set of road surface feature images.

[0083] Finally, using the vehicle's forward direction as a reference, images captured by adjacent cameras at the same time are grouped together. SIFT feature points in overlapping areas are detected for each group of images. False matches are iteratively eliminated using the RANSAC (Random Sample Consensus) algorithm. The number of iterations is estimated by the confidence probability of 0.99 and the inlier ratio, typically set to a maximum of 2000 iterations. A reprojection error threshold of 1 pixel is set; points below this threshold are considered inliers and used to estimate the homography matrix. This matrix transforms the pixel coordinates of the images to be stitched to the coordinate system of the reference image. Overlapping pixels are weighted by distance, with higher weights for closer pixels to the center of the reference image. The weighting function uses linear decay, with a center weight of 1 and an edge weight of 0, thus eliminating brightness abrupt changes at the seams. After stitching, invalid edges are cropped, outputting a rectangular image covering the entire width of the lane, which is the initial road surface image set. SIFT refers to Scale Invariant Feature Transform, which uses a Gaussian difference pyramid to detect extreme points and generate a 128-dimensional descriptor to maintain matching stability under rotation, scale, and brightness changes.

[0084] For example, when a vehicle is traveling at 30 km / h, a 10 Hz acquisition frequency corresponds to a shooting interval of 0.83 m. After simultaneous exposure by four cameras, the images are processed through illumination compensation, median filtering, and weighted fusion of overlapping areas to obtain a preliminary set of road surface images with continuous brightness steps, which can be directly used for subsequent pothole edge extraction and depth calculation.

[0085] In step S12, based on the preliminary road surface image set, environmental variable analysis and image gradient feature extraction are performed to obtain the potential edge locations of potholes, including:

[0086] Based on the preliminary road surface image set, a ray standardization operation is performed to obtain a ray-standardized image set;

[0087] Based on the light-normalized image set, it is determined whether there are weather interference features. If there are interference features, the interference area is filtered to obtain a weather-corrected image set.

[0088] Based on the weather-corrected image set, the spatial change rate of image pixels is calculated and gradient features are extracted to obtain a gradient feature image set.

[0089] Based on the gradient feature image set, the potential edge position of the pit is determined, resulting in a pit edge image set.

[0090] This step, based on the preliminary road surface image set, performs environmental variable analysis and image gradient feature extraction to obtain the potential edge locations of potholes. First, the local grayscale mean is calculated for each image in 64×64 pixel blocks, with the block size corresponding to a road surface of approximately 0.4m×0.4m. This captures slow illumination changes while ignoring crack details. After forming the block mean matrix, bilinear interpolation is performed to the original image size to obtain the illumination component estimate. The original image and the illumination component are divided pixel by pixel, and then multiplied by the overall target mean of 128. 128 is the intermediate grayscale of the 8-bit image. This operation ensures that different frames of images are on the same brightness benchmark, outputting a light-normalized image set.

[0091] Within the HSV color space, connected regions with saturation below 20 and brightness above 180 are marked as candidates for water accumulation or reflection. This threshold range is obtained through statistical analysis of rainy day samples and can cover common water film bright areas. At the same time, the local texture entropy of the candidate region is calculated. A texture entropy less than 0.3 indicates that the gray-level distribution is concentrated and lacks texture, and is therefore determined to be a weather interference region. This region is then Gaussian weighted smoothed to weaken its gradient magnitude, while the pixels in the remaining regions remain unchanged, forming a weather-corrected image set.

[0092] Using a 3×3 Sobel template, the difference is calculated along the horizontal and vertical directions respectively. The template coefficients are combinations of -1, 0, 1 and -1, -2, -1. The sum of the absolute values ​​is taken as the gradient magnitude, which can highlight the edges of the pits and suppress slow changes. In order to eliminate the difference in dimensions, the magnitude is normalized to the maximum and minimum values ​​and mapped to the 0-255 interval to obtain the gradient feature image set.

[0093] A preset dynamic edge threshold is set to 1.5 times the overall mean of the gradient image. This multiple is obtained through cross-validation of rainy and sunny day samples, which can suppress noise while preserving true edges. Pixels with amplitudes higher than this threshold are marked as potential edges. The marked image is then filtered by the area of ​​eight connected components, and fragments with an area less than 50 pixels are deleted. 50 pixels corresponds to a road surface of approximately 0.02m. 2 If the area is smaller than the minimum visible area of ​​a typical pit, the area retained after deletion is the potential edge location of the pit, and the pit edge image set is output.

[0094] For example, in a cloudy scene, after the light is standardized, the average grayscale value of the image is increased from 80 to 128. The gradient amplitude of the water reflection area is reduced by 60% after interference filtering. After detection, only the true pit boundary is retained, avoiding misidentification of the water surface reflection edge as damage.

[0095] In step S13, multi-view fusion is performed based on the multi-angle image data to obtain a pit depth mapping map, including:

[0096] Based on the multi-angle image data, feature points in the road surface image are extracted and the spatial correspondence between feature points is calculated to obtain a set of registered multi-view images.

[0097] Based on the registered multi-view image set, the disparity values ​​of feature points are calculated to obtain a disparity image set;

[0098] Based on the set of disparity images, the disparity values ​​are mapped to depth information and combined with scene complexity parameters to generate a preliminary pit depth mapping map.

[0099] Based on the preliminary pit depth mapping map, a 3D point cloud transformation is performed to obtain the pit depth mapping map.

[0100] First, SIFT feature points are detected for images captured by adjacent cameras at the same time, and initial matching is obtained by the nearest neighbor distance ratio. RANSAC is used to remove mismatches, and point pairs with reprojection errors of less than 1 pixel are retained. The fundamental matrix is ​​calculated and the image is corrected so that corresponding points are located on the same horizontal scan line. The registered multi-view image set is then output.

[0101] For the corrected image pairs, a block matching method is used to search in the horizontal direction. The block size is 11×11, which balances texture richness and computational cost. Smaller sizes are prone to introducing noise, while larger sizes will blur edge details. The search range is set to 0 to 2000 pixels, with the upper limit determined by the maximum road surface undulation and baseline length. 2000 pixels correspond to a depth of about 0.10m, covering the common pothole depth range of urban main roads. The maximum value of normalized cross-correlation is taken as the matching cost, and subpixel-level disparity is obtained through parabolic fitting to form a disparity image set.

[0102] The depth value Z is calculated as follows:

[0103]

[0104] Where B is the camera baseline of 0.2m, f is the normalized focal length of 1000 pixels, and d is the parallax. At the same time, the texture entropy of the reference image is calculated as a scene complexity parameter. Regions with texture entropy lower than 0.4 (texture entropy lower than 0.4 indicates that the texture distribution of the region is relatively regular and does not conform to the distribution characteristics of the pit region, and there may be edge holes) are assigned a low confidence weight, such as 0.4. The weight is written together with the depth value to form a preliminary pit depth mapping map.

[0105] Each pixel is back-projected to 3D coordinates. Local median filtering is performed on points with confidence weights below 0.4. The filtering kernel is 3×3, and the median value within the kernel can remove isolated flying points. The processed point cloud is then reprojected back to the depth map to fill edge holes and output the final pit depth map.

[0106] In step S14, based on the pit depth mapping map and the potential edge positions of the pit, a three-dimensional model is performed to obtain a three-dimensional model of the pit, including:

[0107] Based on the potential edge positions of the pits, the pit edge feature points are identified and the spatial correspondence of the feature points is calculated to obtain an edge feature view set;

[0108] Based on the edge feature view set, the edge features are converted into three-dimensional point cloud data, and combined with the pothole depth mapping map, a preliminary three-dimensional model of the pothole is generated.

[0109] If the point cloud density of the preliminary 3D model of the pit is lower than the preset density threshold, then point cloud data supplementation and edge optimization are performed to obtain the 3D model of the pit.

[0110] First, ORB feature points are detected within pixels marked as potential edges, and the top 500 points with the highest response values ​​are retained. This number is sufficient to maintain matching point pairs on weakly textured surfaces. Errors are eliminated using epipolar constraints between adjacent cameras to obtain corresponding point pairs, and an edge feature view set is output. Triangulation is used to convert these corresponding points into 3D coordinates, and the corresponding pixels in the depth map are back-projected into point clouds. After merging the two point clouds, outliers outside a 0.05m range are removed to obtain a preliminary 3D model of the potholes.

[0111] The density threshold is set to at least 10 points per square decimeter, which translates to approximately 1 cm point spacing, to satisfy the detailed depiction of pothole edges. Points with a density lower than this are considered voids, and Kriging interpolation is performed at the voids. Points with a change in elevation greater than 2 cm after interpolation do not match the actual road slope and are discarded to ensure that the interpolation surface is smooth and physically reasonable. Only interpolation points with an elevation difference of less than 0.02 m from their neighbors are retained. Weighted median filtering is performed on the edge areas with a kernel size of 5×5, and the weights are proportional to the confidence level. After filling, the 3D model of the pothole is output.

[0112] In step 15, based on the 3D model of the pothole, the volume and shape parameters of the pothole are calculated to obtain the corresponding severity index. Then, a sorting operation is performed based on the corresponding severity index to obtain a repair priority sequence, including:

[0113] Based on the three-dimensional model of the pit, the volume and shape parameters of the pit are calculated to obtain a first pit parameter set. If the volume parameter in the first pit parameter set is lower than a preset volume threshold, the volume parameter is smoothed to obtain a second pit parameter set.

[0114] Based on the second pothole parameter set, calculate the pothole severity score to obtain the corresponding severity index;

[0115] Based on the corresponding severity index and combined with the geographical location information of the potholes, they are sorted in descending order to generate a repair priority sequence.

[0116] In one implementation, a 0.01m × 0.01m × 0.01m cube is used as the voxel unit. The number of voxels inside the pit is counted, and the result is multiplied by the volume of a single voxel to obtain the volume parameter. Principal component analysis is used to obtain the major axis, minor axis, and depth, and the aspect ratio and rectangularity are calculated as shape parameters. The volume threshold is set to 0.02m. 3 When the volume is below a threshold, a weighted average of the current pit volume and the volumes of adjacent pits within 5 meters is taken, with greater weight for closer pits (for example, the weights are set to be inversely proportional to the distance), to obtain the second pit parameter set. The volume of a single voxel is 1 cm³. 3It can balance computational efficiency and volumetric accuracy under conditions of 30km / h vehicle speed and 10Hz data acquisition; too small a voxel will increase memory usage, while too large a voxel will underestimate the volume. The volume threshold is set to 0.02m. 3 This corresponds to a shallow pit of approximately 20cm×20cm×5cm. If the value is smaller than this, it is considered to be an initial micro-crack. To avoid single-point noise, spatial weighted smoothing is introduced.

[0117] Volume, aspect ratio, and rectangularity were normalized to the 0-1 range using maximum-minimum value normalization. Volume was weighted at 0.6, aspect ratio at 0.25, and rectangularity at 0.15. These weights were derived from regression analysis of historical maintenance data. Volume had the greatest impact on driving safety, aspect ratio reflected the degree of fragmentation, and rectangularity described the regularity of the shape. The weighted score of these three factors showed the highest correlation with the human assessment results. The weighted sum was then used to obtain the severity score, which is the corresponding severity index.

[0118] The severity score is multiplied by the road grade coefficient, which is provided by the geographic information system. The coefficient is 1.2 for arterial roads, 1.0 for secondary roads, and 0.8 for local roads. The coefficient is set according to the urban road network grade and the ratio of daily traffic volume. The coefficient for arterial roads is the highest to ensure that defects in high-traffic areas receive higher maintenance priority. The repair priority sequence is output by sorting the products from high to low.

[0119] For example, the volume of a pit after modeling is 0.04m³. 3 With an aspect ratio of 1.8 and a rectangularity of 0.7, the severity index calculated after normalization is 0.68. Located on a main road, the product is 0.82, ranking first in the same road segment, and the system will prioritize its maintenance.

[0120] In step S16, the repair priority sequence is dynamically monitored according to the pit depth mapping map to obtain a dynamic monitoring report. Based on the dynamic monitoring report, joint risk verification is performed to obtain the joint risk verification results, including:

[0121] Based on the pothole depth mapping map, the pothole region and background region in the repair priority sequence are separated by combining geographical location distribution information, and environmental noise is filtered to obtain a depth feature set.

[0122] Based on the depth feature set, spatiotemporal sequence analysis is performed and the rate of change of pit depth is calculated to obtain the evolution trend set;

[0123] Based on the set of evolution trends and combined with preset risk level assessment standards, a dynamic monitoring report is generated;

[0124] Based on the dynamic monitoring report, the evolution trend of high-risk potholes in the repair priority sequence is extracted to obtain evolution trend data;

[0125] Based on the evolution trend data and the blurred display details of the potential edge positions of the pits, the edge contours are extracted and noise is filtered to obtain the edge blur distribution;

[0126] Based on the evolution trend data and the edge ambiguity distribution, the edge ambiguity and evolution trend are fused and the joint risk value is calculated to obtain the joint risk verification result.

[0127] First, the depth map is aligned with the lane centerline given by GPS. A rectangle 0.5m outward from the centroid of the pothole is taken as the region of interest, and the remaining pixels are marked as the background. At this time, the side length of the rectangle is 1m, which can completely cover the typical potholes and their broken edges of the main urban road, while avoiding the mixing of vehicle textures from adjacent lanes into the region of interest. A three-dimensional discrete cosine transform is performed on the depth values ​​in the region of interest. After setting the high-frequency coefficients to zero, the inverse transform is performed to suppress random noise caused by illumination jitter and output the depth feature set.

[0128] Using the current frame as a baseline, backtrack the average depth of the same region of interest in the previous n frames. The value of n is related to the inspection cycle. For example, n=14 when inspecting every two weeks. Use least squares fitting to obtain the depth-time slope, i.e. the rate of depth change, in mm / day. Write this rate and the corresponding frame number into the evolution trend set.

[0129] The rate of depth change is normalized to the 0-1 range based on its maximum and minimum values ​​to obtain the temporal risk score; the area of ​​interest is normalized to obtain the spatial risk score; the two are then weighted and summed, with weights of 0.7 and 0.3 respectively. A product greater than 0.6 is marked as rapid deterioration, generating a dynamic monitoring report. It is worth noting that the temporal risk score reflects depth acceleration with a weight of 0.7, while the spatial risk score reflects area expansion with a weight of 0.3. Historical sample regression shows that this combination has the highest correlation with manually determined hazard levels. A value of 0.6 corresponds to a normalized depth rate of approximately 0.4 mm / day, which is at the 80th percentile of the measured samples; values ​​exceeding this are considered to require immediate action.

[0130] The normalized depth change rate is directly read from the dynamic monitoring report as evolution trend data. Potential edge locations are multiplied by the area of ​​interest mask to obtain a pit edge sub-image. The gradient magnitude of the sub-image is calculated, and a 5×5 median filter is applied to the magnitude map. The percentage of pixels with a filtered magnitude lower than 50% of the overall mean is statistically analyzed; this percentage represents the edge blurriness and is recorded in the edge blurriness distribution. The edge blurriness is also normalized to its maximum and minimum values ​​to obtain the spatial blurriness score. The temporal risk score and the spatial blurriness score are weighted and summed with weights of 0.6 and 0.4 respectively to obtain a joint risk value between 0 and 1. This value is continuously updated with each inspection frame to form a joint risk verification result. It is worth noting that edge blurriness represents the uncertainty of boundary degradation. Giving it a weight of 0.4 preserves the dominant role of depth acceleration while also considering visual degradation information, making the joint risk value equally sensitive to scenes where the image is "unclear but sinks quickly."

[0131] For example, the inspection cycle is set to 14 days, meaning one frame is collected every 14 days; the average depth is 30mm during the first inspection and 46mm during the fifth inspection, with a time span of 56 days. The fitting rate is approximately 0.29mm / day (46mm-30mm) / 56 days, and the normalized temporal risk score is 0.73. The edge ambiguity accounts for 45%, and the normalized spatial ambiguity score is 0.45. The joint risk value is 0.73×0.6+0.45×0.4=0.618. Based on this, the system advances the maintenance window from quarterly to within two weeks.

[0132] In step S17, based on the joint risk verification results and the dynamic monitoring report, the road surface database is updated and sensor parameters are optimized to obtain the next cycle's data acquisition strategy, including:

[0133] Obtain the road surface database;

[0134] Based on the dynamic monitoring report and the joint risk verification results, the road surface data and risk values ​​are integrated and the road surface database is updated to obtain the updated road surface database.

[0135] Based on the updated road surface database, the sensor sensitivity and sampling frequency are adjusted to obtain optimized sensor parameters;

[0136] Based on the joint risk verification results and the dynamic monitoring report, the road surface database is updated and sensor parameters are optimized to obtain the next cycle's data acquisition strategy.

[0137] It is worth noting that the road surface database is located at the edge computing unit of the inspection vehicle. Each record contains the road segment ID, historical pothole 3D parameters, the latest severity index, the geographic location polygon, and the update timestamp.

[0138] First, the depth change rate, edge ambiguity, and joint risk value from the dynamic monitoring report are appended to the corresponding road segment record. If the joint risk value is greater than 0.6, this threshold corresponds to a normalized depth rate of approximately 0.4 mm / day, which is at the 80th percentile of historical samples. Values ​​higher than this are considered rapid deterioration, so the risk level field is set to high, and the inspection timestamp is updated to the current date. If the risk value is less than 0.3, it is set to low, corresponding to a normalized rate of less than 0.15 mm / day. The original timestamp is retained, and the updated road surface database is completed.

[0139] Sensitivity was adjusted as follows: for high-risk road sections, the camera gain coefficient was increased by 20%, which can improve the signal amplitude in low-light areas without saturating the image; the laser power was increased by 15% to enhance the intensity of structured light projection and improve the depth signal-to-noise ratio of weakly textured road surfaces; for low-risk road sections, both gain and power were restored to the default values; the sampling frequency was adjusted as follows: the frequency in high-risk areas was increased from 10Hz to 15Hz, which can obtain denser spatial sampling at the same vehicle speed; the frequency in low-risk areas was reduced to 5Hz to reduce data redundancy and storage pressure, resulting in optimized sensor parameters.

[0140] Using the risk level in the updated road surface database as weights, K-means clustering is employed to aggregate high-risk points into three clusters, with the cluster centers serving as new data collection points. A priority score is calculated for each road segment, equal to the product of the joint risk value and the lane flow coefficient. The flow coefficient is determined by historical traffic flow classification (the specific coefficient setting is based on the statistical distribution of traffic flow on the road segment; the principle is to assign a higher flow coefficient to high-flow road segments, with a value range of 1.1-1.3, and a lower flow coefficient to low-flow road segments, with a value range of 0.8-0.9). Data collection points are sorted in descending order of priority score, generating a sequence of collection points and recommended driving routes, which serve as the data collection strategy for the next cycle.

[0141] For example, a section of main road has a joint risk value of 0.72, a flow coefficient of 1.2, and a priority score of 0.86. The system ranks it first, increases the sensitivity gain by 20%, raises the sampling frequency to 15Hz, and adds two supplementary measurement points at the cluster center. The second lowest risk section has a score of 0.25, the frequency is reduced to 5Hz, no new points are added, and the strategy loop is completed.

[0142] In summary, this invention discloses a visual intelligence-based intelligent method for detecting road potholes. By acquiring multi-angle images and performing illumination compensation, multi-view fusion, and 3D modeling, combined with depth evolution and edge degradation dynamic tracking, the method continuously updates the joint risk verification results and back-optimizes sensor parameters, thereby achieving continuous high-precision identification and risk quantification of potholes and solving the problem of low detection accuracy in existing technologies.

[0143] Reference Figure 2 The second embodiment of the present invention provides a visual intelligence-based intelligent road pothole detection device, comprising:

[0144] The data acquisition and preprocessing module is used to acquire multi-angle image data and perform illumination compensation processing on the multi-angle image data to obtain a preliminary road surface image set.

[0145] The edge detection module is used to perform environmental variable analysis and image gradient feature extraction based on the preliminary road surface image set to obtain the potential edge locations of potholes;

[0146] The depth mapping module is used to perform multi-view fusion based on the multi-angle image data to obtain a pit depth mapping map;

[0147] The 3D modeling module is used to perform 3D modeling based on the pit depth mapping map and the potential edge position of the pit to obtain a 3D model of the pit.

[0148] The priority sequence generation module is used to calculate the volume and shape parameters of the potholes based on the three-dimensional model of the potholes, obtain the corresponding severity index, and perform a sorting operation based on the corresponding severity index to obtain a repair priority sequence.

[0149] The risk verification generation module is used to dynamically monitor the repair priority sequence according to the pit depth mapping map, obtain a dynamic monitoring report, perform joint risk verification according to the dynamic monitoring report, and obtain joint risk verification results.

[0150] The data acquisition strategy adjustment module is used to update the road surface database and optimize sensor parameters based on the joint risk verification results and the dynamic monitoring report to obtain the data acquisition strategy for the next cycle.

[0151] It should be noted that the visual intelligence-based road pothole detection device provided in this embodiment of the invention is used to execute all the process steps of the visual intelligence-based road pothole detection method in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0152] This invention also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a vision-based intelligent pothole detection program. When the processor executes the computer program, it implements the steps described in the various vision-based intelligent pothole detection method embodiments above, for example... Figure 1 The step S11 shown. Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-described device embodiments, such as the data acquisition module.

[0153] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0154] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0155] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device via various interfaces and lines.

[0156] The memory can be used to store the computer program or module. The processor implements various functions of the electronic device by running or executing the computer program or module stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0157] If the modules integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0158] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0159] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A visual intelligence-based intelligent method for detecting road potholes, characterized in that, include: Acquire multi-angle image data and perform illumination compensation processing on the multi-angle image data to obtain a preliminary road surface image set; Based on the preliminary road surface image set, environmental variable analysis and image gradient feature extraction are performed to obtain the potential edge locations of potholes; Based on the multi-angle image data, multi-view fusion is performed to obtain a pit depth mapping map; Based on the pothole depth mapping map and the potential edge position of the pothole, a three-dimensional model is performed to obtain the three-dimensional model of the pothole; Based on the three-dimensional model of the pothole, the volume and shape parameters of the pothole are calculated to obtain the corresponding severity index. Then, a sorting operation is performed according to the corresponding severity index to obtain a repair priority sequence. Based on the pit depth mapping map, the repair priority sequence is dynamically monitored to obtain a dynamic monitoring report. Based on the dynamic monitoring report, joint risk verification is performed to obtain the joint risk verification result. Based on the joint risk verification results and the dynamic monitoring report, the road surface database is updated and sensor parameters are optimized to obtain the next cycle's data acquisition strategy. The step of performing environmental variable analysis and image gradient feature extraction based on the preliminary road surface image set to obtain the potential edge location of potholes includes: Based on the preliminary road surface image set, a ray standardization operation is performed to obtain a ray-standardized image set; Based on the light-normalized image set, it is determined whether there are weather interference features. If there are interference features, the interference area is filtered to obtain a weather-corrected image set. Based on the weather-corrected image set, the spatial change rate of image pixels is calculated and gradient features are extracted to obtain a gradient feature image set. Based on the gradient feature image set, the potential edge position of the pit is determined to obtain the pit edge image set; The step of performing multi-view fusion based on the multi-angle image data to obtain a pit depth mapping map includes: Based on the multi-angle image data, feature points in the road surface image are extracted and the spatial correspondence between feature points is calculated to obtain a set of registered multi-view images. Based on the registered multi-view image set, the disparity values ​​of feature points are calculated to obtain a disparity image set; Based on the set of disparity images, the disparity values ​​are mapped to depth information and combined with scene complexity parameters to generate a preliminary pit depth mapping map. Based on the preliminary pit depth mapping map, a 3D point cloud transformation is performed to obtain the pit depth mapping map.

2. The intelligent road pothole detection method based on visual intelligence according to claim 1, characterized in that, The process of acquiring multi-angle image data and performing illumination compensation processing on the multi-angle image data to obtain a preliminary road surface image set includes: Acquire multi-angle image data; Based on the multi-angle image data, illumination compensation processing is performed to obtain an illumination-compensated image set; Based on the illumination-compensated image set, smoothing processing is performed to obtain a clear road surface feature image set; Based on the clear road surface feature image set, multi-angle image fusion is performed to obtain the preliminary road surface image set.

3. The intelligent road pothole detection method based on visual intelligence according to claim 1, characterized in that, The step of performing three-dimensional modeling based on the pit depth mapping map and the potential edge positions of the pit to obtain a three-dimensional model of the pit includes: Based on the potential edge positions of the pits, the pit edge feature points are identified and the spatial correspondence of the feature points is calculated to obtain an edge feature view set; Based on the edge feature view set, the edge features are converted into three-dimensional point cloud data, and combined with the pothole depth mapping map, a preliminary three-dimensional model of the pothole is generated. If the point cloud density of the preliminary 3D model of the pit is lower than the preset density threshold, then point cloud data supplementation and edge optimization are performed to obtain the 3D model of the pit.

4. The intelligent road pothole detection method based on visual intelligence according to claim 1, characterized in that, The process involves calculating the volume and shape parameters of the potholes based on the 3D model to obtain a corresponding severity index, and then sorting the potholes according to the severity index to obtain a repair priority sequence, including: Based on the three-dimensional model of the pit, the volume and shape parameters of the pit are calculated to obtain a first pit parameter set. If the volume parameter in the first pit parameter set is lower than a preset volume threshold, the volume parameter is smoothed to obtain a second pit parameter set. Based on the second pothole parameter set, calculate the pothole severity score to obtain the corresponding severity index; Based on the corresponding severity index and combined with the geographical location information of the potholes, they are sorted in descending order to generate a repair priority sequence.

5. The intelligent road pothole detection method based on visual intelligence according to claim 1, characterized in that, The step involves dynamically monitoring the repair priority sequence based on the pit depth mapping map to obtain a dynamic monitoring report, and then performing joint risk verification based on the dynamic monitoring report to obtain joint risk verification results, including: Based on the pothole depth mapping map, the pothole region and background region in the repair priority sequence are separated by combining geographical location distribution information, and environmental noise is filtered to obtain a depth feature set. Based on the depth feature set, spatiotemporal sequence analysis is performed and the rate of change of pit depth is calculated to obtain the evolution trend set; Based on the set of evolution trends and combined with preset risk level assessment standards, a dynamic monitoring report is generated; Based on the dynamic monitoring report, the evolution trend of high-risk potholes in the repair priority sequence is extracted to obtain evolution trend data; Based on the evolution trend data and the blurred display details of the potential edge positions of the pits, the edge contours are extracted and noise is filtered to obtain the edge blur distribution; Based on the evolution trend data and the edge ambiguity distribution, the edge ambiguity and evolution trend are fused and the joint risk value is calculated to obtain the joint risk verification result.

6. The intelligent road surface pothole detection method based on visual intelligence according to claim 1, characterized in that, The process of updating the road surface database and optimizing sensor parameters based on the joint risk verification results and the dynamic monitoring report to obtain the next cycle's data acquisition strategy includes: Obtain the road surface database; Based on the dynamic monitoring report and the joint risk verification results, the road surface data and risk values ​​are integrated and the road surface database is updated to obtain the updated road surface database. Based on the updated road surface database, the sensor sensitivity and sampling frequency are adjusted to obtain optimized sensor parameters; Based on the optimized sensor parameters and the updated road surface database, the distribution and priority of the collection points for the next cycle are generated, and the collection strategy for the next cycle is obtained.

7. A visual intelligence-based intelligent system for detecting potholes in road surfaces, characterized in that, For implementing the method as described in any one of claims 1-6, comprising: The data acquisition and preprocessing module is used to acquire multi-angle image data and perform illumination compensation processing on the multi-angle image data to obtain a preliminary road surface image set. The edge detection module is used to perform environmental variable analysis and image gradient feature extraction based on the preliminary road surface image set to obtain the potential edge locations of potholes; The depth mapping module is used to perform multi-view fusion based on the multi-angle image data to obtain a pit depth mapping map; The 3D modeling module is used to perform 3D modeling based on the pit depth mapping map and the potential edge position of the pit to obtain a 3D model of the pit. The priority sequence generation module is used to calculate the volume and shape parameters of the potholes based on the three-dimensional model of the potholes, obtain the corresponding severity index, and perform a sorting operation based on the corresponding severity index to obtain a repair priority sequence. The risk verification generation module is used to assess the risk level of the repair priority sequence based on the pit depth mapping map, obtain a dynamic monitoring report, and perform joint risk verification based on the dynamic monitoring report to obtain the joint risk verification result. The data acquisition strategy adjustment module is used to update the road surface database and optimize sensor parameters based on the joint risk verification results and the dynamic monitoring report to obtain the data acquisition strategy for the next cycle.

Images