A large-scale road surface anomaly detection method based on prior information

Abstract

This invention relates to the field of road surface inspection technology and discloses a method for large-scale road surface anomaly detection based on prior information. The method includes: acquiring and filtering a road surface depth image; calculating a gradient field based on the pixel depth values ​​of the depth image; identifying and connecting complete regions representing drainage channels and joints based on prior information about road surface flatness and grooves using the consistency of gradient amplitude and direction; removing these regions from the depth image and extracting anomaly candidate regions from the remaining regions; verifying and extracting complete anomaly regions by performing area filtering and pixel-level depth difference calculation on the candidate regions; and finally, cropping and normalizing the data of these regions and inputting it into a deep learning classification network to identify the anomaly category. This invention removes inherent interference through rapid preprocessing guided by prior knowledge and combines it with deep learning for judgment, utilizing CUDA acceleration to achieve high-precision detection of large-scale spliced ​​road surface anomalies, effectively improving the reliability and efficiency of automated inspection.

Description

Technical Field

[0001] This invention relates to the field of road surface inspection technology and discloses a method for detecting large-scale road surface anomalies based on prior information. Background Technology

[0002] Regular inspections and maintenance of various runways (such as airport runways, test track runways, and large sports field runways) are fundamental to ensuring their structural integrity, operational safety, and extended service life. Currently, runway inspections and maintenance mainly rely on the following methods: first, traditional manual visual inspections, conducted by professionals walking or moving slowly along the runway; second, the deployment of fixed dedicated sensor systems, such as lidar or specific image sensors, in key areas; and third, the use of mobile platforms (such as vehicles) equipped with detection equipment for regular scanning.

[0003] However, existing technical solutions all have significant limitations in practical applications. Manual inspection is inefficient, highly subjective, and difficult to objectively quantify and accurately locate defects. Fixed sensor systems are costly to deploy and lack flexibility, failing to achieve large-scale, full-coverage dynamic detection. While existing mobile inspection platforms improve detection range and flexibility, they have significant shortcomings in detection accuracy and positioning accuracy: on the one hand, detection methods based on LiDAR or FMCW radar are susceptible to environmental interference, costly, and difficult to achieve millimeter-level precision in quantitative detection; on the other hand, vision-based detection methods (especially those relying on deep learning for direct discrimination) have high false positive and false negative rates against complex runway texture backgrounds, and most lack high-precision spatial correlation capabilities with the runway's physical coordinate system, resulting in a serious disconnect between "detection" and "positioning," failing to meet the operational requirements of "accurate repair upon discovery."

[0004] Therefore, the industry urgently needs a mobile detection solution that integrates high-resolution 3D perception, intelligent processing of runway prior information, and high-precision spatial positioning technology. This solution can achieve rapid and accurate anomaly identification in complex environments, while mapping the identification results to the runway's absolute coordinate system in real time. This truly realizes a closed-loop operation process from "detection" to "location" and from "discovery" to "response," providing efficient and reliable technical support for the digital and intelligent maintenance of runways. Summary of the Invention

[0005] This invention discloses a large-scale road surface anomaly detection method based on prior information, which can achieve rapid, integrated detection and millimeter-level accuracy identification of surface defects and foreign objects on various paved roads such as airport runways and test roads, significantly improving the automation level of inspections and the reliability of detection results. This method integrates rapid preprocessing guided by prior information on the inherent geometric features of the road surface with secondary judgment using a deep learning model, and utilizes CUDA acceleration to achieve efficient processing throughout the entire process. This effectively overcomes the systemic challenges of existing technologies, such as slow detection speed, poor environmental adaptability, high false alarm and false negative rates, and difficulty in achieving quantitative and refined detection against complex texture backgrounds.

[0006] To achieve the above objectives, this invention provides a method for detecting large-scale road surface anomalies based on prior information. The large-scale road surface is composed of multiple cement blocks, including: Acquire pavement depth images of the road surface to be detected and perform median filtering to obtain the filtering results; Based on the filtering results, the gradient magnitude and gradient direction of each pixel in the pavement depth image are calculated to obtain the calculation results; Based on prior information about large-scale road surfaces and the calculation results, candidate regions for drainage channels and splicing joints between adjacent cement blocks are generated. Perform region filtering and connectivity processing on the candidate regions to obtain the complete region concerning the drainage ditch and the splicing joint between adjacent cement blocks; The complete region is removed from the pavement depth image after median filtering to obtain the corresponding region to be detected; anomaly candidate regions are obtained by filtering the region to be detected based on gradient magnitude. The area of ​​the candidate abnormal region is filtered and the pixel-level depth is calculated. The results are then verified using a CUDA kernel function to obtain the complete abnormal region. The depth data of the abnormal complete region is normalized, and the processed depth data is input into the classification model to determine the category information of the anomaly.

[0007] Optionally, the step of acquiring the pavement depth image of the road surface to be detected and performing median filtering to obtain the filtering result includes: Acquire pavement depth images of the road surface to be inspected and save them to the host memory; The track depth image is transferred from the host memory to the graphics card memory; In the graphics card memory, the track depth image is subjected to median filtering to eliminate holes in the track depth image, and the filtering result is obtained. The calculation formula for the median filtering process is as follows: In the formula: The coordinates in the depth image are The pixel depth value at that location, This indicates the depth image after median filtering. Pixel depth value at that location, This indicates the preset filtering window. This indicates the midpoint operation.

[0008] Optionally, calculating the gradient magnitude and gradient direction of each pixel in the pavement depth image includes: Using a horizontal Sobel filter kernel With vertical Sobel filter kernel Perform filtering calculations; The horizontal Sobel filter kernel and vertical Sobel filter kernel They are respectively: , For each pixel in the pavement depth image, perform the following calculation: The filter kernel and Perform convolution operations to obtain the gradient values ​​of each pixel along the x-axis of the image coordinate system. and gradient value in the y-axis direction of the image coordinate system ; According to the formula Calculate the gradient magnitude of each pixel. ; According to the formula Calculate the gradient direction of each pixel and will Normalized to the range of 0° to 180°.

[0009] Optional prior information for large-scale road surfaces includes: The normal area of ​​the large-scale road surface is flat, which is represented in the road surface depth image as a depth difference of less than 1 mm between adjacent pixels. The large-scale pavement has grooves, which include joints between adjacent cement blocks and drainage channels on the surface of the cement blocks; the grooves are represented in the pavement depth image as having high gradient amplitude and consistent gradient direction at their edges. The high gradient amplitude specifically refers to the depth of the groove being lower than that of the normal road surface area, with a depth difference greater than 1 mm.

[0010] Optionally, based on prior information about the large-scale road surface and the calculation results, candidate regions for drainage channels and joints between adjacent cement blocks are generated, including: Based on the high gradient amplitude of the groove edge, a first threshold is set, and all pixels with gradient amplitude greater than the first threshold are extracted from the calculation results to form an initial candidate pixel set. The first threshold is 1.2 to 1.5 times the average gradient magnitude of all pixels in the pavement depth image; Based on the consistent gradient direction of the groove edges, a second threshold is set. For each pixel in the initial candidate pixel set, the angle between its gradient direction and that of its neighboring pixels is determined by the CUDA kernel function. If the directional angle is less than the second threshold, the pixel and its adjacent pixels are determined to belong to the same direction, and the regions corresponding to the two are connected. The second threshold is 0°~10°. Repeat the above operations, traverse the initial candidate pixel set, connect the regions corresponding to pixels that satisfy the gradient direction consistency condition, and generate candidate regions for drainage channels and splicing seams between adjacent cement blocks.

[0011] Traverse the initial candidate pixel set, connect the regions corresponding to pixels that satisfy the gradient direction consistency condition, and generate candidate regions for drainage channels and splicing seams between adjacent cement blocks.

[0012] Optionally, region filtering and connectivity processing are performed on the candidate regions to obtain a complete region concerning the drainage ditch and the joint between adjacent cement blocks, including: The regions in the candidate region whose major axis dimension is greater than the third threshold are extracted as the preliminarily determined drainage channel and splicing seam regions, where the third threshold is 1m~2m; For the initially determined drainage channel and splicing seam area, according to its long axis direction, the areas that are broken due to interference from foreign objects, pits or falling pieces in this direction are connected to generate the complete area.

[0013] Optionally, the complete region is removed from the pavement depth image after median filtering to obtain the corresponding region to be detected; anomaly candidate regions are obtained from the region to be detected based on gradient magnitude, specifically including: The area to be detected is obtained by removing the complete area of ​​the drainage ditch and the splicing joint between adjacent cement blocks in the pavement depth image; In the region to be detected, pixel regions with gradient magnitudes greater than the fourth threshold are extracted to generate the anomaly candidate regions. The fourth threshold is 1.8 to 2.2 times the average gradient magnitude of all pixels in the pavement depth image.

[0014] Optionally, area filtering and pixel-level depth calculation are performed on the candidate abnormal regions, and verification is performed using CUDA kernel functions to obtain the complete abnormal region; including: If the area of ​​the abnormal candidate region is less than the fifth threshold, the current detection process is terminated; If the area of ​​the abnormal candidate region is not less than the fifth threshold, then for each pixel in the abnormal candidate region, the difference between the pixel depth value and the average depth of all pixels in the surrounding neighborhood of the pixel is calculated using the CUDA kernel function. The CUDA kernel function is used to judge each pixel in the abnormal candidate region: if the absolute value of the difference is greater than 1 mm, the pixel is determined to be an abnormal pixel. The pixel regions corresponding to all pixels identified as abnormal are merged to obtain the complete abnormal region.

[0015] The fifth threshold is 0.01m²~0.02m².

[0016] Optionally, the depth data of the abnormal complete region is normalized, and the processed depth data is input into a classification model to determine the anomaly category information, including: The depth data of the abnormally complete region is cropped to obtain cropped depth data blocks; Calculate the mean and variance of the depth of the cropped depth data block; Based on the mean and variance of the depth, the cropped depth data block is subjected to grayscale stretching to enhance the contrast between its internal abnormal features and the background, thereby obtaining normalized depth data. The normalized depth data is used to determine abnormal area diseases using any of the following methods: The normalized depth data is input into a deep learning classification network to determine the type of disease in the abnormal area. The normalized depth data is input into a deep learning multi-class segmentation network; a disease category segmentation map for each pixel is output; it is determined whether there is a connected region in the segmentation map that intersects with the abnormally complete region; if so, the disease category corresponding to the intersecting region is determined as the final category of the abnormally complete region.

[0017] Optionally, the depth data of the abnormal complete region is normalized, and the processed depth data is input into a classification model to determine the category information of the anomaly. This also includes the following methods: The depth data of the abnormal complete region is cropped to obtain the cropped abnormal data block. Normal region data blocks are obtained by cropping the adjacent detection region of the abnormal complete region. The depth mean and variance of each data block are calculated. The grayscale of each data block is stretched according to the mean and variance to obtain the normalized abnormal data block and normal region data block. Based on the geometric orientation of the complete region, the abnormal data blocks and normal region data blocks are rotated and corrected to make their orientations consistent; all corrected data blocks are merged, and their main components are extracted by PCA to form a low-dimensional feature vector. The low-dimensional feature vector is input into a pre-trained feature classifier, which outputs the category information of the anomaly; wherein the feature classifier is a support vector machine (SVM) or a multilayer perceptron (MLP).

[0018] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention include at least the following: This invention addresses the inherent contradiction in existing road anomaly detection schemes—the difficulty in balancing detection speed, recognition accuracy, and algorithm generalization—by integrating prior information about the inherent geometric features of the road surface with a deep learning model. By leveraging prior knowledge such as the high gradient magnitude and consistent gradient direction of grooves, it rapidly identifies and filters out interference from inherent regular road surface structures (drainage channels and seams), focusing the computational burden of the deep learning model on a small number of high-confidence anomaly candidate regions, thus achieving synergistic optimization of algorithm efficiency and accuracy. The introduction of CUDA accelerates the entire process of intensive operations such as median filtering, Sobel gradient calculation, region connectivity, and pixel-level depth difference calculation, ensuring real-time processing capabilities for massive amounts of road data. By setting a 1-millimeter quantization threshold for depth difference determination and performing grayscale stretching and normalization based on mean and variance on the cropped region data, it achieves millimeter-level accuracy detection and stable identification of anomaly regions. This method, while ensuring high-precision detection, significantly reduces the processing time for a single (2m×2m area) to less than 50ms, achieving automated inspection with wide coverage, high speed, high accuracy, and strong environmental adaptability. It significantly reduces the reliance on high-cost dedicated sensors or repetitive manual checks, providing an efficient and reliable technical solution for intelligent and preventive maintenance of various paved roads. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 A schematic diagram of the core steps of the detection method provided by the present invention; Figure 2 A detailed algorithm flowchart of the detection method provided by this invention; Figure 3 This is a schematic diagram of the road joint line and drainage channel provided by the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0023] Example 1 As can be seen from the background technology above, existing runway inspection solutions have significant shortcomings in terms of efficiency, accuracy, and practicality. Manual inspection and fixed sensor systems are difficult to achieve efficient and comprehensive dynamic detection. While existing mobile detection platforms have improved coverage, their core technologies still face a dilemma: radar-based solutions are costly, have poor environmental adaptability, and struggle to achieve millimeter-level quantization accuracy; vision-based solutions suffer from high false detection and false negative rates under interference from complex runway textures (such as pavement grooves and seams), and most methods fail to effectively integrate prior geometric information about the runway, resulting in redundant computations, poor real-time performance, and an inability to reliably correlate detection results with high-precision physical coordinates, making it difficult to support the closed-loop operation and maintenance requirements of "detection equals location, location equals handling." Therefore, to achieve rapid, accurate, and localizable detection of runway anomalies, there is an urgent need for an integrated method that can fully integrate three-dimensional depth perception, intelligent processing of inherent prior runway information, and parallel computing acceleration to achieve high-efficiency, high-precision, and low-false-alarm anomaly identification and location in complex environments, truly meeting the requirements of modern digital and intelligent runway operation and maintenance.

[0024] like Figure 3 As shown, taking a typical airport runway surface as an example, the joint lines between the drainage channels and runway blocks are one specific example of the "inherent regular structure of the road surface" as described in this invention. The detection method provided by this invention works based on the geometric prior information of such inherent structures. Figure 1 and Figure 2 As shown, the method includes the following steps: Step S101: Obtain depth image data of the road surface to be detected.

[0025] In this embodiment, a planar array LiDAR or a high-precision structured light 3D camera mounted on a mobile inspection vehicle is used as the data acquisition device. The mobile vehicle travels at a constant speed along the lane, and the sensor continuously acquires 3D point cloud data of the road surface directly below at a certain frame rate. Each frame of data covers a road surface area of ​​approximately 2 meters (in the direction of travel) × 2 meters (lateral width), obtaining data with sufficient point cloud density. The acquisition system synchronously receives spatiotemporal positioning information output by a high-precision positioning system, assigning precise spatiotemporal stamps and geographic coordinates to each frame of data. After real-time processing, the 3D point cloud data generates a road surface depth image that strictly corresponds to the physical scale, where the grayscale value of each pixel represents the vertical height of that point relative to the sensor reference plane, in millimeters, and the image resolution matches the original point cloud density.

[0026] Step S102: Perform parallel preprocessing and gradient calculation on the depth image data, and identify and filter out the inherent grooves on the road surface (i.e., drainage channels and splicing joints between adjacent cement blocks) based on prior information.

[0027] First, the track depth image data obtained in step S101 is asynchronously transferred from the system host memory to the global memory of the graphics processor (GPU).

[0028] Next, a parallel median filtering kernel function is launched on the GPU to preprocess the depth image. A 5×5 (or 3×3) filtering window is selected, and each GPU thread is responsible for calculating the filtered value of one pixel in the output image. The mathematical expression of the median filtering is: ,in and These are the input and output depth images, respectively. This indicates the selected filter window (e.g., 5×5). This operation effectively eliminates depth value holes or extreme noise caused by sensor noise, water surface reflection, or minor obstructions, while preserving the edges of structures such as grooves relatively well.

[0029] Then, CUDA-accelerated gradient field calculations are performed. The horizontal Sobel filter kernel is pre-stored in the GPU's shared memory. Sobel filter kernel in the vertical direction Their specific matrix forms are as follows: , Each thread block is responsible for a segment of the image. Threads within the block collaboratively load pixel data into shared memory, and then... and Perform convolution operations to calculate the gradient components of each pixel in the x and y directions. and Subsequently, according to the public... and Parallel computation of gradient magnitude for each pixel and gradient direction (Normalized to 0°~180°). This step outputs the gradient magnitude and gradient direction.

[0030] Subsequently, based on the prior information of the large-scale road surface and the above calculation results, candidate regions for drainage channels and splicing seams are generated. The prior information includes: the normal road surface area is flat, and the depth difference between adjacent pixels is within 1 mm; the grooves in the depth image are characterized by high gradient amplitude and consistent gradient direction at the edges, and their physical depth is greater than 1 mm lower than that of the normal road surface.

[0031] The specific operations include: setting a first gradient threshold T1 (for example, 1.3 times the average gradient magnitude of the current frame); based on the high gradient magnitude of the groove edges, marking pixels with gradient magnitudes greater than T1 as significant edge candidate points to form an initial candidate pixel set. On this basis, based on the consistent gradient direction of the groove edges, a second threshold is set. (For example, 10°), initiate a region-connected CUDA kernel function: Using each candidate point as the center, examine the pixels within its eight neighborhoods. If a neighboring pixel is also a candidate point, and the absolute difference between their gradient directions is less than... If two points are found to be in the same direction, they are merged into the same connected region. This process is repeated in parallel to generate several connected regions, which are called "candidate regions for drainage channels and seams".

[0032] Finally, region filtering and connectivity processing are performed on the candidate regions to obtain the complete groove region. The minimum bounding rectangle of each candidate region is calculated, and its major axis dimension is taken as the feature length L. A third threshold TL (e.g., 1.5 meters) is set, and regions where L>TL are retained as "preliminarily determined drainage groove and splice seam regions". Since the road surface may have foreign objects, pits, or broken pieces, causing a complete groove to be broken in the image, connectivity repair is required: along the major axis direction of the region, within a certain range (e.g., ±0.5 meters) on both sides, it is searched for other broken candidate regions with similar directions. If they exist and the gap between them is less than the preset connectivity distance (e.g., 0.5 meters), they are merged to form the final, complete "complete region of drainage groove and splice seam between adjacent cement blocks". At this point, the accurate identification and extraction of the inherent groove structure of the road surface is completed.

[0033] Step S103: Remove the complete region from the pavement depth image after median filtering to obtain the corresponding region to be detected; obtain abnormal candidate regions based on gradient magnitude filtering in the region to be detected; perform area filtering and pixel-level depth calculation on the abnormal candidate regions, and verify them through CUDA kernel function to obtain abnormal complete regions.

[0034] First, the pixels covered by the complete area of ​​the drainage ditch and the splicing joint between adjacent cement blocks identified in step S102 are set to invalid values ​​from the pavement depth image after median filtering, resulting in a "clean" area to be detected.

[0035] Subsequently, in the region to be detected, an initial screening is performed based on gradient magnitude to generate anomaly candidate regions. A relatively high fourth threshold T2 is set (for example, 2.0 times the average gradient magnitude of all pixels in the region to be detected), and pixels with gradient magnitudes greater than T2 are extracted to form preliminary anomaly candidate regions.

[0036] Next, area filtering and pixel-level depth calculation are performed on the candidate anomaly regions. First, the area of ​​the candidate anomaly region is calculated. If its area is less than a preset fifth threshold (e.g., 0.015 square meters), it is considered not to constitute a valid anomaly, and the current detection process ends directly. For regions whose area meets the condition, a GPU thread is started for each pixel within it to perform pixel-level depth calculation: a 5×5 (or 3×3) neighborhood window is taken with the pixel as the center, and the arithmetic mean of the depth values ​​of all valid pixels within the window is calculated. Then, the depth value of the center pixel is calculated. and The difference This step is performed in parallel using CUDA kernel functions.

[0037] Then, verification is performed using CUDA kernel functions. Each difference Δ calculated above is evaluated: if its absolute value is greater than 1 millimeter, the pixel is determined to be an anomalous pixel. The connected region formed by all pixels determined to be anomalous is the anomalous complete region.

[0038] Step S104: Perform data normalization processing on the extracted abnormal regions, and input the corresponding classification model for identification and judgment according to different implementation methods.

[0039] First, based on the location and range of the abnormal complete region obtained in step S103, the corresponding depth data block is cropped from the pavement depth image after median filtering to obtain the abnormal data block.

[0040] Next, the abnormal data block is normalized. The mean of the depth values ​​of all valid pixels within the data block is calculated. With variance Based on the mean and variance, the data blocks are subjected to grayscale stretching (e.g., mapping depth values ​​to a standardized range via linear transformation). This processing aims to eliminate overall brightness and contrast differences under different acquisition conditions, enhance the contrast between abnormal areas and the normal pavement background, thereby obtaining normalized abnormal data.

[0041] Finally, depending on different system configurations or application requirements, any of the following methods can be selected to classify and determine the normalized abnormal data: Method 1: Direct determination based on deep learning classification networks.

[0042] The normalized anomalous data (which can be resampled to a fixed size required by the network, such as 224×224 pixels) is input into a pre-trained deep learning classification network (e.g., a lightweight convolutional neural network CNN). After forward inference, the network directly outputs a category label and confidence score, which represents the most likely type of damage to the anomalous intact area (e.g., "metal foreign object", "rubber debris", "pavement damage", etc.).

[0043] Method 2: Verification decision based on deep learning segmentation network.

[0044] The normalized outlier data is input into a pre-trained deep learning multi-class segmentation network (e.g., a network based on the U-Net architecture). The network outputs a disease category segmentation map of the same size as the input, where each pixel is assigned a disease category label.

[0045] Subsequently, the segmentation results are compared with the precise spatial location of the abnormally complete regions extracted in step S103: in the segmentation map, all connected regions that have spatial intersection with the abnormally complete regions are located.

[0046] If such overlapping areas exist, the main disease categories appearing within these overlapping areas are statistically analyzed, and this category is determined as the final category of the abnormally complete area. This method, through cross-validation of the "initial inspection area" and the "segmentation results," can further improve the reliability of the judgment.

[0047] Method 3: Judgment based on traditional machine learning feature classifiers.

[0048] In this approach, joint feature data needs to be prepared: Data preparation: In addition to the abnormal data block mentioned above, a normal data block of similar size needs to be cut out from the area to be tested that is adjacent to the abnormal complete area and has had its grooves removed.

[0049] Joint normalization: Calculate the depth mean and variance of the abnormal and normal data blocks respectively, and then perform grayscale stretching on both independently to complete the normalization.

[0050] Orientation unification: Based on the geometric orientation (such as the major axis direction) of the complete region (i.e. the groove region) obtained in step S102, all normalized data blocks (including abnormal and normal blocks) are rotated and corrected to ensure that the local surface orientation they represent is consistent.

[0051] Feature dimensionality reduction: All data blocks after orientation correction are merged into a feature set, and principal component analysis (PCA) is used to reduce its dimensionality, extracting the most important components of the data to form a set of low-dimensional feature vectors.

[0052] Classification and Recognition: The low-dimensional feature vector is input into a pre-trained feature classifier (e.g., Support Vector Machine (SVM) or Multilayer Perceptron (MLP)). Based on the learned decision boundary, the classifier outputs the category determination result corresponding to the abnormal data block.

[0053] Regardless of the method used, after obtaining the anomaly category information, the system will combine the spatiotemporal positioning information recorded for that frame of data in step S101 to calculate the precise geographical location (polygonal outline or center point coordinates) of the complete anomaly area in the absolute coordinate system of the road surface. Finally, the system outputs a structured inspection report, which includes at least information such as "anomaly category," "confidence level," "physical location," "estimated dimensions," and "protrusion height," thus achieving a complete closed loop from "detection," "classification," to "location," providing a direct and reliable basis for subsequent maintenance and handling.

[0054] Example 2 This embodiment is a further refinement and optimization of step S102 in Embodiment 1, specifically disclosing how to utilize prior information of the runway surface (especially the edge features of the grooves) to achieve efficient and accurate identification and filtering of inherent groove structures on the road surface such as drainage channels and splice joints.

[0055] S201: Parallel preprocessing of depth data.

[0056] First, the same data acquisition process as step S101 in Embodiment 1 is executed to acquire a frame of pavement depth image data to be processed. After the data is transferred from the host memory to the GPU memory, parallel median filtering is initiated for preprocessing. In this embodiment, a 5×5 (or 3×3) filtering window is preferably used. On the GPU, each thread is responsible for calculating the filtered depth value of a target pixel in the output image. Specifically, the depth values ​​of all pixels in a 5×5 (or 3×3) neighborhood centered on the target pixel are read, sorted, and the median is taken as the output value of the target pixel. This operation can effectively suppress impulse noise, fill in small holes, and preserve the sharp edges of inherent structures such as grooves to the greatest extent, providing a high-quality data foundation for subsequent gradient calculations.

[0057] S202: Parallel computation of gradient fields.

[0058] On the depth image after median filtering, CUDA-accelerated gradient calculation is performed. This embodiment defines and uses a horizontal Sobel filter kernel. Sobel filter kernel in the vertical direction as follows: , In the GPU implementation, and Pre-stored in constant memory or shared memory to improve access efficiency. Each thread is responsible for gradient calculation of one pixel: reading the 3×3 neighborhood data of that pixel, and respectively... and Perform a convolution operation to obtain the gradient value in the x-direction of that pixel. and gradient value in the y direction Subsequently, through the formula Calculate the gradient magnitude of this pixel. Through formula Calculate gradient direction and will Normalized to the range of 0° to 180°. The entire computation process is performed in parallel on the GPU, generating gradient magnitude and gradient direction maps of the same size as the original depth image.

[0059] S203: Generate groove candidate regions based on prior information.

[0060] This step is crucial for groove recognition. Its core is to apply the prior information of the large-scale road surface, namely, that the groove edge has a high gradient amplitude and a consistent gradient direction.

[0061] First, based on the high gradient magnitude of the groove edges, a first threshold T1 is set according to the gradient magnitude map to initially screen for significant edges. T1 can be dynamically set to a value between 1.2 and 1.5 times the global mean of the gradient magnitude of the current frame, for example, 1.3 times. Pixels with gradient magnitude G greater than T1 are marked as "significant edge pixels," forming an initial candidate pixel set.

[0062] Subsequently, based on the consistent gradient direction of the groove edges, the set is partitioned and connected. A custom CUDA kernel function is started, iterating through each salient edge pixel (called a "seed pixel"). For each seed pixel, its eight neighbors are checked. If the neighboring pixels are also salient edge pixels, and their gradient directions are consistent... The absolute value of the difference is less than the preset directional consistency threshold. If the threshold (i.e., the second threshold) is reached, then the two pixels are determined to belong to the same direction and are marked as the same connected region. The threshold... The angle can be set according to the regularity of the groove, for example, 10°, and its preferred range can be selected between 0° and 10°. This process connects all adjacent pixels that meet the directional consistency condition in a parallel and iterative manner to form several initial connected regions, which are called "candidate regions for drainage grooves and seams".

[0063] S204: Geometric screening and broken connectivity of candidate regions.

[0064] Geometric analysis is performed on all candidate regions obtained in step S203 to further confirm whether they are grooves that need to be filtered out, and repair is performed to form a complete region.

[0065] First, calculate the minimum bounding rectangle of each candidate region, obtaining its major axis length L and direction. Set a length threshold TL (i.e., the third threshold), retaining only candidate regions where the major axis length L is greater than TL. TL is used to filter structures with significant length characteristics; its value can be set to, for example, 1.5 meters, and can be adjusted within a range of 1 to 2 meters based on the typical length of the actual groove. Regions meeting this condition are identified as the "preliminarily determined drainage channel and splice seam area".

[0066] Next, the fractured sections of the initially identified areas are connected. Due to interference from foreign objects, potholes, or loose blocks on the road surface, a complete groove may appear broken in the depth image. This embodiment employs a connection strategy based on direction and distance: for each initially identified area, the major axis direction of its smallest bounding rectangle is obtained. A search is performed along both sides of this major axis direction within a set connection distance threshold TC (e.g., the search range is set to 0.5 meters, preferably 0.5m to 1m). If other broken candidate area segments with similar major axis directions are found within this range, these two area segments are merged. This process is iterated until all spatially and directionally adjacent fractured segments are connected, ultimately outputting a complete and continuous "complete area regarding the drainage channel and the joint between adjacent cement blocks".

[0067] S205: Filtering out areas with inherent grooved structures.

[0068] All pixels corresponding to the complete region obtained in step S204 are removed from the pavement depth image after median filtering. This removal operation sets the depth values ​​of these pixels to a specific invalid value (NaN). This generates a "region to be detected" image that has had its groove structure interference filtered out; this image will serve as input for subsequent anomaly detection procedures.

[0069] Example 3 This embodiment is a further refinement and optimization of steps S103 and S104 in Embodiment 1, specifically disclosing how to achieve rapid extraction, quantitative verification, and accurate classification of abnormal candidate regions in road surface depth data after filtering out grooved structures.

[0070] S301: Extract abnormal candidate regions from the area to be detected.

[0071] The input for this step is the "region to be detected" output from step S205 of Example 2. First, the average gradient magnitude of all valid pixels in this region image is calculated. A fourth threshold T2 is set to capture significant edges that may be caused by anomalies. T2 can be set to a value between 1.8 and 2.2 times the average gradient magnitude, for example, 2.0 times. Pixels with gradient magnitudes greater than T2 are extracted to form preliminary "anomaly candidate regions".

[0072] S302: Perform area filtering and pixel-level depth calculation on abnormal candidate regions.

[0073] For the abnormal candidate regions obtained in step S301, area filtering is first performed: the total pixel area is calculated. If the area is less than the preset fifth threshold (e.g., 0.015 square meters, with a preferred range of 0.01m² to 0.02m²), the subsequent detection process for that region is terminated directly to improve overall processing efficiency.

[0074] For regions with an area not less than the fifth threshold, pixel-level depth calculation is performed. A GPU thread is started for each pixel within the region to perform the following quantization calculations: A neighborhood window of M×N is defined centered on the pixel. M and N are positive integers and are usually equal. In this embodiment, a 5×5 (or 3×3) square neighborhood window is preferred. For pixels at the image boundary, mirroring or zero-filling is performed.

[0075] Calculate the arithmetic mean of the depth values ​​of all valid pixels within the neighborhood window. .

[0076] Calculate the depth value of the center pixel. Compared with the neighborhood average The difference The difference It directly reflects the physical height of the point relative to its local background, whether it is a bulge or a depression.

[0077] S303: The abnormally complete region was obtained through verification using CUDA kernel functions.

[0078] After completing pixel-level depth calculations, verification is performed using CUDA kernel functions. Each difference Δ obtained from the calculations is evaluated: a physical height threshold is set, which is determined based on the required detection accuracy. In this embodiment, according to the quantization criteria defined by the method, a pixel is considered considered safe if the absolute value of the difference between its depth and the average depth of its neighboring area is greater than 1 millimeter. If so, the central pixel is determined to be a genuine abnormal pixel.

[0079] The connected region formed by all the points identified as anomalous pixels constitutes the "abnormal complete region" after fine screening and verification. This step directly achieves a physical measurement of the anomalous size (area) and protrusion height (Δ) by quantifying local depth differences.

[0080] S304: Normalization of data in abnormal regions To adapt the extracted abnormally complete region data to subsequent deep learning classification models, cropping and normalization processing are required. For each abnormally complete region: First, the corresponding depth data block is cropped from the pavement depth image after median filtering based on its bounding rectangle (which can be appropriately expanded).

[0081] Next, the mean of the depth values ​​of all valid pixels within the data block is calculated. and variance .

[0082] Then, based on the mean and variance, the depth data block is subjected to grayscale stretching, i.e., normalization. For example, the depth values ​​can be mapped to a standardized range through a linear transformation. This processing aims to eliminate overall brightness and contrast differences under different acquisition conditions, enhance the contrast between abnormal areas and the normal pavement background, thereby obtaining normalized depth data.

[0083] S305: GPU-accelerated parallel computing implementation.

[0084] In this embodiment, the pixel-level depth calculation in step S302, the verification judgment in step S303, and some calculations in step S304 (such as mean, variance calculation, and pixel-level transformation) are all accelerated in parallel using CUDA kernel functions, which is the key to achieving fast processing.

[0085] For calculations in S302 and S303: The calculation and judgment task for each pixel is mapped to a thread on the GPU. The thread block size can be configured to 16×16 or 32×32. Each thread independently loads the depth values ​​of the target pixel and its preset neighborhood (e.g., 5×5) into a register, and calculates the neighborhood mean in parallel. Difference It is then compared with a decision threshold (1 mm). This "single-pixel, one-thread" mode perfectly matches the pixel-level parallelism of the image.

[0086] For the normalized calculation of S304: mean and variance The computation can be performed on the GPU using an efficient parallel reduction algorithm. The subsequent grayscale stretching operation (i.e., the transformation of each pixel value) can also be assigned to a thread for parallel computation per pixel.

[0087] By deploying computationally intensive pixel-level and region-level operations on GPUs, their massively parallel processing capabilities are fully utilized, ensuring that the entire anomaly extraction, verification, and preprocessing process maintains extremely high processing speeds even when processing large-scale road image data.

[0088] S306: Deep learning classification of abnormal regions.

[0089] The normalized depth data output from step S304, after being standardized and of uniform size (e.g., resampled to a fixed resolution), is fed into a pre-trained deep learning classification network. This network is a lightweight convolutional neural network (CNN), which, after forward inference, outputs the most likely anomaly category information for the region (e.g., foreign object type or disease type). Simultaneously, combined with the high-precision positioning information acquired synchronously during the data collection in step S101 of Example 1, the precise location of the anomaly region in the absolute geographic coordinate system can be calculated.

[0090] Finally, the system outputs a structured inspection report containing information such as "anomaly category", "confidence level", "physical location (coordinates)", "estimated size" and "protrusion height", thus completing a closed loop from road surface data collection to accurate anomaly identification and location.

[0091] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0092] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0093] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting large-scale road surface anomalies based on prior information, wherein the large-scale road surface is composed of multiple cement blocks, characterized in that, include: Acquire pavement depth images of the road surface to be detected and perform median filtering to obtain the filtering results; Based on the filtering results, the gradient magnitude and gradient direction of each pixel in the pavement depth image are calculated to obtain the calculation results; Based on prior information about large-scale road surfaces and the calculation results, candidate regions for drainage channels and splicing joints between adjacent cement blocks are generated. Perform region filtering and connectivity processing on the candidate regions to obtain the complete region concerning the drainage ditch and the splicing joint between adjacent cement blocks; The complete region is removed from the pavement depth image after median filtering to obtain the corresponding region to be detected; anomaly candidate regions are obtained by filtering the region to be detected based on gradient magnitude. The area of ​​the candidate abnormal region is filtered and the pixel-level depth is calculated. The results are then verified using a CUDA kernel function to obtain the complete abnormal region. The depth data of the abnormal complete region is normalized, and the processed depth data is input into the classification model to determine the category information of the anomaly.

2. The large-scale road anomaly detection method based on prior information according to claim 1, characterized in that, The process of acquiring a pavement depth image of the road surface to be detected and performing median filtering to obtain the filtering result includes: Acquire pavement depth images of the road surface to be inspected and save them to the host memory; The track depth image is transferred from the host memory to the graphics card memory; In the graphics card memory, the track depth image is subjected to median filtering to eliminate holes in the track depth image, and the filtering result is obtained. The calculation formula for the median filtering process is as follows: In the formula: The coordinates in the depth image are The pixel depth value at that location, This indicates the depth image after median filtering. The pixel depth value at that location, This indicates the preset filtering window. This indicates the midpoint operation.

3. The large-scale road anomaly detection method based on prior information according to claim 2, characterized in that, The calculation of the gradient magnitude and gradient direction of each pixel in the pavement depth image includes: Using a horizontal Sobel filter kernel With vertical Sobel filter kernel Perform filtering calculations; The horizontal Sobel filter kernel and vertical Sobel filter kernel They are respectively: ， For each pixel in the pavement depth image, perform the following calculation: The filter kernel and Perform convolution operations to obtain the gradient values ​​of each pixel along the x-axis of the image coordinate system. and gradient value in the y-axis direction of the image coordinate system ; According to the formula Calculate the gradient magnitude of each pixel. ; According to the formula Calculate the gradient direction of each pixel and will Normalized to the range of 0° to 180°.

4. The large-scale road anomaly detection method based on prior information according to claim 1, characterized in that, The prior information of the large-scale road surface includes: The normal area of ​​the large-scale road surface is flat, which is represented in the road surface depth image as a depth difference of less than 1 mm between adjacent pixels. The large-scale pavement has grooves, which include joints between adjacent cement blocks and drainage channels on the surface of the cement blocks; the grooves are represented in the pavement depth image as having high gradient amplitude and consistent gradient direction at their edges. The high gradient amplitude specifically refers to the depth of the groove being lower than that of the normal road surface area, with a depth difference greater than 1 mm.

5. The large-scale road anomaly detection method based on prior information according to claim 4, characterized in that, Based on prior information about large-scale road surfaces and the calculation results, candidate regions for drainage channels and joints between adjacent cement blocks are generated, including: Based on the high gradient amplitude of the groove edge, a first threshold is set, and all pixels with gradient amplitude greater than the first threshold are extracted from the calculation results to form an initial candidate pixel set. The first threshold is 1.2 to 1.5 times the average gradient magnitude of all pixels in the pavement depth image; Based on the consistent gradient direction of the groove edges, a second threshold is set. For each pixel in the initial candidate pixel set, the angle between its gradient direction and that of its neighboring pixels is determined by the CUDA kernel function. If the directional angle is less than the second threshold, the pixel and its adjacent pixels are determined to belong to the same direction, and the regions corresponding to the two are connected. The second threshold is 0°~10°. Repeat the above operations, traverse the initial candidate pixel set, connect the regions corresponding to pixels that satisfy the gradient direction consistency condition, and generate candidate regions for drainage channels and splicing seams between adjacent cement blocks.

6. The large-scale road anomaly detection method based on prior information according to claim 1, characterized in that, The process of performing region filtering and connectivity processing on the candidate regions yields a complete region concerning the drainage ditch and the joint between adjacent cement blocks, including: The regions in the candidate region whose major axis dimension is greater than the third threshold are extracted as the preliminarily determined drainage channel and splicing seam regions, where the third threshold is 1m~2m; For the initially determined drainage channel and splicing seam area, according to its long axis direction, the areas that are broken due to interference from foreign objects, pits or falling pieces in this direction are connected to generate the complete area.

7. The large-scale road anomaly detection method based on prior information according to claim 1, characterized in that, The complete region is removed from the pavement depth image after median filtering to obtain the corresponding region to be detected. Anomaly candidate regions are obtained by filtering based on gradient magnitude within the region to be detected, specifically including: The area to be detected is obtained by removing the complete area of ​​the drainage ditch and the splicing joint between adjacent cement blocks in the pavement depth image; In the region to be detected, pixel regions with gradient magnitudes greater than the fourth threshold are extracted to generate the anomaly candidate regions. The fourth threshold is 1.8 to 2.2 times the average gradient magnitude of all pixels in the pavement depth image.

8. The large-scale road anomaly detection method based on prior information according to claim 1, characterized in that, The process of performing area filtering and pixel-level depth calculation on the candidate abnormal regions, and verifying them using CUDA kernel functions, to obtain the complete abnormal regions includes: If the area of ​​the abnormal candidate region is less than the fifth threshold, the current detection process is terminated; If the area of ​​the abnormal candidate region is not less than the fifth threshold, then for each pixel in the abnormal candidate region, the difference between the pixel depth value and the average depth of all pixels in the surrounding neighborhood of the pixel is calculated using the CUDA kernel function. The CUDA kernel function is used to judge each pixel in the abnormal candidate region: if the absolute value of the difference is greater than 1 mm, the pixel is determined to be an abnormal pixel. The pixel regions corresponding to all pixels identified as abnormal are merged to obtain the complete abnormal region. The fifth threshold is 0.01m²~0.02m².

9. The large-scale road anomaly detection method based on prior information according to claim 8, characterized in that, The step of normalizing the depth data of the abnormal complete region and inputting the processed depth data into a classification model to determine the anomaly category information includes: The depth data of the abnormally complete region is cropped to obtain cropped depth data blocks; Calculate the mean and variance of the depth of the cropped depth data block; Based on the mean and variance of the depth, the cropped depth data block is subjected to grayscale stretching to enhance the contrast between its internal abnormal features and the background, thereby obtaining normalized depth data. The normalized depth data is used to determine abnormal area diseases using any of the following methods: The normalized depth data is input into a deep learning classification network to determine the type of disease in the abnormal area. The normalized depth data is input into a deep learning multi-class segmentation network; a disease category segmentation map for each pixel is output; it is determined whether there is a connected region in the segmentation map that intersects with the abnormally complete region; if so, the disease category corresponding to the intersecting region is determined as the final category of the abnormally complete region.

10. The large-scale road anomaly detection method based on prior information according to claim 9, characterized in that, The step of normalizing the depth data of the abnormal complete region and inputting the processed depth data into the classification model to determine the anomaly category information also includes the following methods: The depth data of the abnormal complete region is cropped to obtain the cropped abnormal data block. Normal region data blocks are obtained by cropping the adjacent detection region of the abnormal complete region. The depth mean and variance of each data block are calculated. The grayscale of each data block is stretched according to the mean and variance to obtain the normalized abnormal data block and normal region data block. Based on the geometric orientation of the complete region, the abnormal data blocks and normal region data blocks are rotated and corrected to make their orientations consistent; all corrected data blocks are merged, and their main components are extracted by PCA to form a low-dimensional feature vector. The low-dimensional feature vector is input into a pre-trained feature classifier, which outputs the category information of the anomaly; wherein the feature classifier is a support vector machine (SVM) or a multilayer perceptron (MLP).

Images