Road defect self-adaptive detection method and system

Abstract

The invention relates to the field of computer vision, in particular to a road defect self-adaptive detection method and system. The method comprises the following steps: acquiring an original road surface image, and generating a plurality of feature views by using the original road surface image; based on the feature view, performing reasoning by using a plurality of pre-constructed defect detection models to obtain an initial detection frame and a corresponding initial confidence coefficient; constructing a detection frame topological association graph by using the initial detection frame and the initial confidence coefficient, and obtaining a connected component of a potential defect target based on the detection frame topological association graph; calculating a fusion defect position and a fusion confidence coefficient based on the connected component; and carrying out adaptive updating on the defect detection model by utilizing the fusion defect position and the fusion confidence coefficient. The problems of insufficient robustness and long-term effectiveness of model identification caused by insufficient environmental adaptability, easy attenuation of model performance due to data drift and single decision-making mechanism in road defect detection in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of computer vision, specifically to an adaptive method and system for detecting road defects. Background Technology

[0002] With the rapid development of the national highway network, both cement and asphalt pavements will develop defects such as cracks, potholes, and loosening after being opened to traffic (e.g., Figure 1 (As shown). Among them, cracks are the most common and earliest-stage defects. If not repaired in time, rainwater intrusion will reduce the bearing capacity of the roadbed and accelerate road damage.

[0003] With the development of science and technology, experts both at home and abroad have proposed many identification algorithms for road crack detection. The detection of road cracks and defects has mainly gone through the following stages of development: Traditional manual inspection: Manual inspection is not only dangerous, costly in terms of manpower and resources, and extremely inefficient, but also highly subjective in its results and easily affected by objective conditions such as weather. Most importantly, it is difficult to meet the demand for rapid inspection of massive amounts of road surface data.

[0004] Traditional image processing: With the development of computer vision technology, researchers have attempted to achieve automated detection using digital image processing algorithms. These methods typically identify defects through manually designed feature extractors, such as thresholding and Sobel edge detection. These methods rely on manually designed features, and their performance is highly dependent on the researcher's prior knowledge of defect features in a specific scene, requiring tedious "feature engineering." The quality of feature extraction directly determines the success or failure of the algorithm. Since manually designed features are usually specific to lighting, road surface materials, and background conditions, their performance deteriorates sharply once the environment changes (e.g., changes in light intensity, wet and reflective road surfaces, the presence of shadows, differences between new and old asphalt pavements). For example, the Sobel operator is sensitive to noise and may misdetect road textures as cracks; thresholding methods are prone to failure under uneven lighting. Moreover, traditional image processing usually requires a series of complex preprocessing, feature extraction, and post-processing operations. The entire process is sequential, like dominoes, making the processing complex and fragile, with poor robustness and weak generalization ability.

[0005] Deep learning-based detection: In recent years, deep learning technologies, represented by convolutional neural networks (CNNs) and single-stage detectors (such as YOLO and SSD), have changed the landscape of road surface defect detection through their powerful end-to-end feature learning capabilities, gradually becoming the mainstream technology. Deep learning models have significant technical advantages; they can automatically learn hierarchical feature representations of defects from massive amounts of data, avoiding complex manual feature design and achieving a qualitative leap in accuracy and efficiency, basically meeting the needs of real-time detection.

[0006] However, despite significant progress made in deep learning-based methods, the following serious challenges remain in practical road maintenance engineering applications: First, there is insufficient environmental adaptability: existing deep learning models typically input raw images directly. This single data source still faces severe challenges when dealing with the complex and ever-changing road environments in the real world. For example, drastic changes in road lighting (complex textures and backgrounds, as well as road noise, strong shadows, rain and snow films, oil stains, and changes in lighting can all seriously interfere with feature extraction, leading to a large number of missed or false detections in complex conditions)

[0007] Secondly, model performance suffers from static limitations and data drift. Most existing detection models are statically trained offline; once trained and deployed, their knowledge base (model parameters) is "frozen." However, road environments are dynamic and diverse, and datasets cannot possibly encompass all road conditions and defects. Furthermore, material aging and the emergence of new defects cause continuous changes in the actual data distribution (i.e., data drift). Static models cannot adapt to these new distributional changes, leading to a gradual decline in detection accuracy over time and extremely high maintenance costs. Ultimately, this stems from a lack of adaptive evolutionary capabilities.

[0008] Meanwhile, the decision-making mechanism is simplistic. Most existing solutions rely on the output of a single model, failing to effectively utilize complementary information from different feature perspectives. When a single model makes a mistake in a specific scenario, the system lacks an effective internal cross-validation and collaborative decision-making mechanism to correct the error. In other words, existing methods lack effective comprehensive utilization of detection results from different feature perspectives, making it difficult to achieve the optimal balance between "complete detection" (recall) and "accurate detection" (precision).

[0009] Therefore, there is an urgent need to develop a road defect detection system that can integrate multi-view features and has online adaptive learning capabilities to solve the above problems. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides an adaptive road defect detection method and system, which solves the problems of insufficient environmental adaptability, easy degradation of model performance due to data drift, and insufficient robustness and long-term effectiveness of model recognition caused by a single decision-making mechanism in existing technologies.

[0011] To achieve the above objectives, the present invention provides an adaptive road defect detection method, the method comprising: acquiring an original road surface image and generating multiple feature views using the original road surface image; based on the feature views, performing inference using multiple pre-constructed defect detection models to obtain initial detection boxes and corresponding initial confidence scores; constructing a detection box topology graph using the initial detection boxes and the initial confidence scores, and obtaining connected components of potential defect targets based on the detection box topology graph; calculating fused defect locations and fused confidence scores based on the connected components; and adaptively updating the defect detection models using the fused defect locations and the fused confidence scores.

[0012] This invention generates multiple feature views by acquiring original road surface images, infers initial detection boxes and confidence scores based on a multi-defect detection model, extracts potential defect connected components through the topological association graph of the detection boxes, calculates and fuses defect locations and confidence scores, and incrementally trains the model. This effectively integrates multi-view features to reduce the impact of environmental interference, improves defect location accuracy and detection reliability, and enables dynamic model evolution to avoid performance degradation due to data drift, significantly improving the robustness and long-term effectiveness of road defect detection.

[0013] Optionally, generating multiple feature views using the original road surface image includes: processing the original road surface image using a contrast-limited adaptive histogram equalization algorithm to generate an illumination-invariant view; extracting edge gradient information from the original road surface image using the Sobel operator to generate a high-frequency geometric texture view; and processing the original road surface image using a bilateral filtering algorithm to generate a high signal-to-noise ratio structure view.

[0014] This invention generates an illumination-invariant view by limiting the contrast adaptive histogram equalization algorithm, which reduces the interference of shadows and strong light on detection. It generates a high-frequency geometric texture view by using the Sobel operator to enhance the edge features of defects such as cracks to remove color interference. It generates a high signal-to-noise ratio structural view by using a bilateral filtering algorithm, which preserves the sharp edges of defects while denoising. The multiple views optimize the expression of road surface image features from different dimensions, which improves the adaptability of the subsequent defect detection model to complex environments and the accuracy of feature extraction.

[0015] Optionally, the step of constructing a detection box topology graph using the initial detection boxes and the initial confidence level, and obtaining the connected components of potential defect targets based on the detection box topology graph includes: calculating the intersection-union ratio (CIU) of any two initial detection boxes, and comparing the CIU with a pre-set CIU threshold; setting the initial detection boxes and the initial confidence level as graph nodes, and setting the connection relationships of the graph nodes according to the comparison result to form a detection box topology graph; and traversing the detection box topology graph using a depth-first search algorithm to obtain the connected components of multiple potential defect targets.

[0016] This invention calculates the intersection-union ratio (CUI) of any two initial detection boxes and compares it with a preset threshold. Using the detection boxes and confidence scores as graph nodes, a topological association graph is formed by establishing connections based on the CUI comparison results. Then, a depth-first search is used to traverse the graph to obtain the connected components of potential defects. This accurately associates the same defect detection boxes output by multiple models, effectively avoiding duplicate counting and misassociations of multi-model inference results, and improving the aggregation accuracy of potential defect targets.

[0017] Optionally, the connected component includes a subset of detection boxes and a subset of confidence scores. Calculating the fusion defect location and fusion confidence score based on the connected component includes: calculating the number of model sources for the subset of detection boxes in the connected component; correcting the subset of confidence scores based on the number of model sources to obtain the fusion confidence score; and using the fusion confidence score to perform weighted fusion of the subset of detection boxes to obtain the fusion defect location.

[0018] This invention first calculates the number of model sources for the subset of detection boxes in the connected components, then adjusts the confidence subset based on this number to obtain the fusion confidence, and finally uses the fusion confidence to weight and fuse the subset of detection boxes to obtain the fusion defect location. This allows defects detected by multi-model consensus to receive confidence rewards, defects detected by single models to be reasonably suppressed, and high-confidence detection boxes to have a higher weight in location fusion. It effectively integrates complementary information from multiple models, improving the accuracy of defect location and the reliability of confidence determination.

[0019] Optionally, the fusion confidence score satisfies the following formula: in, To adjust the confidence level, To achieve the highest confidence level, As a consistent reward factor, For the number of model sources, This is the view penalty coefficient.

[0020] This invention designs a segmented fusion confidence formula, which uses a consistency reward factor to increase confidence for defects detected by multiple models and a view penalty coefficient to reasonably suppress confidence for defects detected by a single model. This allows the confidence result to be directly related to the degree of model consensus, which not only strengthens the reliable defect weight of multi-view verification, but also reduces the interference of false detections by a single model, thereby improving the rationality of defect confidence determination and the credibility of detection results.

[0021] Optionally, the adaptive update of the defect detection model using the fusion defect location and the fusion confidence includes: constructing pseudo-label data based on the fusion confidence using the fusion defect location; obtaining historical core anchor data, and constructing incremental training data using the pseudo-label data and the historical core anchor data; constructing an empirical replay loss function, and adaptively updating the defect detection model using the incremental training data based on the empirical replay loss function.

[0022] This invention constructs pseudo-label data based on fusion confidence, combines it with historical core anchor data to form incremental training data, and then uses the experience replay loss function to adaptively update the defect detection model. This allows the model to absorb defect features in new scenarios while avoiding knowledge forgetting through historical data, thus achieving dynamic evolution of the model. It also prevents the model's performance from degrading due to data drift and improves the long-term adaptability and detection stability of the defect detection model for different road scenarios.

[0023] Optionally, constructing pseudo-label data based on the fusion confidence level using the fusion defect location includes: obtaining a high confidence threshold and a low confidence threshold; comparing the fusion confidence level with the high confidence threshold and the low confidence threshold respectively to determine the sample type of the fusion defect location; and constructing pseudo-label data based on the sample type.

[0024] This invention sets high and low confidence thresholds, compares the fusion confidence with the thresholds to classify the sample types at the fusion defect locations, and then constructs pseudo-label data accordingly. This achieves hierarchical processing of samples with different reliability, which not only retains the effective supervision information of high-confidence samples, but also avoids the introduction of noise by low-confidence samples, thereby improving the quality of pseudo-label data and the effectiveness of incremental training.

[0025] Optionally, the empirical replay loss function satisfies the following formula: in, For the experience replay loss function, For the loss function of the new sample, For model parameters, For the newly generated pseudo-label data, For historical samples, the loss function is... Historical core anchor data, It serves as a forgetting balance factor.

[0026] This invention designs an experience replay loss function that uses a forgetting balance factor to weight and fuse the loss of new pseudo-label data with the loss of historical core anchor data. This allows the model to learn defect features in new scenarios while retaining the detection knowledge accumulated from historical data. This avoids the problem of knowledge forgetting in incremental training, achieves a balance between the training weights of new and old data, and improves the long-term evolution ability and detection performance stability of the defect detection model.

[0027] Optionally, the step of adaptively updating the defect detection model using the incremental training data based on the empirical replay loss function includes: freezing the backbone network parameters of the defect detection model, and adaptively updating the detection head parameters of the defect detection model using the incremental training data based on the empirical replay loss function.

[0028] This invention freezes the backbone network parameters of the defect detection model during adaptive updates, and only performs incremental training and updates on the detection head parameters. This preserves the general feature extraction capabilities already learned by the backbone network, while allowing the detection head to quickly adapt to the defect features of new scenarios. It avoids the historical knowledge disturbance that may be caused by a full model update, while reducing training computation costs and parameter adjustment risks, and improving the efficiency and stability of the model's adaptive updates.

[0029] In another aspect, the present invention provides a road defect adaptive detection system, comprising: a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute a road defect adaptive detection method according to any of the preceding aspects of the present invention.

[0030] The road defect adaptive detection system of the present invention has a compact structure, stable performance, high integration and simple configuration. It can stably execute the road defect adaptive detection method provided in the preceding aspect of the present invention, further improving the overall applicability and practical application capability of the present invention. Attached Figure Description

[0031] Figure 1 This is a flowchart of an adaptive road defect detection method according to an embodiment of the present invention; Figure 2 This is an orthogonal feature attribute view of an embodiment of the present invention; Figure 3 This is a schematic diagram of the dynamic evidence fusion logic in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an adaptive road defect detection system according to an embodiment of the present invention. Detailed Implementation

[0032] Specific embodiments of the present invention will now be described in detail. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. In other instances, well-known circuits, software, or methods have not been specifically described to avoid obscuring the invention.

[0033] Throughout this specification, references to "an embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with that embodiment or example is included in at least one embodiment of the invention. Therefore, the phrases "in an embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, or characteristics can be combined in one or more embodiments or examples in any suitable combination and / or sub-combination. Moreover, those skilled in the art will understand that the illustrations provided herein are for illustrative purposes and are not necessarily drawn to scale.

[0034] Please see Figure 1 To address the problems of the prior art, in one alternative embodiment, as shown in the figure, an adaptive road defect detection method includes the following steps: Step S1: Obtain the original road surface image and generate various feature views using the original road surface image.

[0035] In this embodiment, images are collected on target detection sections such as highways and municipal roads by using high-definition cameras, drones, or fixed road monitoring equipment mounted on inspection vehicles. The equipment is kept stable and moves at a constant speed during the collection process to ensure that the original road surface images are clear, without obvious distortion, and completely cover the surface area of ​​the road surface. This allows for a true reflection of potential defects such as cracks and potholes on the road surface, as well as information about the surrounding environment.

[0036] The generation of multiple feature views using the original road surface image specifically includes the following sub-steps: Step S101: The original road surface image is processed by a contrast-limited adaptive histogram equalization algorithm to generate an illumination-invariant view.

[0037] In this embodiment, the illumination-invariant view refers to the use of Limiting Contrast Adaptive Histogram Equalization (CLAHE) to address common issues in road scenes such as shadow occlusion and strong light reflection. Specifically, the principle involves dividing the image into... The non-overlapping sub-blocks are identified. For each sub-block, its gray-level histogram is calculated. To prevent noise amplification, a shearing threshold is set. Truncate the histogram. Let... For input pixel grayscale, the mapping function within the sub-block The discrete form of the cumulative distribution function: in, To output grayscale levels, This is the grayscale mapping function. The original gray level of the input pixel. The number of gray levels (usually 256). To truncate and renormalize the probability density, the final view LCLAHE is generated.

[0038] Step S102: Extract the edge gradient information of the original road surface image using the Sobel operator to generate a high-frequency geometric texture view.

[0039] In this embodiment, the high-frequency geometric texture view is used to eliminate the interference of road surface color (such as paint, water stains) on crack detection. The Sobel operator is used to extract gradient magnitudes and strengthen the geometric skeleton. Algorithm principle: The horizontal gradient Gx and vertical gradient Gy are calculated separately. The convolution kernel is as follows: Final texture view Determined by the gradient magnitude: in, For output pixels gradient magnitude at that point , These represent the gradient images in the horizontal and vertical directions, respectively. The input is the original road surface image. This is a convolution operation.

[0040] Step S103: Process the original road surface image using a bilateral filtering algorithm to generate a high signal-to-noise ratio structured view.

[0041] In this embodiment, a high signal-to-noise ratio structural view is generated using bilateral filtering to preserve the sharp edges of cracks while denoising. This filter considers both spatial proximity and pixel value similarity. For pixel p, its filtered value... Defined as: in, For output pixels The filtered value, The spatial domain Gaussian kernel has a standard deviation of , For pixel-domain Gaussian kernels, the standard deviation is... , The normalization coefficient is... For the midpoint of the input image The pixel value.

[0042] The specific effects of the above three types of feature views are as follows: Figure 2 As shown: Figure 2 (a) is the unprocessed raw road surface image, which includes uneven lighting and texture interference; Figure 2 (b) The effects of shadows / strong light are reduced in the illumination-invariant view using the CLAHE algorithm; Figure 2 (c) Enhances crack edges in a high-frequency geometric texture view using the Sobel operator; Figure 2 (d) is a high signal-to-noise ratio structure view that removes noise and preserves defective structures through bilateral filtering.

[0043] Step S2: Based on the feature view, use multiple pre-built defect detection models to infer the initial detection box and the corresponding initial confidence level.

[0044] In this embodiment, the feature views specifically include three types of orthogonal feature views: illumination-invariant views, high-frequency geometric texture views, and high signal-to-noise ratio (SNR) structural views. The pre-built multiple defect detection models correspond to three independent deep learning target detection models (denoted as Net-A, Net-B, and Net-C, respectively), and the three types of feature views have a one-to-one correspondence with the three detection models: the illumination-invariant views are input to Net-A, the high-frequency geometric texture views are input to Net-B, and the high SNR structural views are input to Net-C. Each detection model performs defect detection inference on its respective input feature views and outputs the initial defect detection box (containing the coordinates of the rectangle where the defect is located) under the corresponding view and the initial confidence score corresponding to the detection box (characterizing the model's credibility of the detection result). Finally, all the initial detection boxes and corresponding initial confidence scores output by the three models are summarized to form the initial detection box set and the initial confidence score set.

[0045] Net-A was trained on a dataset of illumination-invariant view annotations, which includes multiple sets of illumination-invariant view images under different road surface scenarios, along with bounding boxes for defect locations and defect category labels that match the images one-to-one. Net-B was trained on a dataset of high-frequency geometric texture view annotations, which contains high-frequency geometric texture view images of various road surfaces and corresponding defect annotation information. Net-C was trained on a dataset of high signal-to-noise ratio structured view annotations, which includes high signal-to-noise ratio structured view images under different working conditions and corresponding defect annotation data. All three models are built on the YOLO series of object detection algorithm frameworks.

[0046] Step S3: Construct a topological association graph of the detection boxes using the initial detection boxes and the initial confidence level, and obtain the connected components of the potential defect targets based on the topological association graph of the detection boxes.

[0047] The process of constructing a topological association graph of the detection boxes using the initial detection boxes and the initial confidence level, and obtaining the connected components of the potential defect targets based on the topological association graph includes: Step S301: Calculate the cross-union ratio (CUP) of any two initial detection boxes and compare the CUP with a preset CUP threshold.

[0048] In this embodiment, any two initial detection boxes in the initial detection box set are traversed, and the intersection-to-union ratio (IoU) between them is calculated. First, the intersection area of ​​the rectangular regions corresponding to the two detection boxes is determined (i.e., the area of ​​the overlapping part of the two rectangles). Then, the union area of ​​the two is calculated (i.e., the total area covered by the two rectangles, which is the sum of the areas of the two rectangles minus the intersection area). The IoU is the ratio of the intersection area to the union area. At the same time, an IoU threshold is preset (which can be flexibly adjusted according to the typical size of different road defects). Finally, the IoU calculation result of each pair of initial detection boxes is compared with the preset IoU threshold to preliminarily determine whether the two detection boxes correspond to the same potential defect target.

[0049] Step S302: Set the initial detection box and the initial confidence level as graph nodes, and set the connection relationship of the graph nodes according to the comparison result to form a detection box topology graph.

[0050] In this embodiment, all initial detection boxes and their corresponding initial confidence scores (covering all defect detection boxes output by the three models Net-A, Net-B, and Net-C) are first treated as independent graph nodes in an undirected graph. Then, based on the comparison result of the intersection-union ratio (IU) and a preset threshold, the connection relationship between the graph nodes is determined. If the IU of a pair of initial detection boxes is greater than the preset IU threshold, it is determined that the two detection boxes correspond to the same potential defect target, and an undirected connection edge is established between the graph nodes corresponding to the two detection boxes. If the IU is less than or equal to the preset threshold, no connection edge is established. By sorting out all graph nodes and their corresponding connection relationships one by one, a detection box topology graph is finally formed, with the graph nodes containing the initial detection boxes and their confidence scores as nodes and the spatial association relationship of the detection boxes (corresponding to the same potential defect) as the connecting edges.

[0051] Step S303: Use a depth-first search algorithm to traverse the topological association graph of the detection box to obtain the connected components of multiple potential defect targets.

[0052] In this embodiment, the connected components include a subset of detection boxes and a subset of confidence scores. An unvisited graph node is arbitrarily selected from the topological graph of the detection boxes as a starting point. Using a depth-first search (DFS) algorithm, all graph nodes directly or indirectly connected to the starting node are recursively visited along the established undirected edges. The visit status of these nodes is marked to avoid repeated traversals. When a node has no unvisited adjacent nodes, the current branch traversal stops. At this point, all visited graph nodes constitute an independent node set. Each graph node in this set contains an initial detection box and its corresponding initial confidence score. The initial detection boxes from all graph nodes are extracted to form the subset of detection boxes for the connected components. The initial confidence scores from all graph nodes are extracted to form the subset of confidence scores for the connected components. The process of selecting unvisited nodes, performing DFS traversal, and forming subsets is repeated until all nodes in the topological graph of the detection boxes have been visited, ultimately resulting in multiple independent connected components, each corresponding to a potential defect target.

[0053] Step S4: Calculate the fusion defect location and fusion confidence based on the connected components.

[0054] The calculation of fusion defect location and fusion confidence based on the connected components includes: Step S401: Calculate the number of model sources for the subset of detection boxes in the connected components.

[0055] In this embodiment, the detection box subsets of the connected components are traversed first, and the model source identifier of each initial detection box is extracted one by one. Then, the extracted model source identifiers are deduplicated, that is, the identifier of the same model is kept only once. Finally, the number of model identifiers remaining after deduplication is counted. This number is the number of model sources in the detection box subset of the connected component. For example, if the detection box subset contains detection boxes of Net-A and Net-B, there are 2 model identifiers after deduplication, and the corresponding number of model sources is 2; if the detection box subset only contains detection boxes of Net-C, there is 1 model identifier after deduplication, and the corresponding number of model sources is 1.

[0056] Step S402: Based on the number of model sources, the confidence subset is corrected to obtain the fused confidence.

[0057] For each connected component Instead of simply taking the maximum value, the final confidence level is dynamically calibrated based on the "view source diversity" contained in the cluster. Let the original highest value within the cluster be... (i.e., maximum confidence), this process involves two mutually exclusive logical decisions: Scenario A: Strong consistency goal (for consensus goals). If a connected component contains nodes from at least two different view models ( This indicates that the defect exhibits cross-modal saliency and belongs to the "confidence objective". At this point, the system introduces a consistency reward factor. The confidence level is increased using the following formula: This is the indicator function. This mechanism effectively transforms the "high recall" of multi-model perception into "high confidence" of the final result.

[0058] Scenario B: Isolated Evidence Suppression (for single-view illusion). Conversely, if a connected component consists only of nodes from a single-view model, the system classifies it as "isolated evidence." Considering that single views are easily affected by specific environmental factors (e.g., Sobel views often misjudge road repair strips as cracks), the system introduces a view penalty coefficient. Suppress: in, The value depends on the prior reliability of the view: the lighting view is relatively reliable ( ), while texture views have higher noise ( ), the structure view is centered ( ).

[0059] The fusion confidence level satisfies the following formula: in, To adjust the confidence level, To achieve the highest confidence level, As a consistent reward factor, For the number of model sources, This is the view penalty coefficient.

[0060] Step S403: Use the fusion confidence to perform weighted fusion on the subset of detection boxes to obtain the fusion defect location.

[0061] In this embodiment, the currently connected component to be processed is first locked. Extract each initial detection box within its subset of detection boxes. The two core pieces of information, and their corresponding initial confidence levels. and the coordinates of the rectangle of the detection box. (by , , and This indicates that the coordinates correspond to the top-left x-coordinate, top-left y-coordinate, bottom-right x-coordinate, and bottom-right y-coordinate of the detection box, respectively. Then, a weighted calculation is performed on each of the four dimensions of the coordinates: For the weights, calculate All of The weighted sum of the coordinates, then divided by the weighted sum. All The sum of these values ​​yields the location of the defect after fusion. Similarly, coordinates are calculated sequentially. , , The weighted average of the four dimensions is used, and finally, these four weighted coordinate values ​​are combined to obtain the final bounding box of the fused defect location. This process, by giving higher confidence initial detection boxes (more reliable model detection results) greater weight in the fusion process, not only smooths out the positional jitter of different model detection boxes, but also improves the localization accuracy of the fused defect location, ensuring that the output fused defect location can accurately match the actual area of ​​the real road surface defect.

[0062] The location of the fusion defect satisfies the following formula: in, To merge the rectangular boxes at the defect locations, For the first Connected components For the first An initial detection box, For the first The coordinates of the rectangle of the initial detection box. For the initial detection box The corresponding initial confidence level.

[0063] The above formula fuses all initial detection boxes within connected components using an initial confidence-weighted average, for each dimension of the detection box coordinates. The sum of the products of the initial confidence level and the corresponding coordinate value in each dimension is calculated and then divided by the sum of all initial confidence levels. Finally, the coordinate dimensions of the defect location after fusion are obtained, so that the detection boxes with higher confidence (more reliable model results) have higher weight in the fusion, thereby improving the positioning accuracy of the final defect location.

[0064] The calculation logic for fusion confidence is as follows: Figure 3 As shown: Multi-model detection boxes are clustered into connected components using an IoU topology graph. For strongly consistent clique structures containing two or more model sources, the following method is used: Rewards increase confidence; for 'isolated evidence' from only one model source, use The penalty suppresses false positives, ultimately outputting accurate and reliable fusion results.

[0065] Step S5: Adaptively update the defect detection model using the fused defect location and the fused confidence level.

[0066] The adaptive update of the defect detection model using the fusion defect location and the fusion confidence level specifically includes the following sub-steps: Step S501: Based on the fusion confidence, construct pseudo-label data using the fusion defect locations.

[0067] The construction of pseudo-label data based on the fusion confidence level and the fusion defect location specifically includes the following sub-steps: Step S50101: Obtain the high confidence threshold and the low confidence threshold.

[0068] In this embodiment, the high-confidence threshold and low-confidence threshold can be set empirically, or, to improve accuracy, can be quantified by statistical analysis of a large amount of historical annotation data and verification through engineering practice. First, a historical annotation dataset covering typical pavement types such as asphalt pavement, cement pavement, and aging pavement, as well as defects such as transverse and longitudinal cracks, potholes, and network cracks, is collected. This dataset is input into pre-trained Net-A, Net-B, and Net-C models to obtain the confidence outputs of all real defects and false positives. Second, the output confidence scores are statistically analyzed in layers, and the confidence scores corresponding to all manually annotated real defects are selected. The 25th percentile of their distribution is then used as the high-confidence threshold. Simultaneously, the confidence levels of all manually labeled targets judged as false positives were selected, and the 75th percentile of their distribution was calculated as the low confidence threshold. Finally, through multiple field tests under different road conditions (rainy days, tree shadows, oil stains on the road surface), if the false positive rate was found to be too high in a certain scenario, the high confidence threshold was appropriately increased; if the false negative rate was too high, the low confidence threshold was appropriately decreased. Finally, a stable threshold suitable for most scenarios was determined to ensure that subsequent sample classification can accurately screen highly reliable and confident samples while effectively filtering invalid false positive samples.

[0069] Step S50102: Compare the fusion confidence with the high confidence threshold and the low confidence threshold respectively to determine the sample type of the fusion defect location.

[0070] In this embodiment, the sample types include confident samples and difficult samples. After completing the multi-view feature fusion and outputting the final detection results, a self-supervised closed-loop system is constructed to address the performance degradation of the static model when facing new road conditions. This system utilizes feedback data generated during the inference phase and achieves lifelong learning of the model through a continuous process of "sample hierarchical screening—experience replay construction—adaptive parameter fine-tuning".

[0071] Sample grading strategy and pseudo-label generation: First, the system relies on the obtained fusion confidence level. The detection results for the current frame Uncertainty is quantified and graded. A high-confidence threshold is preset. and low confidence threshold The samples are dynamically divided into two categories: Confirmation Sample: When the test results meet the requirements If the result is deemed highly reliable, it is automatically assigned a pseudo-label to form pseudo-label data.

[0072] Difficult samples: When the detection result is at the decision boundary At this time, it usually means that there are blurry cracks or special lighting interference. Such samples have extremely high learning value. They are stored in the confirmation queue and wait for the time series tracking module or the cloud to perform secondary verification.

[0073] when If the sample is invalid, it cannot be used as a training sample and is discarded directly.

[0074] Step S50103: Construct pseudo-label data based on the sample type.

[0075] In this embodiment, difficult samples are first stored in a confirmation queue, which is a buffer based on time or capacity priority, used to temporarily store difficult samples that need to be verified a second time. The second verification is completed by the time-series tracking module (retrieving continuous frame images of the same detection segment to verify the spatial continuity of the defect) or by manual review in the cloud. After confirmation, pseudo-labels corresponding to the fused defect locations are generated and stored in the cache to form complete pseudo-label data.

[0076] Step S502: Obtain historical core anchor data, and construct incremental training data using the pseudo-label data and the historical core anchor data.

[0077] In this embodiment, the historical core anchor data is a pre-stored key sample data that supports the model in retaining core defect detection knowledge. Its sources include typical defect datasets selected during the model's offline training phase (covering various pavement types such as asphalt, cement, and aging pavements, as well as different defect types such as transverse and longitudinal cracks, potholes, and network cracks), and high-quality defect samples (all with precise defect location bounding boxes) manually verified during the initial deployment process. This effectively prevents the model from forgetting historically learned defect features during incremental training. When acquiring this data, it is directly retrieved from the system's built-in historical sample database, ensuring the data format is complete (including defect images and corresponding annotation information) and undamaged. When constructing incremental training data, the total amount of pseudo-label data is first counted, and then a corresponding number of samples are extracted from the historical core anchor data at a ratio of 7:3 to 8:2. After mixing the two types of data, they are uniformly adjusted to the same data format, and the data arrangement order is randomly shuffled to avoid training bias. Finally, incremental training data is formed that combines new scene defect features (from pseudo-label data) and historical core knowledge (from historical core anchor data).

[0078] Step S503: Construct an empirical replay loss function, and adaptively update the defect detection model using the incremental training data based on the empirical replay loss function.

[0079] The empirical replay loss function satisfies the following formula: in, For the experience replay loss function, For the loss function of the new sample, For model parameters, For the newly generated pseudo-label data, For historical samples, the loss function is... Historical core anchor data, It serves as a forgetting balance factor.

[0080] and The following general loss calculation structure is used in all cases: Bounding Box Regression Loss measures the difference in positional information between the predicted bounding box and the ground truth bounding box (or pseudo-label). CIoU Loss (Complete IoU Loss) is preferred, as it considers not only the overlap area but also the consistency of the center point distance and aspect ratio. The calculation formula is as follows: Where N represents the total number of samples in the current training batch, and i represents the i-th sample in the batch. This shows the intersection-union ratio (IoU) between the predicted bounding box and the target bounding box of the i-th sample. For Euclidean distance, For the first The diagonal distance between the minimum closure regions of two bounding boxes covered by each sample. For the first Penalty term for the aspect ratio of each sample, Indicates the first The center point coordinates of the predicted bounding box for each sample model Indicates the first The coordinates of the center point of each sample truth box (i.e., pseudo-label box or historical annotation box).

[0081] For the first The sample balance coefficient, used to adjust the weight of the aspect ratio penalty term, is defined as follows: For the first The aspect ratio consistency parameter for each sample is used to measure the similarity between the aspect ratios of the predicted bounding box and the ground truth bounding box. It is defined as follows: ,in , For the first The width and height of the predicted bounding box for each sample. and For the first The width and height of the truth boxes for each sample.

[0082] It should be noted that in online learning, the model is in the training state (backpropagation updates parameters). At this time, "new sample" refers to the "confident pseudo-label" generated through multi-model fusion. This is done when calculating the loss function. At that time, we consider this "confidence pseudo-label" as a ground truth. Here... The intersection-over-union ratio (IoU) refers to the ratio between the predicted bounding boxes output by the single model currently being fine-tuned and the pseudo-labeled bounding boxes used as ground truth. It measures how well the model fits the pseudo-labels in the current iteration, rather than the ratio between multiple models. .

[0083] Classification Loss: This measures the accuracy of the target class prediction. Binary Cross Entropy (BCE) or Focal Loss is used to address class imbalance and focus on hard-to-classify samples. The calculation formula is as follows: in, For the first Each sample category label (for pseudo-label data) (This tag was generated by online inference) Predict probabilities for the model.

[0084] The adaptive update of the defect detection model based on the incremental training data using the empirical replay loss function includes: The backbone network parameters of the defect detection model are frozen, and the detection head parameters of the defect detection model are adaptively updated using the incremental training data based on the empirical replay loss function.

[0085] In this embodiment, adaptive parameter update refers to the "small steps, quick updates" principle followed after the data filtering and loss calculation mechanisms are defined. Updates are performed when the number of valid samples accumulated in the short-term memory cache reaches a threshold. Upon activation, silent background training is immediately triggered. The short-term memory cache is a first-in, first-out (FIFO) queue structure used to temporarily store newly generated high-quality pseudo-label data within the current inspection cycle. Its capacity limit can be dynamically adjusted according to the deployment environment, typically ranging from 100 to 500 samples. When the number of samples in the cache reaches a preset trigger threshold, the incremental training process is automatically initiated to adaptively update the model parameters. To ensure the real-time performance and stability of online updates, a freeze-fine-tuning strategy is adopted, freezing the first K layers (i.e., general feature extraction layers) of the YOLO backbone network and only unfreezing and updating the task-sensitive Neck and Head layers. The optimizer uses stochastic gradient descent (SGD), and the learning rate is set to 10% of the initial training learning rate (i.e.,...). This design involves only a small number of epochs (e.g., 5-10 rounds) of iteration. This ensures that the model can quickly converge to the new feature distribution while avoiding model oscillations caused by over-updates.

[0086] The above incremental training process uses a hybrid training strategy of automatic pseudo-label mining and historical core data experience playback. This allows the model to absorb the defect features under new road conditions, while avoiding forgetting historical knowledge through loss function constraints and freeze fine-tuning mechanisms. This effectively solves the data drift problem faced by static models, realizes the lifelong self-evolution of the detection model, and ensures that the detection accuracy does not decay after long-term deployment.

[0087] like Figure 4As shown, in another aspect, the present invention also provides a road defect adaptive detection system, including: a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute relevant steps of a relevant embodiment of the road defect adaptive detection method of the present invention.

[0088] This invention provides an adaptive road defect detection system, in which functional components can be integrated into a single processing unit, or each component can exist independently, or two or more components can be integrated into a single unit. The integrated components can be implemented in hardware or as software functions.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. An adaptive detection method for road defects, characterized in that, The method includes: Acquire the original road surface image and generate various feature views using the original road surface image; Based on the feature view, inference is performed using multiple pre-built defect detection models to obtain the initial detection box and the corresponding initial confidence level; A topological association graph of the detection boxes is constructed using the initial detection boxes and the initial confidence level, and the connected components of the potential defect targets are obtained based on the topological association graph of the detection boxes. The location of fusion defects and the fusion confidence level are calculated based on the connected components. The defect detection model is adaptively updated using the fused defect location and the fused confidence level.

2. The adaptive road defect detection method according to claim 1, characterized in that, The process of generating multiple feature views using the original road surface image includes: The original road surface image is processed by a contrast-limited adaptive histogram equalization algorithm to generate an illumination-invariant view. The edge gradient information of the original road surface image is extracted using the Sobel operator to generate a high-frequency geometric texture view; The original road surface image is processed by a bilateral filtering algorithm to generate a high signal-to-noise ratio structured view.

3. The adaptive road defect detection method according to claim 1, characterized in that, The step of constructing a topological association graph of detection boxes using the initial detection boxes and the initial confidence level, and obtaining the connected components of potential defect targets based on the topological association graph of detection boxes includes: Calculate the cross-union ratio (CUP) of any two initial detection boxes and compare the CUP with a preset CUP threshold. The initial detection box and the initial confidence score are set as graph nodes, and the connection relationship of the graph nodes is set according to the comparison result to form a detection box topology graph; The topological graph of the detection box is traversed using a depth-first search algorithm to obtain the connected components of multiple potential defect targets.

4. The adaptive road defect detection method according to claim 1, characterized in that, The connected components include a subset of detection boxes and a subset of confidence scores. The calculation of the fusion defect location and fusion confidence score based on the connected components includes: Calculate the number of model sources for the subset of detection boxes in the connected components; The fusion confidence score is obtained by correcting the confidence score subset based on the number of model sources. The location of the fusion defect is obtained by weighting and fusing the subset of detection boxes using the fusion confidence score.

5. The adaptive road defect detection method according to claim 4, characterized in that, The fusion confidence level satisfies the following formula: in, To adjust the confidence level, To achieve the highest confidence level, As a consistent reward factor, For the number of model sources, This is the view penalty coefficient.

6. The adaptive road defect detection method according to claim 1, characterized in that, The adaptive update of the defect detection model using the fused defect location and the fused confidence level includes: Based on the fusion confidence level, pseudo-label data is constructed using the fusion defect locations; Obtain historical core anchor data, and construct incremental training data using the pseudo-label data and the historical core anchor data; An experience replay loss function is constructed, and the defect detection model is adaptively updated based on the incremental training data using the experience replay loss function.

7. The adaptive road defect detection method according to claim 6, characterized in that, The step of constructing pseudo-label data based on the fusion confidence level and the fusion defect location includes: Obtain high-confidence thresholds and low-confidence thresholds; The fusion confidence score is compared with the high confidence threshold and the low confidence threshold to determine the sample type of the fusion defect location; Pseudo-label data is constructed based on the sample types described above.

8. The adaptive road defect detection method according to claim 6, characterized in that, The empirical replay loss function satisfies the following formula: in, For the experience replay loss function, For the loss function of the new sample, For model parameters, For the newly generated pseudo-label data, For historical samples, the loss function is... Historical core anchor data, It serves as a forgetting balance factor.

9. The adaptive road defect detection method according to claim 6, characterized in that, The adaptive update of the defect detection model based on the incremental training data using the empirical replay loss function includes: The backbone network parameters of the defect detection model are frozen, and the detection head parameters of the defect detection model are adaptively updated using the incremental training data based on the empirical replay loss function.

10. An adaptive road defect detection system, characterized in that, include: The system includes a processor, an input device, an output device, and a memory, all interconnected. The memory stores a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute a road defect adaptive detection method as described in any one of claims 1 to 9.

Images