Municipal road disease patrol detection method and related device

Abstract

The application relates to a municipal road disease patrol detection method and related device, which comprises the following steps: acquiring a road surface collection image of a patrol vehicle and a real-time lateral offset relative to a lane center line; determining a specific semantic area to which the image belongs according to the offset, and retrieving an associated disease distribution prior probability; generating a dynamic attention mask and a prior confidence gain coefficient according to the semantic area; inputting the above data into a deep neural network, using the mask to perform spatial weighted modulation on image features, using the gain coefficient to adjust a determination threshold, and outputting a detection result. The application uses spatial prior knowledge contained in the vehicle motion trajectory to adaptively detect the disease distribution law of different road surface areas, improves the detection rate of small diseases in the wheel track, effectively suppresses the false alarm in the road edge area, and improves the accuracy and anti-interference ability of automatic patrol.

Description

Technical Field

[0001] This application relates to the field of road inspection, and in particular to a method and related apparatus for inspecting and detecting defects in municipal roads. Background Technology

[0002] With the rapid development of urban infrastructure construction, municipal road maintenance has become an important part of urban management. Traditional road defect inspection mainly relies on manual foot or vehicle patrols, which are inefficient and limited by the subjective experience of the workers. In recent years, automated inspection systems equipped with high-definition cameras and onboard computing units have gradually become more widespread. These systems process the collected road surface images using deep learning algorithms, enabling them to automatically identify common defects such as cracks and potholes, significantly improving inspection coverage.

[0003] Existing computer vision-based road defect detection technologies typically treat video frames captured by vehicle-mounted cameras as independent two-dimensional images. Mainstream target detection algorithms often employ a uniform feature extraction strategy and judgment criteria for the entire image region when extracting image features. For example, the algorithm usually sets a globally fixed confidence threshold; when the model's predicted probability exceeds this threshold, it is considered that a defect exists. This approach ignores the differences in the physical characteristics of municipal road surfaces in their lateral spatial distribution and the statistical patterns of defect occurrence.

[0004] In real-world road engineering scenarios, the probability and type of road surface defects vary significantly across different areas. Wheel tracks, where vehicle tires frequently pass over surfaces, are more prone to longitudinal cracks or subsidence due to structural fatigue, while road edge areas are more susceptible to the accumulation of leaves, debris, or broken curbs. However, existing visual detection algorithms struggle to perceive the real-time lateral position of vehicles within the lane and cannot determine whether the currently acquired image is located in the wheel track or at the road edge. This presents the algorithm with a dilemma when dealing with different areas: lowering the global detection threshold to capture subtle early cracks within the wheel track often leads to non-defective interference at the road edge being incorrectly identified as defects, resulting in numerous false alarms; conversely, raising the global detection threshold to suppress false alarms at the edge can easily miss genuine defects with inconspicuous features within the wheel track.

[0005] Furthermore, while existing automated inspection systems are equipped with high-precision positioning devices, the positioning data is typically only used to mark the geographical location of detected defects and is not involved in the front-end image recognition process. The vehicle's trajectory, lateral offset within the lane, and other kinematic information actually contain rich prior knowledge of the scene. Due to the lack of a mechanism to deeply integrate this non-image-dimensional prior information with the visual feature extraction process, existing detection methods struggle to utilize spatial location information to assist visual algorithms in adaptive weighting or threshold adjustment, limiting further improvements in system accuracy and anti-interference capabilities in complex road environments. Summary of the Invention

[0006] In order to improve the detection rate of minor defects in wheel track areas, effectively suppress false alarms in road edge areas, and enhance the accuracy and anti-interference ability of automatic inspection, this application provides a method and related device for inspecting and detecting defects in municipal roads.

[0007] Firstly, this application provides a method for inspecting and detecting defects in municipal roads, which adopts the following technical solution:

[0008] A method for inspecting and detecting defects in municipal roads includes the following steps:

[0009] S1. Use an onboard camera to acquire road surface images and corresponding timestamps of the inspection vehicle during its journey, and use a positioning device to simultaneously acquire the real-time lateral offset of the inspection vehicle relative to the centerline of the current driving lane.

[0010] S2. Based on the real-time lateral offset, determine the specific semantic region to which the road surface image belongs on the lateral section of the current driving lane, wherein the specific semantic region is associated with the prior probability of the disease distribution stored in a preset prior probability model.

[0011] S3. Based on the specific semantic region, retrieve the prior probability model and generate a dynamic attention mask and prior confidence gain coefficient that are spatially aligned with the road surface image.

[0012] S4. Input the road surface image, the dynamic attention mask, and the prior confidence gain coefficient into the deep neural network. In the feature extraction stage, the dynamic attention mask is used to perform spatial weighted modulation of the image features. In the target detection stage, the prior confidence gain coefficient is used to adjust the judgment threshold. The result of the disease detection includes the physical parameters of the disease and the geographical coordinates of the disease.

[0013] Optionally, step S1 includes the following sub-steps:

[0014] S11. Use the vehicle-mounted camera mounted on the inspection vehicle to extract the lane line coordinates of the current driving lane and construct a local lane coordinate system;

[0015] S12. Combining the real-time pose data of the inspection vehicle output by the positioning device, calculate the vertical distance of the geometric center point of the inspection vehicle relative to the lane line coordinates in the local lane coordinate system to obtain the real-time lateral offset.

[0016] S13. Align the road surface image with the real-time lateral offset according to the acquisition timestamp.

[0017] Optionally, the specific semantic region is a discrete region divided based on the mechanical load characteristics and environmental characteristics of the road's transverse cross-section, including:

[0018] The wheel track area corresponding to the tire crush range of the inspection vehicle;

[0019] The road edge area corresponding to the edge of the current driving lane near the curb;

[0020] The central non-track region corresponds to the range between the two track zones.

[0021] Optionally, step S4 includes the following sub-steps:

[0022] S41. Input the road surface image into the backbone network of the deep neural network, extract the original feature map tensor, and adjust the resolution of the dynamic attention mask to be consistent with the spatial dimension of the original feature map tensor;

[0023] S42. Apply the dynamic attention mask as a spatial weight matrix to the original feature map tensor, adjust the gain of the feature response values ​​in the specific semantic region, and generate a weighted feature map;

[0024] S43. Input the weighted feature map into the classification detection head to calculate the initial confidence of the candidate target, and correct the judgment threshold based on the prior confidence gain coefficient to obtain the disease detection result;

[0025] S44. Based on the intrinsic parameter matrix of the vehicle-mounted camera and the real-time pose data output by the positioning device, perform inverse perspective projection on the target box in the defect detection result, calculate the actual physical size of the candidate target and solve the geographical coordinates of the defect.

[0026] Optionally, step S42 includes the following sub-steps:

[0027] S421. Construct a weight matrix that matches the tensor dimension of the original feature map, configure the elements in the weight matrix corresponding to the specific semantic region as first weight values, and configure the elements in the weight matrix corresponding to the region outside the specific semantic region as second weight values, wherein the first weight value is greater than the second weight value;

[0028] S422. Perform element-wise multiplication or broadcast weighting operations on the original feature map tensor and the weight matrix, so that the feature numerical intensity corresponding to the specific semantic region in the weighted feature map is amplified.

[0029] Optionally, step S43 includes the following sub-steps:

[0030] S431. Establish a negative correlation mapping relationship between the judgment threshold and the prior confidence gain coefficient;

[0031] S432. When the prior probability of the disease distribution retrieved is greater than a preset probability threshold, the prior confidence gain coefficient with a value greater than 1 is used to perform a numerical reduction operation on the judgment threshold.

[0032] S433. When the initial confidence level of the candidate target is higher than the determination threshold after numerical reduction, it is confirmed that a disease target has been detected.

[0033] Optionally, the method further includes step S5:

[0034] S51. In response to the defect detection result indicating the existence of a defect, a maintenance work order is generated that includes the geographical coordinates of the defect and a road surface image.

[0035] S52. Automatically match the assigned jurisdiction grid based on the geographical coordinates of the disease, and push the maintenance work order to the maintenance terminal corresponding to the assigned jurisdiction grid.

[0036] Secondly, the computer device provided in this application adopts the following technical solution:

[0037] A computer device comprising:

[0038] One or more processors;

[0039] Memory;

[0040] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to:

[0041] Perform the above-mentioned methods for inspecting and detecting defects in municipal roads.

[0042] Thirdly, this application provides a computer-readable storage medium that adopts the following technical solution:

[0043] A computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described above.

[0044] The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following:

[0045] The above-mentioned methods for inspecting and detecting defects in municipal roads.

[0046] In summary, this application includes at least one of the following beneficial technical effects:

[0047] 1. This application establishes a logical connection between simple two-dimensional image data and semantic regions of physical space by acquiring the real-time lateral offset of the vehicle. This spatial prior mechanism, which combines kinematic parameters, enables the detection algorithm to distinguish whether the currently captured image is located on a wheel track area frequently trodden by vehicles or on the edge of a road where debris easily accumulates. This provides accurate scene information for subsequent differentiated image processing and makes up for the lack of spatial position awareness in traditional computer vision algorithms.

[0048] 2. This application utilizes a dynamic attention mask to perform spatial weighted modulation on specific regions during the feature extraction stage. By enhancing the feature response values ​​within high-probability disease areas, this technical solution enables the neural network to focus more on key areas such as wheel tracks, making it easier to capture early-stage minor diseases with low texture contrast or indistinct features, effectively improving the detection rate for hidden diseases.

[0049] 3. This application utilizes a priori confidence gain coefficient to adaptively adjust the judgment threshold in the target detection stage. When faced with complex environmental interference in the road edge area, the system can automatically raise the judgment standard based on prior knowledge, thereby effectively filtering false targets caused by fallen leaves, garbage accumulation, or curb joints. While ensuring the detection sensitivity of the core area, it significantly reduces the overall false alarm rate of the system. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a method for inspecting and detecting defects in municipal roads, as provided in an embodiment of this application.

[0051] Figure 2 This is a first scene diagram illustrating the detection of the wheel track area and the results of the defect detection in this embodiment of the application;

[0052] Figure 3This is a second scene diagram illustrating the detection status of the central non-wheel track area and the results of the defect detection in this embodiment of the application;

[0053] Figure 4 This is a third-scene schematic diagram illustrating the detection status of the road edge area and the results of the defect detection in the embodiments of this application;

[0054] Figure 5 This is a schematic diagram of the deep neural network processing architecture and data flow provided in the embodiments of this application;

[0055] Figure 6 This is a schematic diagram of the adaptive threshold determination logic based on a specific semantic region and prior confidence gain coefficient provided in the embodiments of this application;

[0056] Figure 7 A schematic diagram of a computer device according to an embodiment of the present invention is shown. Detailed Implementation

[0057] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the application and are not intended to limit the scope of the application.

[0058] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the inventive concept. As part of this specification, some of the accompanying drawings of this disclosure are block diagrams illustrating structures and devices to avoid complicating the disclosed principles. For clarity, not all features of the actual embodiment need to be described. Furthermore, the language used in this disclosure has been primarily chosen for readability and instructional purposes and may not have been chosen to define or limit the subject matter of the invention, thus requiring the necessary claims to determine such inventive subject matter. References to “an embodiment” or “an embodiment” in this disclosure mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment, and multiple references to “an embodiment” or “an embodiment” should not be construed as necessarily referring to the same embodiment.

[0059] Unless explicitly defined, the terms “a,” “an,” and “the” are not intended to refer to a singular entity, but rather to include a general category whose specific examples can be used for illustration. Therefore, the use of the terms “a” or “an” can mean any number of at least one, including “a,” “one or more,” “at least one,” and “one or more.” The term “or” means any of the options and any combination of the options, including all options unless explicitly indicated that the options are mutually exclusive. The phrase “at least one of” when combined with a list of items refers to a single item in the list or any combination of items in the list. The phrase does not require all items listed unless explicitly defined as such.

[0060] This application discloses a method for inspecting and detecting defects in municipal roads, referring to... Figure 1 This includes the following steps S1-S4.

[0061] S1. Use an onboard camera to acquire road surface images and corresponding timestamps of the inspection vehicle during its journey, and use a positioning device to simultaneously acquire the real-time lateral offset of the inspection vehicle relative to the centerline of the current driving lane.

[0062] Specifically, the vehicle-mounted camera uses an industrial-grade global shutter CMOS camera, rigidly mounted on the center of the roof beam of the inspection vehicle. The lens optical axis faces the road surface and maintains a preset pitch angle to achieve vertical or large-angle overhead shooting of the road surface. The positioning device uses a combined navigation terminal integrating a real-time dynamic differential positioning module and a high-precision inertial measurement unit, and communicates with the vehicle computing unit via a gigabit Ethernet or USB 3.0 high-speed interface. The road surface images are captured in high-resolution format, with a resolution of no less than 1920×1080 pixels, a field of view covering the width of a single standard lane, and a horizontal field of view set to 90 to 120 degrees to cover wheel tracks and edge areas within the lane. The timestamp marking mechanism uses a hardware-triggered synchronization strategy, where the positioning device sends a pulse signal per second to trigger camera exposure, or a precise clock protocol is used to unify the sensor time reference. Typical types of road surface images acquired in actual operations include asphalt road surface images, cement concrete road surface images, shaded road surface images under different lighting conditions, and road surface images containing different dry and wet conditions.

[0063] Based on this, the centerline of the current driving lane is determined by identifying and fitting the lane markings on both sides, or by real-time matching of high-precision map data. The geometric definition of this real-time lateral offset is the vertical distance between the projection of the geometric center point of the inspection vehicle onto the road surface and the centerline of the current driving lane. Given the heterogeneous nature of visual sensor data and positioning device data, synchronization is achieved through alignment using a unified spatiotemporal reference, employing interpolation algorithms to map low-frequency positioning data to the high-frequency image acquisition time. Obtaining this high-precision real-time lateral offset allows the system to accurately determine the vehicle's precise lateral position within the lane, enabling it to distinguish whether the currently acquired image corresponds to the wheel track or the road edge, facilitating subsequent adaptive image processing.

[0064] Furthermore, step S1 includes the following sub-steps S11-S13.

[0065] S11. Use the vehicle-mounted camera mounted on the inspection vehicle to extract the lane line coordinates of the current driving lane and construct a local lane coordinate system.

[0066] Step S11 utilizes the vehicle-mounted camera mounted on the inspection vehicle to extract the lane line coordinates of the currently traveling lane and construct a local lane coordinate system. Techniques for extracting lane line coordinates include edge detection algorithms combined with Hough transform, or semantic segmentation models based on deep learning. This local lane coordinate system takes the projection point of the current frame camera's optical center onto the road surface as its origin, the vehicle's travel direction as the positive Y-axis, and the rightward direction perpendicular to the travel direction as the positive X-axis.

[0067] S12. Combining the real-time pose data of the inspection vehicle output by the positioning device, calculate the vertical distance of the geometric center point of the inspection vehicle relative to the lane line coordinates in the local lane coordinate system to obtain the real-time lateral offset.

[0068] Step S12 combines the real-time pose data of the inspection vehicle output by the positioning device to calculate the vertical distance of the geometric center point of the inspection vehicle relative to the lane line coordinates in the local lane coordinate system, thus obtaining the real-time lateral offset. The real-time pose data includes latitude and longitude coordinates, heading angle, pitch angle, and positioning covariance matrix. The calculation process of the real-time lateral offset involves converting the global coordinates of the lane line to coordinates in the local lane coordinate system based on the vehicle's current position and heading angle, and calculating the Euclidean distance from the origin to the fitted lane centerline.

[0069] S13. Align the road surface image with the real-time lateral offset according to the acquisition timestamp.

[0070] Step S13 aligns the acquired road surface image with the real-time lateral offset according to the acquisition timestamp. The timing alignment methods include nearest neighbor matching, linear interpolation, or hardware synchronization based on FPGA trigger signals. Lateral offset calculation and timing alignment ensure the consistency between the visual data and the vehicle's physical location in the time dimension.

[0071] Next, step S2 is executed: based on the real-time lateral offset, the specific semantic region to which the road surface image belongs on the lateral section of the current driving lane is determined, wherein the specific semantic region is associated with the prior probability of the disease distribution stored in the preset prior probability model.

[0072] Specifically, the logic for determining the specific semantic region to which the captured road image belongs mainly employs either a numerical comparison method based on a preset distance threshold interval, or a normalized determination method based on the lane width ratio. When using the numerical comparison method, the system pre-sets lateral distance threshold intervals corresponding to different semantic regions. The calculated real-time lateral offset is compared with these threshold intervals. If the offset falls within a specific interval, the current image is determined to belong to the semantic region corresponding to that interval. When using the normalized determination method, the system calculates the ratio of the real-time lateral offset to the current lane width, and determines the region to which it belongs based on the percentage range of this ratio.

[0073] The pre-built prior probability model is constructed based on statistical mining of historical road maintenance big data or on rule definitions by road engineering mechanics experts. By analyzing the stress state and historical damage data at different lateral locations of the road surface, the model establishes a quantitative correlation between regions and damage distribution. In this model, different semantic regions are associated with typical prior probabilities of damage distribution. For example, the wheel track area, which corresponds to the area frequently rolled by vehicle tires, is associated with a high probability of longitudinal cracks and subsidence; the road edge area near the curb edge of the current driving lane is associated with a high probability of water accumulation and debris. Establishing this "location-region-probability" mapping relationship provides crucial data support for subsequent targeted image enhancement and adaptive threshold determination, enabling the detection algorithm to adjust its processing strategy based on physical spatial attributes.

[0074] Furthermore, referring to Figure 2-4 The specific semantic region is a discrete region divided based on the mechanical load characteristics and environmental characteristics of the road's transverse cross-section, mainly including the following three categories:

[0075] The wheel track area corresponding to the tire crush range of the inspection vehicle;

[0076] The road edge area corresponding to the edge of the current driving lane near the curb;

[0077] The central non-track region corresponds to the range between the two track zones.

[0078] The mechanical load characteristics of the road's transverse cross-section reflect the differences in the frequency and intensity of vehicle loads at different transverse locations, while environmental characteristics reflect the differences in the distribution of road surface drainage and debris accumulation. The geometric boundary of the wheel track zone is typically defined as two strip-shaped areas with a preset width centered on the standard wheel track. This area mainly bears repeated dynamic loads from vehicles, and typical defects include longitudinal cracks, swells, and ruts. The geometric boundary of the road edge zone is defined as the outermost strip-shaped area of ​​the lane closest to the curb. This area is susceptible to road surface drainage, and typical defects or disturbances include curb damage, water accumulation, and leaf accumulation. The geometric boundary of the central non-wheel track zone is defined as the area between the two wheel track zones. This area is relatively less subjected to direct wheel pressure, and typical characteristics or defects include manhole covers and transverse cracks caused by thermal shrinkage. The specific parameter setting strategy for dividing these discrete zones is usually based on dynamic segmentation according to the national standard lane width. Taking a standard lane width of 3.75 meters as an example, the range from 0 to 0.6 meters from the lane edge can be defined as the edge zone, and the range from 0.6 meters to 1.4 meters can be defined as the wheel track zone. The continuous horizontal space of the road surface is discretized into specific semantic regions to facilitate the construction of a refined prior model, thereby calling the corresponding detection strategy for the physical characteristics of different regions.

[0079] Then, step S3 is executed: based on the specific semantic region, the prior probability model is retrieved, and a dynamic attention mask and prior confidence gain coefficient are generated that are spatially aligned with the road surface image.

[0080] Specifically, the query mechanism for retrieving the prior probability model employs a key-value pair-based hash lookup or a relational database-based SQL query. The system uses the semantic region identifier determined in step S2 as the index key to quickly read the corresponding disease distribution parameters from the pre-loaded memory database. Mathematically, this dynamic attention mask is defined as a weight matrix with the same spatial dimension as the image or feature map, and its data structure is typically a single-channel floating-point tensor. The mask's typical form is a binary mask that hard-segments the region of interest, a Gaussian heatmap that smoothly transitions from the region center to the edge, or a weighted matrix reflecting the disease probability gradient.

[0081] To ensure accurate correspondence between the mask and the acquired image in the pixel coordinate system, specific algorithms for generating a mask aligned with the image space include perspective-transformation-based region-of-interest mapping and resolution-scaling-based bilinear interpolation. The system projects the boundary coordinates of semantic regions in physical space onto the image plane based on camera calibration parameters and the ground plane equation, and generates the corresponding mask region through a filling algorithm. Simultaneously, the numerical definition strategy for the prior confidence gain coefficient follows the principles of inverse suppression and forward enhancement. For high-risk areas (such as wheel tracks), a gain coefficient greater than 1 is set, for example, 1.2, to improve detection sensitivity; for areas prone to false alarms (such as road edges), a suppression coefficient less than 1 is set, for example, 0.8, to tighten the judgment criteria. These two intermediate parameters are generated to connect macroscopic prior knowledge with microscopic image features, transforming statistical laws into specific numerical values ​​that can directly participate in neural network tensor operations.

[0082] Finally, step S4 is executed: the road surface image, the dynamic attention mask, and the prior confidence gain coefficient are input into the deep neural network. In the feature extraction stage, the dynamic attention mask is used to perform spatial weighted modulation of the image features. In the target detection stage, the prior confidence gain coefficient is used to adjust the judgment threshold, and the disease detection result containing the physical parameters of the disease and the geographical coordinates of the disease is output.

[0083] Specifically, refer to Figure 5 The backbone architecture of this deep neural network preferably uses YOLO series (such as YOLOv8) or ResNet series (such as ResNet-50) as the core network to balance detection speed and accuracy. Before being input into the network, the road surface images need to be preprocessed, including adjusting the resolution to the network input size (such as 640×640 pixels) and normalizing the pixel values, mapping the RGB channel values ​​to the range of 0 to 1. The construction of this neural network requires an offline training step, using a sample dataset containing labeled defects for iterative training. During training, the loss function value between the predicted value and the true label is calculated through forward propagation, and backpropagation is performed using stochastic gradient descent or Adam optimizer to continuously update the weights and bias parameters in the convolutional layers, batch normalization layers, and fully connected layers until the model converges, thereby enabling the network to extract road surface texture features and identify defect patterns.

[0084] In the feature extraction stage, the specific algorithm for spatial weighted modulation using a mask is an attention mechanism implemented through Hadamard product. This involves broadcasting the dynamic attention mask along the channel dimension, making its size consistent with the feature map tensor output by the current layer, and then performing element-wise multiplication. Through this operation, the feature map values ​​corresponding to high-weight regions in the mask are amplified, thereby strengthening the feature responses of key regions such as wheel tracks and suppressing background noise.

[0085] In the target detection phase, the specific logic strategy for adjusting the decision threshold using the gain coefficient employs a negative correlation mapping formula between the threshold and the coefficient. The system sets a basic confidence threshold, and the final decision threshold is equal to this basic threshold divided by the prior confidence gain coefficient. When the detection box is located in a high-incidence area, a larger gain coefficient lowers the decision threshold, thereby recalling low-confidence defective targets; when the detection box is located in an interference area, a smaller coefficient raises the decision threshold, thereby filtering out false alarms.

[0086] The calculation method for the physical parameters of road defects is based on inverse perspective transformation (IPM). Camera calibration parameters are used to convert image pixel coordinates into road surface physical coordinates, thereby calculating the actual Euclidean length of cracks or the actual opening area of ​​potholes. The calculation process for the geographical coordinates of the defects involves a cascaded transformation matrix operation: from the pixel coordinate system to the camera coordinate system, then to the vehicle body coordinate system via the vehicle extrinsic parameter matrix, and finally to the global geodetic coordinate system using GNSS positioning data. This dual mechanism of "feature enhancement + threshold correction" highlights key information at the physical feature level and dynamically adapts to environmental differences at the logical decision level, effectively resolving the irreconcilable contradiction between the accuracy and false alarm rate of defect identification in complex road surface backgrounds.

[0087] Further, step S4 is specifically decomposed into the following sub-steps S41-S44 to achieve refined feature processing. S41. Input the road surface acquisition image into the backbone network of the deep neural network, extract the original feature map tensor, and adjust the resolution of the dynamic attention mask to be consistent with the spatial dimension of the original feature map tensor.

[0088] Specifically, the backbone network typically employs mature architectures such as ResNet-50, VGG16, or CSPDarknet from the YOLO series. These networks extract high-dimensional semantic information from images through stacked convolutional kernels and nonlinear activation functions. The output original feature map tensor has a three-dimensional structure of H×W×C, where H and W represent the height and width of the feature map plane, respectively, and C represents the number of channels. Each value in this tensor, i.e., the feature response value, physically represents the activation intensity of a specific visual pattern (such as edges, textures, or shape primitives) within the corresponding receptive field region of the original image. Because the backbone network involves multiple stride convolutions or pooling operations during computation to expand the receptive field, the spatial dimension of the original feature map tensor is significantly smaller than that of the input image and the initially generated dynamic attention mask. Therefore, the mask resolution must be adjusted. Specific algorithms typically employ bilinear interpolation downsampling or adaptive average pooling to compress its spatial size to be completely consistent with H×W, thereby ensuring strict spatial alignment between mask pixels and feature map pixels.

[0089] S42. Apply the dynamic attention mask as a spatial weight matrix to the original feature map tensor, adjust the gain of the feature response values ​​in the specific semantic region, and generate a weighted feature map.

[0090] The principle behind this step is a feature modulation mechanism based on spatial attention, treating the adjusted mask as a two-dimensional gain field. Logically, the system performs element-wise multiplication of the mask matrix and the feature map tensor in the spatial dimension. Through this operation, the feature response values ​​located within a specific semantic region (such as wheel tracks) are multiplied by a weighting coefficient greater than 1, resulting in numerical amplification, while the feature values ​​of the background region remain unchanged or are suppressed by multiplying by a coefficient less than 1. This gain adjustment effectively reconstructs the numerical distribution of the feature map, making the minute crack features within the wheel tracks, which might otherwise be obscured by the rough texture of the road surface, significantly more numerically intense than the surrounding background noise. This improves the signal-to-noise ratio of the effective signal, allowing subsequent detection layers to more accurately capture key targets.

[0091] Specifically, in order to achieve targeted feature enhancement, step S42 is further refined into the following sub-steps S421-S422.

[0092] S421. Construct a weight matrix that matches the tensor dimension of the original feature map, configure the elements in the weight matrix corresponding to the specific semantic region as first weight values, and configure the elements in the weight matrix corresponding to the region outside the specific semantic region as second weight values, wherein the first weight value is greater than the second weight value.

[0093] When constructing this weight matrix, its dimensions are typically determined based on the spatial resolution (height H and width W) of the original feature map tensor. To balance computational efficiency and memory usage, the weight matrix is ​​usually configured as a single-channel structure (H×W×1), relying on a broadcast mechanism to handle all channels of the feature map; or, when channel attention mechanisms are required, it is configured as a multi-channel structure identical to the feature map (H×W×C). Regarding the weight values, the first weight value is usually set to a value greater than 1.0 (e.g., 1.5 or 2.0) to amplify signals in key areas such as wheel tracks; the second weight value is usually set to 1.0 (keeping the background unchanged) or a value less than 1.0 (e.g., 0.5, used to suppress road edge background). The specific assignment logic is as follows: the system iterates through each spatial coordinate $(i,j)$ of the weight matrix, determines whether the coordinate falls within the index range of the specific semantic region determined in step S2, and if so, assigns the first weight value to the matrix element at that position; otherwise, assigns the second weight value.

[0094] S422. Perform element-wise multiplication or broadcast weighting operations on the original feature map tensor and the weight matrix, so that the feature numerical intensity corresponding to the specific semantic region in the weighted feature map is amplified.

[0095] The element-wise multiplication performed in step S422 is mathematically represented as the Hadamard product. During this process, if the weight matrix is ​​single-channel, "broadcast weighting" means that the deep learning framework automatically performs a logical copy of the two-dimensional weight matrix along the channel dimension, expanding it to the same dimension as the input tensor, without requiring physical memory data copying, thus significantly improving the efficiency of GPU parallel computing. From a fundamental mathematical perspective, the effect of amplifying the numerical intensity of features is that, after weighting, when the feature values ​​pass through subsequent non-linear activation functions (such as ReLU), features in key regions are more likely to cross the activation threshold (Bias) due to their larger values, thus avoiding truncation or zeroing. This ensures that small defects within the wheel track band can be transmitted to deeper networks with stronger signal intensity, effectively preventing gradient vanishing.

[0096] Next, step S43 is executed: the weighted feature map is input into the classification detection head to calculate the initial confidence of the candidate target, and the judgment threshold is corrected based on the prior confidence gain coefficient to obtain the disease detection result.

[0097] Specifically, this classification detection head is typically designed as a sub-network structure attached to the end of the backbone network, for example, consisting of several 1x1 convolutional layers or fully connected layers. The physical meaning of its output data includes two dimensions: first, a class probability vector, representing the probability distribution of the target within the current anchor box belonging to various types of defects (such as cracks or craters); and second, a bounding box regression offset, used to fine-tune the center coordinates and dimensions of the preset anchor box. The initial confidence level is the value corresponding to the maximum value in the aforementioned class probability vector. Its mathematical definition is usually the output value after activation by the Softmax function (for mutually exclusive classes) or the Sigmoid function (for independent classes), with a strictly limited range of 0 to 1, representing the degree of certainty the neural network has regarding the existence of the target based solely on visual texture features.

[0098] In this process, the traditional fixed threshold judgment method often faces irreconcilable limitations. Specifically, setting a high global threshold to reduce the false alarm rate in road edge areas can lead to missed detection of early defects with inconspicuous features within the wheel track zone; conversely, lowering the threshold to increase the detection rate introduces a large number of false alarms caused by environmental noise. This application introduces the prior confidence gain coefficient as a threshold adjustment variable, logically incorporating the feedforward control concept from cybernetics. That is, by utilizing the vehicle position as external prior information, the decision boundary is dynamically adjusted before the final decision, thus breaking the one-size-fits-all dilemma of a single threshold. The final defect detection result is encapsulated in a standardized data format, typically including a unique defect category identifier (ID), the final confidence score, the bounding box coordinates (x, y, w, h) of the defect target in the image coordinate system, and the corresponding mask contour point set in the instance segmentation task.

[0099] Specifically, in order to achieve accurate adaptive decision-making, refer to Figure 6 Step S43 includes the following sub-steps S431-S433.

[0100] S431. Establish a negative correlation mapping relationship between the judgment threshold and the prior confidence gain coefficient.

[0101] This mapping relationship is typically defined using division or linear subtraction to ensure that an increase in the gain coefficient drives a decrease in the decision threshold. A typical mathematical formula is T_1 = T_0 / K, where T_1 is the dynamically calculated decision threshold, T_0 is the system-preset base threshold, and K is the prior confidence gain coefficient.

[0102] S432. When the prior probability of the disease distribution retrieved is greater than a preset probability threshold, the prior confidence gain coefficient with a value greater than 1 is used to perform a numerical reduction operation on the judgment threshold.

[0103] The preset probability threshold is typically set based on statistical significance levels. For example, regions with a prior probability P > 0.7 are defined as high-probability regions. Taking a base threshold T_0 of 0.5 as an example, when a vehicle travels in the wheel track area, the system identifies this area as a high-incidence zone for cracks and assigns a gain coefficient of 1.25. At this point, the system performs a numerical reduction operation: 0.5 / 1.25 = 0.4, meaning the dynamic threshold for this region is temporarily adjusted to 0.4.

[0104] S433. When the initial confidence level of the candidate target is higher than the determination threshold after numerical reduction, it is confirmed that a disease target has been detected.

[0105] The technical effectiveness of this mechanism can be seen through the comparative analysis of the above typical scenarios: In scenario one, for a blurry crack located within the wheel track, the initial confidence level of the neural network output is 0.45. If a traditional fixed threshold (such as 0.5) is used, this target will be regarded as background noise and missed; however, in this scheme, since the threshold has been adaptively reduced to 0.4, the target satisfies the condition 0.45>0.4, and is thus successfully detected, effectively improving the recall rate for hidden defects. In scenario two, for a suspected interference object (such as leaves) located in the road edge area, the initial confidence level is 0.55. Since this area is a low-probability area, the system matches an inhibition coefficient of 0.8, causing the dynamic threshold to increase to 0.5 / 0.8=0.625. At this time, 0.55<0.625, the target is judged as a negative sample and filtered, thereby effectively reducing the false alarm rate of the system. This dynamic threshold mechanism achieves a dynamic balance between recall and precision without changing the model network structure, utilizing spatial prior knowledge.

[0106] Finally, step S44 is executed: based on the intrinsic parameter matrix of the vehicle-mounted camera and the real-time pose data output by the positioning device, inverse perspective projection is performed on the target box in the defect detection result, the actual physical size of the candidate target is calculated, and the geographical coordinates of the defect are solved.

[0107] Specifically, the intrinsic parameter matrix of the vehicle-mounted camera is a key parameter describing the mapping relationship between the pixel coordinate system and the camera coordinate system. It mainly includes focal length, principal point coordinates, and radial and tangential distortion coefficients. Before system deployment, technicians typically use a checkerboard calibration board in conjunction with Zhang Zhengyou's calibration method to obtain these parameters to correct lens distortion and establish an accurate projection model. The real-time pose data output by this positioning device plays a dynamic compensation role in the projection transformation. Specifically, the camera's installation height provides the reference distance from the camera's optical center to the road surface, while the dynamic pitch and roll angle data generated during vehicle movement are used to correct projection plane tilt errors caused by vehicle body vibrations, ensuring that the inverse perspective transformation is always based on the current true geometric relationships.

[0108] The specific mathematical principle behind performing inverse perspective projection is based on homography matrix transformation. The system constructs a homography matrix using intrinsic parameter matrices, camera ground height, and real-time rotation matrices, mapping the pixel coordinates in the perspective image to the metric coordinate system of the road surface's physical plane, thereby eliminating near-to-far distortion caused by perspective effects. Based on the transformed physical plane coordinates, the system can calculate the actual physical dimensions of candidate targets using Euclidean geometry formulas. Typical calculation examples include determining the actual length of a longitudinal crack by calculating the Euclidean distance between the endpoints of the crack skeleton, or estimating the opening area of ​​a pothole by statistically analyzing the physical pixel area within the region enclosed by the pothole contour.

[0109] Meanwhile, the process of calculating the geographical coordinates of road defects involves a complete coordinate system transformation chain. The system first converts the pixel coordinates of the defect's center point into spatial coordinates in the camera coordinate system, then uses an extrinsic parameter matrix to transform it to the vehicle coordinate system. Finally, combining the latitude, longitude, and heading information output by the positioning device, it uses rigid body transformation to convert the relative position in the vehicle coordinate system into absolute geographical coordinates in the global geodetic coordinate system. The resulting structured and quantified defect data can be directly integrated with the municipal road maintenance management system, providing a highly reliable data foundation for the subsequent automatic generation of electronic ledgers for municipal road maintenance containing precise locations and engineering quantities, significantly reducing the cost of manual verification.

[0110] Furthermore, in order to achieve a closed-loop flow of detection data, the method further includes step S5. Specifically, step S5 is implemented through the following sub-steps S51-S52.

[0111] S51. In response to the indication of the presence of a defect by the defect detection result, a maintenance work order is generated that includes the geographical coordinates of the defect and a road surface image.

[0112] To optimize maintenance resource allocation, the system incorporates built-in triggering logic and filtering mechanisms for work order generation. For example, the system automatically generates work orders only for defects exceeding a preset threshold in severity (such as potholes deeper than 3 cm or cracks wider than 5 mm), while minor defects are simply recorded in the database without triggering immediate repair processes, thus avoiding invalid work orders. These work orders typically include a unique work order number, defect level, repair timeframe, and a captured image of the site with a detection frame.

[0113] S52. Automatically match the assigned jurisdiction grid based on the geographical coordinates of the disease, and push the maintenance work order to the maintenance terminal corresponding to the assigned jurisdiction grid.

[0114] Regarding the spatial division of jurisdictional grids, electronic fence technology is typically used to divide urban roads into several non-overlapping polygonal responsibility zones. The matching algorithm employs either a point-within-a-polygon determination algorithm or a spatial index query algorithm based on R-Tree, to determine the responsibility grid to which the fault coordinates belong in milliseconds.

[0115] Once the attribution is determined, the system pushes the work order to the corresponding maintenance terminal. Typical types of these terminals include industrial-grade handheld PDAs used by frontline maintenance personnel, dedicated apps installed on smartphones, and web-based management terminals for the maintenance management center. The preferred technology for message push is a message queue push mechanism based on the MQTT protocol, or full-duplex real-time notification technology based on WebSocket, ensuring reliable delivery of work orders even in weak network environments. This automatic case filing and intelligent dispatch mechanism changes the traditional manual inspection and layer-by-layer transfer model, significantly shortening the response time from defect discovery to treatment. It achieves closed-loop management of municipal maintenance from discovery, dispatch, repair to verification, greatly improving the digitalization level and operational efficiency of road maintenance.

[0116] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0117] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database contains data related to municipal road defect inspection and detection methods. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a municipal road defect inspection and detection method.

[0118] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the municipal road defect inspection and detection method described in the above embodiment, for example... Figure 1 S1 to S5 are shown.

[0119] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the municipal road defect inspection and detection method of the above embodiment, for example... Figure 1 S1 to S5 are shown.

[0120] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments of this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0122] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for inspecting and detecting defects in municipal roads, characterized in that, Includes the following steps: S1. Use an onboard camera to acquire road surface images and corresponding timestamps of the inspection vehicle during its journey, and use a positioning device to simultaneously acquire the real-time lateral offset of the inspection vehicle relative to the centerline of the current driving lane. S2. Based on the real-time lateral offset, determine the specific semantic region to which the road surface image belongs on the lateral section of the current driving lane, wherein the specific semantic region is associated with the prior probability of the disease distribution stored in a preset prior probability model. S3. Based on the specific semantic region, retrieve the prior probability model and generate a dynamic attention mask and prior confidence gain coefficient that are spatially aligned with the road surface image. S4. Input the road surface image, the dynamic attention mask, and the prior confidence gain coefficient into the deep neural network. In the feature extraction stage, the dynamic attention mask is used to perform spatial weighted modulation on the image features. In the target detection stage, the prior confidence gain coefficient is used to adjust the judgment threshold and output the disease detection result containing the physical parameters of the disease and the geographical coordinates of the disease. The specific semantic region is a discrete region defined based on the mechanical load characteristics and environmental characteristics of the road's transverse cross-section, including: The wheel track area corresponding to the tire crush range of the inspection vehicle; The road edge area corresponding to the edge of the current driving lane near the curb; The central non-track region corresponds to the range between the two track zones.

2. The method for inspecting and detecting defects in municipal roads according to claim 1, characterized in that, S1 includes the following sub-steps: S11. Use the vehicle-mounted camera mounted on the inspection vehicle to extract the lane line coordinates of the current driving lane and construct a local lane coordinate system; S12. Combining the real-time pose data of the inspection vehicle output by the positioning device, calculate the vertical distance of the geometric center point of the inspection vehicle relative to the lane line coordinates in the local lane coordinate system to obtain the real-time lateral offset. S13. Align the road surface image with the real-time lateral offset according to the acquisition timestamp.

3. The method for inspecting and detecting defects in municipal roads according to claim 2, characterized in that, S4 includes the following sub-steps: S41. Input the road surface image into the backbone network of the deep neural network, extract the original feature map tensor, and adjust the resolution of the dynamic attention mask to be consistent with the spatial dimension of the original feature map tensor; S42. Apply the dynamic attention mask as a spatial weight matrix to the original feature map tensor, adjust the gain of the feature response values ​​in the specific semantic region, and generate a weighted feature map; S43. Input the weighted feature map into the classification detection head to calculate the initial confidence of the candidate target, and correct the judgment threshold based on the prior confidence gain coefficient to obtain the disease detection result; S44. Based on the intrinsic parameter matrix of the vehicle-mounted camera and the real-time pose data output by the positioning device, perform inverse perspective projection on the target box in the defect detection result, calculate the actual physical size of the candidate target and solve the geographical coordinates of the defect.

4. The method for inspecting and detecting defects in municipal roads according to claim 3, characterized in that, S42 includes the following sub-steps: S421. Construct a weight matrix that matches the tensor dimension of the original feature map, configure the elements in the weight matrix corresponding to the specific semantic region as first weight values, and configure the elements in the weight matrix corresponding to the region outside the specific semantic region as second weight values, wherein the first weight value is greater than the second weight value; S422. Perform element-wise multiplication or broadcast weighting operations on the original feature map tensor and the weight matrix, so that the feature numerical intensity corresponding to the specific semantic region in the weighted feature map is amplified.

5. The method for inspecting and detecting defects in municipal roads according to claim 3, characterized in that, S43 includes the following sub-steps: S431. Establish a negative correlation mapping relationship between the judgment threshold and the prior confidence gain coefficient; S432. When the prior probability of the disease distribution retrieved is greater than a preset probability threshold, the prior confidence gain coefficient with a value greater than 1 is used to perform a numerical reduction operation on the judgment threshold. S433. When the initial confidence level of the candidate target is higher than the determination threshold after numerical reduction, it is confirmed that a disease target has been detected.

6. The method for inspecting and detecting defects in municipal roads according to any one of claims 1 to 5, characterized in that, The method further includes step S5: S51. In response to the defect detection result indicating the existence of a defect, a maintenance work order is generated that includes the geographical coordinates of the defect and a road surface image. S52. Automatically match the assigned jurisdiction grid based on the geographical coordinates of the disease, and push the maintenance work order to the maintenance terminal corresponding to the assigned jurisdiction grid.

7. A computer device, characterized in that, It includes: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to: perform the municipal road defect inspection and detection method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the municipal road defect inspection and detection method as described in any one of claims 1 to 6.

Images