Road infrastructure monitoring method and system based on intelligent perception

Abstract

The invention provides a road infrastructure monitoring method and system based on intelligent perception, and the method comprises the steps: extracting a feature map of an image, generating a probability thermodynamic diagram, and generating a noise frame; calculating a denoising budget of the noise frame; in each iteration, the corrected noise box set and the corresponding confidence coefficient are predicted, a consumption value deducted from the denoising budget is determined according to the confidence coefficient, and if the denoising budget of the noise box is consumed completely or the confidence coefficient is lower than a preset value for continuous N times, subsequent denoising iteration of the noise box is stopped; when the number of iterations reaches a preset number of times, eliminating a noise frame with the confidence coefficient lower than a pruning threshold, if the confidence coefficient of the noise frame is higher than a trigger threshold and the size of the noise frame is larger than a size threshold, generating an auxiliary noise frame with the same current denoising budget as the noise frame in a neighborhood of the noise frame, and adding the auxiliary noise frame into a noise frame set of next iteration; and after the denoising step number is iterated, obtaining a monitoring result of the road infrastructure based on the output set.

Description

Technical Field

[0001] This application belongs to the field of monitoring, and in particular relates to a method and system for monitoring highway infrastructure based on intelligent sensing. Background Technology

[0002] The smooth operation of highways is directly related to the safety and efficiency of transportation. Traditional highway infrastructure monitoring mainly relies on manual inspections, where dedicated personnel periodically check and record information on road surfaces, guardrails, road signs, and other facilities. This method is not only costly in terms of manpower and resources and inefficient, but the results are also easily affected by the inspectors' sense of responsibility and fatigue. Furthermore, highway inspection operations themselves carry high safety risks, especially on sections with high traffic volume. To overcome the drawbacks of manual inspections, many companies and institutions have begun to adopt automated monitoring based on computer vision, such as the Faster R-CNN series and the YOLO series. These methods, through end-to-end learning on large amounts of labeled data, can extract high-level semantic features, achieving a qualitative leap in detection accuracy and speed compared to traditional algorithms. However, these methods often rely on manually designed components, such as anchor boxes and non-maximum suppression. The parameter settings of these components are complex, and they are not well-suited to defects in highway infrastructure that are irregular in shape or have large scale variations. Summary of the Invention

[0003] To improve the accuracy of monitoring, this application proposes a highway infrastructure monitoring method based on intelligent sensing, including: An image of the highway to be monitored is acquired, and a feature map of the image is extracted using a backbone network; a probability heatmap of the highway region is generated using the feature map; a noise box is generated on the image of the highway to be monitored based on the probability heatmap; the gradient variance of the noise box in the corresponding region of the feature map is calculated, and the denoising budget of the noise box is obtained based on the gradient variance; Iterative denoising is performed within the number of denoising steps. In each iteration, the image decoder predicts the corrected set of noise boxes and the corresponding confidence level based on the feature map and the set of noise boxes in the current iteration step. The consumption value deducted from the denoising budget is determined according to the confidence level. If the denoising budget of the noise box is exhausted or the confidence level is lower than the preset value for N consecutive times, the subsequent denoising iterations for the noise box are stopped. When the number of iterations reaches a preset number, noise boxes with confidence levels below the pruning threshold are removed. If the confidence level of a noise box is higher than the trigger threshold and its size is greater than the size threshold, an auxiliary noise box with the same denoising budget as the noise box is generated in the neighborhood of the noise box and added to the noise box set for the next iteration. After the denoising steps are completed, the monitoring results of the highway infrastructure are obtained based on the output bounding box set.

[0004] Preferably, generating a probability heatmap of the highway area using the feature map includes: The feature maps extracted by the backbone network are input into a segmentation subnetwork consisting of three 3x3 convolutional layers; The output of the segmentation sub-network is reduced in channel dimension by a 1x1 convolutional layer to obtain a single-channel prediction map; The single-channel prediction map is upsampled to the same resolution as the image of the highway to be monitored, and then processed by the Sigmoid activation function to normalize the value of each pixel in the map to between 0 and 1, thus obtaining a probability heatmap of the highway area.

[0005] Preferably, generating a noise box on the image of the highway to be monitored based on the probability heatmap includes: The values ​​of all pixels in the probability heatmap are normalized so that the sum of all pixel values ​​is 1. N weighted random samplings are performed from the normalized probability heatmap to obtain N pixel coordinate points, where N is the preset number of noise boxes; Centered on each sampled pixel coordinate point, and from the size range [S min ,S max The width and height of the frame are randomly selected from the frame to generate a noise box.

[0006] Preferably, obtaining the denoising budget of the noise box based on the gradient variance includes: The rectangular region covered by the noise box on the feature map is determined. The horizontal and vertical gradient values ​​of all feature pixels within the rectangular region are calculated using the 3x3 Sobel operator, and the gradient magnitude of each pixel is calculated. Calculate the variance V of the gradient magnitude of all pixels within the rectangular region. grad ; The initial denoising budget B for the noise box is calculated using the following formula: B=B min +(B max -B min )*(V grad / (V grad +V norm )); Among them, B min For the minimum noise reduction budget, B max For the maximum noise reduction budget, V norm This is the preset variance normalization constant.

[0007] Preferably, determining the consumption value to be deducted from the denoising budget based on the confidence level includes: The consumption value C to be deducted in this iteration is calculated using the following formula: C = p + α * (1 - s)conf ); where α is a coefficient, p is a fixed consumption value, and s conf , where is the confidence level.

[0008] Preferably, generating an auxiliary noise box with the same denoising budget as the current noise box within the neighborhood of the noise box includes: Obtain the center coordinates and size of the noise box, and generate K new center coordinates by sampling with the center coordinates as the center and a standard deviation of one-quarter of the size; For each newly generated center coordinate point, respectively from the uniform distribution U(0.1*w) p 0.5*w p ) and U(0.1*h p 0.5*h p K auxiliary noise boxes are generated by randomly selecting the width and height of the noise box, and the K auxiliary noise boxes and the noise box have the same current denoising budget; where w p h is the width of the noise frame. p The height of the noise frame.

[0009] Preferably, generating an auxiliary noise box with the same denoising budget as the current noise box within the neighborhood of the noise box includes: Calculate the ratio R of the current denoising budget to the initial denoising budget for the noise box. b And the ratio R of the current size of the noise box to the size threshold. s ; According to the formula K=ceil(C*R) b *(R s -1)) calculates the number of auxiliary noise boxes K, where C is the coefficient and ceil() is the rounding up function; Obtain the center coordinates and size of the noise box, and generate K new center coordinates by sampling with the center coordinates as the center and a standard deviation of one-quarter of the size; For each newly generated center coordinate point, respectively from the uniform distribution U(0.1*w) p 0.5*w p ) and U(0.1*h p 0.5*h p K auxiliary noise boxes are generated by randomly selecting the width and height of the noise box, and the K auxiliary noise boxes and the noise box have the same current denoising budget; where w p h is the width of the noise frame. p The height of the noise frame.

[0010] This application also proposes a highway infrastructure monitoring system based on intelligent sensing, comprising: A feature extraction unit is used to acquire an image of the highway to be monitored and extract a feature map of the image using a backbone network; generate a probability heatmap of the highway region using the feature map; generate a noise box on the image of the highway to be monitored based on the probability heatmap; calculate the gradient variance of the region corresponding to the noise box on the feature map; and obtain the denoising budget of the noise box based on the gradient variance. The iterative unit is used to perform iterative denoising within a number of denoising steps. In each iteration, the image decoder predicts the corrected set of noise boxes and the corresponding confidence level based on the feature map and the set of noise boxes in the current iteration step. The consumption value deducted from the denoising budget is determined according to the confidence level. If the denoising budget of the noise box is exhausted or the confidence level is lower than the preset value for N consecutive times, the subsequent denoising iterations for the noise box are stopped. The optimization unit is used to remove noise boxes with confidence levels lower than the pruning threshold when the number of iterations reaches a preset number. If the confidence level of the noise box is higher than the trigger threshold and the size is greater than the size threshold, an auxiliary noise box with the same denoising budget as the noise box is generated in the neighborhood of the noise box and added to the noise box set for the next iteration. The result output unit is used to obtain the monitoring results of highway infrastructure based on the output bounding box set after the denoising steps are completed.

[0011] Preferably, generating a probability heatmap of the highway area using the feature map includes: The feature maps extracted by the backbone network are input into a segmentation subnetwork consisting of three 3x3 convolutional layers; The output of the segmentation sub-network is reduced in channel dimension by a 1x1 convolutional layer to obtain a single-channel prediction map; The single-channel prediction map is upsampled to the same resolution as the image of the highway to be monitored, and then processed by the Sigmoid activation function to normalize the value of each pixel in the map to between 0 and 1, thus obtaining a probability heatmap of the highway area.

[0012] Preferably, generating a noise box on the image of the highway to be monitored based on the probability heatmap includes: The values ​​of all pixels in the probability heatmap are normalized so that the sum of all pixel values ​​is 1. N weighted random samplings are performed from the normalized probability heatmap to obtain N pixel coordinate points, where N is the preset number of noise boxes; Centered on each sampled pixel coordinate point, and from the size range [S min ,S max The width and height of the frame are randomly selected from the frame to generate a noise box.

[0013] Preferably, obtaining the denoising budget of the noise box based on the gradient variance includes: The rectangular region covered by the noise box on the feature map is determined. The horizontal and vertical gradient values ​​of all feature pixels within the rectangular region are calculated using the 3x3 Sobel operator, and the gradient magnitude of each pixel is calculated. Calculate the variance V of the gradient magnitude of all pixels within the rectangular region. grad ; The initial denoising budget B for the noise box is calculated using the following formula: B=B min +(B max -B min )*(V grad / (V grad +V norm )); Among them, B min For the minimum noise reduction budget, B max For the maximum noise reduction budget, V norm This is the preset variance normalization constant.

[0014] Preferably, determining the consumption value to be deducted from the denoising budget based on the confidence level includes: The consumption value C to be deducted in this iteration is calculated using the following formula: C = p + α * (1 - s) conf ); where α is a coefficient, p is a fixed consumption value, and s conf , where is the confidence level.

[0015] Preferably, generating an auxiliary noise box with the same denoising budget as the current noise box within the neighborhood of the noise box includes: Obtain the center coordinates and size of the noise box, and generate K new center coordinates by sampling with the center coordinates as the center and a standard deviation of one-quarter of the size; For each newly generated center coordinate point, respectively from the uniform distribution U(0.1*w) p 0.5*w p ) and U(0.1*h p 0.5*h p K auxiliary noise boxes are generated by randomly selecting the width and height of the noise box, and the K auxiliary noise boxes and the noise box have the same current denoising budget; where w p h is the width of the noise frame. p The height of the noise frame.

[0016] Preferably, generating an auxiliary noise box with the same denoising budget as the current noise box within the neighborhood of the noise box includes: Calculate the ratio R of the current denoising budget to the initial denoising budget for the noise box.b And the ratio R of the current size of the noise box to the size threshold. s ; According to the formula K=ceil(C*R) b *(R s -1)) calculates the number of auxiliary noise boxes K, where C is the coefficient and ceil() is the rounding up function; Obtain the center coordinates and size of the noise box, and generate K new center coordinates by sampling with the center coordinates as the center and a standard deviation of one-quarter of the size; For each newly generated center coordinate point, respectively from the uniform distribution U(0.1*w) p 0.5*w p ) and U(0.1*h p 0.5*h p K auxiliary noise boxes are generated by randomly selecting the width and height of the noise box, and the K auxiliary noise boxes and the noise box have the same current denoising budget; where w p h is the width of the noise frame. p The height of the noise frame.

[0017] This application guides the generation of noisy bounding boxes in high-probability areas such as road surfaces and road signs using probabilistic heatmaps, thus focusing detection attention on key areas from the outset and improving sensitivity to target defects. Furthermore, it abandons the exploration of invalid background areas during iteration; when a large-area, highly suspicious defect is identified, new bounding boxes can be generated in the neighborhood for reinforcement, enhancing the completeness and accuracy of detection in complex scenes. Attached Figure Description

[0018] Figure 1 This is a flowchart of Example 1; Figure 2 A photograph of the highway and its corresponding probability heat map; Figure 3 This is a schematic diagram of the initial noise box distribution; Figure 4 This is a schematic diagram of the auxiliary noise box. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0020] The terms "first," "second," and corresponding designations in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0021] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. The terms "and / or" or the character " / " in this application are merely descriptions of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B, or A / B, can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0022] In a specific embodiment, this application proposes a method for monitoring highway infrastructure based on intelligent sensing, such as... Figure 1 As shown, it includes: Step 1: Acquire an image of the highway to be monitored and extract a feature map from the image using a backbone network; generate a probability heatmap of the highway region using the feature map; generate a noise box on the image of the highway to be monitored based on the probability heatmap; calculate the gradient variance of the noise box in the corresponding region of the feature map, and obtain the denoising budget of the noise box based on the gradient variance. Images of highways are captured using drones or vehicle-mounted cameras. In one embodiment, the images include not only the highway itself but also other highway infrastructure, such as road signs and guardrails. A backbone network is used to extract feature maps from the captured images and generate a probability heatmap. In one embodiment, the feature maps are input into a global average pooling layer, then connected to a fully connected layer to obtain the output weights. The weighted summation of each channel of the original feature map enhances the parts of the feature map related to highway infrastructure. After upsampling and normalization, the probability heatmap is obtained, indicating the likelihood of different locations in the image belonging to highway infrastructure. Figure 2As shown. In another embodiment, a simple target recognition method or image segmentation is used to segment areas containing highway infrastructure, and then these areas are assigned high heat values ​​to obtain a probability heatmap. Noise boxes are then generated based on the heatmap. In one embodiment, the image is divided into m×n equal-sized grid cells, and the average probability value of all pixels within each grid cell on the probability heatmap is calculated. Based on this average probability value, one or more noise boxes are assigned to each grid cell; cells with higher average probabilities are assigned more noise boxes, such as... Figure 3 As shown, within each cell, a specified number of noise boxes are generated by combining a small random positional perturbation with a size randomly selected from a preset size library, based on its geometric center. The original DiffusionDet model typically distributes the initial noise boxes uniformly or randomly across the entire image, which is redundant for highway monitoring. The noise box generation method in this application focuses more on areas where highway infrastructure is most likely to appear.

[0023] Simultaneously, the gradient variance or Shannon entropy of the feature values ​​within the corresponding region on the feature map is calculated. Higher gradient variance or feature entropy values ​​indicate more complex texture information and greater uncertainty in the region, potentially leading to difficult-to-identify defects. In one embodiment, the calculated feature entropy values ​​are mapped to a preset denoising budget interval using a piecewise linear function, thereby assigning an initial denoising budget related to the feature complexity of each noise box. A larger denoising budget is allocated to complex regions, allowing for fine-tuning, while a smaller budget is used for simple regions, resulting in faster stopping and avoiding redundant computation and potential adverse consequences.

[0024] Step 2: Perform iterative denoising within the number of denoising steps. In each iteration, the image decoder predicts the corrected set of noise boxes and the corresponding confidence level based on the feature map and the set of noise boxes in the current iteration step. The consumption value deducted from the denoising budget is determined according to the confidence level. If the denoising budget of the noise box is exhausted or the confidence level is lower than the preset value for N consecutive times, the subsequent denoising iterations for the noise box are stopped. During the iteration process, when determining the denoising budget for each noise box, the budget is determined based on the confidence level of the noise box. The confidence level of a noise box can be obtained in various ways, such as through an MLP network, and this application does not specifically limit this method. A higher confidence level indicates a greater likelihood that the noise box contains a target, including but not limited to cracks, damaged guardrails, etc. In one embodiment, the budget expenditure value consists of a base expenditure value and a variable adjustment value. The variable adjustment value is related to the confidence level. In one embodiment, a higher confidence level indicates a greater probability that the noise box contains a target, and less budget will be deducted to accurately locate the target. Conversely, a lower confidence level indicates a lower probability that the noise box contains a target, and more budget will be deducted to avoid repeatedly iterating over meaningless noise boxes.

[0025] The lifecycle controller records the remaining budget of the noise box and the correction history of the most recent N iterations. After each iteration, the controller checks two conditions: first, whether the remaining budget has fallen below a minimum value, such as 1; and second, whether the corrections in the correction history have all been less than a correction threshold. If either condition is met, the noise box is marked as terminated and removed from the active iteration set, no longer participating in subsequent denoising calculations. In one embodiment, stopping subsequent denoising iterations for a noise box does not mean directly deleting it, but rather adding these stopped noise boxes to the result set, where subsequent step 4 will further filter them based on confidence level, class probability, IOU, etc.

[0026] Step 3: When the number of iterations reaches the preset number, noise boxes with confidence levels lower than the pruning threshold are removed. If the confidence level of a noise box is higher than the trigger threshold and its size is greater than the size threshold, an auxiliary noise box with the same current denoising budget as the noise box is generated in the neighborhood of the noise box and added to the noise box set for the next iteration. To eliminate invalid noisy boxes early, in one embodiment, denoising is paused at checkpoints such as 30% and 60% of the total iterations, and all noisy boxes still actively iterating are traversed. The confidence score of each box is checked; if its confidence score for identifying as background exceeds a high threshold, such as 0.95, or its confidence score for identifying as any valid defect category is below a low threshold, such as 0.05, then the noisy box is determined to be invalid and permanently removed from the active set. Noisy boxes removed at this point will not proceed to step 4.

[0027] To locate small or overlapping defects, in one embodiment, a quadtree-based method is used to generate auxiliary noise boxes. Specifically, when a noise box is detected with high confidence and large size, the parent box is divided into four equal-sized sub-quadrants along its central horizontal and vertical lines. Within each of these four sub-quadrants, an auxiliary noise box half the size of its parent box is generated. The denoising budget inherited by these newly generated auxiliary noise boxes is a certain proportion of the parent box's remaining budget or its initial budget, such as 70%. Figure 4 A schematic diagram illustrating the generation of the auxiliary noise box is shown. In one embodiment, the parent noise box is discarded after the auxiliary noise box is generated.

[0028] Step 4: After the denoising steps are completed, the monitoring results of the highway infrastructure are obtained based on the output bounding box set.

[0029] The entire iterative process is conducted within a global maximum step limit to prevent infinite loops. It terminates when all boxes have terminated or the global maximum step limit has been reached, yielding a preliminary detection result set. The bounding boxes in the detection result set may overlap; for example, multiple initial noise boxes may converge to the same road crack. To obtain accurate results, further post-processing steps are performed. In one embodiment, a weighted bounding box fusion (WBF) algorithm is used. Specifically, all overlapping boxes pointing to the same object are weighted and averaged according to their confidence scores to calculate the fused new bounding box. The position, size, and final confidence score of the new bounding box are the contributions of all participating boxes. This application can be used not only for detecting road surfaces but also for detecting other infrastructure such as guardrails and traffic lights. In one embodiment, before running this application, image segmentation or semantic segmentation is performed on the captured image to retain the parts related to road infrastructure, which can further improve the accuracy of the monitoring. Those skilled in the art will understand that other processing procedures may also be included.

[0030] In an optional embodiment, generating a probabilistic heatmap of the highway area using the feature map includes: The feature maps extracted by the backbone network are input into a segmentation subnetwork consisting of three 3x3 convolutional layers; The output of the segmentation sub-network is reduced in channel dimension by a 1x1 convolutional layer to obtain a single-channel prediction map; The single-channel prediction map is upsampled to the same resolution as the image of the highway to be monitored, and then processed by the Sigmoid activation function to normalize the value of each pixel in the map to between 0 and 1, thus obtaining a probability heatmap of the highway area.

[0031] Specifically, assuming the highway image to be monitored has a resolution of 1920×1080, a backbone network such as ResNet extracts feature maps from the highway image, for example, a 60×34 feature map with 512 channels. The feature map is input into a segmentation sub-network, preferably composed of three convolutional layers, whose task is to identify features related to highway infrastructure from the 512 channels. Then, a 1×1 convolutional layer fuses the multi-channel features into a single-channel prediction map, resulting in a 60×34 condensed prediction map, where each point represents the probability that the corresponding original region is highway infrastructure. To ensure the small-sized prediction map corresponds to the original image, it is upsampled back to a resolution of 1920×1080. The entire image is then processed using a sigmoid activation function, which transforms the value of each pixel to the range of 0 to 1, thereby generating a probability heatmap, the size of which is similar to the original image. Figure 1 In this diagram, the value of each pixel, such as 0.95, represents the probability that the point belongs to highway infrastructure. The closer the value is to 1, the higher the probability that it is highway infrastructure, and vice versa.

[0032] In an optional embodiment, generating a noise box on the image of the highway to be monitored based on the probability heatmap includes: The values ​​of all pixels in the probability heatmap are normalized so that the sum of all pixel values ​​is 1. N weighted random samplings are performed from the normalized probability heatmap to obtain N pixel coordinate points, where N is the preset number of noise boxes; Centered on each sampled pixel coordinate point, and from the size range [S min ,S max The width and height of the frame are randomly selected from the frame to generate a noise box.

[0033] To ensure the noise bounding boxes are located on road infrastructure as much as possible, the values ​​of all pixels in the probability heatmap are normalized so that the sum of all pixel values ​​is 1. This increases the probability of a pixel with a higher value being selected. For example, a pixel in the center of a sign might have a value of 0.2, while a pixel in the roadside green belt might only have a value of 0.01. If N is 300, 300 samples are taken, with each sample drawing a pixel coordinate point from the image. Since the probability value of road infrastructure pixels is much higher than that of background areas, the vast majority of the 300 selected coordinate points fall on road infrastructure, not in the sky. After obtaining 300 highly correlated center coordinate points, each point is assigned a random size to generate the initial noise bounding box. Assume the preset size range is a minimum of 32 pixels and a maximum of 128 pixels. For the first selected coordinate point, a width and height are randomly selected from this range, for example, a width of 60 pixels and a height of 80 pixels, resulting in a 60x80 noise bounding box.

[0034] In an optional embodiment, obtaining the denoising budget of the noise box based on the gradient variance includes: The rectangular region covered by the noise box on the feature map is determined. The horizontal and vertical gradient values ​​of all feature pixels within the rectangular region are calculated using the 3x3 Sobel operator, and the gradient magnitude of each pixel is calculated. Calculate the variance V of the gradient magnitude of all pixels within the rectangular region. grad ; The initial denoising budget B for the noise box is calculated using the following formula: B=B min +(B max -B min )*(V grad / (V grad +V norm )); Among them, B min For the minimum noise reduction budget, B max For the maximum noise reduction budget, V norm This is the preset variance normalization constant.

[0035] A denoising budget is allocated based on the image complexity of the area covered by the noise box. Specifically, the rectangular region corresponding to the noise box on the feature map is located, and the Sobel operator is used to calculate the drastic change of each feature point within that region, obtaining the gradient magnitude. A higher gradient magnitude indicates a more drastic feature change, such as at the edge of an object or on complex textures. The variance V of all gradient magnitudes within the entire region is then calculated. gradA higher variance value indicates more drastic regional variations, which is characteristic of defects such as cracks; conversely, a very low variance value represents a region with uniform texture, such as the distant sky. Assuming a minimum denoising budget B... min The maximum noise reduction budget is 10, and the maximum noise reduction budget is B. max The value is 100, while the normalization constant V norm Set to 100. A noise box is located on a smooth and intact road surface, and the gradient change of its corresponding feature region is very small. The calculated variance V grad It might only be 0.5, but it obtains an initial denoising budget B of approximately 10.45. Another noise box covers a network of cracks, where the gradient changes significantly in the feature region, resulting in a calculated variance V. grad If the value is 400, then its initial noise reduction budget B is 82.

[0036] In an optional embodiment, determining the consumption value to be deducted from the denoising budget based on the confidence level includes: The consumption value C to be deducted in this iteration is calculated using the following formula: C = p + α * (1 - s) conf ); where α is a coefficient, p is a fixed consumption value, and s conf , where is the confidence level.

[0037] Specifically, suppose there are two noisy boxes in one iteration. Box A represents a crack, with a calculated confidence level (sconf) of 0.95. Box B is located on a complex but defect-free road surface, making it impossible to accurately determine whether it is defective or not, with a sconf of 0.2. If we set a fixed cost value p of 1 and a coefficient α of 100, then the cost value C for box A is 6, while that for box B is 81. Box A, with its high confidence, incurs very low iteration costs, while box B, with its low confidence, incurs high costs. Suppose the initial denoising budget for both boxes A and B is 200. After the current iteration, the remaining budget for box A becomes 194, while the remaining budget for box B drops sharply to 119. Low-quality, uncertain boxes like box B, if their confidence level does not improve rapidly in the next two or three iterations, will quickly exhaust their entire budget due to iteration costs and be prematurely eliminated. The high-quality frame A can sustain iterations at a very low cost, which gives it ample budget and opportunity to continue fine-tuning the position in subsequent iterations.

[0038] In an optional embodiment, generating an auxiliary noise box with the same denoising budget as the current noise box within the neighborhood of the noise box includes: Obtain the center coordinates and size of the noise box, and generate K new center coordinates by sampling with the center coordinates as the center and a standard deviation of one-quarter of the size; For each newly generated center coordinate point, respectively from the uniform distribution U(0.1*w)p 0.5*w p ) and U(0.1*h p 0.5*h p K auxiliary noise boxes are generated by randomly selecting the width and height of the noise box, and the K auxiliary noise boxes and the noise box have the same current denoising budget; where w p h is the width of the noise frame. p The height of the noise frame.

[0039] Some large defect areas may contain multiple defects. For example, a large noise box with a size of 200×80 pixels may contain various defects, such as road cracks or damaged signs. Using the center of this box as a reference, K new center points are generated. The positions of the new center points follow a Gaussian distribution, meaning most are close to the original center, with a few slightly farther away. Their dispersion is determined by one-quarter of the original box size. K smaller auxiliary boxes are generated using these new center points. Their widths are randomly selected between 20 and 100 pixels, and their heights are randomly selected between 8 and 40 pixels, forming a set of sub-region detection boxes. All newly generated auxiliary boxes inherit the current denoising budget of the parent noise box. For example, if the 200×80 pixel noise box has 75 denoising budget remaining when triggering the generation of auxiliary noise boxes, all the newly generated K smaller auxiliary boxes will begin their respective denoising iterations with a budget of 75.

[0040] In an optional embodiment, generating an auxiliary noise box with the same denoising budget as the current noise box within the neighborhood of the noise box includes: Calculate the ratio R of the current denoising budget to the initial denoising budget for the noise box. b And the ratio R of the current size of the noise box to the size threshold. s ; According to the formula K=ceil(C*R) b *(R s -1)) calculates the number of auxiliary noise boxes K, where C is the coefficient and ceil() is the rounding up function; Obtain the center coordinates and size of the noise box, and generate K new center coordinates by sampling with the center coordinates as the center and a standard deviation of one-quarter of the size; For each newly generated center coordinate point, respectively from the uniform distribution U(0.1*w) p 0.5*w p ) and U(0.1*h p 0.5*h p K auxiliary noise boxes are generated by randomly selecting the width and height of the noise box, and the K auxiliary noise boxes and the noise box have the same current denoising budget; where w p h is the width of the noise frame.p The height of the noise frame.

[0041] In this embodiment, the number of auxiliary noise boxes is related to the denoising budget and the noise box size. If the denoising budget is low, generating too many auxiliary noise boxes is not very meaningful. Similarly, if the noise box size is small, generating too many auxiliary noise boxes may even introduce errors. Assume that at a certain iteration stage, the noise box size is three times the preset size threshold, and its remaining denoising budget is 80. Calculate the budget ratio of 0.8 and the size ratio of 3, assuming a coefficient of 2, and calculate using the above formula, rounding up to determine that four new auxiliary noise boxes need to be generated. Other parts are the same as in the previous embodiment and will not be repeated here.

[0042] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some feature data may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0043] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0044] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0045] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for monitoring highway infrastructure based on intelligent sensing, characterized in that, include: Acquire images of the highway to be monitored, and extract feature maps from the images using a backbone network; A probability heatmap of the highway area is generated using the feature map; A noise box is generated on the image of the highway to be monitored based on the probability heatmap; Calculate the gradient variance of the region corresponding to the noise box on the feature map, and obtain the denoising budget of the noise box based on the gradient variance; Iterative denoising is performed within the number of denoising steps. In each iteration, the image decoder predicts the corrected set of noise boxes and the corresponding confidence level based on the feature map and the set of noise boxes in the current iteration step. The consumption value deducted from the denoising budget is determined according to the confidence level. If the denoising budget of the noise box is exhausted or the confidence level is lower than the preset value for N consecutive times, the subsequent denoising iterations for the noise box are stopped. When the number of iterations reaches a preset number, noise boxes with confidence levels below the pruning threshold are removed. If the confidence level of a noise box is higher than the trigger threshold and its size is greater than the size threshold, an auxiliary noise box with the same denoising budget as the noise box is generated in the neighborhood of the noise box and added to the noise box set for the next iteration. After the denoising steps are completed, the monitoring results of the highway infrastructure are obtained based on the output bounding box set.

2. The method according to claim 1, characterized in that, The step of generating a probability heatmap of the highway area using the feature map includes: The feature maps extracted by the backbone network are input into a segmentation subnetwork consisting of three 3x3 convolutional layers; The output of the segmentation sub-network is reduced in channel dimension by a 1x1 convolutional layer to obtain a single-channel prediction map; The single-channel prediction map is upsampled to the same resolution as the image of the highway to be monitored, and then processed by the Sigmoid activation function to normalize the value of each pixel in the map to between 0 and 1, thus obtaining a probability heatmap of the highway area.

3. The method according to claim 1, characterized in that, The step of generating a noise box on the image of the highway to be monitored based on the probability heatmap includes: The values ​​of all pixels in the probability heatmap are normalized so that the sum of all pixel values ​​is 1. N weighted random samplings are performed from the normalized probability heatmap to obtain N pixel coordinate points, where N is the preset number of noise boxes; Centered on each sampled pixel coordinate point, and from the size range [S min ,S max The width and height of the frame are randomly selected from the frame to generate a noise box.

4. The method according to claim 1, characterized in that, The step of obtaining the denoising budget of the noise box based on the gradient variance includes: The rectangular region covered by the noise box on the feature map is determined. The horizontal and vertical gradient values ​​of all feature pixels within the rectangular region are calculated using the 3x3 Sobel operator, and the gradient magnitude of each pixel is calculated. Calculate the variance V of the gradient magnitude of all pixels within the rectangular region. grad ; The initial denoising budget B for the noise box is calculated using the following formula: B=B min +(B max -B min )*(V grad / (V grad +V norm )); Among them, B min For the minimum noise reduction budget, B max For the maximum noise reduction budget, V norm This is the preset variance normalization constant.

5. The method according to claim 1, characterized in that, The step of determining the consumption value to be deducted from the denoising budget based on the confidence level includes: The consumption value C to be deducted in this iteration is calculated using the following formula: C = p + α * (1 - s) conf ); where α is a coefficient, p is a fixed consumption value, and s conf , where is the confidence level.

6. The method according to claim 1, characterized in that, The step of generating an auxiliary noise box within the neighborhood of the noise box that has the same denoising budget as the current noise box includes: Obtain the center coordinates and size of the noise box, and generate K new center coordinates by sampling with the center coordinates as the center and a standard deviation of one-quarter of the size; For each newly generated center coordinate point, respectively from the uniform distribution U(0.1*w) p 0.5*w p ) and U(0.1*h p 0.5*h p K auxiliary noise boxes are generated by randomly selecting the width and height of the noise box, and the K auxiliary noise boxes and the noise box have the same current denoising budget; where w p h is the width of the noise frame. p The height of the noise frame.

7. The method according to claim 1, characterized in that, The step of generating an auxiliary noise box within the neighborhood of the noise box that has the same denoising budget as the current noise box includes: Calculate the ratio R of the current denoising budget to the initial denoising budget for the noise box. b And the ratio R of the current size of the noise box to the size threshold. s ; According to the formula K=ceil(C*R) b *(R s -1)) calculates the number of auxiliary noise boxes K, where C is the coefficient and ceil() is the rounding up function; Obtain the center coordinates and size of the noise box, and generate K new center coordinates by sampling with the center coordinates as the center and a standard deviation of one-quarter of the size; For each newly generated center coordinate point, respectively from the uniform distribution U(0.1*w) p 0.5*w p ) and U(0.1*h p 0.5*h p K auxiliary noise boxes are generated by randomly selecting the width and height of the noise box, and the K auxiliary noise boxes and the noise box have the same current denoising budget; where w p h is the width of the noise frame. p The height of the noise frame.

8. A highway infrastructure monitoring system based on intelligent sensing, characterized in that, include: The feature extraction unit is used to acquire images of the highway to be monitored and extract feature maps of the images using the backbone network; A probability heatmap of the highway area is generated using the feature map; A noise box is generated on the image of the highway to be monitored based on the probability heatmap; Calculate the gradient variance of the region corresponding to the noise box on the feature map, and obtain the denoising budget of the noise box based on the gradient variance; The iterative unit is used to perform iterative denoising within a number of denoising steps. In each iteration, the image decoder predicts the corrected set of noise boxes and the corresponding confidence level based on the feature map and the set of noise boxes in the current iteration step. The consumption value deducted from the denoising budget is determined according to the confidence level. If the denoising budget of the noise box is exhausted or the confidence level is lower than the preset value for N consecutive times, the subsequent denoising iterations for the noise box are stopped. The optimization unit is used to remove noise boxes with confidence levels lower than the pruning threshold when the number of iterations reaches a preset number. If the confidence level of the noise box is higher than the trigger threshold and the size is greater than the size threshold, an auxiliary noise box with the same denoising budget as the noise box is generated in the neighborhood of the noise box and added to the noise box set for the next iteration. The result output unit is used to obtain the monitoring results of highway infrastructure based on the output bounding box set after the denoising steps are completed.

9. The system according to claim 8, characterized in that, The step of generating a probability heatmap of the highway area using the feature map includes: The feature maps extracted by the backbone network are input into a segmentation subnetwork consisting of three 3x3 convolutional layers; The output of the segmentation sub-network is reduced in channel dimension by a 1x1 convolutional layer to obtain a single-channel prediction map; The single-channel prediction map is upsampled to the same resolution as the image of the highway to be monitored, and then processed by the Sigmoid activation function to normalize the value of each pixel in the map to between 0 and 1, thus obtaining a probability heatmap of the highway area.

10. The system according to claim 8, characterized in that, The step of generating a noise box on the image of the highway to be monitored based on the probability heatmap includes: The values ​​of all pixels in the probability heatmap are normalized so that the sum of all pixel values ​​is 1. N weighted random samplings are performed from the normalized probability heatmap to obtain N pixel coordinate points, where N is the preset number of noise boxes; Centered on each sampled pixel coordinate point, and from the size range [S min ,S max The width and height of the frame are randomly selected from the frame to generate a noise box.

Images