Unmanned aerial vehicle road disease detection method and system based on sub-band reweighting

Abstract

The application discloses a UAV road disease detection method and system based on sub-band reweighting, and belongs to the field of road engineering intelligent detection and computer vision. In order to solve the problems of frequency aliasing and high-frequency detail loss caused by step convolution, insufficient adaptability of equal-weight multi-scale fusion, and interference of low-quality prediction box on training, a weighted bidirectional feature pyramid is arranged in the road disease detection network, and a learnable sub-band reweighting Haar wavelet down-sampling is used in the bottom-up path, the sub-band and the fusion branch are respectively adjusted through non-negative normalized weights, and the sample gradient contribution is adjusted by using a WIoUv3 during training, so that the structure, edge and texture information of the road disease are considered, and the stability of multi-scale detection and positioning is improved.

Description

Technical Field

[0001] This invention relates to the fields of intelligent road engineering inspection and computer vision, and particularly to a method and system for detecting road defects using unmanned aerial vehicles (UAVs) based on sub-band weighting. Background Technology

[0002] With the development of drone inspection and computer vision technology, using drones to collect road surface images or videos and then identifying defects such as cracks, fissures, repairs, and potholes through object detection networks has become an important technical approach for road maintenance informatization. Existing detection networks typically reduce the resolution of feature maps step by step through stride convolution and combine semantic information at different scales through feature pyramids.

[0003] However, stride convolution lacks explicit separation of different frequency information during downsampling, easily leading to frequency aliasing and attenuation of details such as crack edges, pothole contours, and local textures. From a UAV perspective, the scale differences in the damage are significant; thin cracks have weak textures and their boundaries are easily affected by complex road surface backgrounds, making it difficult for conventional equal-weighted stitching fusion to determine the contribution of branches at different scales based on the input content. Furthermore, fixed-boundary-box regression loss can easily allow gradients from outlier prediction boxes to affect network parameter updates, reducing localization stability.

[0004] Therefore, there is a need for a method and system for detecting road defects using unmanned aerial vehicles (UAVs) based on subband reweighting that can overcome the shortcomings of the existing technologies. Summary of the Invention

[0005] One objective of this invention is to propose a method and system for detecting road defects using unmanned aerial vehicles (UAVs) based on subband reweighting. Addressing the shortcomings of existing technologies, such as the lack of frequency separation during feature downsampling, the inability of conventional multi-scale fusion to adaptively determine branch contributions, and the impact of outlier prediction boxes on regression training, this invention constructs a processing chain with a road defect detection network as the main component. Non-negative and normalized learnable branch weights are configured in the top-down and bottom-up propagation paths of the neck network. A learnable subband reweighting Haar wavelet downsampling module is set in the two-stage bottom-up downsampling paths to decompose the input features into four subbands, which are then reweighted, concatenated, and projected. During training, WIoUv3 is used to adjust the gradient contribution based on the outlier degree of the samples. This invention can preserve low-frequency structure and high-frequency edge texture while controlling the channel size, allowing the contributions of features at different scales to adaptively adjust during training, and improving the localization stability of road defect bounding boxes.

[0006] This invention provides a method for detecting road defects using unmanned aerial vehicles (UAVs) based on subband reweighting, comprising:

[0007] S1. Acquire road surface images or videos collected by the UAV, and perform scale adaptation on the image frames of the road surface images or videos to obtain the road image to be detected;

[0008] S2. Input the road image to be detected into the trained road defect detection network, and extract P3 scale features, P4 scale features and P5 scale features through the backbone network;

[0009] S3. In the top-down propagation path of the neck network, according to the non-negative and normalized learnable branch weights, the P5 scale features are upsampled and fused with the P4 scale features to obtain the P4 scale intermediate features. Then, the P4 scale intermediate features are upsampled and fused with the P3 scale features to obtain the P3 scale fused features.

[0010] S4. In the bottom-up propagation path of the neck network, the P3 scale fusion feature and the subsequently formed P4 scale fusion feature are sequentially input into the learnable subband reweighted Haar wavelet downsampling module; the module uses Haar wavelet transform to obtain four types of subbands, and performs reweighting, channel splicing and projection according to non-negative and normalized learnable subband weights; the downsampling features at each level are fused with the corresponding scale features according to the learnable branch weights to obtain the P4 scale fusion feature and the P5 scale fusion feature in sequence;

[0011] S5. Input the fused features from each scale into the detection head to obtain road defect candidate boxes, defect categories, and confidence levels, thus forming the road defect detection results.

[0012] Optionally, S1 includes: reading the file format and size information of the input data, performing a decodeability check on the image file, extracting image frames from the video file according to a preset frame interval or timestamp sequence, and recording an input anomaly flag for input data that fails to be decoded and stopping subsequent processing of the input data.

[0013] The images or image frames that have passed the verification are scaled proportionally and edge-filled according to the input size of the road defect detection network, and the scaling ratio and filling offset are recorded to form the road image to be detected and its coordinate restoration parameters.

[0014] Optionally, S2 includes: using the road defect detection network constructed and trained based on YOLO11s as a feature extraction network;

[0015] The backbone network sequentially performs convolution and feature extraction on the road image to be detected, and outputs the P3 scale features, the P4 scale features, and the P5 scale features with downsampling ratios of eight times, sixteen times, and thirty-two times respectively relative to the road image to be detected;

[0016] Furthermore, the training of the road defect detection network includes: using WIoUv3 bounding box regression loss, and determining the outlier degree of the sample based on the cross-union ratio loss of the current training sample and the moving average of the cross-union ratio loss;

[0017] Non-monotonic focusing coefficients are generated based on the outlier degree of the samples, and the non-monotonic focusing coefficients are used to adjust the bounding box regression gradient contribution of the current training samples to reduce the impact of outlier prediction boxes on network parameter updates.

[0018] Optionally, S3 includes: the neck network constitutes a weighted bidirectional feature pyramid network;

[0019] The top-down propagation path upsamples the P5 scale features and fuses them with the P4 scale features to obtain the P4 scale intermediate features, and then upsamples the P4 scale intermediate features and fuses them with the P3 scale features to obtain the P3 scale fused features.

[0020] The bottom-up propagation path processes the P3-scale fused feature through the learnable sub-band reweighted Haar wavelet downsampling module, then fuses it with the P4-scale intermediate feature and the P4-scale feature to obtain the P4-scale fused feature. The P4-scale fused feature is then processed by the learnable sub-band reweighted Haar wavelet downsampling module and fused with the P5-scale feature to obtain the P5-scale fused feature.

[0021] Each fusion node is configured with learnable parameters for its input branches, positive constraints are applied using the Softplus function and normalized according to the sum of positive values, and the weighted sum of each input branch is calculated using the weights of the learned branches.

[0022] Optionally, S4 includes: the module is provided with a spatial component partitioning layer and a fixed Haar decomposition layer;

[0023] The spatial component partitioning layer divides each two-by-two local region of the input feature map into four spatial components according to the positional relationship of odd row odd column, odd row even column, even row odd column, and even row even column;

[0024] The fixed Haar decomposition layer uses a fixed Haar transform kernel to perform sum-difference combinations on the four spatial components to generate the low-frequency approximate subband, horizontal high-frequency detail subband, vertical high-frequency detail subband, and diagonal high-frequency subband.

[0025] Furthermore, the learnable subband reweighted Haar wavelet downsampling module also includes a subband reweighting layer and a projection layer;

[0026] The subband reweighting layer configures learnable parameters for each of the four subbands, uses the Softplus function to convert the learnable parameters into positive values, and normalizes each positive value by the sum of the four positive values ​​to obtain the learnable subband weights.

[0027] Multiply each learnable subband weight element-wise with the corresponding subband and then concatenate them along the channel dimension;

[0028] The projection layer sequentially performs cross-band information interaction and channel compression on the splicing result through one-to-one convolution, batch normalization, and activation functions, and outputs the corresponding downsampled features.

[0029] Optionally, S5 includes: the detection head performs classification prediction and bounding box regression on the P3 scale fusion feature, the P4 scale fusion feature and the P5 scale fusion feature respectively, outputs the category probability and bounding box parameters of each road disease candidate box for each disease category, determines the disease category corresponding to the category probability with the largest value as the disease category of the road disease candidate box, and determines the category probability with the largest value as the confidence of the road disease candidate box;

[0030] Read the scaling ratio and fill offset recorded during scale adaptation in step S1, and map the bounding box coordinates of the road defect candidate box to the original coordinate system of the road surface image or the video according to the scaling ratio and the fill offset.

[0031] Furthermore, the categories of defects include longitudinal cracks, transverse cracks, alligator cracks, diagonal cracks, repairs, and potholes;

[0032] The road defect candidate boxes with confidence scores not less than a preset confidence threshold are sorted in descending order of confidence scores. The road defect candidate box at the top of the current ranking is retained. The remaining road defect candidate boxes with the same defect category and whose intersection-union ratio with the top-ranked road defect candidate box is greater than a preset overlap threshold are deleted. The sorting, retention and deletion processes are repeated for the unprocessed road defect candidate boxes.

[0033] Generate labeled images or videos based on the retained road defect candidate boxes, structured label files containing defect categories, bounding box coordinates and confidence scores, and statistical results summarized by defect category.

[0034] On the other hand, the present invention also provides a UAV road defect detection system based on subband reweighting, comprising:

[0035] The system comprises the following modules: an input processing module for acquiring road surface images or videos collected by a drone and performing format verification, image frame extraction, and scale adaptation to obtain the road image to be detected; a multi-scale feature extraction module for extracting P3, P4, and P5 scale features using the backbone network of the trained road defect detection network; a bidirectional weighted fusion module for obtaining P4-scale intermediate features and P3-scale fusion features in a top-down propagation path according to the non-negative and normalized learnable branch weights of the fusion nodes, and obtaining P4-scale fusion features and P5-scale fusion features in a bottom-up propagation path; a sub-band reweighted downsampling module for sequentially performing Haar wavelet decomposition, sub-band reweighting, channel stitching, and projection on the P3-scale and P4-scale fusion features, and outputting the corresponding downsampling features to the bottom-up propagation path; and a defect detection output module for generating road defect candidate boxes, defect categories, and confidence scores based on the P3-scale, P4-scale, and P5-scale fusion features, and obtaining road defect detection results after confidence score filtering and overlapping candidate box suppression.

[0036] The beneficial effects of this invention are:

[0037] 1. By using a fixed Haar transform, the downsampled input is explicitly decomposed into a low-frequency approximate subband, two high-frequency detail subbands in two directions, and a diagonal high-frequency subband. The subbands are then reweighted using learnable non-negative normalized weights, which helps to preserve road structure information, crack edges, pothole contours, and local texture information while reducing the feature map resolution.

[0038] 2. By setting learnable branch weights at the fusion nodes of the top-down and bottom-up propagation paths, the P3, P4, and P5 scale features can determine the contribution of each input branch according to the training results, thereby enhancing the collaborative expression between semantic, edge, and texture information at different scales.

[0039] 3. By using WIoUv3 to determine the outlier degree of the samples based on the crossover ratio loss and its moving average, and by using the non-monotonic focusing coefficient to adjust the contribution of the bounding box regression gradient, it is beneficial to reduce the impact of outlier prediction boxes on parameter updates and improve the stability of training for the location and detection of irregular road defects. Attached Figure Description

[0040] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0041] Figure 1 This is a flowchart of the UAV road defect detection method based on subband reweighting according to the present invention.

[0042] Figure 2This is a flowchart of the S4 learnable subband reweighted Haar wavelet downsampling module of the present invention. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0044] refer to Figures 1-2 A method for detecting road defects using unmanned aerial vehicles (UAVs) based on subband reweighting includes:

[0045] S1. Acquire road surface images or videos collected by the UAV, and perform scale adaptation on the image frames of the road surface images or videos to obtain the road image to be detected;

[0046] S2. Input the road image to be detected into the trained road defect detection network, and extract P3 scale features, P4 scale features and P5 scale features through the backbone network;

[0047] S3. In the top-down propagation path of the neck network, according to the non-negative and normalized learnable branch weights, the P5 scale features are upsampled and fused with the P4 scale features to obtain the P4 scale intermediate features. Then, the P4 scale intermediate features are upsampled and fused with the P3 scale features to obtain the P3 scale fused features.

[0048] S4. In the bottom-up propagation path of the neck network, the P3 scale fusion feature and the subsequently formed P4 scale fusion feature are sequentially input into the learnable subband reweighted Haar wavelet downsampling module; the module uses Haar wavelet transform to obtain four types of subbands, and performs reweighting, channel splicing and projection according to non-negative and normalized learnable subband weights; the downsampling features at each level are fused with the corresponding scale features according to the learnable branch weights to obtain the P4 scale fusion feature and the P5 scale fusion feature in sequence;

[0049] S5. Input the fused features from each scale into the detection head to obtain road defect candidate boxes, defect categories, and confidence levels, thus forming the road defect detection results.

[0050] In this specific embodiment, S1 includes:

[0051] The UAV road inspection processing terminal reads road surface images or videos from the flight mission catalog and writes the mission number, flight segment number, absolute file path, file extension, number of bytes, shooting timestamp, and camera intrinsic parameter version into the input index record. In this specific embodiment, the image files are in JPEG or PNG format, and the video files are in MPEG version 4 video encapsulation format. The inspection camera collects data in a vertically downward or tilted at no more than 15 degrees relative to the road normal. The processing terminal removes duplicate files according to the combination key of shooting timestamp and flight segment number, and keeps the original files read-only.

[0052] The processing terminal calls the decoder corresponding to the file format to read the image width, image height, number of channels, bit depth, video frame rate, total number of video frames, and encoding type. It performs complete decoding on the image and verifies that the decoded pixel matrix is ​​not empty. For the video, it sequentially verifies the file header, keyframe index, and the first and last decodeable frames. When the file extension is inconsistent with the decoder's recognition type, either the width or height is zero, the pixel matrix is ​​missing, or three consecutive keyframes fail to decode, the file identifier, failure stage, and error code are written to the input anomaly record, the input anomaly flag is set, and subsequent processing of the input data is stopped.

[0053] For videos that pass verification, the processing terminal establishes a target timestamp sequence at a preset frame interval of 0.5 seconds, and decodes to the target timestamp starting from the nearest keyframe no later than each target timestamp, obtaining a road image frame with a flight segment number, video number, frame sequence number, and original timestamp. The processing terminal verifies the decoding return code, actual arrival timestamp, and pixel matrix for each target timestamp one by one. When any target timestamp results in a decoder return error, failure to reach the target timestamp, or an empty pixel matrix, the source file number, target timestamp, failure stage S1, and error code are written to the input exception record, and the input data is set. If an input anomaly flag is detected, the remaining target timestamp of the video and processing from S2 to S5 are stopped, and the failure status is registered in the task result list. For the failed video, only the troubleshooting index of the decoded image frames before the failure is retained. They are not written to the inference cache, not passed to S2, and no labeled media, disease object tags, or disease statistics are generated, thus distinguishing them from the normal frame extraction results. When the actual frame rate of the video changes, the target timestamp is converted according to the time base instead of being extracted according to the fixed frame number, so that the acquisition time difference between adjacent output frames is kept at 0.5 seconds. Image frames with a tail of less than 0.5 seconds and that are repeated with the previous output frame are discarded.

[0054] The processing terminal converts the verified single image or extracted image frame into a three-channel RGB pixel matrix, retaining the original width, original height and original coordinate origin, and using the target width of 640 pixels and the target height of 640 pixels in the detection network configuration file as the scale adaptation benchmark; for single-channel input, the grayscale channel is copied to form a three-channel matrix; for input containing transparent channels, channel synthesis is first completed according to an opaque black background, and when the pixel value exceeds the limit, it is truncated to 0 to 255;

[0055] Scale adaptation module according to Calculate the scaling ratio and follow... and Calculate the scaled width and height; symbol This represents the scaling ratio shared in the horizontal and vertical directions of the same image, and is a dimensionless positive number. (Symbol: ...) Indicates the width of the target in the detection network, expressed in pixels, symbol Indicates the target height detected by the network, expressed in pixels, symbol Represents the width of the original image in pixels, symbol Represents the height of the original image in pixels, symbol This represents the scaled image width, which is at least one pixel, expressed in pixels. (symbol) The function represents the height of the scaled image, which is at least one pixel in size and is expressed in pixels. The function represents the minimum of two positive values. The function returns the maximum of the two input values. This indicates taking the nearest integer pixel value, a constant. This indicates the minimum number of pixels allowed in a single direction after scaling; the processing terminal uses bilinear interpolation to achieve proportional scaling without changing the aspect ratio.

[0056] Scale adaptation module according to and Determine the edge fill offset; symbol This represents the actual left-side integer fill amount, in pixels, written to the coordinate restoration parameter record and used by S5. (symbol) This represents the actual integer fill amount, in pixels, written to the coordinate restoration parameter record and used by S5. Indicates rounding down; right fill amount is... The amount of filler on the lower side is taken The fill value is set to 114, thus obtaining a 640x640 pixel image of the road to be detected.

[0057] The processing terminal loads a disease category dictionary bound to the network version. The dictionary sequentially registers six category names, category numbers, drawing colors, and statistical field names for longitudinal cracks, transverse cracks, alligator cracks, diagonal cracks, repairs, and pits. If any category is missing or a duplicate category number appears in the category dictionary, the network version is marked as invalid and the task is stopped to enter the inference phase.

[0058] The processing terminal simultaneously reads the post-processing configuration record. The record fields include the preset confidence threshold, preset overlap threshold, sorting direction, tie-breaking rules, and effective network version. The higher the confidence value, the greater the probability that the candidate box belongs to the determined disease category. Therefore, the post-processing always uses descending confidence. When the confidence values ​​are the same, the candidate box area is used to sort in descending order and the generation number is used to sort in ascending order. The resulting rules are written into the post-processing configuration reference field of the coordinate restoration parameter record.

[0059] The post-processing configuration specifies that only road defect candidate boxes with a confidence level not less than a preset confidence threshold are sent to overlap suppression. The road defect candidate box currently ranked first is retained, and other road defect candidate boxes with the same defect category and an intersection-union ratio greater than a preset overlap threshold with the currently ranked first candidate box are deleted. Unprocessed road defect candidate boxes are repeatedly sorted, retained, and deleted. The input of this filtering rule is the candidate box set generated by S5, and the output is the retained candidate box set. The candidate box with low confidence is not used to replace the ranked first candidate box.

[0060] The coordinate restoration parameter record adopts a one-to-many structure of image-level master record and candidate box-level result sub-record. The image-level master record is associated with an original road surface image or a video frame through the source file number and frame sequence number. It only saves the original size, scaling size, scaling ratio, horizontal and vertical fill amount, original timestamp and post-processing configuration reference, and sets the candidate box result array. S5 creates a candidate box-level result sub-record in the array for each final retained candidate box according to the candidate box unique key. The disease category, the original coordinates upper left horizontal coordinate, upper left vertical coordinate, lower right horizontal coordinate, lower right vertical coordinate and confidence score are added to the sub-record as a consistent submission. Fields of other candidate boxes are not reused or overwritten. Only after successful submission is the sub-record added to the set of output candidate box-level records and used uniformly for drawing, labeling and statistics. When submission fails, the corresponding candidate box is excluded from the three types of output. Its unique key, failure stage and error code are recorded, and the failure is not interpreted as normal no disease.

[0061] The task output definition record specifies that the output candidate box level record set formed by the consistent write-back and aggregation in S5 is used as the sole data source. It generates labeled images or videos, structured label files containing disease categories, bounding box coordinates and confidence scores, and quantity statistics summarized by disease category. The retained candidate box set after overlap suppression is only used as an intermediate set before write-back. Candidate boxes that fail to write back are not included in drawing, labeling or statistics. The same candidate box is uniquely keyed by the task number, source file number, frame number and candidate box generation number. This unique key is verified in drawing, label writing and quantity summarization to avoid the same disease candidate box being counted repeatedly.

[0062] The processing terminal establishes a master record of image-level coordinate reconstruction parameters for each road image to be detected. The record fields include task number, flight segment number, source file number, frame number, original width, original height, scaled width, scaled height, scaling ratio, horizontal fill amount, vertical fill amount, original timestamp, post-processing configuration reference, and an initially empty candidate box result array. The road image to be detected is named with the master record primary key and written to the inference cache. The image-level master record is written to the task-level metadata file. Both are passed to S2 with the same primary key. S5 uses the master record to complete the coordinate reconstruction and appends zero or more candidate box level result sub-records to the array. The successfully submitted sub-records in the array are used to generate labeled images or videos, structured label files, and classification quantity statistics.

[0063] In this specific embodiment, S2 includes:

[0064] The multi-scale feature extraction module takes the road image to be detected written to the inference cache by S1 as input, reads the network version record bound to the task and loads the road disease detection network built on YOLO11s as the base network and trained on road disease samples; the network version record includes weight file hash, category dictionary version, input size, channel order, normalization parameters, training data version and enabling status. If the hash verification fails or the category dictionary does not contain the six types of road diseases, the inference of this version is terminated and the model loading anomaly is recorded.

[0065] The module converts a 640x640 pixel RGB pixel matrix into a floating-point tensor with a batch size of one, three channels, and a height and width of 640, and then follows the... Complete pixel-by-pixel normalization; symbol Indicates channel On the go List The normalized pixel value at that location is a dimensionless number between zero and one, with the sign... Represents the eight-bit pixel value at the same position, with values ​​ranging from zero to 255, sign Represents the RGB channel index, ranging from zero to two, with the sign... Represents the image row index, ranging from zero to 639, symbol Represents the image column index, ranging from zero to 639, a constant. Indicates the maximum value of an eight-bit pixel;

[0066] The backbone network performs forward propagation in the following order: convolutional downsampling layer, C3k2 feature extraction layer, spatial pyramid pooling layer, and channel spatial attention layer of YOLO11s. The convolutional layers use a combination of convolution, batch normalization, and SiLU activation. The convolution with a stride of 2 is responsible for halving the resolution. The C3k2 layer maintains the transmission of shallow texture information and deep semantic information through cross-stage partial connections. In this specific implementation, the network inference uses 32-bit floating-point weights, and the input batch does not mix images from other flight segments.

[0067] After the road image to be detected is successively downsampled three, four, and five times by a factor of 2, the backbone network outputs from the corresponding layers respectively. Scale characteristics Scale features and Scale features; the downsampling ratios of the three relative to the input image are eight times, sixteen times and thirty-two times respectively. In this specific embodiment, the spatial dimensions are eighty by eighty, forty by forty and twenty by twenty respectively. The number of output channels is determined by the network version record. Subsequently, they are all projected to two hundred and fifty-six channels through a one-to-one convolution for neck network fusion.

[0068] The feature extraction module encapsulates each scale feature into a multi-scale feature record. The record fields include task primary key, level identifier, downsampling factor, tensor shape, data type, network version, and tensor cache address. It also verifies that the input width and input height bound to the network version record are both 640 pixels, and that the height and width of adjacent levels satisfy a 2x relationship. When the input width or input height of the network version record is not 640 pixels, the module marks the network version as invalid, records the abnormal model input size configuration, and stops the frame from entering S3. If the size relationship between any adjacent levels is not valid or is not finite, the feature extraction of the frame is marked as failed and does not enter S3. At the same time, the failure stage S2 and the corresponding error code are written into the frame processing status record, and the task result list is registered. The labeled media, disease object tags, or disease statistics items for the frame are not generated.

[0069] The training samples for the road defect detection network were collected by the same road inspection camera under sunny, cloudy and different illumination conditions. The labeled objects were limited to longitudinal cracks, transverse cracks, alligator cracks, diagonal cracks, repairs and potholes. The label file recorded the defect category and the bounding rectangle. The data was divided into training set, validation set and test set according to road segments, with a ratio of 7:2:1. Adjacent frames of the same segment only entered the same set to avoid leakage of similar texture frames across sets. During training, random brightness adjustment, horizontal flipping and scale jitter were performed and the bounding boxes were updated synchronously.

[0070] During the training phase, classification loss, distribution focus loss, and WIoUv3 bounding box regression loss are used to jointly update the network parameters. The batch size is set to 16, the initial learning rate is set to 0.1%, and cosine decay is used for training for 300 epochs. The early stopping condition is that the average accuracy of the validation set does not improve for 20 consecutive epochs. Temperature calibration is performed on the confidence of the validation set. The resulting temperature parameters and six class dictionaries are written into the network version record along with the weight file. Only versions with finite validation set loss and consistent weight file hash are enabled.

[0071] After the multi-scale feature extraction module completes the forward propagation, it will... Scale characteristics Scale features and The scale features, along with references to the network version, task key, and coordinate reconstruction parameters, are written into the feature cache. Scale features preserve the edges of fine cracks and the texture of small objects. Scale characteristics support medium receptive field structures. Scale features carry the semantics of the road surface area. The three are passed to S3 as an indivisible feature group of the same frame. If any scale is missing, the entire frame will not enter the subsequent neck network.

[0072] In this specific embodiment, S3 includes:

[0073] The neck network in steps S3 and S4 forms a weighted bidirectional feature pyramid network, and the neck network fusion module reads the same frame generated in step S2. Scale characteristics Scale features and After verifying that the three tasks, primary keys, network versions, and batch identifiers are consistent, the scale features enter the top-down propagation path of the weighted bidirectional feature pyramid network. Each input feature is first unified into 256 channels through an independent one-to-one convolution. After convolution, batch normalization and SiLU activation are performed to ensure that the branches to be added have the same data type, number of channels, and spatial size.

[0074] The larger the normalized branch weight value in each fusion node, the greater the contribution ratio of the corresponding input branch in the weighted sum; the smaller the weight value, the smaller the corresponding contribution ratio. The branch weight is only used for feature weighting in the same spatial location and the same channel, and does not represent risk, confidence or candidate priority. Input branches are not deleted based on this, and the fusion result is fixedly written to the current scale feature cache.

[0075] for Scale features to To transfer the scale, the fusion module uses nearest neighbor interpolation to scale the 20x20 matrix. The scale feature upsampling is 40x40, and the upsampling coordinates are back-mapped to the source grid along the center of the target grid, copying the feature values ​​of the nearest position; the upsampling result is compared with the channel-projected result. The scale features form the two input branches of the first fusion node. If the spatial size of the two branches differs by one pixel, only the right or bottom boundary is cropped. If the size difference exceeds one pixel, the fusion size is recorded as abnormal and the processing of the frame is stopped. At the same time, the failure stage S3 and the corresponding error code are written into the frame processing status record and the task result list is registered. The labeled media, disease object label or disease statistics item of the frame is not generated.

[0076] Nearest neighbor sampling according to Copy source feature value; symbol This indicates that the upsampling result is in the target row. Target Column and channels The dimensionless network internal eigenvalues ​​at the location, with symbols This represents the dimensionless internal feature value of the source feature at the corresponding row, column, and channel within the network. The symbol is... Represents the target feature row index, ranging from zero to thirty-nine, with the symbol... Indicates the index of the target feature column, ranging from zero to thirty-nine, with the sign... The function represents the channel index, ranging from zero to 255. This indicates rounding down; this rule does not introduce a new numerical scale, the output tensor is written to the upsampling buffer and the primary key recorded with the source features is used to maintain the association;

[0077] The first fusion node targets branches. according to Obtain a non-negative response, and follow the instructions. Obtain the normalized branch weights; symbol and symbols Both represent branch indexes and take one of them. The first fusion node Take two, symbol Indicates the first The positive responses of each branch are dimensionless, and the sign is... Indicates the first The learnable parameters of each branch are dimensionless real numbers, with the sign... This represents a dimensionless constant used to prevent a zero response, and in this specific embodiment, it is set to 10 to the power of negative 4. The symbol is... Indicates the first The dimensionless normalized weights of each branch are greater than zero and satisfy the condition that the sum of the weights is one. (Signature) The function represents the number of input branches of the current fusion node. Represents the natural logarithm, a function This represents the natural exponential function; the learnable parameters are initially set to zero and updated via backpropagation during network training.

[0078] Fusion module according to calculate Scale intermediate features; symbols express The scale intermediate features are dimensionless network internal feature tensors, with the sign... This represents a combination of 3x3 convolution with a stride of 1 and padding of 1, batch normalization, and SiLU activation. The symbol is... express Dimensionless normalized weights of the upsampling branch, sign express Dimensionless normalized weights of the input branch, function This indicates an upsampling operation performed using twice the nearest neighbor interpolation, denoted by [symbol]. Indicates the channel projection Scale features and are dimensionless internal feature tensors of the network, with sign... Indicates the channel projection The scale features are dimensionless internal feature tensors of the network; the convolution output is written into an intermediate feature buffer of 40 x 40 x 256.

[0079] The fusion module will The intermediate features are upsampled by a factor of 2 to 80x80 and then projected onto the channels. Scale features form the second fusion node, which uses independent learnable parameters and repeats the positive value constraint and normalization process; according to... get Scale fusion features, symbols express Scale-fused features are dimensionless internal feature tensors of the network, with sign... express Dimensionless normalized weights of the scale intermediate feature upsampling branch, sign express Dimensionless normalized weights of the input branch, sign Following the aforementioned interpretation of the dimensionless network internal feature tensor, the notation... Indicates the channel projection The node has scale characteristics and is a dimensionless network internal feature tensor; both weights of this node are non-negative and their sum is one.

[0080] During the training of the road defect detection network, the training module will... The set of valid regression sample indices consists of the sample indices in the current training batch that have valid labeled bounding boxes and have completed the forward computation of the regression branch, corresponding to the parameter update. For each ,according to Calculate the crossover ratio loss; symbol Indicates the training parameter update sequence number, symbol Indicates the first The secondary parameter update uses a set of valid regression sample indices, and the set members are unitless and symbolic. Indicates the sample index within this set, symbol Indicates the first When the second parameter is updated, the first The dimensionless intersection-union loss of a valid regression sample, taking values ​​from zero to one, is a function. This represents the dimensionless intersection-union ratio (IUU) between the predicted bounding box and the labeled bounding box, with values ​​ranging from zero to one. The sign is... This represents the predicted bounding box of the effective regression sample, denoted by [symbol]. The bounding box of the effective regression sample is represented; the larger the intersection-union ratio, the smaller the intersection-union loss, and the loss of each sample is written into the training batch statistics record;

[0081] The training modules respectively obtained the first The predicted bounding boxes and labeled bounding box center coordinates of each valid regression sample are obtained, and the width and height of the smallest bounding rectangle that can simultaneously enclose both are obtained, according to... Calculate the distance attention factor; symbol Indicates the first When the second parameter is updated, the first The WIoU distance attention factor of each effective regression sample, which is a dimensionless positive number, with sign... and symbols These represent the x and y coordinates of the center of the predicted bounding box for this sample, respectively, in pixels. The symbol is... and symbols These represent the x and y coordinates of the center of the bounding box labeled in the sample, respectively, in pixels. and symbols These represent the width and height of the smallest bounding rectangle of the predicted bounding box and the labeled bounding box, respectively, in pixels. ,symbol Its index range follows the aforementioned definition, function Represents the natural exponential function, superscript Indicates will This is used as the scale for stopping the gradient; the actual value of this scale is used during forward propagation, but the gradient is not transmitted to the bounding box coordinates through this scale during backward propagation.

[0082] Training module according to Forming the WIoUv1 basic regression loss; symbol Indicates the first When the second parameter is updated, the first Each effective regression sample is derived from the dimensionless WIoUv1 basic regression loss modulated by the distance attention factor, denoted by [symbol missing]. and symbols Following the aforementioned interpretation, this basic loss still provides computable geometric modulation through the center distance attention factor when the predicted box and the labeled box do not intersect, and serves as the input for the subsequent WIoUv3 non-monotonic focusing mechanism;

[0083] The training module establishes a stable reference outside the current batch using the moving average of the batch intersection-union ratio loss, and sets the initial value in the training state record to an uninitialized moving average initialization flag; when the set of valid regression sample indices is not empty, it follows... Determine the number of valid regression samples and follow Calculate the first The batch crossover ratio loss mean is updated for the secondary parameter; when the initialization flag is uninitialized, let... And change the flag to initialized. When the flag is initialized, proceed according to... Update the moving average; symbol Represents a set The number of members is a positive integer, and the sign is... Represents a set cardinality, sign Indicates the first The dimensionless batch intersection-union ratio loss mean of the secondary parameter update, where the summation operation represents the summation of the set All sample indexes Summation, symbol Indicates the first The dimensionless crossover ratio loss moving average after secondary parameter update, sign This represents the dimensionless moving average coefficient set to 0.9 in this specific embodiment, with the symbol... This represents the dimensionless cross-union ratio loss moving average after the previous parameter update; it is not created when the set is empty and the initialization flag is uninitialized. Furthermore, outlier assessment, focus coefficient, and regression loss calculations are not performed. When the set is empty and the initialization flag is initialized, the updated parameters from the previous parameter are used. As the current moving average state;

[0084] The training module for each according to Determine the degree of outlier in the sample, and follow... Generate non-monotonic focusing coefficients, and then according to... Apply this focusing coefficient to the WIoUv1 basic regression loss to obtain the sample-level WIoUv3 regression loss; function This represents the stopping gradient operation where the input value remains unchanged during forward propagation and the derivative is always zero during backward propagation. (Symbol: ...) Indicates the first When the second parameter is updated, the first The outlier of each effective regression sample relative to the moving average crossover ratio loss is a dimensionless, non-negative number, with the sign of... This represents a dimensionless constant, set to 10 to the power of negative 6, that guarantees the sum of the moving average loss and this constant is greater than zero. This indicates that the effective regression sample after gradient stopping has non-monotonic focusing coefficients that are dimensionless and non-negative, with the sign... This represents the dimensionless normalization scale and exponential shift parameter set to three, with the symbol... This indicates that a dimensionless exponential base parameter of 1.9 is set to determine the position of the focusing peak, and the outlier degree corresponding to the peak value of the focusing coefficient is... ,symbol Indicates the first When the second parameter is updated, the first The dimensionless sample-level regression loss after focusing a valid regression sample using WIoUv3, denoted as: Following the aforementioned interpretation; during reverse propagation Only used as the gradient of the WIoUv1 basic regression loss with fixed scalar scaling, without... or Calculating the branch produces additional derivative terms;

[0085] Before a joint loss is formed, when When the value is greater than zero, the training module follows the... Calculate batch-level regression loss; symbol Indicates the first The dimensionless batch regression loss is obtained by calculating the arithmetic mean of the WIoUv3 regression losses of all samples within the valid regression sample index set during secondary parameter updates. (Signature) ,symbol ,gather The range of summation follows the aforementioned definition; when When it equals zero, and All values ​​are set to zero, and the gradient contributions of the bounding box regression branch and the distribution focus loss branch are skipped. The classification loss branch still calculates the gradients for the background anchors and class targets in the current training batch. The training module is based on Formation of joint loss; symbol Indicates the first The dimensionless joint loss for secondary parameter updates, sign... This represents the dimensionless classification loss weight set to 0.5, with the symbol... This represents the dimensionless classification loss of the current training batch, with the symbol... This represents the focal loss weight set to a dimensionless distribution of 1.5, with the symbol... This represents the dimensionless distribution focal loss of the current training batch, with the sign... This represents the dimensionless bounding box regression loss weight set to 7.5; the joint loss is passed to the optimizer to calculate the gradient of the network parameters.

[0086] The optimizer uses the joint loss as the objective function and adopts the AdamW parameter update rule. It performs gradient calculation and parameter update once for each training batch and controls the update step size with the initial learning rate and cosine decay plan set by S2. When the joint loss, gradient norm or updated weights have non-finite values, the parameter update of this batch is canceled, the checkpoint before the update is restored, and the batch primary key is written to the training anomaly record.

[0087] The training module uses joint loss for backpropagation, allowing the learnable branch parameters of the two fusion nodes from top to bottom to be updated together with the backbone network, neck network, and detector head. After each training round, it verifies that all normalized branch weights are finite and that the weights and biases do not exceed ten to the power of negative six. During the inference phase, the parameters after training convergence are fixed, and finally... Scale intermediate features and The scale-fused features, along with the weights of each node, feature shapes, and network version, are written into the top-down fusion record. The scale fusion features are passed to the first-level Haar wavelet downsampling module of S4. The intermediate scale features serve as the corresponding scale inputs for the first-level bottom-up fusion nodes of S4.

[0088] In this specific embodiment, S4 includes:

[0089] The neck network in steps S3 and S4 forms a weighted bidirectional feature pyramid network, and the bottom-up pass-through module reads the output of S3. Scale fusion features Scale intermediate features and S2 cache Scale features and Scale characteristics, and lock the same frame of data according to the task key; the module first divides the 80x80... The scale-fused features are input into the first-level learnable sub-band reweighted Haar wavelet downsampling module to obtain 40x40 first-level downsampling features, which are then used to form... The scale fusion features are used as the second-level input to obtain a 20x20 second-level downsampled feature;

[0090] The larger the subband weight value, the greater the contribution of the corresponding frequency band features to the channel stitching result; the larger the fusion node branch weight value, the greater the contribution of the corresponding scale input to the weighted sum; both types of weights are non-negative normalized weights, which only control the contribution of feature amplitude and do not represent risk, confidence or candidate priority. All four types of subbands and each scale branch participate in subsequent projection or fusion.

[0091] For any level of input features The spatial component partitioning layer is based on the upper left corner of each 2x2 local region, according to... , , and Divide into four spatial components; symbol This represents the current-level input features with axis order of batch, channel, row, and column, and is a dimensionless internal feature tensor of the network. In the slicing expression, the first two single colons indicate that all batch and channel indices are retained. The number two after the double colons indicates that sampling begins at a specified row or column starting index with a step size of two. Both the row and column starting indices are either zero or one. The symbol represents the internal feature tensor of the dimensionless network composed of odd-row and odd-column positions. The symbol represents the internal feature tensor of the dimensionless network composed of odd-row and even-column positions. The symbol represents the internal feature tensor of the dimensionless network consisting of even-row and odd-column positions. The dimensionless network internal feature tensor represents the even-numbered rows and even-numbered columns. The two subscripts represent the row position index and column position index in the local region, respectively, with zero representing the first position and one representing the second position. The tensor index is counted starting from zero, so the odd-numbered rows and even-numbered rows mentioned in the text correspond to this zero-based index according to the human row number.

[0092] Fixed Haar decomposition layer according to , , and Generate four types of subbands; symbols This represents a low-frequency approximate subband and is a dimensionless feature tensor within the network; the symbol is... This represents a horizontal high-frequency detail subband and is a dimensionless feature tensor within the network; the symbol is... Represents the vertical high-frequency detail subbands and is a dimensionless feature tensor within the network, denoted by [symbol]. Represents a diagonal high-frequency subband and is a dimensionless feature tensor within the network; symbol ,symbol ,symbol and symbols Using the aforementioned interpretation of the internal feature tensor of the dimensionless network, the four types of subbands reduce the spatial width and height to half, and the constant... The scale normalization factor represents the orthogonal Haar basis; the fixed transformation kernel does not participate in gradient updates;

[0093] When the height or width of the input feature is odd, the spatial component partitioning layer only copies the outermost row or column on the bottom or right side, making the padded width and height even, and writes the padded flag and the original size into the downsampling record; when the road defect detection network uses a 640-pixel input, Scale fusion features and The spatial dimensions of the scale fusion features are 80x80 and 40x40 respectively, without triggering the completion branch, and the four types of sub-bands maintain a one-to-one correspondence in the channel dimension and spatial dimension.

[0094] Subband reweighting layer is the subband index Configure independent learnable parameters, according to Obtain a positive value, and follow Obtain learnable subband weights; symbol and symbols All represent sub-band indices, ranging from one to four, corresponding to low-frequency, horizontal high-frequency, vertical high-frequency, and diagonal high-frequency sub-bands, respectively. The symbols are... Indicates the first The positive response of the subband is dimensionless, and the sign is... Indicates the first The learnable parameters of the subband are dimensionless real numbers, with the sign of... Indicates the first The dimensionless, non-negative, normalized weights of the subbands are such that the sum of the four weights is one; this formula follows the complete interpretation of the S3 isomorphic positive value constraint formula, with the symbol... The function represents a dimensionless constant, set to the power of 10 to the power of negative 4. Represents the natural logarithm, a function This represents the natural exponential function; all four parameters are initially set to zero, and the first and second level modules do not share parameters.

[0095] The reweighted layer multiplies the weight of each learnable subband element-wise with the corresponding subband, and then... Complete the splicing along the channel dimension; symbol The function represents the channel splicing tensor of the four types of reweighted subbands, which is a dimensionless feature tensor of the network. This indicates a channel-dimensional splicing operation that maintains the batch and spatial position unchanged. (symbol) The dimensionless weight of the low-frequency approximate subband is represented by the symbol. The dimensionless weights of the horizontal high-frequency detail subbands are represented by the symbol. The dimensionless weights of the vertical high-frequency detail subbands are represented by the symbol. Represents the dimensionless weight of the diagonal high-frequency subband; symbol ,symbol ,symbol and symbols Using the aforementioned interpretation of the internal feature tensor of the dimensionless network, the number of channels in the spliced ​​tensor is four times the number of input channels;

[0096] Projection layer according to Perform cross-band information exchange and channel compression; symbols This represents the downsampled features output by the current-level Haar wavelet downsampling module, which are dimensionless internal feature tensors of the network. The symbol is... Following the aforementioned interpretation of the internal feature tensor of dimensionless networks, the function The function represents a 1x1 convolution with a stride of 1, compressing the number of channels to 256. This indicates the use of batch normalization of the moving mean and moving variance during the training phase. The function... This represents the SiLU activation function; the batch normalization parameter is fixed during inference, and the output space size is half the input space size;

[0097] First-level downsampling features, Scale intermediate features and channel projection The scale features form three input branches. The fusion node is configured with independent learnable parameters for each of the three branches and follows the positive value constraints and normalization rules of S3. get Scale fusion features; symbols express Scale-fused features are dimensionless internal feature tensors of the network, with sign... The dimensionless normalized weights of the first-level downsampling feature branches are represented by the symbol. express Dimensionless normalized weights of intermediate feature branches at scale, sign express Dimensionless normalized weights of scale input feature branches, sign Indicates by The first-level downsampled features generated by scale fusion are dimensionless internal feature tensors of the network, with the sign... and symbols Following the interpretation of the dimensionless network internal feature tensor described in S3, all three weights are non-negative and the sum of the weights is one.

[0098] The bottom-up transmission module will The scale-fused features are input into the second-level learnable subband reweighted Haar wavelet downsampling module. This second-level module repeats the spatial component partitioning, fixed Haar decomposition, subband reweighting, channel stitching, and projection processes. The resulting second-level downsampling features are then compared with the channel-projected features. Scale features enter the two-branch fusion node and follow... get Scale fusion features, symbols express Scale-fused features are dimensionless internal feature tensors of the network, with sign... The dimensionless normalized weights of the second-level downsampling feature branches are represented by the symbol. express Dimensionless normalized weights of scale input feature branches, sign Indicates by The scale-fusion feature generates a second-level downsampled feature, which is a dimensionless internal feature tensor of the network, with the sign... Using the interpretation of the dimensionless network internal feature tensor described in S3, both weights are non-negative and the sum of the weights is one;

[0099] During network training, the regression gradient contribution of bounding boxes is adjusted according to the outlier degree and non-monotonic focusing coefficient of WIoUv3 samples determined by S3. This regression gradient is backpropagated through the detection head and neck network to the subband parameters, projection layer parameters, and branch parameters of the bottom-up fusion node of the two Haar wavelet downsampling modules. The gradient contribution of severely outlier predicted boxes is suppressed, samples with moderate outlier degree retain the main update role, and the parameter gradient of the fixed Haar transform kernel is always turned off.

[0100] After each training epoch, the training module checks that the weights of the four subbands and the branch weights of each fusion node are finite non-negative values, and checks that the deviation of the corresponding weights from one does not exceed ten to the power of negative six. If the check fails, the parameters of that epoch are discarded and the weights of the previous validation epoch are restored. When the validation set loss improves, the weights of the first-level subband, the second-level subband, and... Merge node branch weights The branch weights of the fusion nodes, along with the network version, are written into the model checkpoints, enabling the contributions of each frequency band and scale branch to be reproduced in the same version during the inference phase.

[0101] After the bottom-up pass is completed, the module will Scale fusion features Scale fusion features and The scale fusion features are encapsulated into a fusion feature group, recording the spatial size, number of channels, cache address, two-level subband weights, branch weights of each fusion node, network version, and task key of each frame. The three scale tensors should be 80x80x256, 40x40x256, and 20x20x256, respectively. If any frame is missing or has a non-finite value, the entire frame output is invalid and will not be passed to S5. At the same time, the failure stage S4 and the corresponding error code are written into the frame processing status record, and the task result list is registered. The labeled media, disease object label, or disease statistics item for that frame is not generated. When the verification passes, the fusion feature group is passed to the detection head of S5.

[0102] In this specific embodiment, S5 includes:

[0103] The disease detection output module reads the same frame generated by S4. Scale fusion features Scale fusion features and The scale fusion features are fed into the decoupled detection head. Each scale detection head includes a classification branch and a bounding box regression branch. The classification branch outputs the classification logic value for each grid position for the six disease categories, and the regression branch outputs the four-sided distance distribution relative to the grid center. The output record saves the scale identifier, grid row and column, network version and source image primary key.

[0104] The bounding box regression branch outputs sixteen discrete logic values ​​for each of the four directions: left, top, right, and bottom; for each direction... and gear First, according to Obtain a numerically stable baseline, and then follow Calculate the Softmax probability, and follow... Calculate the desired gear distance; symbol Indicates left, top, right, or bottom direction and units are not applicable; symbol and symbols Represents a discrete gear index, all ranging from zero to... ,symbol This represents the total number of discrete gears, and in this specific implementation, it is taken as sixteen. (symbol) and symbols This represents the dimensionless regression logistic value for the corresponding direction and gear, with the symbol... Indicates direction The dimensionless maximum value among all finite logic values, sign Indicates direction In gear The dimensionless probability on the same direction sums to one, sign Indicates direction The dimensionless expected gear distance and satisfying ,function This function retrieves the maximum value within a specified index range. This represents the natural exponential function, and the summation operation covers all explicitly stated gear indices. When a non-finite value appears in any direction of the logical value, the output module removes the four-directional regression results for that grid position, writes the scale identifier, grid row and column, and non-finite error code for the regression logical value, and does not generate a candidate box for that grid position.

[0105] For grid positions where all logical values ​​are finite, the output module follows... Convert the desired gear distance to pixel distance, and then... and Calculate the grid center in the coordinate system of the road image to be detected, and then divide the grid into four corresponding directions: left, top, right, and bottom. Recorded in sequence , , and ,according to , , and Get candidate boxes; symbol Indicates direction The predicted distance in pixels, symbol This represents the downsampling rate at the current detection scale, expressed in pixels per grid interval. , and The scales are taken as eight, sixteen, and thirty-two respectively, with the symbol... Represents a dimensionless grid column index that satisfies ,symbol Represents a dimensionless grid row index that satisfies ;exist , and Scale, symbol and symbols The integer values ​​of are respectively zero to seventy-nine, zero to thirty-nine, and zero to nineteen; the sign and symbols Representing the x and y coordinates of the grid center, respectively, in pixels, constants. This represents a dimensionless half-grid offset from the top left boundary of the grid to the center of the grid, denoted by [symbol]. ,symbol ,symbol and symbols The symbols represent the predicted left, up, right, and down distances in pixels, respectively. ,symbol ,symbol and symbols The coordinates of the left, top, right, and bottom boundaries of the candidate box are represented in pixels, respectively.

[0106] Classification branches according to Convert the category logical values ​​into temperature-calibrated category probabilities, and then... and Determine the disease type and confidence level; symbols Indicate category The dimensionless class probability takes values ​​from zero to one, with the sign... Indicate category The classification logical value is a dimensionless number, with the sign... This represents the dimensionless positive temperature parameter calibrated by the validation set and stored in the network version record, with the symbol... Represents a set of disease categories Candidate disease categories and meet the following conditions As a member of a set, its unit is not applicable; the symbol here... This refers to local categorical variables within the S5 classification branch, distinct from the corresponding letters in S2 and S3 that represent RGB channel or feature channel indices; (symbol) The set represents the disease category corresponding to the highest category probability. Represents a set of six categories of road defects, symbolized by... The function represents the confidence score of a dimensionless candidate box with values ​​ranging from zero to one. This function returns the category with the highest probability. When two or more categories have the same highest probability, it returns the first category in ascending order of category index in the S1 category dictionary. This represents the largest dimensionless probability in the set; this parallel processing rule is only used for deterministic single-valued transformation when the class probabilities are equal.

[0107] The six categories of road defects are longitudinal cracks, transverse cracks, alligator cracks, diagonal cracks, repairs, and potholes. The output module parses the configuration referenced in the S1 coordinate restoration parameter record and verifies that the effective network version of the configuration is consistent with the network version of the current fused feature group. If the versions are inconsistent, the failure stage S5 and the configuration version inconsistency error code are written into the frame processing status record. The frame is registered in the task result list and post-processing is stopped. The labeled media, defect object label, or defect statistics item of the frame is not generated, and the failure is not interpreted as normal and defect-free. The preset confidence threshold and preset overlap threshold in the post-processing configuration are used for candidate screening and similar overlap suppression, respectively. In this specific embodiment, the preset confidence threshold is set to 0.25. Only road defect candidate boxes with a confidence of not less than the threshold and a bounding box width and height greater than two pixels are retained. Candidate boxes with a confidence of less than the threshold, degraded bounding boxes, or coordinates containing non-finite values ​​are not included in the subsequent overlap suppression.

[0108] The output module uniformly adopts a continuous pixel boundary coordinate convention: the top left boundary of the original image is zero, the right boundary can be equal to the width of the original image, and the bottom boundary can be equal to the height of the original image; the output module reads the S1 coordinate reconstruction parameter record and follows... and Map the candidate bounding box coordinates to the original coordinate system; symbol Indicates the boundary endpoint index, taking either one or two, symbol Represents the coordinate system of the road image to be detected. A set of horizontal continuous boundary coordinates, in pixels, with symbol... Represents the coordinate system of the road image to be detected. A series of vertical continuous boundary coordinates, with units of pixels, and symbols. Represents the first in the original coordinate system A set of horizontal continuous boundary coordinates, in pixels, with symbol... Represents the first in the original coordinate system A continuous vertical boundary coordinate system, with units of pixels, function This means truncating the values ​​to a given closed interval; the mapped coordinates are reordered according to the upper left and lower right endpoints, and before calculating the intersection-union ratio, it is verified that all four coordinates are finite values, the right boundary is greater than the left boundary and the lower boundary is greater than the upper boundary. Candidate boxes that do not meet the conditions are removed from the candidate set and written into the post-processing exception record.

[0109] For two candidate boxes with the same disease category and The output module is in accordance with Calculate the intersection-union ratio; symbol This represents the first candidate box for road defects in the current ranking, indicated by the symbol. This represents the candidate boxes for similar road defects to be compared, and the function... Represents the dimensionless intersection-union ratio with values ​​from zero to one, set operations. Set operations represent the intersection region of two candidate boxes at continuous pixel boundary coordinates. This represents the union region of two candidate boxes under the same coordinate convention, denoted by [symbol]. Represents the area of ​​the intersecting region in square pixels, symbol The area of ​​the union region is represented in square pixels; the area of ​​each candidate box and the intersecting region is calculated by multiplying the width obtained by subtracting the left boundary from the right boundary and the height obtained by subtracting the upper boundary from the lower boundary, without adding a pixel to the endpoints, and the union area is calculated by subtracting the intersection area from the sum of the areas of the two candidate boxes; in this specific embodiment, the preset overlap threshold is set to 0.45;

[0110] The output module sorts the candidate boxes that have passed the confidence level screening and original coordinate validity verification in descending order of confidence level. When the confidence levels are the same, the order is determined by descending candidate box area and ascending generation number. The first candidate box in the current sort is retained, and the remaining candidate boxes with the same disease category and whose intersection-union ratio with the candidate box is greater than the preset overlap threshold in the post-processing configuration are deleted. In this specific embodiment, the threshold is 0.45. The sorting, retention and deletion process is repeated for unprocessed candidate boxes until the candidate set is empty. Candidate boxes of different disease categories are not deleted from each other due to spatial overlap. The output module appends an independent sub-record to the candidate box result array of the S1 image-level master record for each finally retained candidate box using a unique key composed of task number, source file number, frame number and candidate box generation number. The disease category, original coordinate bounding box and confidence level are written into the sub-record. The sub-records that are successfully written back are added to the output candidate box level record set. A single failed write-back only excludes the corresponding candidate box and does not cover other sub-records in the same image.

[0111] For a single image, the rendering module reads the set of output candidate bounding boxes one by one, configures bounding boxes of fixed colors on the original road surface image according to the disease category, and writes the disease category name and confidence level to three decimal places. For video, the rendering module writes the processed image frames back to the output video according to the original timestamp. No detection results are generated for time periods that are not extracted. The output video frame rate follows the reciprocal of the frame extraction time interval and records the change in the task metadata.

[0112] The structured output module generates a label record for each sub-record in the same set of outputtable candidate boxes. Fields include task number, flight segment number, source file number, frame number, original timestamp, defect category, top-left x-coordinate, top-left y-coordinate, bottom-right x-coordinate, bottom-right y-coordinate, confidence level, network version, and coordinate restoration parameter version. This label is written to a JSON label file in UTF-8 encoding. The statistics module counts defects by defect category only for the same set of records, then summarizes the number of defects across six categories by flight segment. A unique key for each candidate box is used to avoid duplicate counting. The drawing, labeling, and statistics outputs all use successfully written-back candidate box-level sub-records as the sole data source.

[0113] When a frame completes processing S2 to S5 normally but no candidate box with confidence reaching the threshold is found, the frame processing status record is written to the successful no-candidate-box state. The output module still generates an empty object array label file and writes the count to zero. This state is distinguished from the failure stage of writing S2 to S4 and the upstream processing failure state of the error code. Finally, the labeled image or video, structured label file, quantity statistics results and processing log are registered as road defect detection results and written to the result list with the task primary key.

[0114] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0115] This invention combines learnable subband reweighted Haar wavelet downsampling with a weighted bidirectional feature pyramid, enabling the downsampled features after frequency band decomposition to enter the corresponding scale fusion node along a bottom-up propagation path. The learnable branch weights of the fusion node adjust the contribution relationship between the downsampling features and the semantic features, thereby making downsampling, cross-scale propagation, and detection output form a continuous technical processing chain.

[0116] Compared to the neck network that uses stride convolution and equal weight concatenation, this invention changes the internal frequency band processing structure and multi-scale fusion method of two-level downsampling, and adjusts the gradient contribution of training samples through WIoUv3, thereby structurally solving the problems of high-frequency detail attenuation, fixed scale contribution and outlier prediction box interference.

Claims

1. A method for detecting road defects using unmanned aerial vehicles (UAVs) based on sub-band reweighting, characterized in that, include: S1. Acquire road surface images or videos collected by the UAV, perform scale adaptation on the image frames of the road surface images or videos, and obtain the road image to be detected. S2. Input the road image to be detected into the trained road defect detection network, and extract P3 scale features, P4 scale features and P5 scale features through the backbone network; S3. In the top-down propagation path of the neck network, according to the non-negative and normalized learnable branch weights, the P5 scale features are upsampled and fused with the P4 scale features to obtain the P4 scale intermediate features. Then, the P4 scale intermediate features are upsampled and fused with the P3 scale features to obtain the P3 scale fused features. S4. In the bottom-up propagation path of the neck network, the P3 scale fusion feature and the subsequently formed P4 scale fusion feature are sequentially input into the learnable subband reweighted Haar wavelet downsampling module. The module uses Haar wavelet transform to obtain four types of subbands, and performs reweighting, channel splicing and projection according to non-negative and normalized learnable subband weights. The downsampling features at each level are fused with the corresponding scale features according to the learnable branch weights to obtain the P4 scale fusion feature and the P5 scale fusion feature in sequence. S5. Input the fused features from each scale into the detection head to obtain road defect candidate boxes, defect categories, and confidence levels, thus forming the road defect detection results.

2. The UAV road defect detection method based on sub-band reweighting according to claim 1, characterized in that, S1 includes: The system reads the file format and size information of the input data, verifies the decodeability of the image file, extracts image frames from the video file according to a preset frame interval or timestamp sequence, records an input anomaly flag for input data that fails to be decoded, and stops further processing of the input data; it scales the verified image or image frame proportionally and fills its edges according to the input size of the road defect detection network, and records the scaling ratio and filling offset to form the road image to be detected and its coordinate restoration parameters.

3. The UAV road defect detection method based on sub-band reweighting according to claim 1, characterized in that, S2 includes: The road defect detection network, constructed and trained based on YOLO11s, is used as the feature extraction network. The backbone network sequentially performs convolution and feature extraction on the road image to be detected, and outputs the P3 scale feature, the P4 scale feature, and the P5 scale feature with downsampling ratios of eight, sixteen, and thirty-two times respectively relative to the road image to be detected.

4. The UAV road defect detection method based on sub-band reweighting according to claim 1, characterized in that, In step S4, the module is configured with a spatial component partitioning layer and a fixed Haar decomposition layer. The spatial component partitioning layer divides each 2x2 local region of the input feature map into four spatial components according to the positional relationship of odd row odd column, odd row even column, even row odd column, and even row even column. The fixed Haar decomposition layer uses a fixed Haar transform kernel to perform sum-difference combination on the four spatial components to generate the low-frequency approximate subband, horizontal high-frequency detail subband, vertical high-frequency detail subband, and diagonal high-frequency subband.

5. The UAV road defect detection method based on sub-band reweighting according to claim 4, characterized in that, The learnable subband reweighted Haar wavelet downsampling module further includes a subband reweighting layer and a projection layer. The subband reweighting layer configures learnable parameters for each of the four subbands, uses the Softplus function to convert the learnable parameters into positive values, and normalizes each positive value by the sum of the four positive values ​​to obtain the learnable subband weights. The learnable subband weights are then multiplied element-wise with the corresponding subband and concatenated along the channel dimension. The projection layer sequentially performs cross-band information interaction and channel compression on the concatenated result through one-to-one convolution, batch normalization, and activation functions to output the corresponding downsampling features.

6. The UAV road defect detection method based on sub-band reweighting according to claim 5, characterized in that, In steps S3 and S4, the neck network forms a weighted bidirectional feature pyramid network; the top-down propagation path upsamples the P5-scale features and fuses them with the P4-scale features to obtain the P4-scale intermediate features, and then upsamples the P4-scale intermediate features and fuses them with the P3-scale features to obtain the P3-scale fused features. The bottom-up propagation path processes the P3-scale fused feature through the learnable sub-band reweighted Haar wavelet downsampling module, then fuses it with the P4-scale intermediate feature and the P4-scale feature to obtain the P4-scale fused feature. The P4-scale fused feature is then processed by the learnable sub-band reweighted Haar wavelet downsampling module and fused with the P5-scale feature to obtain the P5-scale fused feature. Each fusion node is configured with learnable parameters for its input branches, positive constraints are applied using the Softplus function and normalized according to the sum of positive values, and the weighted sum of each input branch is calculated using the weights of the learned branches.

7. The UAV road defect detection method based on sub-band reweighting according to claim 6, characterized in that, In step S5, the detection head performs classification prediction and bounding box regression on the P3 scale fusion feature, the P4 scale fusion feature and the P5 scale fusion feature respectively, and outputs the category probability and bounding box parameters of each road defect candidate box for each defect category. The defect category corresponding to the category probability with the largest value is determined as the defect category of the road defect candidate box, and the category probability with the largest value is determined as the confidence of the road defect candidate box. The scaling ratio and fill offset recorded during scale adaptation in step S1 are read, and the bounding box coordinates of the road defect candidate box are mapped to the original coordinate system of the road surface image or the video according to the scaling ratio and the fill offset.

8. The UAV road defect detection method based on sub-band reweighting according to claim 6, characterized in that, The training of the road defect detection network includes: using WIoUv3 bounding box regression loss, determining the outlier degree of the sample based on the cross-union ratio loss of the current training sample and the moving average of the cross-union ratio loss; generating non-monotonic focusing coefficients based on the outlier degree of the sample, and using the non-monotonic focusing coefficients to adjust the bounding box regression gradient contribution of the current training sample, so as to reduce the impact of outlier prediction boxes on network parameter updates.

9. The UAV road defect detection method based on sub-band reweighting according to claim 7, characterized in that, The disease categories include longitudinal cracks, transverse cracks, alligator cracks, diagonal cracks, repairs, and potholes. Candidate road disease boxes with a confidence level not less than a preset confidence threshold are sorted in descending order of confidence level. The candidate road disease box currently ranked first is retained, and other candidate road disease boxes with the same disease category and whose intersection-union ratio with the ranked candidate road disease box is greater than a preset overlap threshold are deleted. The sorting, retention, and deletion processes are repeated for the unprocessed candidate road disease boxes. Based on the retained candidate road disease boxes, labeled images or videos, structured label files containing disease categories, bounding box coordinates, and confidence levels, and statistical results summarized by disease category are generated.

10. A UAV road defect detection system based on subband reweighting, used to execute the UAV road defect detection method based on subband reweighting as described in any one of claims 1 to 9, characterized in that, include: The input processing module is used to acquire road surface images or videos collected by the UAV and perform format verification, image frame extraction and scale adaptation to obtain the road image to be detected. The multi-scale feature extraction module is used to extract P3-scale, P4-scale, and P5-scale features using the backbone network of the trained road disease detection network; the bidirectional weighted fusion module is used to obtain P4-scale intermediate features and P3-scale fusion features in the top-down propagation path according to the non-negative and normalized learnable branch weights of the fusion nodes, and to obtain P4-scale fusion features and P5-scale fusion features in the bottom-up propagation path. The subband reweighted downsampling module is used to sequentially perform Haar wavelet decomposition, subband reweighting, channel splicing, and projection on the P3-scale fusion features and the P4-scale fusion features, and output the corresponding downsampling features to the bottom-up propagation path; the disease detection output module is used to generate road disease candidate boxes, disease categories, and confidence scores based on the P3-scale fusion features, the P4-scale fusion features, and the P5-scale fusion features, and obtain road disease detection results after confidence score filtering and overlapping candidate box suppression.

Images