Pavement disease high-precision identification method and system based on multi-scale feature fusion

Abstract

The invention discloses a pavement disease high-precision identification method and system based on multi-scale feature fusion, relates to the technical field of road inspection multi-source vision and intelligent identification, and is used for solving the problem of realizing stable image acquisition, strong binding supervision and high-precision disease identification under tail-flow speckles and cross-domain disturbance. Three ends of a vehicle, an unmanned aerial vehicle and a ground site are synchronously collected, time and geometric alignment is completed, the vehicle is put in storage in a quaternary binding mode, corridor route lateral overlapping and linear polarization light control are matched, and positioning and posture tracks are recorded; triggering a micro time window frame cluster in a tail-flow speckle scene, collecting shear reciprocal and phase inversion images, constructing convection discrimination and a mask, executing directional line integration and reverse displacement compensation to obtain an image-stabilized image, and performing gating continuous collection by a coherent image-stabilized score; and then standardized cleaning, multi-scale augmentation, semi-automatic labeling and consistency checking are carried out, a multi-scale feature pyramid is constructed, three-path decoding is carried out, cross-domain training and model compression are combined, and a unified result and a rule list are output.

Description

Technical Field

[0001] This invention relates to the field of multi-source vision and intelligent recognition for road inspection, and more specifically, to a method and system for high-precision identification of road surface defects based on multi-scale feature fusion. Background Technology

[0002] Road inspection often relies on single-end data collection from vehicles or drones. This data is susceptible to issues such as motion blur, underexposure, and cross-device geometric inconsistencies due to variations in vehicle speed, strong reflections, shadows, and changing viewpoints. This makes direct alignment and incremental updates of the data sequence difficult. Near the rear of heavily loaded vehicles, a short-term speckle layer caused by dust or droplets appears above the road surface, exhibiting high-speed drift and sudden increases in contrast. This interferes with the visibility of fine cracks and disrupts the texture continuity between adjacent frames. Conventional multi-exposure, oblique side lighting, and short-term median fusion amplify random high frequencies or produce brightness reversals, leading to distortion in the comparison of corresponding frames. Existing pipelines also suffer from weak data binding and a lack of unified spatial mapping. After geometric or photometric augmentation, masks and bounding boxes are prone to misalignment, resulting in insufficient annotation consistency and traceability. Furthermore, the training side often ignores cross-domain differences, making it difficult to simultaneously capture both close-range fine cracks from vehicles and large potholes from drones at long distances.

[0003] To address the above problems, this invention proposes a solution. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a method and system for high-precision identification of pavement defects based on multi-scale feature fusion, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: In a preferred embodiment, it includes: Establish a unified acquisition and registration base with cross-device temporal and geometric constraints, perform four-element binding according to acquisition timestamp, device identifier, pose index and trajectory number, and acquire and store spatiotemporally consistent multi-source image sequences. Within a micro-time window frame cluster, shearing inverse and phase inversion images are acquired. After alignment, shearing difference and exposure difference are calculated, and convection discrimination is synthesized according to baseline weight. Then, the principal direction is determined by the heading projection and the gradient consistency difference is thresholded to form a mask. The mask area is integrated along the orthogonal line and compensated for by the reverse displacement of the long exposure. It is then fused with the median of the short exposure truncation. If the image stabilization score is insufficient, the image is acquired again. After cleaning and augmentation, a strong binding supervision is formed, and the spatial isomorphism matrix is ​​recorded as a unified mapping and back mapping rule. Based on the multi-scale feature pyramid and three-way decoding, joint reasoning for classification, detection and segmentation is completed.

[0006] In a preferred embodiment, the time reference across devices is aligned by three-terminal synchronization and a precision time synchronization protocol. The internal and external parameters of the camera are obtained with the calibration board and aligned to the vehicle coordinates. Then, a multi-source image sequence with consistent time and space is collected by vehicle speed-linked exposure, corridor-style flight path lateral overlap and ground intersection angle continuous shooting. The linear polarization is unified and the polarization angle is set according to the solar altitude angle before distortion is removed and the image is stored in the database. Next, segmented multi-exposure, oblique side lighting, and short-time multi-frame fusion acquisition of explicit supplementary sequences are triggered, and then training input is generated.

[0007] In a preferred embodiment, when the vehicle is in low-speed following and the attitude change rate given by the inertial measurement unit falls within the stable threshold range, micro-time window frame cluster acquisition is triggered at the same location and the same trajectory number, and multiple consecutive frames are written into the quaternary binding extension domain with frame cluster tags to form a replayable quinary registration.

[0008] In a preferred embodiment, two frames of cut-inverse images with opposite readout slopes are set within the above frame cluster. The readout slope is defined as the ratio of the readout time of adjacent lines to the total readout time of a single frame, and the range of values ​​is limited by the vehicle linear velocity and inertial angular velocity to counteract the deformation caused by rigid body planar motion. Next, a set of phase inversion exposure images are acquired within the same frame cluster. Short exposures are used to preserve instantaneous details, while long exposures are used to perform time averaging of speckle drift, while keeping the predetermined polarization angle unchanged. Subpixel alignment was then performed using the joint geometric constraints of the camera and the inertial measurement unit, retaining only the non-rigid body disturbances caused by the wake, and the shear reciprocal differential response and the exposure phase differential response were calculated respectively.

[0009] In a preferred embodiment, two weights are given based on the offline calibration of the wakeless base line segment. The grid search and cross-validation are performed with the goal of minimizing false alarms in steady-state texture. When the baseline is missing, it is initialized with equal weights. Then, the two normalized results are linearly synthesized into a pixel-level convection discriminant by weights.

[0010] In a preferred embodiment, within the same frame cluster, the principal direction angle is obtained by projecting the heading angle given by the inertial measurement unit onto the image plane through extrinsic and intrinsic parameters, and its orthogonal direction angle is taken; then the gradient consistency difference is calculated according to the principal direction and the orthogonal direction, and the convection discrimination threshold is combined to generate a convection susceptibility mask. In the region where the mask is one, a finite-length directional linear integral averaging is performed along the orthogonal direction based on the flow cues given by the shear inverse difference to suppress random speckle. Then, microscale reverse displacement compensation is applied to the long-time exposure image and it is fused with the short-time exposure image by truncating the median. Short exposures are retained in areas where the mask is zero to maintain the sharpness of fine band textures, resulting in frame-cluster stabilized images.

[0011] In a preferred embodiment, the frame cluster coherent image stabilization score is defined as the proportion of non-convection-dominant pixels within the effective field of view. When the score is lower than the score threshold, a new frame cluster is continuously acquired at the same location, and the shearing, phase inversion, mask partitioning, and convection compensation processes are repeated.

[0012] In a preferred embodiment, four-image stitching and regular mesh occlusion expansion context are introduced into the target sparse batch and missing elements are created. The isomorphic matrix is ​​calculated and written simultaneously. After augmentation, if the mask and bounding box are misaligned, the process is rolled back and redone. After cleaning and augmentation, crack skeleton candidates are generated by fine structure enhancement and line segment detection, and pit candidates are generated by texture measurement and morphological closing operation. Pixel masks are obtained by interactive segmentation and minimum bounding rectangles are extracted. The four categories are labeled according to a unified dictionary and the intersection-union ratio and coverage consistency are checked.

[0013] In a preferred embodiment, the model constructs a multi-scale feature pyramid and introduces attention, and the three-way decoding simultaneously outputs image-level classification, object-level detection and pixel-level segmentation. The sorting adopts the intersection-union ratio perceptual score and category adaptation. The training implements cross-domain batch balancing and curriculum-based advancement, and the loss weight and learning rate are adaptively adjusted according to the validation curve and semi-supervised retraining is introduced. The deployment adopts knowledge distillation and channel pruning to derive a unified inference list.

[0014] In a preferred embodiment, the module includes: an acquisition and registration base module, a frame cluster convection discrimination and image stabilization synthesis module, a strong binding supervision and multi-task inference module, and a data storage module, with signal connections between the modules. The data storage module is used to store all data during the storage process; The acquisition and registration base module is used to establish an acquisition and registration base with unified time and geometric constraints across devices. It performs four-element binding according to the acquisition timestamp, device identifier, pose index and trajectory number to acquire and store spatiotemporally consistent multi-source image sequences. The frame cluster convection discrimination and image stabilization synthesis module is used to acquire shearing inverse and phase inversion images within a micro-time window frame cluster. After alignment, shearing difference and exposure difference are calculated, and convection discrimination quantity is synthesized according to baseline weight. Then, the principal direction is determined by the heading projection and the gradient consistency difference is thresholded to form a mask. The mask area is integrated along the orthogonal line and compensated for by the reverse displacement of the long exposure. It is then fused with the median of the short exposure truncation. If the image stabilization score is insufficient, the image is acquired again. The strong binding supervision and multi-task reasoning module is used to form strong binding supervision after cleaning and augmentation, record the spatial isomorphism matrix as a unified mapping and back mapping rule, and complete the joint reasoning of classification, detection and segmentation based on multi-scale feature pyramid and three-way decoding.

[0015] The technical effects and advantages of the high-precision pavement distress identification method and system based on multi-scale feature fusion of this invention are as follows: This invention constructs a unified temporal geometry base through three-terminal synchronous acquisition and quaternary binding, ensuring sequence traceability and cross-viewpoint alignment. In wake speckle scenarios, micro-temporal window frame clusters are introduced, along with shearing inversion and phase inversion joint encoding, forming a convection discrimination and direction consistency mask. Stable images are obtained using directional line integrals and reverse displacement compensation, suppressing random high-frequency and brightness-dark reversals at the source. On the data side, isomorphic matrices are used to integrate augmentation and annotation, providing strong binding supervision and consistency verification. On the model side, multi-scale feature pyramids and three-way decoding are linked, combined with cross-domain training and adaptive scheduling, accommodating both fine cracks and large potholes. On the deployment side, distillation and pruning are combined to improve edge latency and energy efficiency, forming a verifiable unified inference and rule list. Thus, texture continuity and supervision consistency are maintained in highly reflective, shadow-interlaced, and high-noise road sections, significantly improving recall, positioning accuracy, and cross-domain stability, while reducing annotation and maintenance costs. Attached Figure Description

[0016] Figure 1 This is a time sequence diagram of the high-precision identification method and system for road surface defects based on multi-scale feature fusion of the present invention.

[0017] Figure 2 This is a schematic diagram of the high-precision identification method and system module for road surface defects based on multi-scale feature fusion according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example This invention discloses a high-precision method for identifying pavement defects based on multi-scale feature fusion, such as... Figure 1 As shown, it includes: First, image acquisition is activated simultaneously at three points: the road inspection vehicle, the drone, and the ground-based location. To ensure consistency of the time reference throughout the entire chain, the clocks of the vehicle-mounted camera, the inertial measurement unit, and the vehicle-mounted industrial control computer are aligned using the PTP precision time protocol. At the same time, the timestamps of the drone camera and the remote control are aligned.

[0020] To fix the imaging geometry and eliminate cross-device differences, the intrinsic parameters and relative pose calibration of the vehicle-mounted camera are completed sequentially using a checkerboard or AprilTag calibration board. The camera coordinate system is then aligned to the vehicle coordinate system by solving the extrinsic parameters. The UAV gimbal is then quickly calibrated in a centering attitude to ensure that the projection models at both the air and ground ends are in a reproducible state with the position of the principal optical axis.

[0021] After completing rapid intrinsic parameter calibration, each image frame is bound and written using a four-element binding system based on the acquisition timestamp, device identifier, pose index, and trajectory number. This ensures a one-to-one correspondence between the image and the acquisition context on the data side, allowing for backtracking. Through this step, time alignment, geometric locking, and context solidification are merged into a verifiable acquisition base.

[0022] Based on the completion of the four-element binding, continuous data collection is performed according to a closed-loop strategy of motion, light control, and tracing. Specifically, On the vehicle side, the exposure linkage curve automatically adjusts the electronic shutter and gain based on speed to avoid motion blur or underexposure caused by changes in vehicle speed; on the drone side, a corridor-style aerial photography strategy combining contour flight paths and lateral overlap is adopted, combined with RTK recording of flight path points and attitude trajectories to ensure coverage and seam control of the aerial viewpoint; on the ground, three consecutive shots at cross angles cover backlight, shadow, and light-dark boundary scenes.

[0023] To suppress specular reflections from the road surface, all three cameras are uniformly equipped with linear polarizing filters, and the polarization angle is automatically set according to the solar altitude angle to minimize reflections at low solar altitude angles. After acquisition, distortion correction and reconstruction are performed on all images to eliminate radial and tangential distortions, and then the images are stored in the database according to their trajectory numbers, forming a spatiotemporally consistent multi-source image sequence. Through this process, motion blur and strong reflections are suppressed at the source using speed-linked exposure and polarization control, making subsequent texture information more stable, while ensuring that the sequence organization can be directly used for cross-view alignment and incremental updates.

[0024] In road sections with high reflectivity, weak texture, and particle noise, explicit acquisition supplementary sequences are triggered to improve the visibility of defects and preserve detail. In areas with alternating reflectivity and shadows, multi-exposure segmented enclosure is used to acquire high-exposure frames and low-exposure frames at the same location. In road sections with fine cracks and shallow texture, oblique side lighting is briefly activated to project shallow shadow outlines and enhance fine line structures. On asphalt surfaces with significant high-frequency particle noise, short-time multi-frame synthesis using temporal alignment and median fusion is used to suppress random noise and preserve static structures.

[0025] All co-located frames are precisely paired with timestamps, and the exposure package number, compositing strategy, and fill light parameters are written into the local context label, so that subsequent training can be restored according to the original ambient lighting and texture conditions when needed.

[0026] After acquisition and visualization, the images undergo standardized cleaning and multi-scale augmentation for playback. First, channel mean and variance parameters are written to each frame. Then, the illuminance and reflectance components are separated within the logarithmic Retinex framework to reduce the interference of uneven illumination on texture recognition. For frames with motion blur, deconvolution based on blur kernel estimation is used to restore sharpness. For high-noise frames, median fusion of short-time multi-frames is used to suppress random noise.

[0027] The input scale pyramid is then constructed in a fixed order: first, scale jitter is applied to form a global view with different sampling rates; then, random cropping is performed to enhance the local context; then, a small affine transformation is applied to simulate slight pitch and rotation; and finally, a light perspective transformation is applied to simulate lane geometry changes.

[0028] It should be noted that during inspections near the rear of heavily loaded vehicles in dry, hot, or slightly humid road conditions, the vehicle's wake can excite a short-lived speckle layer of sub-millimeter-sized dust or droplets above the road surface. This speckle exhibits a random texture on the imaging plane that lasts only tens of milliseconds, with rapid positional drift and abruptly high contrast. Even after completing clock alignment via the PTP precision time protocol, relative pose calibration of the camera and gimbal's internal components, and the four-element binding and distortion-reconstruction and database entry based on the acquisition timestamp, device identifier, pose index, and trajectory number, under the aforementioned speckle-dominated conditions, an unsteady state phenomenon will still occur in adjacent frames under the same trajectory number, where fine textures are intermittently obscured and local micro-deformations are superimposed. If, at this point, the predetermined speed is maintained, with exposure linkage, contour corridor aerial photography, cross-angle three-shot bursts, and continuous acquisition using linearly polarized light control, and an explicit acquisition supplementary sequence designed for reflective / weak texture / grain noise is directly triggered, the following deficiencies and their consequences will be exposed:

[0029] The short lifetime of speckle and rapid phase drift, along with the repeated flipping of the brightness order of the same spatial location within an exposure pack due to multiple exposures, cause the intensity contrast between high / low exposure frames to no longer correspond to the real road surface texture, resulting in the destruction of the contrast relationship between frames in the same position. Side lighting will simultaneously amplify the brightness gradient of speckle, causing random high frequencies that should be suppressed to be mistakenly identified as objects of visibility enhancement and thus amplified. Short-time multi-frame median fusion can only remove some transient noise when the frame rate is limited and speckle drift is high speed, and still leaves some sporadic bright fragments.

[0030] This directly causes multi-source image sequences organized by trajectory number to lose their homogeneity in time and exhibit fragmented high-contrast point sets in space. This weakens the intended effect of using trajectory numbered images for cross-view alignment and incremental updates, and makes it difficult to maintain the verifiability of texture continuity under these circumstances, given that temporal alignment, geometric locking, and contextual fixation on the acquisition side.

[0031] Therefore, in this embodiment, while keeping the acquisition device and data registration structure unchanged, firstly, when the vehicle speed enters the low-speed following zone and the attitude change rate given by the inertial measurement unit is in the stable threshold range, micro-time window frame cluster acquisition is triggered at the same location and the same trajectory number. Several consecutive frames are written into the quadruple binding extension field with frame cluster marker κ, so that timestamp—device identifier—pose index—trajectory number—frame cluster marker correspond one-to-one and can be replayed.

[0032] Next, spatiotemporal shearing coding acquisition is enabled within this frame cluster: taking advantage of the settable line readout direction and interline delay of the camera's electronic shutter, two shearing inverse frames are continuously acquired within an extremely short time window, with readout slopes set to +s and -s, respectively. The readout slope is determined by the ratio of interline time delay to frame height, and the amplitude of sss is constrained by the IMU angular velocity and forward velocity to make the effects of shearing on the rigid body's planar motion cancel each other out.

[0033] The readout slope is defined as follows: ; in The time interval for reading adjacent rows This represents the total readout time for a single frame. To avoid amplifying the deformation of rigid body planar motion, let...

[0034] ; Where H is the number of image rows, Let v and ω be the equivalent pixel scale of the ground, v and ω be the linear and angular velocities of the vehicle calculated by the IMU, respectively, and r be the equivalent imaging radius. , ∈(0,1) is the safety factor set during calibration. Finally, the above parameters and the judgment result are written into the frame cluster context.

[0035] Subsequently, a pair of phase inversion exposure pairs were acquired within the same frame cluster. Without changing the predetermined polarization angle setting, the short exposure maintained the instantaneous sharpness of the fine band, and the long exposure was used to perform time averaging of the speckle drift trajectory. Furthermore, after performing sub-pixel alignment using joint geometric constraints from the camera and IMU, and retaining only the non-rigid perturbations caused by the wake to complete the geometric stabilization of the frame cluster, a convection discrimination tensor is constructed within the frame cluster. and will Defined as a two-dimensional field coupled with a shear antisymmetric response term and an exposure phase difference response term: First, calculate the differential response of the cut inverse frames in aligned coordinates: ; Where Aκ(⋅) is the subpixel geometric alignment operator within the frame cluster. , These are two inversely cropped original images; geometric alignment and radiometric consistency processing were completed before the difference was performed.

[0036] When the pixel intensity changes antisymmetrically with the readout direction, it indicates that it is mainly affected by line-time-dependent convection disturbances. Secondly, calculate the differential response of the phase inversion exposure in aligned coordinates: ; in, , These are the original images for short and long exposures, respectively, where tshort and tlong represent the corresponding exposure times. The denominator is used for exposure time normalization to avoid intensity bias introduced by the difference in exposure time. For robustness, , Obtained by range normalization within the frame cluster ^, ^, and write the normalized interval into the frame cluster context.

[0037] When pixel intensity is significantly inconsistent between short and long exposures, it indicates that there is a rapid intensity phase flip within the same micro-time window.

[0038] Furthermore, after normalizing the two items, they are weighted... , Linear synthesis with ≥0 yields: ; Among them, weight , The baseline was obtained through offline calibration with no wake: the target was to minimize the steady-state texture false alarm rate by searching and cross-validating within the [0,2] grid; if the baseline was missing, it was initially set according to the principle of equal energy. = =1. Threshold τD is taken from the baseline segment. The 95th percentile of the pixel distribution, updated automatically by intraday exponential moving average:

[0039] ; ρ∈[0.8,0.95]; Used to highlight the spatiotemporal shear sensitivity related to the readout slope Used to highlight the rapid phase flip associated with short / long exposure inconsistency; The higher the value, the more likely the pixel is to be dominated by wake convection rather than stable road surface texture within that frame cluster.

[0040] Subsequently, to ensure directional consistency, the gradient consistency difference between the main direction of vehicle travel θ∗ and its orthogonal direction θ⊥ is calculated simultaneously: ; The binding source for θ* is as follows: The vehicle heading angle is provided by the IMU and is denoted as . Construct the unit vector on the ground plane:

[0041] ; Then, the image is projected onto the image plane using the camera's extrinsic parameter [R|t] and intrinsic parameter K: The principal direction θ* of the image plane is defined as the angle between u and the image x-axis; where, .

[0042] when When the value is greater than 0, it indicates that random high-frequency patterns dominate in orthogonal directions, which is more consistent with speckle characteristics. Ultimately, and Thresholding superposition forms a convection-susceptible mask Mκ(x,y)∈{0,1}, where 1 represents the convection speckle-dominant region.

[0043] After obtaining Mκ, convection compensation synthesis is performed within the same frame cluster. For the region Mκ(x,y)=1, instead of directly using the conventional median, a finite-length directional line integral averaging is performed in the θ⊥ direction based on the convection direction clues given by the sheared inverse frame difference Δs, to suppress random speckle in the orthogonal direction;

[0044] The length of the line integral is set as follows: ; Where v is the linear velocity of the IMU calculation, Where p is the camera metadata frame duration, p is the ground equivalent pixel scale, and H is the number of image rows; Subsequently, microscale reverse displacement compensation was performed on the long-exposure image for this region. The displacement amplitude was obtained by converting |s| to the IMU velocity dimension and then robustly fused with the short-exposure image using the truncated median. For the region Mκ(x,y)=0, the short-exposure image was retained to maintain the sharpness of the fine band texture.

[0045] After being fused using the aforementioned local strategy, a frame cluster stabilized image Mκ(x,y) is generated.

[0046] Subsequently, the frame cluster marker κ, exposure packet number ε, readout slope s and direction, and reciprocal exposure order are set. , θ∗ and the mask number are synchronously written into the local context label, and the generated The Mκ is compressed quickly and then stored, along with {κ,ε,s,H,Δtrow, ,tshort,tlong,θ*,L, , , The normalized interval, moving average coefficient ρ, and τΨ} are written together into the frame cluster context label, and simultaneously recorded. ^, The normalized interval, directional line integral, and parameter caliber of micro-displacement compensation are defined; and the above fields are strongly bound to the images in the database according to the acquisition timestamp, device identifier, pose index, trajectory number, and frame cluster mark to ensure that the parameter source and judgment caliber can be verified and played back.

[0047] Finally, to complete the quality self-verification at the acquisition side, a coherent image stabilization score for frame clusters is defined: ; Where Ω represents the set of effective field-of-view pixels. For convection susceptibility threshold, ∈[0,1] and the closer to 1, the fewer regions are dominated by convection. If If the value is below the lower threshold τΨ, a new frame cluster is acquired at the same location, and the process of shearing reciprocity—phase inversion—mask partitioning—convection compensation is repeated, with the newly generated Mκ(x,y) covering the context record with the same name. After synthesis and scoring are completed, Mκ(x,y) along with the quaternion binding and extended context are stored in the database according to the original trajectory number, maintaining complete consistency with the above-mentioned established database storage and tracing rules;

[0048] The frame cluster scoring threshold τΨ is taken as the mean of the wakeless baseline scoring distribution minus one standard deviation: τΨ=μbase−σbase, and is updated on a weekly rolling window basis; then μbase, σbase, the window period and the timestamp are written into the context.

[0049] Furthermore, four-image stitching is introduced in batches with sparse target instances to expand the scene context, and regular grid occlusion is introduced in occluded and defective samples to construct structural defects.

[0050] To ensure that any subsequent annotation or inference is completely aligned with the spatial transformation, the isomorphic matrix is ​​calculated and written synchronously for each geometric or photometric transformation, serving as a unified spatial mapping and back-mapping rule for the training, validation, and visualization stages. If spatial misalignment between the mask and the bounding box is detected after augmentation, the frame is immediately rolled back to redo the augmentation and annotation, and the corresponding matrix record is updated.

[0051] To generate labels that can be directly used for high-precision supervision, a semi-automatic labeling pipeline is initiated after cleaning and augmentation. For crack candidates, the response of thin structures is enhanced by Frangi filtering or structural tensor, and line segment detection is combined to generate crack skeleton candidates; for pit candidates, roughness is measured by texture description operators such as LBP or GLCM, and then connected regions are generated by morphological closing operations.

[0052] The two types of candidates are then fed into an interactive segmentation network, where pixel-level masks are quickly generated using point or box cues. Subsequently, a boundary tracking algorithm is used to extract the outer contour of the mask and generate the minimum bounding rectangle as the bounding box.

[0053] All samples are labeled as longitudinal cracks, transverse cracks, network cracks, or pits according to a unified disease category dictionary. To ensure consistency among image-level, target-level, and pixel-level supervision, a dual-indicator check is performed on each frame, including masking, bounding box intersection-union, and skeletonization, as well as mask coverage. Samples that do not meet the threshold are automatically pushed back to interactive revision until they pass the consistency check. After passing the check, the image-level category, bounding box, and mask are strongly bound under the same unique image identifier. The entire process of candidate generation, interactive segmentation, and approval is recorded in the annotation log for post-audit and review.

[0054] Under the premise of strong binding supervision, a feature extraction and decoding process with multi-scale fusion as the main axis is constructed. The backbone encoder arranges standard convolution branches and multi-dilation rate dilated convolution branches in parallel in the shallow layer, so that the short receptive field and the long receptive field are sampled simultaneously, thereby capturing fine crack textures and large pit contours at the same time. In the middle layer, a lightweight weight generator with channel attention and spatial attention is introduced to suppress the background texture channels and amplify the structural channels related to the disease. In the high layer, local window self-attention is embedded to characterize the long-distance dependence of crack cross-pixel connectivity. At the end, spatial pyramid pooling is concatenated to aggregate the context of different windows to form multi-scale base features.

[0055] To incorporate scale information into the output, a feature pyramid is built on this foundation, and a learnable weighted bidirectional fusion strategy is adopted: high semantic information is backpropagated along the top-down path, and high-resolution details are backpropagated along the bottom-up path; before each cross-layer fusion, the number of channels is aligned by a one-to-one convolution, and then the spatial resolution is aligned by upsampling or downsampling, and finally the weighted fusion is completed with learnable weights; when there is a scale bias in the cross-layer features, the bias is eliminated by repeated fusion iterations to obtain a fused feature pyramid with consistent scale and sufficient context.

[0056] A three-task decoder is deployed in parallel on top of the fused feature pyramid, allowing multi-scale information to be integrated into the final output. The image-level classification path performs gating and concatenation of the fused features from the highest and second-highest layers, and then outputs the prior information on the presence of disease and the dominant disease category after global average pooling and two-level fully connected layers. The gating weights and pyramid fusion weights are updated together to maintain scale consistency.

[0057] The target-level detection path deploys decoupled detection heads on feature maps at three scales. The location branch uses a continuous regression strategy with geometric constraints to regress boundary parameters, while the category branch uses IoU-aware classification loss to learn a classification score consistent with the localization quality. During the sorting phase, IoU-aware classification scores are used instead of raw scores to prioritize candidates with high localization quality. During inference, class-adaptive suppression is used first to merge overlapping candidates, and non-maximum suppression is used when class overlap is severe. The pixel-level segmentation path recovers resolution layer by layer along the sequence of upsampling, convolution, normalization, activation, and skip connections, and cross-scale alignment is achieved by concatenating the fused features of adjacent two scales before decoding at each level. During training, the sparse foreground distribution is stabilized by the joint constraint of Dice loss and binary cross-entropy, and thin strip clipping supervision is introduced for fine crack samples to enhance skeleton connectivity.

[0058] Considering the spatial and semantic uniformity of the three outputs, firstly, the instances are aligned by masking, bounding box intersection, and threshold comparison, so that the mask and bounding box are geometrically mutually constrained. Then, the competition among multiple masks is resolved by rules that are consistent with the dominant class at the image level and have higher detection scores, thus avoiding class conflicts. Next, morphological refinement of the mask is performed on the open and closed parts to remove burrs, and the continuity of narrow cracks is restored by shortest path skeletonization. Finally, class-related screening is performed based on geometric priors, so that cracks tend to the minimum bounding rectangle skeleton shape with high aspect ratio, and pits tend to the shape of near-circular or closed compact regions.

[0059] To maintain a stable recall rate across different acquisition domains, cross-domain batch balancing and curriculum-based training were implemented during the training process. The UAV sub-queue increased the probability of scale jitter and context stitching to amplify the visibility of small targets, while the vehicle-mounted sub-queue added perspective perturbation and slight blur to simulate pitch and motion blur. The two domains were dynamically balanced according to the batch sampling ratio to maintain the prior stability of the four types of defects.

[0060] At the end of each training cycle, the average detection accuracy, segmentation intersection-over-union ratio, and F1 score are simultaneously calculated on the two validation subsets using the same script. The index curves are then fed back into the scheduling logic of loss weights and learning rate to generate an adaptive weight and learning rate plan for the next cycle.

[0061] To unify the optimization objective and suppress training oscillations, an end-to-end joint optimization strategy is adopted. The total loss is defined as:

[0062] ; in, For image-level classification loss, For target-level detection loss, For pixel-level segmentation loss, , , These are the loss weights for the three branches.

[0063] In the initial training phase, initial weight values ​​are set, and then the weights are periodically fine-tuned in small steps according to the validation curve, so that the three tasks converge collaboratively under the overall objective. Parameter updates employ an adaptive optimizer with weight decay, and the learning rate is linearly warmed up to enter the stable region, and then smoothly decreased using cosine annealing. During the batch construction phase, multi-scale mixed inputs are always open, and rare classes are resampled. In the later stages of training, semi-supervised retraining is introduced, where only high-confidence soft labels are fed back into the training queue, and consistency constraints are used as a gate for filtering pseudo-labels, thereby improving cross-domain generalization ability and stability for small objectives.

[0064] When complex texture backgrounds still trigger local false detections, a cascaded selection of high-frequency and low-frequency attention is introduced at the mid-to-high-level fusion points of the detection path, and features from the last two layers of the backbone encoder are limited to participate in uplink and downlink fusion to reduce interference from irrelevant textures. At the same time, the candidate box selection threshold and the category threshold are decoupled, and an adaptive step is performed using a cost function of recall and precision, so that the two can be optimized independently in different scenarios. In scenarios where real-time performance is prioritized, the traditional suppression stage is replaced by end-to-end high-quality query selection to minimize ranking uncertainty and maintain frame rate.

[0065] In parallel with this path, multi-scale auxiliary supervision with edge enhancement is introduced into the segmentation path: Sobel, Prewitt or Laplacian edge responses are calculated in the shallow and middle feature maps, and are concatenated with the corresponding scale decoded features as sparse constraints, and included in the total loss with small weights; when training oscillates, the weight of this branch is temporarily suspended or reduced, and gradually increased after the main loss stabilizes, so that the edges and mask boundaries gradually fit together and maintain cross-scale consistency.

[0066] After training convergence, knowledge distillation and channel pruning are performed to meet the dual constraints of deployment latency and accuracy. Specifically, feature distillation is performed in the intermediate layer based on teacher and student strategies, and mask contour distillation is performed at the output end, so that the student model can approach the performance of the teacher model in terms of edge continuity and small target recall. Channel pruning is implemented based on channel importance and attention sparsity, and only the key convolutional kernels and attention weights in the three-task decoder are fine-tuned in a short range to ensure that the single-scale inference latency meets the predetermined upper limit threshold. Subsequently, the ONNX or TorchScript inference graph is exported, and the category dictionary, geometric prior, threshold table, consistency constraint configuration, and isomorphic matrix playback rules are compiled into a unified list for field reproduction, transfer, and sampling.

[0067] During the deployment and acceptance phases, integrated results are generated based on a unified inference process. Candidate detections are merged using class-adaptive suppression, followed by mask and bounding box consistency rules to eliminate boundary overflow and internal holes. When multiple masks overlap at the same pixel location, inter-class rejection with the dominant image class and higher detection score is applied. Retained instances are filtered based on geometric priors, and mask refinement is performed in the order of fine opening, fine closing, skeletonization, and boundary smoothing. Finally, the image-level presence / absence and dominant class, target-level bounding boxes, and pixel-level masks are written into a unified result interface. Simultaneously, the exported list is used to sample and check thresholds and playback rules. Evaluation logs, weight versions, thresholds, and priors are archived as the starting point for subsequent domain adaptation and incremental training.

[0068] This invention also proposes a high-precision pavement distress identification system based on multi-scale feature fusion, such as... Figure 2 As shown, it includes: an acquisition and registration base module, a frame cluster convection discrimination and image stabilization synthesis module, a strong binding supervision and multi-task inference module, and a data storage module, with signal connections between the modules; The data storage module is used to store all data during the storage process; The data storage module is used to store all data during the storage process; The acquisition and registration base module is used to establish an acquisition and registration base with unified time and geometric constraints across devices. It performs four-element binding according to the acquisition timestamp, device identifier, pose index and trajectory number to acquire and store spatiotemporally consistent multi-source image sequences. The frame cluster convection discrimination and image stabilization synthesis module is used to acquire shearing inverse and phase inversion images within a micro-time window frame cluster. After alignment, shearing difference and exposure difference are calculated, and convection discrimination quantity is synthesized according to baseline weight. Then, the principal direction is determined by the heading projection and the gradient consistency difference is thresholded to form a mask. The mask area is integrated along the orthogonal line and compensated for by the reverse displacement of the long exposure. It is then fused with the median of the short exposure truncation. If the image stabilization score is insufficient, the image is acquired again. The strong binding supervision and multi-task reasoning module is used to form strong binding supervision after cleaning and augmentation, record the spatial isomorphism matrix as a unified mapping and back mapping rule, and complete the joint reasoning of classification, detection and segmentation based on multi-scale feature pyramid and three-way decoding.

[0069] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0070] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0071] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0072] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0073] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0074] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-precision identification method for pavement defects based on multi-scale feature fusion, characterized in that, include: Establish a unified acquisition and registration base with cross-device temporal and geometric constraints, perform four-element binding according to acquisition timestamp, device identifier, pose index and trajectory number, and acquire and store spatiotemporally consistent multi-source image sequences. Within a micro-time window frame cluster, shearing inverse and phase inversion images are acquired, aligned, and then shearing difference and exposure difference are calculated. Convection discrimination is synthesized according to baseline weights. Then, the principal direction is determined by heading projection and the gradient consistency difference is thresholded to form a mask. After integrating the mask area along the orthogonal line and compensating for the reverse displacement of the long exposure, it is fused with the median of the short exposure truncation. If the image stabilization score is insufficient, the image is continued. After cleaning and augmentation, a strong binding supervision is formed, and the spatial isomorphism matrix is ​​recorded as a unified mapping and back mapping rule. Based on the multi-scale feature pyramid and three-way decoding, joint reasoning for classification, detection and segmentation is completed.

2. The high-precision pavement distress identification method based on multi-scale feature fusion according to claim 1, characterized in that: By aligning the time reference across devices through three-terminal synchronization and precision time synchronization protocol, and using the calibration board to obtain the internal and external parameters of the camera and align them to the vehicle coordinates, a multi-source image sequence with consistent time and space is collected through vehicle speed-linked exposure, corridor-style flight path lateral overlap and ground intersection angle continuous shooting. The linear polarization is unified and the polarization angle is set according to the solar altitude angle before distortion is removed and the image is stored in the database. Next, segmented multi-exposure, oblique side lighting, and short-time multi-frame fusion acquisition of explicit supplementary sequences are triggered, and then training input is generated.

3. The high-precision pavement distress identification method based on multi-scale feature fusion according to claim 1, characterized in that: When the vehicle is following another vehicle at low speed and the attitude change rate given by the inertial measurement unit falls within the stable threshold range, micro-time window frame cluster acquisition is triggered at the same location and the same trajectory number, and multiple consecutive frames are written into the quaternary binding extension domain with frame cluster tags to form a replayable quinary registration.

4. The high-precision pavement distress identification method based on multi-scale feature fusion according to claim 3, characterized in that: Within the aforementioned frame cluster, two frames of cut-inverse images with opposite readout slopes are set. The readout slope is defined as the ratio of the readout time of adjacent lines to the total readout time of a single frame, and its range is limited by the combined vehicle linear velocity and inertial angular velocity to counteract the deformation caused by rigid body planar motion. Next, a set of phase inversion exposure images are acquired within the same frame cluster. Short exposures are used to preserve instantaneous details, while long exposures are used to perform time averaging of speckle drift, while keeping the predetermined polarization angle unchanged. Subpixel alignment was then performed using the joint geometric constraints of the camera and the inertial measurement unit, retaining only the non-rigid body disturbances caused by the wake, and the shear reciprocal differential response and the exposure phase differential response were calculated respectively.

5. The high-precision pavement distress identification method based on multi-scale feature fusion according to claim 4, characterized in that: Based on the offline calibration of the wakeless base line segment, two weights are given. The grid search and cross-validation are performed with the goal of minimizing false alarms in steady-state texture. When the baseline is missing, it is initialized with equal weights. Then, the two normalized results are linearly synthesized into a pixel-level convection discriminant according to the weights.

6. The high-precision pavement distress identification method based on multi-scale feature fusion according to claim 5, characterized in that: Within the same frame cluster, the principal orientation angle is obtained by projecting the heading angle given by the inertial measurement unit onto the image plane through external and internal parameters, and its orthogonal orientation angle is taken; then the gradient consistency difference is calculated according to the principal orientation and the orthogonal orientation, and the convection discrimination threshold is combined to generate a convection susceptibility mask; In the region where the mask is one, a finite-length directional linear integral averaging is performed along the orthogonal direction based on the flow cues given by the shear inverse difference to suppress random speckle. Then, microscale reverse displacement compensation is applied to the long-time exposure image and it is fused with the short-time exposure image by truncating the median. Short exposures are retained in areas where the mask is zero to maintain the sharpness of fine band textures, resulting in frame-cluster stabilized images.

7. The high-precision pavement distress identification method based on multi-scale feature fusion according to claim 6, characterized in that: The frame cluster coherent image stabilization score is defined as the proportion of non-convection-dominant pixels within the effective field of view. When the score is lower than the score threshold, a new frame cluster is continuously acquired at the same site, and the shearing, phase inversion, mask partitioning, and convection compensation processes are repeated.

8. The method for high-precision identification of pavement defects based on multi-scale feature fusion according to claim 1, characterized in that: In the target sparse batch, four-image stitching and regular grid occlusion expansion context are introduced and missing elements are created. The isomorphic matrix is ​​calculated and written simultaneously. After augmentation, if the mask and bounding box are misaligned, the process is rolled back and redone. After cleaning and augmentation, crack skeleton candidates are generated by fine structure enhancement and line segment detection, and pit candidates are generated by texture measurement and morphological closing operation. Pixel masks are obtained by interactive segmentation and minimum bounding rectangles are extracted. The four categories are labeled according to a unified dictionary and the intersection-union ratio and coverage consistency are checked.

9. The method for high-precision identification of pavement defects based on multi-scale feature fusion according to claim 8, characterized in that: The model constructs a multi-scale feature pyramid and introduces attention. Three-way decoding simultaneously outputs image-level classification, object-level detection, and pixel-level segmentation. The ranking adopts the intersection-union ratio perceptual score and category adaptation. The training implements cross-domain batch balancing and curriculum-based advancement. The loss weight and learning rate are adaptively adjusted according to the validation curve and semi-supervised retraining is introduced. The deployment adopts knowledge distillation and channel pruning to derive a unified inference list.

10. A high-precision pavement distress identification system based on multi-scale feature fusion, characterized in that, Includes: a data acquisition and registration base module, a frame cluster convection discrimination and image stabilization synthesis module, a strong binding supervision and multi-task inference module, and a data storage module, as well as signal connections between the modules; The data storage module is used to store all data during the storage process; The acquisition and registration base module is used to establish an acquisition and registration base with unified time and geometric constraints across devices. It performs four-element binding according to the acquisition timestamp, device identifier, pose index and trajectory number to acquire and store spatiotemporally consistent multi-source image sequences. The frame cluster convection discrimination and image stabilization synthesis module is used to acquire shearing inverse and phase inversion images within a micro-time window frame cluster. After alignment, shearing difference and exposure difference are calculated, and convection discrimination quantity is synthesized according to baseline weight. Then, the principal direction is determined by the heading projection and the gradient consistency difference is thresholded to form a mask. The mask area is integrated along the orthogonal line and compensated for by the reverse displacement of the long exposure. It is then fused with the median of the short exposure truncation. If the image stabilization score is insufficient, the image is acquired again. The strong binding supervision and multi-task reasoning module is used to form strong binding supervision after cleaning and augmentation, record the spatial isomorphism matrix as a unified mapping and back mapping rule, and complete the joint reasoning of classification, detection and segmentation based on multi-scale feature pyramid and three-way decoding.

Images