An image recognition-based asphalt pavement construction quality detection method

Abstract

This invention discloses an image recognition-based method for inspecting the construction quality of asphalt pavement, relating to the field of image recognition technology. The method includes: performing illumination normalization and texture feature modeling on a uniformly lit pavement orthophoto image to generate an illumination-normalized image and candidate anomaly response maps; inputting the illumination-normalized image into a multi-task segmentation network to perform multi-task segmentation inference, and combining it with the candidate anomaly response maps to perform differential operations to generate a reliable weighted defect probability map; performing temporal fusion and integral scoring on the reliable weighted defect probability map, and superimposing it onto the uniformly lit pavement orthophoto image to generate a quality inspection image report. This invention achieves efficient and automated inspection of asphalt pavement construction quality.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method for detecting the construction quality of asphalt pavement based on image recognition. Background Technology

[0002] Quality inspection of asphalt pavement construction is a crucial aspect of road construction and maintenance, vital for ensuring road performance and driving safety. Traditional inspection methods primarily rely on manual on-site inspections and fixed-point measurements, which suffer from efficiency bottlenecks and subjective biases. With advancements in computer vision technology, image recognition-based non-destructive testing methods have become a significant development direction in this field. Conventional techniques typically involve deploying imaging equipment to continuously acquire images of the road surface, followed by image preprocessing, feature extraction, and pattern classification to identify and locate common defects such as cracks and segregation. The core of these methods lies in utilizing image processing algorithms to transform visual information into quantifiable quality indicators, enhancing the automation level of the inspection process and providing technical support for online monitoring and evaluation of construction quality.

[0003] To address the limitations of traditional detection methods, existing research faces two key issues requiring improvement. First, image registration often employs fixed-parameter homography matrix estimation models, which struggle to adapt to nonlinear deformations caused by road surface deformation and equipment vibration during construction. This leads to local misalignment errors in the stitched images, affecting the accuracy of subsequent defect location. Second, illumination compensation mechanisms rely excessively on global statistical features, failing to effectively separate the grayscale differences between shadowed areas and actual defects, resulting in a higher rate of missed defects in weakly reflective areas. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an image recognition-based method for inspecting the construction quality of asphalt pavement, which solves the problems of insufficient registration accuracy and weak light adaptability in the inspection of asphalt pavement construction quality.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a method for inspecting the construction quality of asphalt pavement based on image recognition. The method includes: acquiring construction images, performing calibration and standardization processing to generate clear and effective images, optimizing registration through feature matching and residual constraints, and generating orthophotos of the pavement with consistent brightness; performing illumination normalization and texture feature modeling on the orthophotos of the pavement with consistent brightness to generate an illumination normalized image and a candidate anomaly response map; inputting the illumination normalized image into a multi-task segmentation network to perform multi-task segmentation inference, and combining it with the candidate anomaly response map to perform differential operations to generate a reliable weighted defect probability map; performing temporal fusion and integral scoring on the reliable weighted defect probability map, and superimposing it onto the orthophotos of the pavement with consistent brightness to generate a quality inspection image report.

[0007] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the steps of acquiring construction images and performing calibration and consistency processing are as follows: Collect original construction images and organize and store them as a sequence of original construction images; Radial and tangential distortion corrections are performed on the original construction image sequence to generate a distortion-corrected image sequence; Based on the channel index of the distortion-corrected image sequence, the corresponding channel grayscale response calibration curve is retrieved, and a radiometrically uniform image is generated through piecewise polynomial inverse transform.

[0008] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the steps for generating a valid and clear image are as follows: Laplacian filtering is performed on the radiometrically consistent image to obtain the Laplacian response map, and the sharpness score of each frame is obtained from the Laplacian response map and aggregated into a sharpness score sequence; Traverse the sharpness score sequence, remove image frames with sharpness scores lower than the sharpness qualification threshold, and rearrange the index to form a valid sharp image.

[0009] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the feature matching and residual constraint optimization registration steps are as follows: Local feature extraction and feature matching are performed on adjacent image frames in the effective clear image to generate matching feature point pairs and remove outliers to form a feature correspondence record table. Using the matching feature point pairs in the feature correspondence record table as constraints, the homography matrix is ​​solved by random sampling consistency estimation and summarized to form the initial registration parameter set.

[0010] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the steps for generating a pavement orthophoto image with consistent brightness are as follows: Using the homography matrix in the initial registration parameter set as the iteration variable, the parameters of the homography matrix are updated by performing damped nonlinear least squares iteration to generate an optimized registration parameter set. Based on the optimized registration parameter set, the effectively clear images are resampled and mapped to a unified pixel coordinate system through perspective transformation to generate a road surface orthophoto mosaic image group; Gain compensation for overlapping areas and multi-band fusion are performed on the road orthophoto mosaic image group to obtain road orthophoto images with consistent brightness.

[0011] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the steps of performing illumination normalization and texture feature modeling on the orthophoto image of the pavement with consistent brightness to generate an illumination normalized image and candidate anomaly response map are as follows. Perform illumination component estimation and reflection component reconstruction on orthophotos of roads with uniform brightness to generate illumination-normalized images; Local binary pattern feature maps, Gabor amplitude feature maps, and gradient amplitude feature maps are obtained from illumination-normalized images and stitched together to form a multi-scale texture feature map. The multi-scale texture feature maps are aggregated at the same pixel coordinate to generate a road surface texture baseline feature map. Feature difference is performed between the multi-scale texture feature map and the road surface texture baseline feature map, and the structural similarity difference value is calculated and mapped to the candidate anomaly response map.

[0012] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the steps of inputting the illumination-normalized image into a multi-task segmentation network to perform multi-task segmentation inference are as follows: Training response blocks are cropped from historical illumination normalized images and historical candidate anomaly response maps, and defects are labeled to generate defect-labeled training samples. The defect-annotated training samples are input into the multi-task segmentation network, and the trained multi-task segmentation network is obtained through joint supervised training. The illumination-normalized image is input into the trained multi-task segmentation network to perform forward inference, and the network outputs a defect probability map, a pixel uncertainty map, and a multi-task segmentation mask.

[0013] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the steps for generating a reliable weighted defect probability map are as follows: Perform a difference operation on the defect probability map and the candidate anomaly response map to obtain the probability residual value, and identify the pixel positions where the probability residual value is greater than the gate threshold, and normalize them into a pixel reliability weight map. The pixel reliability weight map and the pixel uncertainty map are merged into a fused pixel reliability weight map, and the defect probability map is weighted to generate a reliable weighted defect probability map.

[0014] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the steps of performing time-series fusion and integral scoring on the reliable weighted defect probability map are as follows: Perform weighted recursive fusion on the reliable weighted defect probability map to generate a time-series fused defect probability map; Based on the temporal fusion defect probability map, time integration is performed on time segments where the temporal fusion defect probability is not less than the defect probability threshold to generate defect exposure integral values, and these values ​​are then summarized to form a defect exposure integral map. The defect exposure integral map is input into the monotonic mapping operator and mapped to a pavement quality score map.

[0015] As a preferred embodiment of the image recognition-based asphalt pavement construction quality inspection method of the present invention, the steps for generating a quality inspection image report by superimposing a pavement orthophoto with consistent brightness are as follows: Defect connected regions are extracted from the multi-task segmentation mask, and the region confidence is calculated by combining the temporal fusion defect probability map to form a defect instance polygon group. Pixel matrices are extracted from orthophotos of roads with uniform brightness, and road quality rating maps are superimposed onto the pixel matrices. Meanwhile, polygon groups of defect instances are written into the pixel matrices to generate a quality inspection image report.

[0016] The beneficial effects of this invention are as follows: by performing illumination normalization and texture feature modeling on orthophotos of roads with consistent brightness, the estimation of illumination components and the reconstruction of reflection components are realized, the fluctuation of ambient illumination is suppressed, and an illumination-invariant characterization basis is provided for defect detection; by inputting the illumination-normalized image into a multi-task segmentation network to perform multi-task segmentation inference and differential operations, parallel prediction of defect probability and pixel uncertainty and reliability weight fusion are realized, improving image recognition accuracy, and finally realizing efficient and automated detection of asphalt pavement construction quality. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of an image recognition-based method for inspecting the construction quality of asphalt pavement.

[0019] Figure 2 This is a flowchart of image acquisition and preprocessing.

[0020] Figure 3 A flowchart for image registration and orthophoto generation.

[0021] Figure 4 This is a flowchart for defect detection and quality assessment. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for detecting the construction quality of asphalt pavement based on image recognition, including the following steps: S1. Collect construction images and perform calibration and standardization processing to generate valid and clear images.

[0026] Collect original construction images and organize and store them as a sequence of original construction images.

[0027] Furthermore, along the construction section, imaging acquisition is triggered at fixed time intervals along the construction direction to continuously acquire original construction image frames (e.g., the time interval is 0.2 seconds, and each batch contains no less than 300 frames). A timestamp, spatial order identifier, imaging posture parameters, exposure parameters, and integrity hash are written for each original construction image frame. The original construction image frames are sorted by timestamp and a frame index table is established using the construction batch identifier. The frame index table is then bound to the storage path to generate a sequence of original construction images.

[0028] Radial and tangential distortion corrections are performed on the original construction image sequence to generate a distortion-corrected image sequence.

[0029] Furthermore, the edge pixel trajectories of road markings are extracted from the original construction image sequence and converged into straight line constraints. The radial and tangential distortion coefficients are jointly solved by minimizing the bending residuals of the straight line constraints, and the radial and tangential distortion coefficients are fixed to the original construction image sequence. Based on the radial and tangential distortion coefficients, the original construction image sequence is resampled frame by frame using inverse mapping to generate distortion-corrected image frames (e.g., the radial distortion order is second and the number of tangential distortion coefficients is two). The frames are rearranged according to the original timestamps and bound to the frame index table to output the distortion-corrected image sequence.

[0030] It should be noted that the bending residual refers to the sum of squares of the orthogonal distances from each pixel to the corresponding ideal straight line after fitting the pixel trajectory of the road marking edge to an ideal straight line in the pixel coordinate system; the radial distortion coefficient is a set of parameters describing the radial nonlinear offset caused by the change in the distance from the pixel to the imaging center, and an exemplary value range is, for example, -0.5 to 0.5; the tangential distortion coefficient is a set of parameters describing the lateral and longitudinal pixel offsets caused by the asymmetric adjustment of the imaging plane, and an exemplary value range is, for example, -0.1 to 0.1.

[0031] Based on the channel index of the distortion-corrected image sequence, the corresponding channel grayscale response calibration curve is retrieved, and a radiometrically uniform image is generated through piecewise polynomial inverse transform.

[0032] Furthermore, based on the channel index of the distortion-corrected image sequence, a key-value search is performed in the gray-level response calibration curve mapping table to locate the corresponding gray-level response calibration curve. The gray-level response calibration curve is then divided into several monotonic segments according to the gray-level interval (e.g., the number of gray-level intervals is 4), and polynomial inverse transform coefficients are determined for each segment. The consistency of gray-level distribution in the overlapping area of ​​adjacent image frames in the same channel is statistically analyzed, and the polynomial inverse transform coefficients of each segment are corrected. Based on the corrected piecewise polynomial inverse transform coefficients, gray-level inverse mapping is performed frame by frame and pixel by pixel on the distortion-corrected image sequence to maintain the stability of the gray-level ratio between channels and eliminate exposure drift (the phenomenon that the overall gray-level distribution of adjacent image frames shifts due to changes in illumination and slight fluctuations in exposure parameters within the same construction batch). The images are then rearranged according to the original timestamps and bound to the frame index table to generate a radiometrically consistent image.

[0033] It should be noted that the grayscale response calibration curve mapping table is set based on the correspondence between the input grayscale value and the output pixel grayscale value obtained by the imaging channel through a standard grayscale target under multiple exposure parameter settings before construction; the polynomial inverse transform coefficients are set based on the least squares inverse function fitting of the grayscale response calibration curve in each grayscale interval, and the exemplary value range is usually -2 to 2.

[0034] Laplacian filtering is performed on the radiometrically consistent image to obtain the Laplacian response map, and the sharpness score of each frame is obtained from the Laplacian response map and aggregated into a sharpness score sequence.

[0035] Furthermore, the pixel grayscale values ​​in the radiometrically consistent image are read and a fixed-scale Laplacian operator (e.g., a Laplacian operator size of 3×3) is applied in the pixel coordinate domain. The second derivative of the pixel grayscale value is convolved to generate a Laplacian response map, which reflects the Laplacian response map of road surface details (aggregate particles, compaction marks, and micro-cracks, etc.) in the high-frequency components. The absolute value distribution of the pixel response values ​​of the entire frame is statistically analyzed in the Laplacian response map, and the mean of the absolute values ​​is used as the single-frame sharpness score to characterize the degree of preservation of edge and texture details in the image. The single-frame sharpness score corresponding to each image frame is written into the indexed sequence in the original timestamp order to form a sharpness score sequence.

[0036] Traverse the sharpness score sequence, remove image frames with sharpness scores lower than the sharpness qualification threshold, and rearrange the index to form a valid sharp image.

[0037] Furthermore, the sharpness score sequence is traversed in timestamp order. The median of the sharpness scores is used as the sharpness reference value, and the attenuation coefficient of the sharpness reference value is used to set the sharpness qualification threshold, which is used to characterize the minimum retention standard with stable high-frequency detail response in the construction batch. The sharpness score corresponding to each image frame is compared with the sharpness qualification threshold. Image frames with a sharpness score not less than the sharpness qualification threshold are marked as valid frames, and the remaining image frames are removed. The frame index of the retained image frames is rearranged according to the original timestamp order, and the frame index table and image storage reference relationship are updated synchronously to form a valid sharp image.

[0038] It should be noted that the attenuation factor is a dimensionless proportional parameter used to proportionally reduce the sharpness reference value to form a sharpness pass threshold. An exemplary attenuation factor is 0.6 to 0.8. The sharpness pass threshold is usually 0.6 to 0.8 times the sharpness reference value, used to achieve a stable balance between retaining valid information and eliminating blurry frames.

[0039] S2. Based on the effective clear image, the initial registration parameter set is obtained through feature matching, and the initial registration parameter set is optimized through residual constraint registration to generate a road orthophoto with consistent brightness.

[0040] Local feature extraction and feature matching are performed on adjacent image frames in the effective clear image to generate matching feature point pairs and remove outliers to form a feature correspondence record table.

[0041] Furthermore, within the pixel coordinate domain of adjacent image frames in the effective clear image, for each image frame, a multi-directional gradient distribution is constructed from the radiometrically consistent grayscale matrix in the radiometrically consistent image. This multi-directional gradient distribution is then normalized and encoded in the neighborhood of keypoints (points in a single image where the multi-directional gradient distribution exhibits stable extreme values ​​in the spatial neighborhood), yielding a feature description vector. Using the Euclidean distance between feature description vectors as a similarity metric, a one-to-one nearest neighbor search is performed on feature points in adjacent image frames to generate initial feature point pairs. Geometric consistency constraints are introduced into the initial feature point pairs (constraints that ensure spatial consistency between feature point pairs after mapping transformation, assuming the road surface satisfies planarity between adjacent image frames). The affine consistency relationship is estimated through iterative random sampling, and the reprojection error is obtained. Feature point pairs with reprojection errors exceeding the tolerance threshold are identified as outliers and removed. The remaining feature point pairs are retained and indexed according to spatial distribution uniformity to form a feature correspondence record table.

[0042] It should be noted that the tolerance threshold is based on the pixel-level positioning stability corresponding to the single-frame sharpness score of the effective sharp image, and is set by statistically analyzing the median of the reprojection error distribution on the initial feature point pairs of adjacent image frames. An exemplary value range is 1 to 3 pixels.

[0043] Using the matching feature point pairs in the feature correspondence record table as constraints, the homography matrix is ​​solved by random sampling consistency estimation and summarized to form the initial registration parameter set.

[0044] Furthermore, randomly select matching feature point pairs (e.g., 4 groups) from the feature correspondence record table, and solve the homography matrix based on the pixel coordinate correspondence of the matching feature point pairs; use the homography matrix to perform reprojection calculation on all matching feature point pairs in the feature correspondence record table to generate registration reprojection error values, and use the number of matching feature point pairs whose registration reprojection error values ​​do not exceed the tolerance threshold as the number of inliers; repeat the process (e.g., repeat 1000 times) and select the homography matrix with the largest number of inliers, and aggregate the homography matrix with the largest number of inliers according to the timestamp and image frame index to form the initial registration parameter set.

[0045] Using the homography matrix in the initial registration parameter set as the iteration variable, the parameters of the homography matrix are updated by performing damped nonlinear least squares iteration to generate an optimized registration parameter set.

[0046] Furthermore, the homography matrix in the initial registration parameter set is selected as the iteration variable, and the target residual is constructed based on the pixel coordinate difference formed by each feature point pair before and after homography mapping in the feature correspondence record table. Under the constraint of the target residual, the parameters of the homography matrix are iteratively updated using a damped nonlinear least squares method. In each iteration, the change of the residual with the parameter perturbation of the homography matrix is ​​calculated and a damping factor (e.g., the initial value of the damping factor is 0.01) is applied to suppress divergence, so that the sum of squares of the target residual gradually decreases. When the change of the sum of squares of the residuals in two adjacent iterations is less than the convergence threshold, the iteration is terminated. The converged homography matrix is ​​aggregated in timestamp order to generate an optimized registration parameter set.

[0047] It should be noted that the convergence threshold is set based on the decrease of the target residual sum of squares in two adjacent iterations. The initial value of the target residual sum of squares is statistically analyzed on the feature correspondence record table of the construction batch and a fixed proportion (e.g., 0.001) is taken as the termination threshold. An exemplary value range is 0.0001 times to 0.01 times the initial target residual sum of squares.

[0048] Based on the optimized registration parameter set, the effectively clear images are resampled and mapped to a unified pixel coordinate system through perspective transformation to generate a road surface orthophoto mosaic image group.

[0049] Furthermore, based on the optimized registration parameter set and the effective clear image, the pixel coordinate mapping relationship is defined by the homography matrix. The pixel coordinates of each image frame in the effective clear image are mapped to a pre-defined unified pixel coordinate system through perspective transformation. In the unified pixel coordinate system, the mapped pixel positions are solved by inverse coordinate and resampled to obtain orthorectified image frames with continuous pixel coordinates and consistent scale. All orthorectified image frames are organized in time stamp order and indexed to generate a road surface orthorectified mosaic image group for brightness compensation and fusion processing.

[0050] It should be noted that the unified pixel coordinate system is set based on the pixel coordinate range of the first frame image of the construction section and the corresponding actual road surface scale. It is defined by taking the upper left pixel of the first frame image as the coordinate origin and establishing a two-dimensional pixel grid. For example, each pixel is set to correspond to an actual road surface size of 5 mm.

[0051] Gain compensation for overlapping areas and multi-band fusion are performed on the road orthophoto mosaic image group to obtain road orthophoto images with consistent brightness.

[0052] Furthermore, the effective pixels of the preceding and following frames in adjacent orthophoto frames of the road surface orthophoto mosaic group are mapped to a unified pixel coordinate system. The spatial overlap region between adjacent orthophoto frames is identified, and the gray-level mean difference and gray-level ratio relationship of corresponding pixel positions in the overlap region are statistically analyzed. The gray-level ratio relationship is used as a gain compensation factor to perform gain correction on the gray-level of pixels in the overlap region, so that the gray-level distribution of the overlap region remains consistent within the sequence dimension. After completing the gain compensation, the orthophoto frame is decomposed into multi-band components of different spatial frequency scales. Smooth transition fusion is performed in the low-frequency components, and the continuity of texture details is maintained in the high-frequency components. The components at each scale are reconstructed to generate the fused orthophoto frame. The gain compensation and fusion are repeated on the fused orthophoto frame to generate a road surface orthophoto image with consistent brightness.

[0053] It should be noted that the gain compensation factor is a grayscale scaling factor obtained by statistically analyzing the grayscale values ​​of corresponding pixels within the spatially overlapping area. It is used to adjust the grayscale of the overlapping area of ​​the next frame to the grayscale reference of the overlapping area of ​​the previous frame. The exemplary value range is usually 0.8 to 1.2.

[0054] S3. Perform illumination normalization and texture feature modeling on the orthophoto of the road surface with consistent brightness to generate illumination normalized image and candidate anomaly response map.

[0055] Illumination component estimation and reflection component reconstruction are performed on orthophotos of roads with uniform brightness to generate illumination-normalized images.

[0056] Furthermore, the pixel grayscale matrix of each orthophoto frame is extracted from the orthophoto of the road surface with uniform brightness, and the spatially varying illumination components are estimated using a large-scale low-pass smoothing operator in the pixel coordinate domain to obtain the illumination component map of the corresponding image frame. Based on the illumination component map, the influence of the illumination component map is eliminated pixel by pixel in the pixel grayscale matrix of the orthophoto frame and amplitude normalization is performed to reconstruct the reflection component map. The illumination component estimation and reflection component reconstruction are repeated for all image frames to generate an illumination-normalized image with suppressed illumination changes and stable texture contrast.

[0057] It should be noted that the large-scale low-pass smoothing operator refers to a smoothing kernel that performs neighborhood weighted averaging on pixel grayscale values ​​within a large window in the pixel coordinate domain, used to extract spatial gradient components as illumination components.

[0058] Local binary pattern feature maps, Gabor amplitude feature maps, and gradient amplitude feature maps are obtained from illumination-normalized images and stitched together to form a multi-scale texture feature map.

[0059] Furthermore, the pixel grayscale matrix is ​​extracted from the illumination-normalized image, and a fixed-radius neighborhood sampling point (e.g., 8 neighborhood samples) is taken centered on the target pixel in the pixel coordinate domain. Local binary mode encoding is performed on each pixel to obtain a local binary mode feature map. A multi-scale, multi-directional Gabor kernel (e.g., 3 Gabor scales) is applied to the pixel grayscale matrix, and the response amplitude is extracted to obtain a Gabor amplitude feature map. The gradient magnitude is calculated on the pixel grayscale matrix to obtain a gradient amplitude feature map. The local binary mode feature map, Gabor amplitude feature map, and gradient amplitude feature map are aligned and stitched together in the channel dimension to form a multi-scale texture feature map.

[0060] By aggregating the multi-scale texture feature maps at the same pixel coordinates using the median, a road surface texture baseline feature map is generated.

[0061] Furthermore, under a unified pixel coordinate system, the corresponding feature values ​​of the same pixel coordinate in the time and scale dimensions are sorted and the median is taken as the representative value of the steady-state texture to suppress the interference of instantaneous noise, local occlusion and occasional anomalies during the construction process on texture modeling; median aggregation is performed on local binary mode features, Gabor amplitude features and gradient amplitude features respectively (for example, the median aggregation window covers texture feature maps of no less than 5 frames), and they are reorganized according to the original feature channel structure to form a road surface texture reference feature map that represents the normal texture morphology of the construction section.

[0062] Feature difference is performed between the multi-scale texture feature map and the road surface texture baseline feature map, and the structural similarity difference value is calculated and mapped to the candidate anomaly response map.

[0063] Furthermore, based on the multi-scale texture feature map and the road surface texture reference feature map, the corresponding channels are aligned pixel by pixel in a unified pixel coordinate system and the texture feature difference value is obtained. Within a local window (e.g., the window size is 11×11 pixels), the structural similarity difference value between the current texture feature and the reference texture feature (the median of the time dimension at the same pixel coordinate in the multi-scale texture feature map) is calculated and mapped to the pixel-level anomaly response intensity. The difference and structural similarity difference calculation are repeatedly performed on all timestamp image frames to generate candidate anomaly response maps, which represent the consistency response of texture and structural mutations independent of training samples. These maps are used to perform difference operations with the defect probability map to generate probability residual values.

[0064] The expression for calculating the structural similarity difference value is: ; in, In a multi-scale texture feature graph, the sequence of local texture feature values ​​corresponding to the current image frame at the same pixel coordinate is represented. This is the sequence of baseline texture feature values ​​obtained by aggregating the median over the time dimension at the same pixel coordinates in the baseline feature map of road texture. This represents the structural similarity difference between the current texture feature and the baseline texture feature. For the sequence of texture feature values The mean obtained through statistical analysis; For the reference texture feature value sequence The mean obtained through statistical analysis; As a constant used to stabilize the denominator of the mean term, it is set as a fixed proportion of the square of the dynamic range of the texture feature values ​​(the difference between the upper and lower bounds of the texture feature values ​​within a local window) to stabilize the denominator of the mean term. An example value range is 10 times the square of the dynamic range. -4 ~10 -2 ; This is a constant used to stabilize the denominators of the variance and covariance terms. It is set as a fixed proportion of the square of the dynamic range, based on the same dynamic range, to stabilize the denominators of the variance and covariance terms. An exemplary value range is 10 times the square of the dynamic range. -3 ~10 -1 ; For texture feature value sequence The variance; Reference texture feature value sequence The variance; For texture feature value sequence Compared with the baseline texture feature value sequence The covariance.

[0065] S4. Input the illumination-normalized image into the multi-task segmentation network to perform multi-task segmentation inference, and combine it with the candidate anomaly response map to perform differential operation to generate a reliable weighted defect probability map.

[0066] Training response blocks are cropped from historical illumination normalized images and historical candidate anomaly response maps, and defect annotations are performed to generate defect-annotated training samples.

[0067] The DeepLabv3+ network structure is used as a shared encoder and a shared decoder, and a multi-task output head is added to construct a multi-task segmentation network.

[0068] Furthermore, the defect-annotated training samples are used as input tensors and fed into the encoding path of the DeepLabv3+ network structure. Multi-scale dilated convolutional feature extraction is performed to output high-dimensional semantic feature representations. At the output of the encoding path, the semantic features are upsampled and concatenated to form a unified decoding feature tensor. On the decoding feature tensor, a defect probability output head, a pixel uncertainty output head, and a segmentation category output head are set in parallel, so that each output head shares the encoded and decoded features and only distinguishes the task semantics at the end convolutional mapping layer. This forms a multi-task segmentation network with shared encoding, shared decoding, and multi-task parallel prediction capabilities.

[0069] The defect-annotated training samples are input into the multi-task segmentation network, and the trained multi-task segmentation network is obtained through joint supervised training.

[0070] Furthermore, the illumination-normalized image blocks and pixel-level defect annotation masks in the defect-annotated training samples are read in batches (e.g., batch size of 8) and forward propagation is performed. The defect probability prediction, pixel uncertainty prediction, and segmentation category prediction are obtained through the multi-task output head. The cross-entropy loss is calculated based on the defect probability prediction and defect annotation mask, and the region consistency loss is calculated based on the segmentation category prediction and defect annotation mask. The cross-entropy loss and region consistency loss are weighted and converged with pixel uncertainty prediction to form a joint supervision loss. Based on the joint supervision loss, the parameters of the multi-task segmentation network are updated in reverse and iterated until the convergence condition or the upper limit is reached (e.g., the number of iterations is 50), and the trained multi-task segmentation network is output.

[0071] It should be noted that the convergence condition is set to the decrease in the joint supervised loss of a number of consecutive rounds (e.g., 50 iterations) being less than the loss threshold. The loss threshold is determined by using a fixed proportion of the initial joint supervised loss as a threshold, with an exemplary range of 0.0001 to 0.01 times the initial joint supervised loss.

[0072] The illumination-normalized image is input into the trained multi-task segmentation network to perform forward inference, and the network outputs a defect probability map, a pixel uncertainty map, and a multi-task segmentation mask.

[0073] Furthermore, the illumination-normalized image is segmented frame by frame into image block tensors consistent with the training phase, while maintaining the pixel coordinate indices. These are then input into the trained multi-task segmentation network for forward propagation. Multi-scale semantic features are extracted in the shared encoding path and restored to the original spatial resolution via the shared decoding path. Parallel mapping is performed on the decoded feature tensors through the defect probability output head, pixel uncertainty output head, and segmentation category output head to obtain the defect occurrence probability value (the defect probability value is limited to the range of 0 to 1), pixel uncertainty value, and segmentation category label corresponding to each pixel position. The three types of outputs from the same image frame are recombined according to the original pixel coordinates and converged in timestamp order to form a defect probability map (representing the multi-task segmentation network's learned probability prediction result of defect occurrence, serving as the verification input of the reliable weighted defect probability map, and also as the update input of the temporal fusion defect probability map), a pixel uncertainty map, and a multi-task segmentation mask.

[0074] Perform a difference operation on the defect probability map and the candidate anomaly response map to obtain the probability residual value, and identify the pixel positions where the probability residual value is greater than the gate threshold, and normalize them into a pixel reliability weight map.

[0075] Furthermore, after aligning the timestamps and pixel coordinates of the defect probability map and the candidate abnormal response map, the difference between the defect probability value and the candidate abnormal response value is obtained frame by frame and pixel by pixel to obtain the probability residual value. The probability residual value is then compared with the gate threshold to filter out high-consistency pixel positions where the probability residual value is greater than the gate threshold. Intra-frame min-max normalization is performed on the probability residual value and the normalization result is retained at the high-consistency pixel positions. Pixel positions where the probability residual value is less than the gate threshold are assigned zero to generate a pixel reliability weight map.

[0076] It should be noted that the gating threshold is set based on the statistical distribution of the probability residual values ​​within a single frame, and is determined by adding a fixed percentage standard deviation to the mean of the probability residual values. An exemplary value range is typically 0.2 to 0.6 after normalization of the probability residual values.

[0077] The pixel reliability weight map and the pixel uncertainty map are merged into a fused pixel reliability weight map, and the defect probability map is weighted to generate a reliable weighted defect probability map.

[0078] Furthermore, after aligning the pixel reliability weight map, pixel uncertainty map, and defect probability map frame by frame according to timestamps and pixel coordinates, the pixel uncertainty value is inverted to obtain the uncertainty suppression weight, which is then combined with the pixel reliability weight value pixel by pixel to obtain the fused pixel reliability weight. At the same pixel coordinate, the defect probability value is weighted pixel by pixel using the fused pixel reliability weight to generate a reliable weighted defect probability map.

[0079] S5. Perform time-series fusion and integral scoring on the reliable weighted defect probability map, and overlay it onto the orthophoto of the road surface with consistent brightness to generate a quality inspection image report.

[0080] A weighted recursive fusion is performed on the reliable weighted defect probability map to generate a time-series fused defect probability map.

[0081] Furthermore, the first frame of the reliable weighted defect probability map is used as the initial frame of the temporal fusion defect probability map. For each subsequent frame, at the same pixel coordinate, a weighted recursive fusion update is performed between the pixel value of the temporal fusion defect probability map of the previous frame and the pixel value of the reliable weighted defect probability map of the current frame. The historical state weight and the current frame weight satisfy the non-negativity constraint and the sum of the weights is 1. For example, the historical state weight is 0.7 and the current frame weight is 0.3. When there is a missing frame in the reliable weighted defect probability map, the pixel value of the temporal fusion defect probability map of the previous frame remains unchanged and subsequent timestamp fusion continues to generate the temporal fusion defect probability map.

[0082] Based on the temporal fusion defect probability map, time integration is performed on time segments where the temporal fusion defect probability is not less than the defect probability threshold to generate defect exposure integral values, which are then summarized to form a defect exposure integral map.

[0083] Furthermore, based on the temporal fusion defect probability map, the pixel values ​​of the temporal fusion defect probability map are traversed and continuous time segments are selected by identifying those whose pixel values ​​are not less than the defect probability threshold. Within the selected time segments, the pixel values ​​of the temporal fusion defect probability map are accumulated at timestamp intervals to obtain the defect exposure integral value. Then, the defect exposure integral value is backfilled into the corresponding pixel coordinates to form the defect exposure integral map.

[0084] It should be noted that the defect probability threshold is set based on the quantile of the time-series fused defect probability map on all pixels and all timestamps, and is set at a fixed quantile (e.g., 0.75 quantile), with a value range of 0 to 1.

[0085] The defect exposure integral map is input into the monotonic mapping operator and mapped to a pavement quality score map.

[0086] Furthermore, within the entire map, the defect exposure integral value in the defect exposure integral map is subjected to min-max normalization and numerically mapped to a pavement quality score using a monotonic mapping operator (e.g., the pavement quality score ranges from 0 to 100). This results in an increase in the defect exposure integral value corresponding to a decrease in the pavement quality score, and a decrease in the defect exposure integral value corresponding to an increase in the pavement quality score. The pavement quality score map is output while keeping the pixel coordinates unchanged.

[0087] Defect connected regions are extracted from the multi-task segmentation mask, and the region confidence is calculated by combining the temporal fusion defect probability map to form a defect instance polygon group.

[0088] Furthermore, based on the multi-task segmentation mask, connectivity scanning and aggregation are performed on the segmentation category labels according to the eight-neighbor connectivity to obtain the defect connected region pixel group, and the outer boundary points of the region are extracted to generate the initial polygon of the defect region; the region range is defined by the initial polygon of the defect region, and the pixel values ​​of the temporal fusion defect probability map within the region (e.g., the number of region pixels is not less than 50) are read from the temporal fusion defect probability map and the time mean value is obtained to obtain the region confidence value, and the defect instance polygon group is bound and output.

[0089] Pixel matrices are extracted from orthophotos of roads with uniform brightness, and road quality rating maps are superimposed onto the pixel matrices. Meanwhile, polygon groups of defect instances are written into the pixel matrices to generate a quality inspection image report.

[0090] Furthermore, orthophotos are read from orthophotos of roads with consistent brightness and pixel matrices are extracted. At the same pixel coordinates, the score values ​​of the road quality rating map are written into the pixel matrix according to color mapping to complete the overlay. Then, the vertex coordinates of the defect instance polygon group are rasterized and written into the pixel matrix to form defect boundary markers. The quality inspection image report is then aggregated and output according to the timestamp.

[0091] In summary, this invention achieves the estimation of illumination components and the reconstruction of reflection components by performing illumination normalization and texture feature modeling on orthophotos of roads with consistent brightness, thus suppressing ambient light fluctuations and providing an illumination-invariant characterization basis for defect detection. Furthermore, by inputting the illumination-normalized image into a multi-task segmentation network to perform multi-task segmentation inference and differential operations, it achieves parallel prediction of defect probability and pixel uncertainty, as well as reliability weight fusion, improving image recognition accuracy and ultimately realizing efficient and automated detection of asphalt pavement construction quality.

[0092] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for inspecting the construction quality of asphalt pavement based on image recognition, characterized in that: include, The acquired construction images are calibrated and standardized to generate effective and clear images. The registration is optimized through feature matching and residual constraints to generate orthophotos of the road surface with consistent brightness. Perform illumination normalization and texture feature modeling on orthophotos of roads with uniform brightness to generate illumination normalized images and candidate anomaly response maps; The illumination-normalized image is input into the multi-task segmentation network to perform multi-task segmentation inference, and the candidate anomaly response map is combined to perform differential operation to generate a reliable weighted defect probability map. Perform temporal fusion and integral scoring on the reliable weighted defect probability map, and overlay it onto the orthophoto of the road surface with consistent brightness to generate a quality inspection image report.

2. The image recognition-based asphalt pavement construction quality inspection method as described in claim 1, characterized in that: The acquired construction images undergo calibration and standardization processing, following these steps: Collect original construction images and organize and store them as a sequence of original construction images; Radial and tangential distortion corrections are performed on the original construction image sequence to generate a distortion-corrected image sequence; Based on the channel index of the distortion-corrected image sequence, the corresponding channel grayscale response calibration curve is retrieved, and a radiometrically uniform image is generated through piecewise polynomial inverse transform.

3. The image recognition-based asphalt pavement construction quality inspection method as described in claim 2, characterized in that: The steps for generating a valid and clear image are as follows. Laplacian filtering is performed on the radiometrically consistent image to obtain the Laplacian response map, and the sharpness score of each frame is obtained from the Laplacian response map and aggregated into a sharpness score sequence; Traverse the sharpness score sequence, remove image frames with sharpness scores lower than the sharpness qualification threshold, and rearrange the index to form a valid sharp image.

4. The image recognition-based asphalt pavement construction quality inspection method as described in claim 3, characterized in that: The feature matching and residual constraint optimization registration process involves the following steps. Local feature extraction and feature matching are performed on adjacent image frames in the effective clear image to generate matching feature point pairs and remove outliers to form a feature correspondence record table. Using the matching feature point pairs in the feature correspondence record table as constraints, the homography matrix is ​​solved by random sampling consistency estimation and summarized to form the initial registration parameter set.

5. The image recognition-based asphalt pavement construction quality inspection method as described in claim 4, characterized in that: The steps for generating a road surface orthophoto with consistent brightness are as follows. Using the homography matrix in the initial registration parameter set as the iteration variable, the parameters of the homography matrix are updated by performing damped nonlinear least squares iteration to generate an optimized registration parameter set. Based on the optimized registration parameter set, the effectively clear images are resampled and mapped to a unified pixel coordinate system through perspective transformation to generate a road surface orthophoto mosaic image group; Gain compensation for overlapping areas and multi-band fusion are performed on the road orthophoto mosaic image group to obtain road orthophoto images with consistent brightness.

6. The image recognition-based asphalt pavement construction quality inspection method as described in claim 5, characterized in that: The steps for performing illumination normalization and texture feature modeling on the orthophoto image of a road surface with consistent brightness to generate an illumination normalized image and candidate anomaly response map are as follows. Perform illumination component estimation and reflection component reconstruction on orthophotos of roads with uniform brightness to generate illumination-normalized images; Local binary pattern feature maps, Gabor amplitude feature maps, and gradient amplitude feature maps are obtained from illumination-normalized images and stitched together to form a multi-scale texture feature map. The multi-scale texture feature maps are aggregated at the same pixel coordinate to generate a road surface texture baseline feature map. Feature difference is performed between the multi-scale texture feature map and the road surface texture baseline feature map, and the structural similarity difference value is calculated and mapped to the candidate anomaly response map.

7. The image recognition-based asphalt pavement construction quality inspection method as described in claim 6, characterized in that: The steps for inputting the illumination-normalized image into the multi-task segmentation network to perform multi-task segmentation inference are as follows: Training response blocks are cropped from historical illumination normalized images and historical candidate anomaly response maps, and defects are labeled to generate defect-labeled training samples. The defect-annotated training samples are input into the multi-task segmentation network, and the trained multi-task segmentation network is obtained through joint supervised training. The illumination-normalized image is input into the trained multi-task segmentation network to perform forward inference, and the network outputs a defect probability map, a pixel uncertainty map, and a multi-task segmentation mask.

8. The image recognition-based asphalt pavement construction quality inspection method as described in claim 7, characterized in that: The steps for generating the reliable weighted defect probability map are as follows: Perform a difference operation on the defect probability map and the candidate anomaly response map to obtain the probability residual value, and identify the pixel positions where the probability residual value is greater than the gate threshold, and normalize them into a pixel reliability weight map. The pixel reliability weight map and the pixel uncertainty map are merged into a fused pixel reliability weight map, and the defect probability map is weighted to generate a reliable weighted defect probability map.

9. The image recognition-based asphalt pavement construction quality inspection method as described in claim 8, characterized in that: The steps for performing time-series fusion and integral scoring on the reliable weighted defect probability map are as follows: Perform weighted recursive fusion on the reliable weighted defect probability map to generate a time-series fused defect probability map; Based on the temporal fusion defect probability map, time integration is performed on time segments where the temporal fusion defect probability is not less than the defect probability threshold to generate defect exposure integral values, and these values ​​are then summarized to form a defect exposure integral map. The defect exposure integral map is input into the monotonic mapping operator and mapped to a pavement quality score map.

10. The image recognition-based asphalt pavement construction quality inspection method as described in claim 9, characterized in that: The steps for generating a quality inspection image report by superimposing the road surface orthophoto with consistent brightness are as follows: Defect connected regions are extracted from the multi-task segmentation mask, and the region confidence is calculated by combining the temporal fusion defect probability map to form a defect instance polygon group. Pixel matrices are extracted from orthophotos of roads with uniform brightness, and road quality rating maps are superimposed onto the pixel matrices. Meanwhile, polygon groups of defect instances are written into the pixel matrices to generate a quality inspection image report.

Images