A Method and System for Highway Pavement Defect Detection Based on UAV Image Analysis

Abstract

A method and system for detecting highway pavement defects based on UAV image analysis is disclosed, relating to the general field of image data processing. In this method, a UAV is controlled to acquire an initial pavement image; environmental interference areas are determined based on the initial pavement image and grayscale analysis; the UAV is controlled to acquire supplementary pavement images of the environmental interference areas; spatial projection transformation is performed on the initial pavement image and the supplementary pavement image according to a preset angle to establish pixel correspondences of the environmental interference areas; based on the pixel correspondences, the grayscale difference of corresponding pixels in the two acquired images of the environmental interference areas is calculated, and a difference matrix is ​​generated based on the grayscale difference; the environmental interference areas in the initial pavement image are compensated using the difference matrix to obtain a compensated pavement image; feature matching is performed on the compensated pavement image based on preset feature parameters to obtain pavement defect information with a similarity greater than a preset threshold to the preset feature parameters. This application aims to improve the accuracy of pavement defect detection.

Description

Technical Field

[0001] This application belongs to the general field of image data processing, and in particular relates to a method and system for detecting highway pavement defects based on UAV image analysis. Background Technology

[0002] Traditional highway pavement defect detection mainly relies on manual inspections on foot or by car. This method suffers from low efficiency, high labor intensity, subjective results, and susceptibility to environmental factors such as weather and lighting. Furthermore, due to the high traffic volume and speed on highways, manual inspection also poses significant safety hazards, making it difficult to guarantee the personal safety of inspection personnel.

[0003] In recent years, with the development of deep learning technology, intelligent pavement defect detection systems based on high-definition video acquisition and deep learning have been gradually applied in the field of highway maintenance. This system uses high-definition cameras mounted on dedicated inspection vehicles to continuously acquire pavement images while the vehicles are in motion. Deep learning algorithms are then used to analyze the acquired images in real time, automatically identifying and locating pavement defects. This achieves intelligent and automated pavement defect detection, improving detection efficiency and accuracy.

[0004] However, since deep learning relies heavily on training with massive amounts of labeled data, in practical applications, it is often necessary to retrain the model for image data under different lighting, weather and other environmental conditions. This makes it difficult for the system to respond quickly and process image data under various complex environments. At the same time, the large amount of data labeling and model training work also increases the system maintenance cost, thereby reducing the practicality and accuracy of road surface defect detection. Summary of the Invention

[0005] This application provides a method and system for detecting pavement defects on highways based on UAV image analysis, which improves the practicality and accuracy of pavement defect detection.

[0006] In the first aspect, this application provides a method for detecting highway pavement defects based on UAV image analysis, which controls the UAV to collect initial pavement images of the highway pavement along a preset route;

[0007] The environmental interference area was determined based on the initial road surface image and grayscale analysis.

[0008] The drone is controlled to adjust its flight attitude and perform a second image acquisition of the area with environmental interference to obtain supplementary road surface images. The acquisition angle of the supplementary road surface images and the acquisition angle of the initial road surface images form a preset angle.

[0009] Based on the preset angle, the initial road surface image and the supplementary road surface image are spatially projected and transformed to establish the pixel correspondence of the environmental interference area;

[0010] The grayscale difference of corresponding pixels in the environmental interference area in two acquired images is calculated based on the pixel correspondence, and a difference matrix is ​​generated based on the grayscale difference.

[0011] The environmental interference areas in the initial road surface image are compensated based on the difference matrix to obtain the compensated road surface image;

[0012] Based on preset feature parameters, feature matching is performed on the compensated road surface image to obtain road surface distress information with a similarity greater than a preset threshold to the preset feature parameters.

[0013] By employing the aforementioned technical solution, a drone is controlled to acquire images of the environmentally disturbed area twice from different angles, obtaining an initial road surface image and a supplementary road surface image with a preset angle. Spatial projection transformation is used to establish pixel correspondences, and the gray-level differences of corresponding pixels are calculated to generate a difference matrix, thereby compensating for the environmentally disturbed area. This compensation method based on dual-angle acquisition can eliminate the influence of environmental lighting, shadows, and other factors on the road surface image quality, improving the clarity and contrast of the road surface image. Feature matching of the compensated image can more accurately identify road surface defects, reducing misjudgments and omissions caused by environmental factors. The compensated image has a higher signal-to-noise ratio and more stable image features, improving the reliability and accuracy of defect detection results and providing more reliable data support for subsequent road maintenance decisions.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, the environmental interference area is determined based on the initial road surface image and grayscale analysis, specifically including:

[0015] Perform grayscale analysis on the initial road surface image to obtain a grayscale image;

[0016] Divide the grayscale image into multiple image blocks of equal size;

[0017] Calculate the mean gray level and standard deviation of gray level for each image patch;

[0018] Image blocks with a grayscale standard deviation greater than a first preset threshold and a grayscale mean greater than a second preset threshold are marked as candidate interference regions.

[0019] The candidate interference region is expanded by boundary extension to obtain the expanded region;

[0020] The area within the extended region whose grayscale change value is greater than the third preset threshold is identified as the environmental interference area.

[0021] By employing the aforementioned technical solution, grayscale analysis and block processing are performed on the initial road surface image. Combined with grayscale mean and standard deviation, multi-level screening and boundary expansion of interference areas are conducted, achieving precise localization of environmental interference areas. The grayscale standard deviation reflects the degree of grayscale fluctuation in a local area of ​​the image, while the grayscale mean characterizes the overall brightness level of the area. The combined use of these two metrics can identify areas affected by environmental factors such as lighting and shadows. By expanding the boundaries of candidate areas and analyzing grayscale changes, the complete range of interference areas can be accurately captured, avoiding missed detections and ambiguous boundary judgments. This multi-index combined judgment method improves the accuracy and completeness of environmental interference area identification, making the compensation effect more precise and effective.

[0022] In conjunction with some embodiments of the first aspect, in some embodiments, the initial road surface image and the supplementary road surface image are spatially projected and transformed according to a preset angle to establish the pixel correspondence of the environmental interference area, specifically including:

[0023] The boundary points of the environmental interference area in the initial road surface image are extracted as the first feature point set;

[0024] Extract the boundary points of the regions corresponding to the environmental interference areas in the supplementary road surface image as the second feature point set;

[0025] Establish a geometric transformation matrix between the first feature point set and the second feature point set based on a preset angle;

[0026] The supplementary road surface image is spatially projected using a geometric transformation matrix to obtain the transformed supplementary road surface image;

[0027] The transformed supplementary road surface image is matched with the initial road surface image in the environmental interference area to obtain the pixel matching result.

[0028] The pixel correspondence in the environmental interference area is established based on the pixel matching results.

[0029] By employing the above technical solution, precise alignment and pixel matching between images acquired from different angles are achieved by extracting boundary feature points of the environmental interference area and establishing a geometric transformation matrix for spatial projection transformation. The extraction of boundary feature points ensures the representativeness and stability of the transformation reference, while the geometric transformation matrix guarantees the accuracy of the spatial projection transformation. The transformed supplementary road surface image and the initial road surface image achieve precise pixel-level correspondence. Pixel-level precise matching eliminates geometric distortion caused by viewing angle differences, improves the calculation accuracy of the difference matrix, and thus enhances the effect of environmental interference compensation, resulting in a clearer and more accurate final compensated road surface image.

[0030] In conjunction with some embodiments of the first aspect, in some embodiments, before calculating the grayscale difference of corresponding pixels in the environmental interference region in two acquired images based on pixel correspondence, the method further includes:

[0031] Acquire historical image data of areas with environmental interference, and extract image feature change sequences from the historical image data;

[0032] Analyze the temporal correlation of image feature change sequences to obtain the feature change patterns;

[0033] Determine the feature classification threshold based on the feature variation pattern;

[0034] The features of the environmentally disturbed area are reclassified using a feature classification threshold to obtain the feature reclassification results;

[0035] The weights are calculated by setting grayscale differences for pixels within the environmental interference area based on the feature reclassification results.

[0036] The weights calculated from the grayscale differences are used as correction parameters for pixel correspondence.

[0037] By employing the aforementioned technical solution, and analyzing the feature change sequences in historical image data, temporal correlations and change patterns were extracted, establishing a feature classification system based on historical data. The analysis of feature change patterns revealed the dynamic evolution characteristics of environmental interference areas, providing statistical support for feature classification. The feature classification thresholds determined based on these change patterns exhibit greater adaptability and rationality, and the feature reclassification results more closely reflect actual conditions. By assigning weights to the grayscale difference calculations of pixels within the environmental interference area, a correction mechanism considering historical change characteristics was established. This weight correction mechanism based on historical data improves the accuracy of grayscale difference calculations, making the environmental interference compensation effect more consistent with actual road conditions and enhancing the reliability of defect detection results.

[0038] In conjunction with some embodiments of the first aspect, in some embodiments, the feature classification threshold is determined based on the feature variation pattern, specifically including:

[0039] Calculate the fluctuation range of characteristic values ​​in the characteristic change pattern, and determine the stable interval within the fluctuation range;

[0040] Calculate the mean and standard deviation of the eigenvalues ​​within the stability interval;

[0041] The characteristic distribution intervals are divided based on the mean and standard deviation;

[0042] The boundary values ​​of the feature distribution interval are determined as the feature classification threshold.

[0043] By employing the above technical solution, the fluctuation range of feature values ​​in the feature variation pattern is calculated and a stable interval is determined. Then, the mean and standard deviation of feature values ​​within the stable interval are calculated. Based on this, feature distribution intervals are divided, and boundary values ​​are determined as feature classification thresholds, thus improving the representativeness and reliability of the thresholds. Through the calculation of the mean and standard deviation, the central tendency and dispersion of feature values ​​can be accurately grasped, reflecting the distribution pattern of features more accurately. This improves the accuracy of subsequent feature classification, reduces classification errors, and makes the reclassification results of features in environmentally disturbed areas more accurate and reliable, thereby enhancing the precision of grayscale difference calculation.

[0044] In conjunction with some embodiments of the first aspect, in some embodiments, after obtaining pavement distress information with a similarity greater than a preset threshold to preset feature parameters, the method further includes:

[0045] Obtain the pavement structure parameters of the area corresponding to pavement distress information;

[0046] Calculate the stress distribution value of the damaged area in the pavement distress information to obtain a stress distribution map, which represents the stress magnitude at each point within the damaged area;

[0047] Determine the stress transmission direction based on the changing trend of stress values ​​in the stress distribution diagram;

[0048] The potential expansion area of ​​the disease is determined based on the stress transmission direction and pavement structure parameters;

[0049] Calculate the cumulative stress value within the potential extended region;

[0050] Risk levels of pavement distress information are classified based on cumulative stress values.

[0051] By adopting the above technical solutions, a pavement distress development prediction model was established using mechanical analysis methods, enabling distress assessment to move beyond mere visual characteristics. Calculating stress distribution and transmission direction allows for accurate prediction of distress expansion trends, while combining analysis with pavement structural parameters provides a more accurate assessment of the actual impact of distress on the pavement structure. Risk level classification based on cumulative stress values ​​quantifies the severity of distress from a mechanical perspective, providing a more accurate basis for risk assessment and facilitating the development of more targeted maintenance plans.

[0052] In conjunction with some embodiments of the first aspect, in some embodiments, the potential expansion area of ​​the defect is determined based on the stress transfer direction and pavement structure parameters, specifically including:

[0053] The stress-affected sector is divided along the stress transmission direction. The stress-affected sector is a fan-shaped area that starts from the diseased area and expands along the stress transmission direction.

[0054] Extract road surface structural parameters within the stress-affected sector;

[0055] Calculate the stress attenuation coefficient based on the road structure parameters within the stress-affected sector;

[0056] Determine the stress influence boundary based on the stress attenuation coefficient;

[0057] The region enclosed by the stress-affected boundary is defined as the potential extension region.

[0058] By employing the above technical solution, stress influence sectors are divided along the stress transmission direction, and the stress attenuation coefficient is calculated in conjunction with pavement structure parameters to determine the stress influence boundary and potential expansion area. This solution considers the actual stress transmission characteristics in the pavement structure, and the sector division method better reflects the actual laws of stress diffusion. Calculating the stress attenuation coefficient in conjunction with pavement structure parameters accurately reflects the influence of different pavement structures on stress transmission, making the determination of the stress influence range more precise. Determining the potential expansion area through the stress influence boundary considers both the directionality of stress transmission and the actual situation of the pavement structure, making the prediction of damage expansion more realistic and improving the accuracy and reliability of the prediction.

[0059] In a second aspect, embodiments of this application provide a highway pavement defect detection system based on UAV image analysis. The highway pavement defect detection system based on UAV image analysis includes: one or more processors and a memory; the memory is coupled to one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the system to perform the method described in the first aspect and any possible implementation thereof.

[0060] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a system, cause the system to perform the method described in the first aspect and any possible implementation thereof.

[0061] Fourthly, embodiments of this application provide a computer program product that, when run on a system, causes the system to execute the method described in any possible implementation of the first aspect.

[0062] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0063] 1. This application provides a method for detecting highway pavement defects based on UAV image analysis. By controlling a UAV to acquire images of environmentally disturbed areas from two different angles, an initial pavement image and a supplementary pavement image with a preset angle are obtained. Spatial projection transformation is used to establish pixel correspondences, and the gray-level differences of corresponding pixels are calculated to generate a difference matrix, thereby compensating for environmentally disturbed areas. This dual-angle acquisition-based compensation method can eliminate the influence of environmental lighting, shadows, and other factors on pavement image quality, improving the clarity and contrast of the pavement images. By performing feature matching on the compensated images, pavement defect information can be identified more accurately, reducing misjudgments and omissions caused by environmental factors. The compensated images have a higher signal-to-noise ratio and more stable image features, improving the reliability and accuracy of defect detection results and providing more reliable data support for subsequent pavement maintenance decisions.

[0064] 2. This application provides a method for detecting highway pavement defects based on UAV image analysis. By analyzing the feature change sequences in historical image data, temporal correlations and change patterns are extracted, and a feature classification system based on historical data is established. The analysis of feature change patterns reveals the dynamic evolution characteristics of environmental interference areas, providing statistical support for feature classification. The feature classification threshold determined based on the change patterns has stronger adaptability and rationality, and the feature reclassification results are more consistent with the actual situation. By setting grayscale difference calculation weights for pixels in environmental interference areas, a correction mechanism considering historical change features is established. This weight correction mechanism based on historical data improves the accuracy of grayscale difference calculation, making the environmental interference compensation effect more consistent with the actual pavement condition and enhancing the reliability of defect detection results.

[0065] 3. This application provides a method for detecting pavement distress on highways based on UAV image analysis. A pavement distress development prediction model is established through mechanical analysis, allowing distress assessment to move beyond mere visual characteristics. By calculating stress distribution and transmission direction, the expansion trend of distress can be accurately predicted. Furthermore, combining analysis with pavement structural parameters allows for a more accurate assessment of the actual impact of distress on the pavement structure. Risk level classification based on cumulative stress values ​​enables a quantitative assessment of the severity of distress from a mechanical perspective, providing a more accurate basis for risk assessment and facilitating the development of more targeted maintenance plans. Attached Figure Description

[0066] Figure 1 This is a flowchart illustrating a method for detecting highway pavement defects based on UAV image analysis in an embodiment of this application.

[0067] Figure 2This is another flowchart illustrating a method for detecting highway pavement defects based on UAV image analysis in an embodiment of this application.

[0068] Figure 3 This is a schematic diagram of the physical device structure of a highway pavement defect detection system based on UAV image analysis provided in an embodiment of this application. Detailed Implementation

[0069] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0070] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0071] The following example is used in conjunction with Figure 1 This application describes a method for detecting highway pavement defects based on UAV image analysis in its embodiments:

[0072] Please see Figure 1 This is a flowchart illustrating a method for detecting highway pavement defects based on UAV image analysis in an embodiment of this application.

[0073] S101. Control the UAV to collect initial road surface images of the highway along a preset route, and determine the environmental interference area based on the initial road surface images and grayscale analysis.

[0074] The system controls a drone to collect initial road surface images of the highway along a preset route. Based on the initial road surface images and grayscale analysis, environmental interference areas are determined. Specifically, determining environmental interference areas based on the initial road surface images and grayscale analysis includes: performing grayscale analysis on the initial road surface images to obtain grayscale images; dividing the grayscale images into multiple image blocks of equal size; calculating the grayscale mean and grayscale standard deviation of each image block; marking image blocks with a grayscale standard deviation greater than a first preset threshold and a grayscale mean greater than a second preset threshold as candidate interference areas; expanding the boundaries of the candidate interference areas to obtain expanded areas; and determining the areas within the expanded areas with grayscale variation values ​​greater than a third preset threshold as environmental interference areas.

[0075] In practice, the system pre-designs the drone's flight path to ensure coverage of the highway section to be detected. The flight path design needs to consider factors such as the highway's direction, length, and width, as well as the drone's flight performance and battery life.

[0076] Drones can be equipped with high-definition digital cameras or professional aerial cameras to capture high-quality road images. Camera selection requires consideration of parameters such as resolution, focal length, and aperture to meet the needs of subsequent image processing and analysis.

[0077] During actual flight, the system uses the drone's flight control system, such as GPS and IMU, to control the drone to fly along a preset route in real time, and adjusts the flight altitude and speed as needed.

[0078] During flight, the drone automatically triggers its camera to capture a series of images of the highway surface, i.e., initial road surface images, according to preset time intervals or spatial intervals.

[0079] Gray-scale analysis and determination of environmental interference areas:

[0080] The initial road surface images are preprocessed, such as image cropping and correction, to eliminate distortion and interference from non-road areas.

[0081] Converting the preprocessed initial road surface image to a grayscale image involves converting the RGB value of each pixel in the color image into a single grayscale value. Common grayscale conversion methods include averaging and weighted averaging.

[0082] The grayscale image is divided into multiple image blocks of equal size. The size of the image blocks can be set according to actual needs and computational efficiency, such as 16x16, 32x32, etc.

[0083] For each image patch, calculate its mean gray value and standard deviation. The mean gray value reflects the average brightness of the image patch, and the standard deviation reflects the dispersion of pixel gray values ​​within the image patch.

[0084] Two preset thresholds are set. The first preset threshold is used to determine the standard deviation of gray levels in an image patch, and the second preset threshold is used to determine the mean gray level of the image patch. Image patches that simultaneously satisfy both the standard deviation of gray levels being greater than the first preset threshold and the mean gray level being greater than the second preset threshold are marked as candidate interference regions. These two thresholds can be determined through empirical values ​​or statistical analysis.

[0085] Image patches marked as candidate interference regions undergo boundary expansion, i.e., expansion within a certain range around them, resulting in an expanded region. The purpose of expansion is to cover any potentially existing complete interference region. The expansion range can be set according to the actual situation and requirements.

[0086] Within the extended region, the grayscale change value between adjacent pixels, i.e., the grayscale gradient, is calculated. Regions with grayscale change values ​​greater than a third preset threshold are identified as the final environmental interference region. The third preset threshold is used to determine whether the grayscale change between pixels is drastic and can be set through experimentation or experience.

[0087] Through the above steps, the system can automatically identify environmental interference areas from the initial road surface images collected by the UAV, providing a foundation for subsequent image compensation and road surface defect detection.

[0088] In the actual implementation process, the algorithm's parameters and thresholds can be adjusted and optimized according to actual needs and conditions to improve the accuracy and robustness of identifying environmental interference areas. Simultaneously, other image processing and analysis techniques, such as image segmentation and edge detection, can be introduced to further enhance the algorithm's performance and adaptability.

[0089] S102. Control the UAV to adjust its flight attitude and perform a second image acquisition of the environmental interference area to obtain supplementary road surface images;

[0090] The system controls the drone to adjust its flight attitude and perform a second image acquisition of the area with environmental interference to obtain supplementary road surface images. The acquisition angle of the supplementary road surface images and the acquisition angle of the initial road surface images form a preset angle.

[0091] After identifying the area of ​​environmental interference, the system controls the drone to adjust its flight attitude and perform a second image acquisition of the interference area to obtain supplementary road surface images. The drone can focus on capturing images of the interference area based on preset flight parameters, such as altitude, speed, and angle. The acquisition angle of the supplementary road surface image and the acquisition angle of the initial road surface image form a preset angle to ensure that road surface information is obtained from different perspectives.

[0092] In addition to adjusting flight attitude, drones can also use zoom lenses or multiple cameras to acquire supplementary images with different resolutions and field of view. Simultaneously, drones can dynamically adjust shooting parameters based on the specific location and size of environmental interference areas to obtain supplementary images of optimal quality.

[0093] When acquiring supplementary road surface images, the system may encounter problems such as inaccurate UAV positioning and image distortion, leading to difficulties in registering the supplementary images with the initial images. To address this issue, the system can incorporate technologies such as high-precision GPS positioning and inertial navigation to improve the UAV's positioning accuracy. Furthermore, the system can use image distortion correction algorithms, such as radial distortion correction, to eliminate the impact of lens distortion on image registration.

[0094] S103. Based on the preset angle, perform spatial projection transformation on the initial road surface image and the supplementary road surface image to establish the pixel correspondence of the environmental interference area.

[0095] The system performs spatial projection transformation on the initial road surface image and the supplementary road surface image according to a preset angle, and establishes the pixel correspondence relationship of the environmental interference area. Specifically, this includes: extracting the boundary points of the environmental interference area in the initial road surface image as the first feature point set; extracting the boundary points of the corresponding area in the supplementary road surface image as the second feature point set; establishing a geometric transformation matrix between the first feature point set and the second feature point set according to the preset angle; performing spatial projection transformation on the supplementary road surface image using the geometric transformation matrix to obtain the transformed supplementary road surface image; performing pixel matching between the transformed supplementary road surface image and the initial road surface image in the environmental interference area to obtain the pixel matching result; and establishing the pixel correspondence relationship of the environmental interference area based on the pixel matching result.

[0096] In practice, for the initial road surface image, the boundary points of the environmental interference areas are extracted as the first feature point set. Boundary points can be obtained using image processing algorithms, such as edge detection and contour extraction. Commonly used edge detection algorithms include the Canny operator and the Sobel operator.

[0097] For the supplementary road surface image, the boundary points of the region corresponding to the environmental interference area are extracted as the second feature point set. Since the supplementary road surface image is taken from different angles, it is necessary to determine the approximate location of the corresponding region in the supplementary road surface image based on the preset angle and the location of the environmental interference area in the initial road surface image, and then extract the boundary points within that region.

[0098] Establishment of the geometric transformation matrix:

[0099] Based on a preset angle, a geometric transformation matrix is ​​established between the first feature point set and the second feature point set. The geometric transformation matrix describes the spatial transformation relationship between the two images. Common transformation types include translation, rotation, scaling, and affine transformation.

[0100] Feature point matching algorithms, such as RANSAC (Random Sample Consensus), can be used to select a subset of points from a first and a second set of feature points as control points to estimate the parameters of the transformation matrix. RANSAC can effectively eliminate mismatched points, improving the robustness of the transformation matrix estimation.

[0101] The specific form of the transformation matrix depends on the type of transformation chosen. For example, for an affine transformation, the transformation matrix is ​​a 3x3 matrix containing parameters such as rotation, scaling, and translation.

[0102] Spatial projection transformation:

[0103] Using the established geometric transformation matrix, a spatial projection transformation is performed on the supplementary road surface image. The purpose of the spatial projection transformation is to convert the supplementary road surface image to the same coordinate system as the initial road surface image, so that the two images can be aligned.

[0104] For each pixel in the supplementary road surface image, its corresponding position in the coordinate system of the initial road surface image is calculated using a geometric transformation matrix, and the pixel value is mapped to the corresponding position to obtain the transformed supplementary road surface image.

[0105] Common spatial projection transformation methods include forward mapping and backward mapping. Forward mapping maps pixels from the source image to the target image, which may result in holes and overlaps. Backward mapping, on the other hand, finds the corresponding position of each pixel in the source image through an inverse transformation and then interpolates it, thus avoiding holes and overlaps.

[0106] Pixel matching:

[0107] The transformed supplementary road surface image is matched pixel-by-pixel with the initial road surface image within the environmental interference area. The purpose of the matching is to find corresponding pixel pairs in the two images and establish the correspondence between them.

[0108] Image similarity measurement methods, such as normalized cross-correlation (NCC) and sum of squared differences (SSD), can be used to calculate the similarity of pixel blocks within a local window in two images and find the pixel pair with the highest similarity.

[0109] To improve the accuracy and efficiency of matching, multi-scale and pyramid strategies can be adopted, first performing coarse matching at low resolution and then fine matching at high resolution.

[0110] Establishing pixel correspondence:

[0111] Based on the pixel matching results, a pixel correspondence relationship is established between the initial road surface image and the supplementary road surface image within the environmental interference area.

[0112] For each pixel in the initial road surface image, find its corresponding pixel in the supplementary road surface image and record the correspondence between them. This can be achieved by creating a mapping table or using a sparse matrix.

[0113] The establishment of pixel correspondence provides a foundation for subsequent image fusion, feature extraction and other operations, enabling the transfer and integration of information between multiple images.

[0114] Through the above steps, the system can perform spatial projection transformation on the initial road surface image and the supplementary road surface image according to a preset angle, and establish the pixel correspondence within the environmental interference area. This provides important information and a foundation for subsequent image compensation and road surface distress analysis.

[0115] In the actual implementation process, appropriate algorithms and strategies such as feature extraction, feature matching, and spatial transformation can be selected according to actual needs and conditions to improve processing accuracy and efficiency. Simultaneously, optimization measures such as feature point filtering, mismatch removal, and multi-view fusion can be introduced to further enhance the robustness and adaptability of the algorithm.

[0116] S104. Calculate the grayscale difference of corresponding pixels in the two acquired images based on the pixel correspondence relationship, and generate a difference matrix based on the grayscale difference.

[0117] The system calculates the grayscale difference between corresponding pixels in the initial and supplementary road surface images of the environmental interference area based on pixel correspondence, and generates a difference matrix based on the grayscale difference. The specific implementation of this step is as follows:

[0118] Obtain the pixel correspondence in the area of ​​environmental interference:

[0119] Using the pixel correspondence established in step S103, the position information of corresponding pixels in the initial road surface image and the supplementary road surface image within the environmental interference area is obtained.

[0120] For each pixel in the initial road surface image, the coordinates of its corresponding pixel in the supplementary road surface image are found through the pixel correspondence relationship.

[0121] Grayscale value extraction:

[0122] For each pair of corresponding pixels within the environmental interference area, their grayscale values ​​are extracted from the initial road surface image and the supplementary road surface image, respectively.

[0123] If the image is in color, it needs to be converted to grayscale first. Common methods for converting color images to grayscale include weighted averaging and average averaging, for example, grayscale value = 0.299 * R + 0.587 * G + 0.114 * B.

[0124] The extracted grayscale values ​​are usually integers between 0 and 255, representing the brightness information of the pixel.

[0125] Gray-scale difference calculation:

[0126] For each pair of corresponding pixels, calculate their grayscale difference in the initial road surface image and the supplementary road surface image.

[0127] The grayscale difference can be obtained through a simple subtraction operation, that is: difference = initial image grayscale value - supplementary image grayscale value.

[0128] The grayscale difference can be positive, negative, or zero, representing the brightness difference between two pixels. A positive value indicates that the pixel in the initial image is brighter than the pixel in the supplementary image, a negative value indicates that the pixel in the initial image is darker than the pixel in the supplementary image, and zero indicates that the two pixels have the same brightness.

[0129] Difference matrix generation:

[0130] Based on the calculated grayscale difference, a difference matrix with the same size as the environmental interference area is generated.

[0131] Each element of the difference matrix corresponds to a pixel within the environmental interference area, and its value is the grayscale difference of that pixel in the initial road surface image and the supplementary road surface image.

[0132] The difference matrix has the same number of rows and columns as the height and width of the environmental interference region, and the position of each element corresponds to the position of a pixel within the environmental interference region.

[0133] Applications of the difference matrix:

[0134] The generated difference matrix can be used for subsequent image compensation and pavement distress analysis.

[0135] By analyzing the values ​​in the difference matrix, brightness variations and abnormal areas within the environmental interference zone can be identified. Larger positive or negative values ​​may indicate that the area has been affected by environmental factors, such as shadows or changes in lighting.

[0136] The difference matrix can be used as input, combined with other feature information, to train machine learning models or deep learning models for automatic detection and identification of road surface defects.

[0137] Through the above steps, the system can calculate the grayscale difference of corresponding pixels in the initial and supplementary road surface images within the environmental interference area based on pixel correspondence, and generate a difference matrix. The difference matrix provides quantitative information on the brightness changes of pixels within the environmental interference area, offering crucial input and basis for subsequent image compensation and road surface damage analysis.

[0138] In the specific implementation process, appropriate grayscale value extraction methods and difference calculation methods can be selected according to actual needs and conditions to improve the accuracy and efficiency of the calculation. Simultaneously, preprocessing steps such as image smoothing and denoising can be introduced to reduce the impact of noise and interference on the difference matrix. Furthermore, other feature representation methods, such as texture features and edge features, can be explored and combined with grayscale differences to improve the accuracy and robustness of pavement distress analysis.

[0139] S105. Compensate the environmental interference area in the initial road surface image according to the difference matrix to obtain the compensated road surface image;

[0140] Region segmentation is performed on both the initial road surface image and the supplementary road surface image. A graph-based segmentation algorithm is used, and its main steps include:

[0141] (1) Represent the image as an undirected graph G=(V,E), where V is the set of pixels and E is the set of edges between pixels.

[0142] (2) Calculate the weight of each edge based on the grayscale difference between pixels. The weight of the edge can be calculated using the Gaussian function, i.e., w(e)=exp(-||I(p)-I(q)||^2 / (2σ^2)), where I(p) and I(q) are the grayscale values ​​of the two pixels p and q connected by edge e, respectively, and σ is the standard deviation of the Gaussian function.

[0143] (3) The graph is segmented according to the edge weights. Initially, each pixel is treated as an independent region. Then, adjacent regions are merged step by step according to the size of the edge weights until the preset number of regions or the region size threshold is reached. The merging criterion is: if the edge weight between two regions is less than the maximum value of the edge weights within each region, then the two regions are merged.

[0144] After region segmentation, both the initial road surface image and the supplementary road surface image are divided into N small regions, denoted as {R1, R2, ..., RN}.

[0145] For each small region Ri, calculate the average of its corresponding elements in the difference matrix D, which serves as the measure of interference level di for that region. That is:

[0146] di = sum(D(x, y)) / |Ri|, where (x, y) are the pixel coordinates within region Ri, and |Ri| is the number of pixels in region Ri.

[0147] Based on the interference level di, the fusion weights of the initial image and the supplementary image within region Ri are adaptively determined. A piecewise linear mapping function is designed as follows to map di to the fusion weights wi of the supplementary image:

[0148] wi = 0, if di <= λ1;

[0149] wi = (di - λ1) / (λ2 - λ1), if λ1 <di<λ2;

[0150] wi = 1, if di>= λ2;

[0151] Here, λ1 and λ2 are two empirical thresholds that can be set according to the characteristics and noise level of the road surface image. When di is less than or equal to λ1, the weight of the supplementary image is 0, meaning the initial image is used entirely; when di is greater than or equal to λ2, the weight of the supplementary image is 1, meaning the supplementary image is used entirely; when di is between λ1 and λ2, the weight of the supplementary image increases linearly. The fusion weight of the initial image is 1-wi.

[0152] For each pixel (x, y) in a small region Ri, the gray values ​​of the initial image I0 and the supplementary image I1 are weighted and averaged according to the fusion weight wi to obtain the compensated gray value Ic: Ic(x, y) = (1-wi) * I0(x, y) + wi *I1(x, y);

[0153] By stitching together all the compensated small areas, the final compensated road surface image Ic is obtained.

[0154] During compensation processing, the system may encounter problems such as insufficient or over-compensation, leading to a decrease in the quality of the compensated road surface image. To address this issue, the system can introduce a compensation effect evaluation mechanism. By calculating indicators such as the similarity and sharpness of the images before and after compensation, the system can automatically adjust the compensation parameters to achieve the optimal compensation effect. For example, the system can use the gradient mean as a sharpness indicator, optimizing the compensation parameters by maximizing the gradient mean to ensure that the sharpness of the compensated road surface image reaches its best.

[0155] S106. Based on preset feature parameters, perform feature matching on the compensated road surface image to obtain road surface distress information with a similarity to the preset feature parameters greater than a preset threshold.

[0156] After obtaining the compensated pavement image, the system performs feature matching on the compensated pavement image based on preset feature parameters to detect and identify pavement distress information. The preset feature parameters can include features such as the shape, size, and texture of the distress, used to describe the typical characteristics of different types of distress. The system calculates the similarity between features in the compensated pavement image and the preset feature parameters, identifies image regions with a similarity greater than a preset threshold, and identifies them as the corresponding type of pavement distress.

[0157] In addition to matching using preset feature parameters, the system can also employ machine learning and deep learning methods to automatically learn and extract features of pavement defects. For example, the system can use a convolutional neural network (CNN) to train on a large number of labeled pavement defect samples to learn deep feature representations of different types of defects, and then use the trained CNN model to identify and classify defects in the compensated pavement images.

[0158] In the above embodiments, by controlling a drone to acquire images of the environmentally disturbed area twice from different angles, an initial road surface image and a supplementary road surface image with a preset angle are obtained. Spatial projection transformation is used to establish pixel correspondence, and the gray-level difference of corresponding pixels is calculated to generate a difference matrix, thereby compensating for the environmentally disturbed area. This compensation processing method based on dual-angle acquisition can eliminate the influence of environmental lighting, shadows, and other factors on the quality of the road surface image, improving the clarity and contrast of the road surface image. By performing feature matching on the compensated image, road surface defects can be identified more accurately, reducing misjudgments and omissions caused by environmental factors. The compensated image has a higher signal-to-noise ratio and more stable image features, improving the reliability and accuracy of defect detection results and providing more reliable data support for subsequent road maintenance decisions.

[0159] Furthermore, before calculating the grayscale difference of corresponding pixels in the two acquired images based on pixel correspondence in step S104 of the above embodiment, the system acquires historical image data of the environmental interference area and extracts image feature change sequences from the historical image data.

[0160] Analyze the temporal correlation of image feature change sequences to obtain the feature change patterns;

[0161] Determining the feature classification threshold based on the feature variation pattern specifically includes: calculating the fluctuation range of feature values ​​in the feature variation pattern and determining the stable interval within the fluctuation range; calculating the mean and standard deviation of feature values ​​within the stable interval; dividing the feature distribution interval based on the mean and standard deviation; and determining the boundary value of the feature distribution interval as the feature classification threshold.

[0162] The features of the environmentally disturbed area are reclassified using a feature classification threshold to obtain the feature reclassification results;

[0163] The weights are calculated by setting grayscale differences for pixels within the environmental interference area based on the feature reclassification results.

[0164] The weights calculated from the grayscale differences are used as correction parameters for pixel correspondence.

[0165] The system retrieves historical image data corresponding to the current environmental interference area from a historical database. This historical image data can be images of the same road segment previously acquired, or images of other road segments with similar environmental characteristics. The historical image data should undergo preprocessing, such as image alignment and scale normalization, to ensure comparability with images of the current environmental interference area.

[0166] Image features related to environmentally disturbed areas, such as brightness, contrast, and texture, are extracted from historical image data. Time series analysis is then performed on the extracted image features to obtain feature change sequences. These sequences reflect the trends and patterns of image feature changes over historical periods.

[0167] Time series analysis methods, such as autocorrelation analysis and cross-correlation analysis, are used to study the temporal correlation of image feature change sequences. By analyzing the periodicity, trend, and stationarity of the feature change sequences, the regularity and patterns of feature changes are obtained.

[0168] Based on the characteristic variation pattern, calculate the fluctuation range of the characteristic values, that is, the maximum and minimum values ​​of the characteristic values ​​within a historical period. Within the fluctuation range, determine a stable interval, representing the range where the characteristic values ​​are relatively stable and concentrated. The stable interval can be determined using statistical methods, such as truncated mean or median. Calculate the mean and standard deviation of the characteristic values ​​within the stable interval to characterize the degree of concentration and dispersion of the characteristic values. Based on the mean and standard deviation, divide the characteristic distribution intervals. For example, the characteristic values ​​can be divided into low-value intervals, medium-value intervals, and high-value intervals, corresponding to the range of mean minus standard deviation, mean, and mean plus standard deviation, respectively. Determine the boundary values ​​of the characteristic distribution intervals as feature classification thresholds for subsequent feature reclassification.

[0169] Using a defined feature classification threshold, image features within the current environmental interference area are reclassified. Pixels within the interference area are categorized into different classes based on the range of their feature values, yielding the feature reclassification result. This result reflects the attribution and similarity of pixels within historical feature variation patterns.

[0170] Based on the feature reclassification results, weights are assigned to pixels within environmental interference areas for grayscale difference calculation. Different weight values ​​can be assigned to pixels of different categories, reflecting their importance and reliability in grayscale difference calculation. For example, pixels belonging to stable ranges can be assigned higher weights, while pixels belonging to fluctuating ranges can be assigned lower weights. These grayscale difference calculation weights can serve as correction parameters for pixel correspondence, used to adjust and optimize the correspondence between pixels and the calculation of grayscale differences.

[0171] When establishing pixel correspondences and calculating grayscale differences, the grayscale difference calculation weight is used as a correction parameter to adjust the original calculation results. For pixel pairs with higher weights, the correspondences and grayscale difference calculation results will be more preserved and enhanced; while for pixel pairs with lower weights, the correspondences and grayscale difference calculation results will be somewhat suppressed and weakened. By introducing grayscale difference calculation weights, the influence of environmental interference factors on pixel correspondences and grayscale difference calculations can be reduced to a certain extent, improving the reliability and accuracy of the results.

[0172] The system incorporates historical image data analysis and feature reclassification mechanisms into the original pixel correspondence and grayscale difference calculation processes to optimize and improve the results. This method leverages the feature change patterns and temporal correlations inherent in historical data. By reclassifying and weighting image features in areas of current environmental interference, it achieves dynamic adjustment and optimization of pixel correspondence and grayscale difference calculations.

[0173] In the implementation process, appropriate historical data ranges, feature extraction methods, time series analysis techniques, and threshold determination strategies can be selected based on actual needs and conditions to improve the algorithm's adaptability and robustness. Simultaneously, other optimization measures, such as feature selection and weight learning, can also be considered to further enhance the algorithm's performance and effectiveness.

[0174] In the above embodiments, by analyzing the feature change sequences in historical image data, temporal correlations and change patterns are extracted, and a feature classification system based on historical data is established. The analysis of feature change patterns reveals the dynamic evolution characteristics of environmental interference areas, providing statistical support for feature classification. The feature classification threshold determined based on these change patterns has stronger adaptability and rationality, and the feature reclassification results are more consistent with the actual situation. By setting grayscale difference calculation weights for pixels within the environmental interference area, a correction mechanism considering historical change characteristics is established. This weight correction mechanism based on historical data improves the accuracy of grayscale difference calculation, making the environmental interference compensation effect more consistent with the actual road surface conditions and enhancing the reliability of the defect detection results.

[0175] The first embodiment described above primarily illustrates a pavement distress detection method based on dual-angle image acquisition and environmental interference compensation using unmanned aerial vehicles (UAVs). This method can effectively identify the location and morphological characteristics of pavement distress. However, to further assess the severity and development trend of pavement distress, in-depth analysis of the detected distress is necessary. The following section combines... Figure 2 Another method for detecting highway pavement defects based on UAV image analysis is described in the embodiments of this application:

[0176] Please see Figure 2 This is another flowchart illustrating a method for detecting highway pavement defects based on UAV image analysis in an embodiment of this application.

[0177] S201. Obtain the pavement structure parameters of the area corresponding to the pavement distress information;

[0178] After obtaining pavement distress information, the system needs to further acquire the corresponding pavement structural parameters for the distressed areas. These parameters can include information such as pavement material type, pavement thickness, and subgrade type. These parameters are crucial for analyzing the causes and development trends of the distress. The system can obtain these pavement structural parameters by querying pavement engineering design documents and conducting on-site surveys.

[0179] In addition to directly acquiring pavement structure parameters, the system can also indirectly infer or estimate pavement structure parameters through techniques such as image analysis and sensor measurements. For example, image texture analysis can be used to determine the type of pavement material, and lidar can be used to measure pavement thickness.

[0180] S202. Calculate the stress distribution value of the damaged area in the pavement distress information to obtain the stress distribution map;

[0181] The system calculates the stress distribution values ​​in the damaged areas of the pavement, obtaining a stress distribution map. This map characterizes the stress magnitude at each point within the damaged area. After acquiring the pavement structural parameters, the system calculates the stress distribution in the damaged areas, generating a stress distribution map. This map visually displays the stress state within the damaged area, providing a basis for analyzing the severity and development trend of the damage.

[0182] The system can utilize methods such as finite element analysis and numerical simulation to establish a mechanical model of the pavement structure based on pavement structure parameters and damage information, and calculate the stress distribution values ​​in the damaged area. By visualizing and rendering the stress distribution values, a stress distribution map is generated, which intuitively shows the stress magnitude and distribution at each point within the damaged area.

[0183] In addition to static analysis, the system can also incorporate dynamic loads and environmental factors to simulate the stress changes in the affected area under actual working conditions, obtaining a dynamic stress distribution map. This dynamic analysis can more realistically reflect the stress characteristics and evolution patterns of the defects.

[0184] When calculating stress distribution values, the system may encounter problems such as large computational load and long processing time, affecting analysis efficiency. To address this, the system can employ techniques such as parallel computing and optimization algorithms to improve the speed and efficiency of stress calculation. For example, GPUs can be used to accelerate finite element analysis, and adaptive mesh generation can be used to reduce the computational scale. Simultaneously, the system can pre-build a stress distribution template library for different fault types. For common fault types, the corresponding templates can be directly called, avoiding redundant calculations and improving analysis efficiency.

[0185] S203. Determine the stress transmission direction based on the changing trend of stress values ​​in the stress distribution diagram;

[0186] Based on the stress distribution map, the system can further analyze the characteristics of stress transmission and diffusion within the affected area, and determine the main direction of stress transmission. The system can identify areas of high and low stress distribution and concentration by analyzing the gradient changes in stress values ​​on the stress distribution map, and thus infer the direction of stress transmission. Generally, stress will transmit from high-stress areas to low-stress areas, diffusing along the direction of the maximum stress gradient.

[0187] In addition to analyzing stress gradients, the system can also utilize mechanical concepts such as principal stress directions and stress streamlines to more accurately characterize the direction and path of stress transmission. For example, by calculating the principal stress direction vector field, the dominant direction of stress transmission can be obtained; by tracing stress streamlines, the detailed path of stress transmission can be obtained.

[0188] S204. Determine the potential expansion area of ​​the disease based on the stress transmission direction and pavement structure parameters;

[0189] The system determines the potential expansion area of ​​the distress based on the stress transmission direction and pavement structure parameters. Specifically, this includes: dividing the stress influence sector along the stress transmission direction, where the stress influence sector is a fan-shaped area that starts from the distress area and expands along the stress transmission direction; extracting the pavement structure parameters within the stress influence sector; calculating the stress attenuation coefficient based on the pavement structure parameters within the stress influence sector; determining the stress influence boundary based on the stress attenuation coefficient; and defining the area enclosed by the stress influence boundary as the potential expansion area.

[0190] After determining the direction of stress transfer, the system can further predict the potential expansion area of ​​the disease. The potential expansion area refers to the region where new diseases may develop or existing diseases may be exacerbated under the influence of the current disease. Identifying potential expansion areas is crucial for timely prevention and control of disease spread and for developing maintenance strategies.

[0191] The system first divides stress-affected sectors according to the direction of stress transmission. A stress-affected sector is a fan-shaped area that radiates outwards from the damaged area along the direction of stress transmission, representing the range of stress transmission influence. Then, the system extracts pavement structural parameters within the stress-affected sector, such as pavement material, thickness, and subgrade condition, and analyzes the impact of these parameters on stress transmission and attenuation. Next, the system calculates the stress attenuation coefficient within the stress-affected sector. The stress attenuation coefficient represents the rate of stress attenuation during transmission and is closely related to the pavement structural parameters. The system can establish a relationship model between the stress attenuation coefficient and pavement structural parameters using empirical formulas, finite element analysis, and other methods, and perform quantitative calculations. Finally, based on the stress attenuation coefficient, the system determines the boundary of the stress influence.

[0192] S205. Calculate the cumulative stress value within the potential extended region;

[0193] After identifying potential expansion areas of the disease, the system needs to further quantify the stress accumulation within these areas to provide a basis for assessing the severity of the disease. The stress accumulation value reflects the overall stress level and trend within the expansion area and is a crucial indicator for determining the risk of disease expansion.

[0194] The system can use methods such as numerical integration and finite element analysis to calculate the cumulative stress value within the potential extended region. Specifically, the extended region is divided into several discrete elements, the stress in each element is integrated, and then the stress integral values ​​of all elements are summed to obtain the cumulative stress value of the entire extended region.

[0195] In addition to spatial accumulation, the system can also consider the stress accumulation effect over time. Due to the long-term effects of traffic loads, environmental factors, etc., the stress in the extended area will continuously accumulate and evolve. The system can incorporate methods such as time series analysis and fatigue damage models to simulate the stress accumulation process over time and predict the stress evolution trend in the extended area.

[0196] S206. Classify the risk level of pavement distress information based on the cumulative stress value.

[0197] Based on the calculated cumulative stress values, the system can classify pavement defects into risk levels, intuitively assessing the severity of the defects and their treatment priorities. Risk level classification is of significant guiding importance for developing maintenance plans and rationally allocating resources.

[0198] The system can pre-set risk level thresholds for accumulated stress values. Based on the relationship between the accumulated stress value and the threshold, the system classifies the defects into different risk levels, such as low risk, medium risk, and high risk. The risk level thresholds can be differentiated according to factors such as road grade, traffic volume, and climate conditions to adapt to the actual conditions of different road sections.

[0199] In addition to classifying risk levels based on fixed thresholds, the system can also use a relative comparison method to classify risk levels according to the distribution of stress accumulation values ​​for the same type of disease. For example, for a certain type of disease, stress accumulation values ​​in the top 20% are classified as high risk, 20%-50% as medium risk, and those below 50% as low risk, without any restrictions here.

[0200] In the above embodiments, a pavement distress development prediction model was established using mechanical analysis methods, enabling distress assessment to move beyond mere visual characteristics. By calculating stress distribution and transmission direction, the expansion trend of distress can be accurately predicted. Furthermore, combining analysis with pavement structural parameters allows for a more accurate assessment of the actual impact of distress on the pavement structure. Risk level classification based on cumulative stress values ​​provides a quantitative assessment of the severity of distress from a mechanical perspective, offering a more accurate basis for risk assessment and facilitating the development of more targeted maintenance plans.

[0201] Furthermore, this embodiment proposes an early warning method for pavement defects based on brightness transfer network analysis. This method not only focuses on the compensation effect of environmental disturbances but also further mines the pavement structure information contained in dual-angle images. By analyzing brightness transfer characteristics and light intensity abrupt change patterns, a mapping relationship between pavement structure state and image features is established, enabling early identification of potential defects. Specifically:

[0202] Analyzing the brightness distribution network of the initial road surface image and the supplementary road surface image yields a brightness transfer feature map:

[0203] The system first performs brightness analysis on the initial and supplementary road surface images, extracting the brightness values ​​of each pixel and constructing a brightness distribution matrix. Then, by calculating the brightness difference between adjacent pixels in the brightness distribution matrix, a brightness transfer vector is obtained, representing the direction and intensity of brightness transfer in the image. Finally, the brightness transfer vector is mapped onto the image plane to generate a brightness transfer feature map, which visually reflects the characteristics of brightness transfer and distribution in the image.

[0204] In addition to directly calculating the brightness difference, the system can also introduce other brightness descriptors, such as brightness gradient and brightness direction, to more comprehensively characterize the brightness transfer features. Furthermore, it can employ multi-scale analysis and image pyramid techniques to extract brightness transfer features at different scales and resolutions, capturing brightness change information at different levels.

[0205] Identify regions of abrupt changes in light intensity within the brightness transfer feature map and extract the boundaries of these changes.

[0206] Based on the luminance transfer feature map, the system can further identify regions of abrupt changes in light intensity within the image. These regions refer to areas where the luminance transfer vector changes drastically, typically corresponding to discontinuities or anomalies in the road surface structure. The system can automatically detect and locate these regions by analyzing the direction and magnitude changes in the luminance transfer vector and setting a threshold for these abrupt changes.

[0207] After identifying regions of abrupt changes in light intensity, the system extracts the boundaries of these regions, obtaining the light intensity change boundaries. These boundaries serve as the dividing line between the abrupt light intensity change region and its surrounding area, reflecting the edge of the change in road surface structure. The system can employ edge detection algorithms, such as the Canny operator and the Sobel operator, to extract the edges of the light intensity change regions, obtaining the pixel coordinate sequence of the light intensity change boundaries.

[0208] Calculate the spatiotemporal evolution characteristics of light intensity abrupt change boundaries to identify weak areas in the road surface structure:

[0209] After extracting the light intensity abrupt change boundary, the system further analyzes the spatiotemporal evolution characteristics of the light intensity abrupt change boundary to identify weak areas in the pavement structure. The spatiotemporal evolution characteristics reflect the changing patterns of the light intensity abrupt change boundary in time and space, and are closely related to the stability of the pavement structure.

[0210] The system can calculate the spatiotemporal evolution characteristics of light intensity abrupt change boundaries by comparing and analyzing them over multiple time periods. For example, the displacement of the light intensity abrupt change boundary reflects the deformation or movement trend of the pavement structure; the deformation of the light intensity abrupt change boundary reflects the stress concentration or relaxation of the pavement structure; and the density of the light intensity abrupt change boundary reflects the integrity and uniformity of the pavement structure.

[0211] By comprehensively analyzing the spatiotemporal evolution characteristics of light intensity abrupt change boundaries, the system can identify weak areas in the pavement structure. Weak areas in the pavement structure refer to areas with low structural strength, poor stability, and susceptibility to damage. They are usually manifested as abnormal changes in light intensity abrupt change boundaries, such as large displacements, sudden deformations, and increased density.

[0212] Based on the weak areas of the pavement structure and the spatiotemporal evolution characteristics, the initial pavement image is used to identify the disease-sensitive areas:

[0213] Based on the identified weak areas in the pavement structure, the system identifies distress-sensitive areas in the initial pavement image. Distress-sensitive areas refer to regions that, despite having weak pavement structures, are prone to damage due to environmental factors. By identifying these distress-sensitive areas, targeted early warning and maintenance decisions can be made.

[0214] The system first maps weak areas of the pavement structure onto the initial pavement image, obtaining the pixel locations of these weak areas. Then, by incorporating the spatiotemporal evolution characteristics of light intensity abrupt change boundaries, such as displacement direction and deformation trends, the system expands and refines the weak areas to obtain disease-sensitive regions. These disease-sensitive regions typically cover the weak areas of the pavement structure and their surrounding areas, while also considering the potential impact of environmental factors.

[0215] When calibrating disease-sensitive areas, the system can use algorithms such as image segmentation and region growing to automatically delineate the boundaries of these areas. Simultaneously, expert knowledge and empirical rules can be incorporated to verify and correct the calibration results for disease-sensitive areas, improving the accuracy and reliability of the calibration.

[0216] Determine environmental compensation parameters based on the distribution characteristics of disease-sensitive areas:

[0217] After identifying the disease-sensitive areas, the system further analyzes the distribution characteristics of these areas to determine the environmental compensation parameters. These parameters are set by adjusting the compensation algorithm's parameters based on the characteristics of the disease-sensitive areas during the initial pavement image compensation process, in order to achieve better compensation results.

[0218] The system can calculate the distribution characteristics of sensitive areas for road damage, such as area, shape, and location, and assess the proportion and impact of these areas in road surface images. Then, based on these distribution characteristics, the system adaptively determines environmental compensation parameters, such as compensation intensity, compensation range, and compensation direction. For example, for sensitive areas with large areas and wide impact, the system can appropriately increase the compensation intensity and range; for sensitive areas with irregular shapes and scattered locations, the system can adopt a local compensation strategy for targeted compensation processing.

[0219] When determining environmental compensation parameters, the system can employ techniques such as heuristic search and parameter optimization. By establishing a compensation effect evaluation function, it can automatically search for and optimize compensation parameters to achieve the best compensation effect. Simultaneously, an adaptive learning mechanism can be introduced to dynamically adjust compensation parameters based on changes in the distribution characteristics of disease-sensitive areas, achieving adaptive optimization of the compensation treatment.

[0220] Selective compensation processing is performed on the initial road surface image using optimized environmental compensation parameters to obtain the compensated road surface image:

[0221] Based on optimized environmental compensation parameters, the system performs selective compensation processing on the initial road surface image. Selective compensation refers to applying different compensation strategies to different areas of the image according to the distribution of areas sensitive to road damage, in order to achieve more accurate and effective compensation results.

[0222] Specifically, the system first divides the initial pavement image into distress-sensitive and non-sensitive areas. Then, for distress-sensitive areas, the system employs optimized environmental compensation parameters, such as a larger compensation intensity and a wider compensation range, for focused compensation processing. For non-sensitive areas, the system uses a relatively weaker compensation strategy, such as a smaller compensation intensity and a narrower compensation range, for general compensation processing. Through selective compensation processing, the system can ensure the compensation quality of sensitive areas while reducing over-compensation of non-sensitive areas, thereby improving overall compensation efficiency.

[0223] When performing selective compensation processing, the system can employ various compensation algorithms, such as histogram equalization, contrast enhancement, and color balance, flexibly selecting and combining these algorithms according to the characteristics and needs of different areas. Simultaneously, image fusion technology can be introduced to fuse the initial road surface image with other auxiliary information, such as illumination distribution maps and depth maps, to further improve the compensation processing effect.

[0224] Feature matching is performed on the compensated pavement image to obtain early warning information of pavement distress:

[0225] After obtaining the compensated pavement image, the system performs feature matching to identify potential pavement defects and obtain early warning information for these defects. Feature matching involves comparing the features in the compensated pavement image with a pre-established pavement defect feature template to identify areas with high similarity, which are then designated as potential defect areas.

[0226] The system first extracts features from the compensated pavement image, obtaining feature descriptors such as texture, color, and shape. Then, the system performs similarity matching between the extracted features and templates in a pavement distress feature template library, calculating a similarity score between each template and the image features. Finally, based on the similarity scores and a preset threshold, the system identifies potential pavement distress areas and generates early warning information for pavement distress, including distress type, location, and severity.

[0227] When performing feature matching, the system can employ various similarity metrics, such as Euclidean distance, cosine similarity, and Hamming distance, selecting the appropriate metric based on the feature type and matching requirements. Simultaneously, machine learning algorithms, such as support vector machines and random forests, can be introduced to automatically learn and optimize feature matching rules by training on pavement defect sample data, thereby improving the accuracy and generalization ability of defect identification.

[0228] Furthermore, the system can integrate other information, such as pavement materials, traffic flow, and weather conditions, to conduct comprehensive analysis and risk assessment of early warning information for pavement defects, providing more comprehensive and reliable early warning results. Simultaneously, the system can interface with the pavement maintenance management system to promptly push early warning information to relevant departments and personnel, enabling early detection, early warning, and early treatment of pavement defects, minimizing their impact on road safety and performance.

[0229] In the above embodiments, by utilizing the rich information contained in dual-angle road surface images and analyzing the brightness transmission characteristics and light intensity abrupt change patterns, weak areas of the road surface structure are identified. Combined with environmental compensation processing and feature matching technology, early warning and risk assessment of road surface defects are achieved, improving the accuracy and timeliness of defect identification and providing important data support for road maintenance decisions.

[0230] The system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference needed]. Figure 3 This is a schematic diagram of the physical device structure of a highway pavement defect detection system based on UAV image analysis provided in an embodiment of this application.

[0231] It should be noted that, Figure 3 The structure of the system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0232] like Figure 3 As shown, the system includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 302 or a program loaded from storage portion 308 into Random Access Memory (RAM) 303, such as executing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.

[0233] The following components are connected to I / O interface 305: input section 306 including a camera, infrared sensor, etc.; output section 307 including a liquid crystal display (LCD) and speakers, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0234] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the various functions defined in the present invention.

[0235] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. The transmitted data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.

[0236] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0237] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the system described in the above embodiments; or it may exist independently and not assembled into the system. The storage medium carries one or more computer programs that, when executed by a processor of a system, cause the system to implement the methods provided in the above embodiments.

[0238] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0239] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0240] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0241] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for detecting pavement defects on highways based on UAV image analysis, characterized in that, include: Control the drone to collect initial road surface images of the highway along a preset flight path; The environmental interference area is determined based on the initial road surface image and grayscale analysis. The drone is controlled to adjust its flight attitude and perform a second image acquisition on the environmental interference area to obtain a supplementary road surface image. The acquisition angle of the supplementary road surface image and the acquisition angle of the initial road surface image form a preset angle. Based on the preset angle, the initial road surface image and the supplementary road surface image are spatially projected and transformed to establish the pixel correspondence of the environmental interference area; Historical image data of the environmental interference area is obtained, and image feature change sequences are extracted from the historical image data; The temporal correlation of the image feature change sequence is analyzed to obtain the feature change pattern; Determine the feature classification threshold based on the aforementioned feature change pattern; The features of the environmental interference region are reclassified using the aforementioned feature classification threshold to obtain the feature reclassification result. Based on the feature reclassification results, the grayscale difference of the pixels in the environmental interference area is set to calculate the weight; The weights calculated from the grayscale differences are used as correction parameters for the pixel correspondence. Based on the pixel correspondence, the grayscale difference of the corresponding pixels in the two acquired images of the environmental interference area is calculated, and a difference matrix is ​​generated based on the grayscale difference. The environmental interference area in the initial road surface image is compensated according to the difference matrix to obtain a compensated road surface image; Based on preset feature parameters, feature matching is performed on the compensated road surface image to obtain road surface defect information with a similarity greater than a preset threshold to the preset feature parameters.

2. The method according to claim 1, characterized in that, The determination of environmental interference areas based on the initial road surface image and grayscale analysis specifically includes: Perform grayscale analysis on the initial road surface image to obtain a grayscale image; The grayscale image is divided into multiple image blocks of equal size; Calculate the mean gray level and standard deviation gray level for each image block; Image blocks whose grayscale standard deviation is greater than a first preset threshold and whose grayscale mean is greater than a second preset threshold are marked as candidate interference regions. The candidate interference region is extended to obtain the extended region; The region within the extended area whose grayscale change value is greater than a third preset threshold is defined as an environmental interference region.

3. The method according to claim 1, characterized in that, The step of performing spatial projection transformation on the initial road surface image and the supplementary road surface image according to the preset angle to establish the pixel correspondence of the environmental interference area specifically includes: Extract the boundary points of the environmental interference area in the initial road surface image as the first feature point set; Extract the boundary points of the region corresponding to the environmental interference area in the supplementary road surface image as the second feature point set; Establish a geometric transformation matrix between the first feature point set and the second feature point set based on the preset included angle; The supplementary road surface image is spatially projected and transformed using the geometric transformation matrix to obtain the transformed supplementary road surface image. The transformed supplementary road surface image and the initial road surface image are matched pixel by pixel within the environmental interference area to obtain the pixel matching result. The pixel correspondence in the environmental interference area is established based on the pixel matching results.

4. The method according to claim 1, characterized in that, The step of determining the feature classification threshold based on the feature change pattern specifically includes: Calculate the fluctuation range of the characteristic values ​​in the characteristic change pattern, and determine the stable interval within the fluctuation range; Calculate the mean and standard deviation of the eigenvalues ​​within the stability interval; The characteristic distribution intervals are divided based on the mean and the standard deviation; The boundary values ​​of the feature distribution interval are determined as the feature classification threshold.

5. The method according to claim 1, characterized in that, After obtaining pavement distress information with a similarity greater than a preset threshold to the preset feature parameters, the method further includes: Obtain the pavement structure parameters of the area corresponding to the pavement distress information; Calculate the stress distribution value of the damaged area in the pavement distress information to obtain a stress distribution map, which represents the stress magnitude at each point in the damaged area; The stress transmission direction is determined based on the changing trend of stress values ​​in the stress distribution diagram. The potential expansion area of ​​the disease is determined based on the stress transmission direction and the pavement structure parameters. Calculate the cumulative stress value within the potential extended region; The risk level of the pavement distress information is classified based on the accumulated stress value.

6. The method according to claim 5, characterized in that, The step of determining the potential expansion area of ​​the defect based on the stress transmission direction and the pavement structure parameters specifically includes: The stress-affected sector is divided along the stress transmission direction. The stress-affected sector is a fan-shaped area that starts from the diseased area and extends along the stress transmission direction. Extract the road surface structure parameters within the stress-affected sector; Calculate the stress attenuation coefficient based on the road surface structure parameters within the stress-affected sector; The stress influence boundary is determined based on the stress attenuation coefficient. The region enclosed by the stress-affected boundary is defined as the potential extended region.

7. A highway pavement defect detection system based on UAV image analysis, characterized in that, The system includes: One or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the system to perform the method as described in any one of claims 1-6.

8. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the system, the system performs the method as described in any one of claims 1-6.

9. A computer program product, characterized in that, When the computer program product is run on the system, the system performs the method as described in any one of claims 1-6.

Images