A method for detecting damage rate of highway traffic marking based on aerial images of unmanned aerial vehicle

Abstract

This invention relates to a method for detecting the damage rate of road traffic markings based on UAV aerial images. The invention pertains to the fields of computer vision and road traffic facility detection technology, and particularly to a method for detecting the damage rate of road traffic markings. The purpose of this invention is to address the problem of low accuracy in identifying road traffic markings from UAV aerial images using existing methods. The process is as follows: 1. Acquire aerial images of the road, simultaneously recording the UAV's altitude, attitude, and heading angle data; 2. Output the bounding box and mask of the road area using the model; 3. Obtain the edge map; 4. Obtain the cleaned point set; 5. Output the category ID and bounding box position information of each marking target within the road area; 6. Determine the size and position coordinates of the dashed line detection box and the solid line detection box; 7. Crop the images of arrow-type markings and dashed / solid line markings; 8. Obtain the cleaned binary image of the markings; 9. Calculate the average damage rate of all arrow-type markings, all dashed line markings, and all solid line markings.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and highway traffic facility inspection technology, and in particular to a method for detecting the damage rate of highway traffic markings based on drone aerial images. Background Technology

[0002] Highway traffic markings are a crucial infrastructure for highway traffic safety management. Their integrity and clarity directly impact drivers' access to highway information and the efficiency of traffic organization. Road markings are susceptible to wear and tear, breakage, and other damage due to factors such as vehicle traffic, rain and snow erosion, UV aging, and construction quality, necessitating periodic inspections and quantitative assessments.

[0003] Current inspection methods largely rely on manual on-site measurement or vehicle-mounted data acquisition equipment, resulting in low operational efficiency, difficulties in traffic management, high personnel safety risks, and high costs. Drones, with their high mobility, flexible viewing angles, and wide coverage, are suitable for rapid inspection of road markings. In recent years, deep learning-based object detection and instance segmentation methods have also been widely used in road scene understanding. YOLOv8, as an end-to-end detection / segmentation framework, features fast inference speed, convenient deployment, and good detection accuracy under complex lighting and multi-scale target conditions, making it suitable for rapid localization of road areas and traffic markings in drone aerial photography scenarios.

[0004] During drone inspections, issues may arise in marking recognition and extraction due to varying degrees of wear, complex environmental interference, and significant changes in the perspective and scale of aerial images, thus affecting the accuracy of the results. Furthermore, different types of markings (including arrows, dashed lines, and solid lines) differ significantly in their structural morphology, making it difficult to achieve stable assessments using uniform rules. To address these problems, this invention combines algorithmic innovation with drone inspection technology to propose an assessment method that is more robust to complex scenarios and can automatically output the marking damage rate, thereby improving assessment accuracy and engineering applicability. Summary of the Invention

[0005] The purpose of this invention is to solve the problem of low accuracy in identifying road traffic markings from UAV aerial images using existing methods, and to propose a method for detecting the damage rate of road traffic markings based on UAV aerial images.

[0006] The specific process of a method for detecting the damage rate of road traffic markings based on drone aerial images is as follows:

[0007] Step 1: The drone acquires aerial images of the highway and records the drone's altitude, attitude, and heading angle data.

[0008] The aerial images of the highway captured by the drone are visible light RGB images;

[0009] Step 2: Input the highway aerial images acquired by the UAV in Step 1 into the pre-trained YOLOv8 instance segmentation model. The YOLOv8 instance segmentation model outputs the bounding box and mask of the highway area.

[0010] Step 3: Construct edge bands based on the mask Based on edge band And the drone-acquired highway aerial images from step one. Obtain a constrained graph Based on constrained graphs Obtain the edge map ;

[0011] Step 4: From the edge map Extract the set of edge points ;

[0012] The RANSAC method is used to analyze the set of edge points. Outlier removal is performed to obtain a clean set of points. ;

[0013] Based on purification point set The highway boundary is represented as a two-dimensional polynomial curve.

[0014] Step 5: Input the aerial images of the highway acquired by the UAV in Step 1 into the pre-trained YOLOv8 target recognition model. The YOLOv8 target recognition model outputs the category ID of each road marking target within the highway area and the bounding box location information of each road marking target.

[0015] If the bounding box of the road marking target intersects with the bounding box of the highway area output by the YOLOv8 instance segmentation model in step two, causing the target to be truncated, the road marking target will be judged as an anomaly and removed.

[0016] The category IDs are: white dashed line, white solid line, yellow dashed line, yellow solid line, straight arrow, straight left arrow, straight right arrow, left turn arrow, and right turn arrow;

[0017] Step 6: Determine the size and position coordinates of the dashed and solid line detection boxes;

[0018] The dashed detection box includes white dashed lines and yellow dashed lines;

[0019] Solid line detection boxes include white solid lines and yellow solid lines;

[0020] Step 7: For arrow-shaped markings, use the bounding boxes of the arrow-shaped marking targets from the aerial highway images acquired by the UAV in Step 1. Cropping out the image of the arrow-shaped marking area;

[0021] Arrow markings include three categories: the first category is straight arrows, the second category is straight left arrows and straight right arrows, and the third category is left-turn arrows and right-turn arrows.

[0022] For dashed and solid line markings, the detection boxes for dashed and solid line marking targets are obtained from the highway aerial images acquired by the UAV in step one. Cropping out the image of areas with dashed or solid line markings;

[0023] The types of dashed and solid line markings are white dashed lines, yellow dashed lines, white solid lines, and yellow solid lines;

[0024] Step 8: Convert the cropped image to grayscale. grayscale image Linear enhancement is performed to obtain an enhanced grayscale image. Based on enhanced grayscale images Calculate threshold Based on threshold Obtain the binary image of the caliper. Based on the binary map of the gradation line Obtain the purified binary map of the caliper. ;

[0025] Step 9: Based on the purified binary map of the road markings, calculate the average damage rate of all arrow-type markings; calculate the average damage rate of all dashed-line-type markings; calculate the average damage rate of all solid-line-type markings.

[0026] The beneficial effects of this invention are as follows:

[0027] This invention, based on drone aerial photography and deep learning recognition models, enables automated inspection and quantitative assessment of damage rates of highway traffic markings, significantly improving operational efficiency and reducing personnel safety risks.

[0028] This invention is based on edge band constraint boundary detection and fitting of highway segmentation mask. It extracts edge points and constructs boundary lines only in the edge region of the mask, which can effectively suppress false edges caused by vehicles, shadows and textures inside the highway and improve the stability of highway boundary positioning.

[0029] This invention designs differentiated damage rate calculation strategies for different marking structures such as arrows, dashed lines, and solid lines, and can output comparable quantitative indicators within the same framework;

[0030] In the preprocessing of the grading lines, the present invention adopts an "enhancement + adaptive threshold" binarization strategy, which makes the threshold adaptively adjusted according to the brightness distribution of the grading area, thereby improving the robustness in scenes with shadows, glare and exposure changes.

[0031] This invention, through a detection box construction and inter-frame update mechanism, can reduce the interference of noise points, local occlusion, and recognition effects on dashed / solid line recognition and statistics, thereby improving the stability of the results. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0033] Figure 2 This is a schematic diagram illustrating the extraction effect in the highway area.

[0034] Figure 3 This is a schematic diagram illustrating the effect of lane marking recognition in a highway area.

[0035] Figure 4 This is a schematic diagram of the area indicated by the straight arrows.

[0036] Figure 5 A schematic diagram illustrating the adaptive threshold binarization effect on the straight arrow region;

[0037] Figure 6 This is a schematic diagram of the area indicated by the left arrow.

[0038] Figure 7 A schematic diagram illustrating the adaptive threshold binarization effect on the region with the straight left arrow.

[0039] Figure 8 This is a schematic diagram of the dashed detection box area;

[0040] Figure 9 A schematic diagram illustrating the effect of adaptive threshold binarization on the dashed detection box region;

[0041] Figure 10 This diagram illustrates the proportional relationship between the standard size of a straight arrow marking and a reference scale. It is used to explain the relative proportional relationship between the length and area of ​​different types of arrow markings. The dimensions and proportional parameters shown in the diagram are preferred embodiments and are only used to illustrate the scale construction principle of the present invention. They do not constitute a limitation of the present invention.

[0042] Figure 11 This diagram illustrates the proportional relationship between the standard size of a right-turn arrow marking and a reference scale. It is used to explain the relative proportional relationship between the length and area of ​​different types of arrow markings. The dimensions and proportional parameters shown in the diagram are preferred embodiments and are only used to illustrate the scale construction principle of the present invention. They do not constitute a limitation of the present invention.

[0043] Figure 12 This diagram illustrates the proportional relationship between the standard size of a right-hand arrow marking and a reference scale. It is used to explain the relative proportional relationship between the length and area of ​​different types of arrow markings. The dimensions and proportional parameters shown in the diagram are preferred embodiments and are only used to illustrate the scale construction principle of the present invention. They do not constitute a limitation of the present invention.

[0044] Figure 13 A schematic diagram of constructing the dashed detection box;

[0045] Figure 14This is a schematic diagram of the dashed line detection frame before inter-frame update.

[0046] Figure 15 This is a schematic diagram showing the update of the dashed line detection frame between frames;

[0047] Figure 16 A schematic diagram of the solid line detection box construction. Detailed Implementation

[0048] Specific implementation method one: Combining Figure 1 , 2 16. This embodiment describes a method for detecting the damage rate of road traffic markings based on UAV aerial images. The specific process is as follows:

[0049] Step 1: The drone acquires aerial images of the highway and records the drone's altitude, attitude, and heading angle data.

[0050] The aerial images of the highway captured by the drone are visible light RGB images;

[0051] Step 2: Input the highway aerial images acquired by the UAV in Step 1 into the pre-trained YOLOv8 instance segmentation model (a YOLOv8 instance segmentation model trained in advance using highway aerial images with bounding boxes and mask labels for the highway area). The YOLOv8 instance segmentation model outputs the bounding boxes and masks for the highway area.

[0052] Subsequent road marking recognition and area statistics are limited to the highway area to reduce background interference;

[0053] Step 3: Construct edge bands based on the mask Based on edge band And the drone-acquired highway aerial images from step one. Obtain a constrained graph Based on constrained graphs Obtain the edge map ;

[0054] Step 4: From the edge map Extract the set of edge points ;

[0055] The RANSAC method is used to analyze the set of edge points. Outlier removal is performed to obtain a clean set of points. ;

[0056] Based on purification point set The highway boundary is represented as a two-dimensional polynomial curve.

[0057] Step 5: Input the aerial images of the highway acquired by the UAV in Step 1 into the pre-trained YOLOv8 target recognition model. The YOLOv8 target recognition model outputs the category ID of each road marking target within the highway area and the bounding box location information of each road marking target.

[0058] If the bounding box of the road marking target intersects with the bounding box of the highway area output by the YOLOv8 instance segmentation model in step two, causing the target to be truncated, the road marking target will be judged as an anomaly and removed.

[0059] The category IDs are: white dashed line, white solid line, yellow dashed line, yellow solid line, straight arrow, straight left arrow, straight right arrow, left turn arrow, and right turn arrow;

[0060] Step 6: Determine the size and position coordinates of the dashed and solid line detection boxes;

[0061] The dashed line detection box includes white dashed lines and yellow dashed lines; the solid line detection box includes white solid lines and yellow solid lines.

[0062] Step 7: For arrow-shaped markings, use the bounding boxes of the arrow-shaped marking targets from the aerial highway images acquired by the UAV in Step 1. Cropping out the image of the arrow-shaped marking area;

[0063] Arrow markings include three categories: the first category is straight arrows, the second category is straight left arrows and straight right arrows, and the third category is left-turn arrows and right-turn arrows.

[0064] For dashed and solid line markings, the detection boxes for dashed and solid line marking targets are obtained from the highway aerial images acquired by the UAV in step one. Cropping out the image of areas with dashed or solid line markings;

[0065] The types of dashed and solid line markings are white dashed lines, yellow dashed lines, white solid lines, and yellow solid lines;

[0066] Step 8: Convert the cropped image to grayscale. grayscale image Linear enhancement is performed to obtain an enhanced grayscale image. Based on enhanced grayscale images Calculate threshold Based on threshold Obtain the binary image of the caliper. Based on the binary map of the gradation line Obtain the purified binary map of the caliper. ;

[0067] Step 9: Based on the purified binary map of the road markings, calculate the average damage rate of all arrow-type markings; calculate the average damage rate of all dashed-line-type markings; calculate the average damage rate of all solid-line-type markings.

[0068] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that: in step three, edge bands are constructed based on masks. Based on edge band And the drone-acquired highway aerial images from step one. Obtain a constrained graph Based on constrained graphs Obtain the edge map The specific process is as follows:

[0069] Step 3.1: Mask the highway region output by the YOLOv8 instance segmentation model from Step 2. First, morphological dilation is performed, and then the morphologically dilated mask is eroded to obtain the edge band. ;

[0070] , Represents pixels;

[0071] Step 3.2, Based on edge bands And the drone-acquired highway aerial images from step one. Obtain a constrained graph ; indicates as:

[0072]

[0073] Step 3: For the constrained diagram Execute the Canny operator to obtain the edge map. .

[0074] The other steps and parameters are the same as in Specific Implementation Method 1.

[0075] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that: in step four, from the edge map... Extract the set of edge points ;

[0076] The RANSAC method is used to analyze the set of edge points. Outlier removal is performed to obtain a clean set of points. ;

[0077] Based on purification point set The highway boundary is represented as a two-dimensional polynomial curve.

[0078] The specific process is as follows:

[0079] Step 41: From the edge map Extract the set of edge points ;

[0080] edge point set ;

[0081] in, Represents the set of edge points The first edge point in the middle, Represents the set of edge points The second edge point in the middle, Represents the set of edge points The Middle An edge point, Represents the set of edge points The Middle One edge point; Represents the set of edge points Total number of mid-edge points;

[0082] Step 4.2: Use the RANSAC method to process the set of edge points. Outlier removal is performed to obtain a clean set of points. ;

[0083] Step 4.3: Based on the set of purification points The highway boundary is represented as a two-dimensional polynomial curve. The specific process is as follows:

[0084] Based on purification point set Solving parameters using least squares ,based on and Obtaining a two-dimensional polynomial curve ;

[0085] Among them, based on the set of purification points Solving parameters using least squares The process is as follows:

[0086] Based on purification point set Minimize squared error ,get When taking the minimum value, the corresponding The expression is:

[0087]

[0088] in, Represents the set of purification points The Middle The coordinates of the points Represents the set of purification points The Middle The x-coordinates of the points Represents the set of purification points The Middle The ordinates of the points; Represents the coefficient. , Represents the coefficients of a polynomial; Indicates the order index of a polynomial; Represents the set of purification points The Middle x-coordinates of points of Power of 1.

[0089] Other steps and parameters are the same as in specific implementation method one or two.

[0090] Specific Implementation Method Four: This implementation method differs from one of Specific Implementation Methods One to Three in that: in step six, the size and position coordinates of the dashed line detection box and the solid line detection box are determined;

[0091] The dashed line detection box includes white dashed lines and yellow dashed lines; the solid line detection box includes white solid lines and yellow solid lines.

[0092] The specific process is as follows:

[0093] Step 61: Take the first 30 images from the drone's aerial photos of the highway as the positioning images;

[0094] Step 6.2: After step 5, the category ID and bounding box location information of each target within the localization image are obtained.

[0095] Step 63: Count the position of each dashed bounding box; Count the position of each solid bounding box;

[0096] Step 64: Filter out the maximum height and maximum width within the dashed bounding box;

[0097] Set the height of the dashed detection box to 1.2 times the maximum height, and set the width of the dashed detection box to 3 times the maximum width;

[0098] Set the height of the solid line detection box to 1.5 times that of the dashed line detection box, and set the width of the solid line detection box to 1.2 times that of the dashed line detection box;

[0099] Step 65: Obtain the horizontal and vertical coordinates of each detection box.

[0100] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0101] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that: in step six-five, the horizontal and vertical coordinates of each detection box are obtained; the specific process is as follows:

[0102] Step 651: Set the lateral distance deviation threshold The recommended value range is 1 / 30 to 1 / 20 of the width of the drone-acquired highway aerial image in step one;

[0103] Step 652: Cluster all dashed and solid line bounding boxes; the process is as follows:

[0104] The absolute value of the difference between the x-coordinates of the center points of any two bounding boxes is less than a threshold. The bounding boxes are grouped together;

[0105] Step 653: For each group of markings, there is one detection box (the size of the detection box is determined by Step 64, and the position of each detection box is determined here). The x-coordinate of the center of the detection box is the average of the x-coordinates of all bounding boxes in the group; the y-coordinate of the detection box is the center of the highway aerial image obtained by the UAV in Step 1, and all detection boxes have the same y-coordinate.

[0106] The other steps and parameters are the same as in any of the specific implementation methods one to four.

[0107] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that: in step eight, the cropped image is converted to a grayscale image. grayscale image Linear enhancement is performed to obtain an enhanced grayscale image. Based on enhanced grayscale images Calculate threshold Based on threshold Obtain the binary image of the caliper. Based on the binary map of the gradation line Obtain the purified binary map of the caliper. The specific process is as follows:

[0108] Step 81: Convert the cropped image to grayscale. grayscale image Linear enhancement is performed to obtain an enhanced grayscale image. ;

[0109] The linear enhancement method is as follows:

[0110]

[0111] in, This is the contrast enhancement factor. For brightness bias;

[0112] Step 82: Statistically enhance the grayscale image within the cropped image. mean with standard deviation ;

[0113] When standard deviation At that time, based on the mean with standard deviation Calculate threshold ; indicates as:

[0114]

[0115] in, This is an adjustable coefficient. This is set as the lower limit of the threshold to avoid the threshold being too low;

[0116] When standard deviation At that time, the threshold Represented as:

[0117]

[0118] in, This is the brightness threshold. The standard deviation threshold is used to avoid incorrect extraction of road markings due to excessive light intensity or overly uniform road surface grayness.

[0119] Step 83: Based on threshold Obtain the binary image of the caliper. ; indicates as:

[0120]

[0121] in, To enhance the grayscale image;

[0122] Step 84: Based on the binary map of the datum line Obtain the purified binary map of the caliper. The process is as follows:

[0123] Binary map of the calibration line Perform a morphological closing operation to obtain the closing result;

[0124] Set a threshold for the area of ​​connected components;

[0125] The area threshold for connected components is 0.5% to 2% of the total area of ​​the corresponding bounding box.

[0126] Compare the area of ​​each connected component in the closing operation result with the connected component area threshold;

[0127] If the area of ​​a connected component is less than the area threshold of the connected component, then the corresponding connected component is deleted from the closing operation result.

[0128] If the area of ​​a connected component is greater than or equal to the area threshold of the connected component, then the corresponding connected component is retained in the closing operation result;

[0129] The final closing operation result is the purified binary map of the calibrated lines. .

[0130] The other steps and parameters are the same as those in any of the specific implementation methods one to five.

[0131] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that: in step nine, based on the purified binary map of the markings, the average damage rate of all arrow-type markings is calculated; the average damage rate of all dashed-line-type markings is calculated; and the average damage rate of all solid-line-type markings is calculated. The specific process is as follows:

[0132] Step 91: Calculate the damage rate of each arrow-type marking;

[0133] Step 92: Calculate the damage rate of each dashed line type marking;

[0134] Step 93: Calculate the damage rate of each solid line type marking;

[0135] Step 94: Calculate the average damage rate of all arrow-type markings; calculate the average damage rate of all dashed-line markings; calculate the average damage rate of all solid-line markings.

[0136] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0137] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One through Seven in that: in step 91, the damage rate of each arrow-type marker is calculated; the specific process is as follows:

[0138] Step 911: For each arrow-type marking within the highway area obtained in Step 5, count the number of arrow-type markings. And the arrow types for each arrow category. ;

[0139] Arrow types are denoted as , , Indicates a straight arrow. Indicates left-turn and right-turn arrows. Indicates a straight left arrow and a straight right arrow;

[0140] Step 912, Statistics The first arrow type The purified binary image of the arrow-type markings. White pixel area ;

[0141] Statistical straight arrows The purified binary image of the arrow-type markings. length ;

[0142] Step 913: Due to the different design speeds of different highways, the standard dimensions of the guide arrows in national standard traffic markings also differ, although the length-to-width ratio remains the same. Furthermore, changes in the drone's flight altitude and focal length also affect the arrow area. Therefore, by utilizing the relationship between the guide arrow length and area, the maximum length of the white region in the binary image of the straight arrows obtained from the aerial image is used as the initial reference value. This is then used to calculate the area reference values ​​for various arrows, thereby achieving adaptive detection.

[0143] exist The maximum length is selected as the initial reference value A. It's a variable, there are many. ;

[0144] If a video segment or intersection lacks a straight-ahead arrow, or the straight-ahead arrow is truncated, obscured, or falsely detected, making it impossible to reliably obtain the initial reference value A, then the ratio between pixels and meters is estimated based on the drone's altitude Z and the equivalent focal length in the camera's intrinsic parameters. (meters / pixel);

[0145] For example, using the equivalent focal length in the camera's intrinsic parameters. (pixel) approximation (Or further adjustments can be made based on pitch angle and camera mounting angle); in this case, it can be directly adopted. Convert the actual physical dimensions of road markings to standard pixel dimensions;

[0146] Step 914: Based on the actual length of each type of arrow and actual area The actual coefficient values ​​of each type of arrow are obtained by proportional calculation. ; indicates as:

[0147]

[0148] Here, a 6-meter-long guide arrow is used as the standard for calculation. The area of ​​a straight arrow is 2.16 square meters, the area of ​​a left (right) turn arrow is 2.7791 square meters, and the area of ​​a straight-left (straight-right) arrow is 3.72 square meters. The above arrow length, area, and proportional parameters are preferred embodiments based on current traffic marking design specifications and experimental statistics, used to illustrate the standard area construction method. The specific values ​​can be adjusted according to the actual road grade, design speed, and imaging conditions, and do not constitute a limitation of the present invention.

[0149] Step 915: Calculate the area reference value of each type of arrow based on the actual coefficient values ​​and the initial reference value A. ; indicates as:

[0150]

[0151] Step 916, based on the first The first arrow type The purified binary image of the arrow-type markings. White pixel area and the Area benchmark values ​​for each arrow type Calculate the first Damage rate of each arrow ; indicates as:

[0152] .

[0153] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0154] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One to Eight in that step Nine Two calculates the damage rate of each dashed line type marking; the specific process is as follows:

[0155] Step 921: Set half the width of each detection box as the offset distance range (each detection box is the same size, and there is only one offset distance range).

[0156] Step 922: When a new dashed line is detected, if the horizontal coordinate distance between the bounding box of the new dashed line and the detection box is within the set offset distance range, it is considered to be the same dashed line lane marking and an offset has occurred. Based on the position of the newly detected dashed line, the horizontal coordinate position of the dashed line detection box is updated to the horizontal coordinate position of the new dashed line; otherwise, the existing detection box is retained.

[0157] When a dashed line changes to a solid line at a road intersection or other road segment, the detection box is updated to a solid line.

[0158] Step 923, Dashed Line State Tracking; the specific process is as follows:

[0159] Step 9231: Set the ratio threshold (Recommended value range: 0.1-0.15);

[0160] Step 9232: Based on the proportional threshold and detection frame length Calculate the length threshold ; indicates as:

[0161]

[0162] Step 9233: Calculate the length of the discrete position in the main direction occupied by white pixels within each dashed detection box. ;

[0163] The process is as follows:

[0164] Binary image of the cleaned marking line within each dashed detection box. Statistical set of white pixels Using the long side of the detection frame as the main direction, Projecting onto the principal direction axis, calculate the length of the discrete position in the principal direction occupied by white pixels corresponding to each dashed detection box. ;

[0165] Steps 9, 2, 3, and 4: The first frame from the previous frame... The length of the main direction discrete position occupied by the white pixels in the dashed detection box With length threshold Compare;

[0166] The length of the discrete position of the main direction occupied by white pixels within the same dashed detection box in the current frame and the previous frame. With length threshold Compare;

[0167] like , If the same dashed line detection box in the current frame and the previous frame are not yet in the tracking state, it is determined that the dashed line has entered the detection box, and then the tracking of the dashed line begins and the peak value is initialized, and the peak value is updated in subsequent frames.

[0168] The initial peak value is the first peak value in the previous frame. The larger of the area of ​​the white pixel in the dashed detection box and the area of ​​the white pixel in the corresponding dashed detection box in the current frame;

[0169] like If the same dashed line detection box in the current frame and the previous frame are in tracking mode, it is considered that the dashed line has left the detection box. At this time, the peak value is recorded, and the peak value is the [value]th [value] in the previous frame. The larger value between the area of ​​white pixels in the dashed detection box and the area of ​​white pixels in the corresponding dashed detection box in the current frame is used to stop tracking the dashed line, and the peak value is taken as the area of ​​the dashed line.

[0170] Continue tracking until all dashed detection boxes are completed;

[0171] Step 9235, according to The standard area of ​​the dashed line is obtained by converting the actual dimensions and area of ​​the road dashed line into pixel areas. Based on the standard area of ​​the dashed line and the area of ​​each dashed line Calculate the first Damage rate (in dashed line) ; indicates as:

[0172] .

[0173] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0174] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that step Nine-Three calculates the damage rate of each solid line type marking; the specific process is as follows:

[0175] Step 931: Generate a binary image of the purified marking line within each solid line detection box. Calculate the area of ​​white pixels;

[0176] Using the short side of each solid-line detection box as the horizontal direction, calculate the centroid of all white pixels in the horizontal direction. ;

[0177] Based on the centroid Centered with each solid line detection box , to obtain the lateral offset ; indicates as:

[0178]

[0179] Step 932: Set the width of each detection frame For each solid-line detection box, the short side pixel length (or the horizontal projection length) is given when... If a deviation of the solid line is detected (e.g., a shift in flight direction or a change in the direction of the marking), the horizontal coordinate of the detection box center is moved towards the centroid. Directional movement ;

[0180] when It was determined that the solid line had not shifted.

[0181] in, The adjustable scaling factor is recommended to be between 0.15 and 0.35.

[0182] Step 933: Since the standard area of ​​the solid line is difficult to determine, the quantiles of the statistical results are used to determine it. This involves statistically analyzing the highway aerial images acquired by the UAV in Step 1. The sequence of white pixel areas within all solid-line detection boxes in all frames (a frame may contain multiple detection boxes) is obtained. ;

[0183] in, This represents the area of ​​the white pixels within the first solid-line detection box;

[0184] This represents the area of ​​the white pixel within the second solid-line detection box;

[0185] Indicates the first The area of ​​the white pixels corresponding to each solid line detection box;

[0186] Indicates the first The area of ​​the white pixels corresponding to each solid line detection box;

[0187] This represents the total number of solid-line detection boxes contained in all frames (a frame may contain multiple detection boxes);

[0188] Step 934: Based on the white pixel area sequence Determine the standard area ; indicates as:

[0189]

[0190] in, Indicates taking The 9th percentile;

[0191] Step 935: Based on the area of ​​the white pixel corresponding to each solid-line detection box and the standard area Calculate the solid line damage rate within each solid line detection box; expressed as:

[0192]

[0193] in, Indicates the first The solid line damage rate within each solid line detection box.

[0194] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0195] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A method for detecting the damage rate of highway traffic markings based on drone aerial images, characterized in that: The specific process of the method is as follows: Step 1: The drone acquires aerial images of the highway and records the drone's altitude, attitude, and heading angle data. The aerial images of the highway captured by the drone are visible light RGB images; Step 2: Input the highway aerial images acquired by the UAV in Step 1 into the pre-trained YOLOv8 instance segmentation model. The YOLOv8 instance segmentation model outputs the bounding box and mask of the highway area. Step 3: Construct edge bands based on the mask Based on edge band And the aerial images of the highway acquired by the drone in step one. Obtain a constrained graph Based on constrained graphs Obtain the edge map ; Step four, extracting edge point set from edge map ;​ RANSAC method is used to remove outliers from the edge point set and obtain the purified point set ; Based on purification point set The highway boundary is represented as a two-dimensional polynomial curve. Step 5: Input the aerial images of the highway acquired by the UAV in Step 1 into the pre-trained YOLOv8 target recognition model. The YOLOv8 target recognition model outputs the category ID of each road marking target within the highway area and the bounding box location information of each road marking target. If the bounding box of the road marking target intersects with the bounding box of the highway area output by the YOLOv8 instance segmentation model in step two, causing the target to be truncated, the road marking target will be judged as an anomaly and removed. The category IDs are: white dashed line, white solid line, yellow dashed line, yellow solid line, straight arrow, straight left arrow, straight right arrow, left turn arrow, and right turn arrow; Step 6: Determine the size and position coordinates of the dashed and solid line detection boxes; The dashed detection box includes white dashed lines and yellow dashed lines; Solid line detection boxes include white solid lines and yellow solid lines; Step 7: For arrow-shaped markings, use the bounding boxes of the arrow-shaped marking targets from the aerial highway images acquired by the UAV in Step 1. Cropping out the image of the arrow-shaped marking area; Arrow markings include three categories: the first category is straight arrows, the second category is straight left arrows and straight right arrows, and the third category is left-turn arrows and right-turn arrows. For dashed and solid line markings, the detection boxes for dashed and solid line marking targets are obtained from the aerial highway images acquired by the UAV in step one. Cropping out the image of areas with dashed or solid line markings; The types of dashed and solid line markings are white dashed lines, yellow dashed lines, white solid lines, and yellow solid lines; Step 8: Convert the cropped image to grayscale. grayscale image Linear enhancement is performed to obtain an enhanced grayscale image. Based on enhanced grayscale images Calculate threshold Based on threshold Obtain the binary image of the caliper. Based on the binary map of the gradation line Obtain the purified binary map of the caliper. ; Step 9: Based on the purified binary map of the road markings, calculate the average damage rate of all arrow-type markings; calculate the average damage rate of all dashed-line-type markings; calculate the average damage rate of all solid-line-type markings. 2.The method of claim 1, wherein the method further comprises: In step three, edge bands are constructed based on masks. Based on edge band And the aerial images of the highway acquired by the drone in step one. Obtain a constrained graph Based on constrained graphs Obtain the edge map The specific process is as follows: Step 3.1: Mask the highway region output by the YOLOv8 instance segmentation model from Step 2. First, morphological dilation is performed, and then the morphologically dilated mask is eroded to obtain the edge band. ; , Represents pixels; Step 3.2, Based on edge bands And the aerial images of the highway acquired by the drone in step one. Obtain a constrained graph ; indicates as: Step three, constraining the graph Canny operator is executed to obtain an edge map . 3.The method of claim 2, wherein the method further comprises: The step four extracts the edge point set from the edge map The step four extracts the edge point set from the edge map ; RANSAC method is used to remove outliers from the edge point set and obtain the purified point set ; Based on a set of purification points Expressing a road boundary as a two-dimensional polynomial curve; The specific process is as follows: Step four i. Extracting edge point set from edge map ;​ edge point set ; in, Represents the set of edge points The first edge point in the middle, Represents the set of edge points The second edge point in the middle, Represents the set of edge points The Middle An edge point, Represents the set of edge points The Middle One edge point; Represents the set of edge points Total number of mid-edge points; Step four two, adopt RANSAC method to edge point set Outlier rejection is carried out to obtain the purified point set ; Step 4.3: Based on the set of purification points The highway boundary is represented as a two-dimensional polynomial curve; the specific process is as follows: Based on purification point set Solving parameters using least squares ,based on and The two-dimensional polynomial curve is obtained; wherein the purification point set is based on The parameters are solved by least squares The process is: Based on a set of purification points Minimizing squared error , resulting in The corresponding , the expression is: in, Represents the set of purification points The Middle The coordinates of the points Represents the set of purification points The Middle The x-coordinates of the points Represents the set of purification points The Middle The ordinates of the points; Represents the coefficient. , Represents the coefficients of a polynomial; Indicates the order index of a polynomial; Represents the set of purification points The Middle x-coordinates of points of Power of 1.

4. The method for detecting the damage rate of highway traffic markings based on UAV aerial images according to claim 3, characterized in that: In step six, the size and position coordinates of the dashed and solid detection boxes are determined. The dashed line detection box includes white dashed lines and yellow dashed lines; the solid line detection box includes white solid lines and yellow solid lines. The specific process is as follows: Step 61: Take the first 30 images from the drone's aerial photos of the highway as the positioning images; Step 6.2: After step 5, the category ID and bounding box location information of each target within the localization image are obtained. Step 63: Count the position of each dashed bounding box; Count the position of each solid bounding box; Step 64: Filter out the maximum height and maximum width within the dashed bounding box; Set the height of the dashed detection box to 1.2 times the maximum height, and set the width of the dashed detection box to 3 times the maximum width; Set the height of the solid line detection box to 1.5 times that of the dashed line detection box, and set the width of the solid line detection box to 1.2 times that of the dashed line detection box; Step 65: Obtain the horizontal and vertical coordinates of each detection box.

5. The method for detecting the damage rate of highway traffic markings based on UAV aerial images according to claim 4, characterized in that: In step six-five, the horizontal and vertical coordinates of each detection box are obtained; the specific process is as follows: Step 651: Set the lateral distance deviation threshold ; Step 652: Cluster all dashed and solid line bounding boxes; the process is as follows: The absolute value of the difference between the x-coordinates of the center points of any two bounding boxes is less than a threshold. The bounding boxes are grouped together; Step 653: For each group of markings, there is one detection box. The x-coordinate of the center of the detection box is the average of the x-coordinates of all bounding boxes in the group; the y-coordinate of the detection box is the center of the highway aerial image acquired by the UAV in Step 1. 6.The method of claim 5, wherein the method further comprises: In step eight, the cropped image is converted to a grayscale image. grayscale image Linear enhancement is performed to obtain an enhanced grayscale image. Based on enhanced grayscale images Calculate threshold Based on threshold Obtain the binary image of the caliper. Based on the binary map of the gradation line Obtain the purified binary map of the caliper. ; The specific process is as follows: Step 81: Convert the cropped image to grayscale. grayscale image Linear enhancement is performed to obtain an enhanced grayscale image. The linear enhancement method is as follows: in, This is the contrast enhancement factor. For brightness bias; Step 82: Statistically enhance the grayscale image within the cropped image. mean with standard deviation ; When standard deviation At that time, based on the mean with standard deviation Calculate threshold ; indicates as: in, This is an adjustable coefficient. This is the lower limit of the threshold. When standard deviation At that time, the threshold Represented as: in, The brightness threshold; The standard deviation threshold; Step 83: Based on threshold Obtain the binary image of the caliper. ; indicates as: in, To enhance the grayscale image; Step 84: Based on the binary map of the datum line Obtain the purified binary map of the caliper. The process is as follows: Binary map of the calibration line Perform a morphological closing operation to obtain the closing result; Set a threshold for the area of ​​connected components; Compare the area of ​​each connected component in the closing operation result with the connected component area threshold; If the area of ​​a connected component is less than the area threshold of the connected component, then the corresponding connected component is deleted from the closing operation result. If the area of ​​a connected component is greater than or equal to the area threshold of the connected component, then the corresponding connected component is retained in the closing operation result; The final closing operation result is the purified binary map of the calibrated lines. .

7. The method for detecting the damage rate of highway traffic markings based on UAV aerial images according to claim 6, characterized in that: In step nine, based on the purified binary map of the road markings, the average damage rate of all arrow-type markings is calculated; the average damage rate of all dashed-line-type markings is calculated; and the average damage rate of all solid-line-type markings is calculated. The specific process is as follows: Step 91: Calculate the damage rate of each arrow-type marking; Step 92: Calculate the damage rate of each dashed line type marking; Step 93: Calculate the damage rate of each solid line type marking; Step 94: Calculate the average damage rate of all arrow-type markings; calculate the average damage rate of all dashed-line markings; calculate the average damage rate of all solid-line markings.

8. The method for detecting the damage rate of road traffic markings based on UAV aerial images according to claim 7, characterized in that: In step nine-one, the damage rate of each arrow-type marker is calculated; the specific process is as follows: Step 911: For each arrow-type marking within the highway area obtained in Step 5, count the number of arrow-type markings. And the arrow types for each arrow category. ; Arrow types are denoted as , , Indicates a straight arrow. Indicates left-turn and right-turn arrows. Indicates a straight left arrow and a straight right arrow; Step 912, Statistics The first arrow type The purified binary image of the arrow-type markings White pixel area ; Statistical straight arrows The purified binary image of the arrow-type markings length ; Step 913, in The maximum length is selected as the initial reference value A. If a video segment or intersection lacks a straight-ahead arrow, or if the straight-ahead arrow is truncated or obscured, making it impossible to reliably obtain the initial reference value A, then the ratio between pixels and meters is estimated based on the drone's altitude Z and the equivalent focal length in the camera's intrinsic parameters. ; use Convert the actual physical dimensions of road markings to standard pixel dimensions; Step 914: Based on the actual length of each type of arrow and actual area The actual coefficient values ​​of each type of arrow are obtained by proportional calculation. ; indicates as: Step 915: Calculate the area reference value of each type of arrow based on the actual coefficient values ​​and the initial reference value A. ; indicates as: Step 916, based on the first The first arrow type The purified binary image of the arrow-type markings White pixel area and the Area benchmark values ​​for each arrow type Calculate the first Damage rate of each arrow ; indicates as: 。 9. The method for detecting the damage rate of highway traffic markings based on UAV aerial images according to claim 8, characterized in that: Step nine-two involves calculating the damage rate of each dashed line type marking; the specific process is as follows: Step 921: Set half the width of each detection frame as the offset distance range; Step 922: When a new dashed line is detected, if the horizontal coordinate distance between the bounding box of the new dashed line and the detection box is within the set offset distance range, it is considered to be the same dashed line lane marking and an offset has occurred. Based on the position of the newly detected dashed line, the horizontal coordinate position of the dashed line detection box is updated to the horizontal coordinate position of the new dashed line; otherwise, the existing detection box is retained. Step 923: Tracking the state of the dashed line; The specific process is as follows: Step 9231: Set the ratio threshold ; Step 9232: Based on the proportional threshold and detection frame length Calculate the length threshold ; indicates as: Step 9233: Calculate the length of the discrete position in the main direction occupied by white pixels within each dashed detection box. ; The process is as follows: Binary image of the cleaned marking line within each dashed detection box. Statistical set of white pixels Using the long side of the detection frame as the main direction, Projecting onto the principal direction axis, calculate the length of the discrete position in the principal direction occupied by white pixels corresponding to each dashed detection box. ; Steps 9, 2, 3, and 4: The first frame from the previous frame... The length of the main direction discrete position occupied by the white pixels in the dashed detection box With length threshold Compare; The length of the discrete position of the main direction occupied by white pixels within the same dashed detection box in the current frame and the previous frame. With length threshold Compare; like , If the same dashed line detection box in the current frame and the previous frame are not yet in the tracking state, it is determined that the dashed line has entered the detection box, and then the tracking of the dashed line begins and the peak value is initialized. The initialization peak value is the larger one of the white pixel area in the dashed detection frame in the last frame and the white pixel area in the corresponding dashed detection frame in the current frame. The initialization peak value is the larger one of the white pixel area in the dashed detection frame in the last frame and the white pixel area in the corresponding dashed detection frame in the current frame. like If the same dashed detection box in the current frame and the previous frame are in tracking mode, it is considered that the dashed line has left the detection box. At this time, the peak value is recorded, and the peak value is the [value]th [value] in the previous frame. The larger value between the area of ​​white pixels in the dashed detection box and the area of ​​white pixels in the corresponding dashed detection box in the current frame is used to stop tracking the dashed line, and the peak value is taken as the area of ​​the dashed line. Continue tracking until all dashed detection boxes are completed; Step 9235, according to The standard area of ​​the dashed line is obtained by converting the actual dimensions and area of ​​the road dashed line into pixel areas. Based on the standard area of ​​the dashed line and the area of ​​each dashed line Calculate the first Damage rate (in dashed line) ; indicates as: 。 10.The method of claim 9, wherein the method further comprises: Step nine-three involves calculating the damage rate of each solid line type of marking; the specific process is as follows: Step nine three one, binarize the cleaned reticle image in each solid line detection box Count the white pixel area; With each solid line detection frame short side direction as the lateral direction, the centroid of all white pixels in the lateral direction is calculated ; Based on the centroid Centered with each solid line detection box , to obtain the lateral offset ; indicates as: Step nine three two, set each detection frame width For each solid line detection frame short side pixel length, when The solid line is determined to be offset, and the detection frame center horizontal coordinate is moved to the center of mass Direction ; When the solid line is determined not to have shifted; wherein, is an adjustable proportionality factor; Step 933: Analyze the aerial images of the highway acquired by the UAV in Step 1. The white pixel area corresponding to all solid-line detection boxes contained in all frames is obtained by generating a sequence of white pixel areas corresponding to all solid-line detection boxes. ; wherein, represents the area of the corresponding white pixels within the first solid detection frame; represents the area of white pixels within the 2nd solid detection box; represents the area of white pixels within the i-th solid detection box; and represents the area of white pixels within the i-th solid detection box; and representing the area of white pixels within the corresponding solid detection box; represents the total number of solid detection boxes contained in all frames; Step nine three four, based on the sequence of white pixel areas Determining the standard area ; is represented as: wherein, represents the 9th quantile of the 9th quantile of Step nine three five, based on the corresponding white pixel area in each solid line detection frame and the standard area , the solid line damage rate in each solid line detection frame is calculated; represented as: wherein, represents the solid line damage rate within the first solid line detection box.

Images