Bridge engineering anomaly identification method and system based on unmanned aerial vehicle image acquisition

Abstract

The application discloses a bridge engineering anomaly identification method and system based on unmanned aerial vehicle image acquisition, relates to the technical field of image processing, and comprises the following steps: according to a bridge form, preset observation points are observed, and the observation points are subjected to multi-unmanned aerial vehicle cooperative path planning; original image data are collected in real time based on unmanned aerial vehicle cooperative paths, a visual inertial estimation mechanism is set, the visual angle of the unmanned aerial vehicle is synchronously modified to complete image coverage splicing, bridge image data are generated, and a three-dimensional space view is updated; a bridge surface crack segmentation model is constructed, a bridge anomaly position is identified, and anomaly coordinate data are generated; damage quantization index division is performed on the identified damage area, key damage evaluation indexes are obtained, and the digital intelligent transformation and upgrading of unmanned aerial vehicle bridge detection are promoted, so that the safety of the bridge is effectively ensured and the service life of the bridge is prolonged.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for anomaly identification in bridge engineering based on UAV image acquisition. Background Technology

[0002] As an important part of the transportation network, bridges are increasingly damaged due to the growing prevalence of overloading. In reality, the detection and location of bridge anomalies often rely on manual information collection and assessment, leading to problems such as lagging detection technology and insufficient workers in bridge maintenance, making it impossible to detect and repair bridge anomalies in a timely manner.

[0003] Currently, a Chinese invention application with application number 202410557619.6 discloses a method for detecting anomalies in bridge engineering based on UAV image acquisition. This method mainly includes: analyzing and processing historical bridge data to obtain the average pixel value and actual displacement data of the bridge image; using a UAV to acquire images of the bridge, obtaining real-time bridge images; and eliminating anomalies based on the analysis results. This method involves acquiring bridge images via UAV and reading historical bridge data, performing binarization processing on the acquired images and historical data to obtain grayscale and deep grayscale images, calculating the grayscale and deep grayscale images, comparing images from different times, determining whether the bridge has anomalies based on the comparison results, and then manually detecting and eliminating anomalies.

[0004] The above methods do not achieve high-precision image detection, fail to accurately detect images acquired by drones, and lack three-dimensional geometric information evaluation. Summary of the Invention

[0005] The technical problem solved by this invention is that high-precision image detection has not been achieved, images acquired by UAVs have not been accurately detected, and three-dimensional geometric information evaluation is lacking.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] The first aspect is a method for anomaly identification in bridge engineering based on UAV image acquisition, which includes the following steps:

[0008] Step S1: Based on the bridge shape, preset observation points and perform multi-UAV collaborative path planning for the observation points;

[0009] Step S2: Real-time acquisition of raw image data based on UAV collaborative path, setting up visual inertial estimation mechanism, synchronously modifying UAV viewpoint to complete image overlay stitching, generating bridge image data and updating 3D spatial view;

[0010] Step S3: Construct a segmentation model of cracks on the bridge surface, identify abnormal locations on the bridge, and generate abnormal coordinate data;

[0011] Step S4: Divide the identified damaged areas into damage quantification indicators to obtain key damage evaluation indicators.

[0012] Preferably, step S1 includes the following sub-steps:

[0013] Step S11: Use a drone to take images of the bridge from four directions: top, bottom, left, and right, to obtain a preliminary view of the bridge shape. The bridge shape includes length, width, height, span, number of piers, and geometric shape.

[0014] Step S12: Construct a three-dimensional spatial view with the bridge length as the x-axis, width as the y-axis, and height as the z-axis. Divide the three-dimensional spatial view into three layers horizontally. Mark three vertical observation points for each pier from high to low. The observation points are spaced at the same distance and are parallel to the z-axis. Mark several horizontal observation points according to the length of the bridge. The horizontal observation points are parallel to the x-axis.

[0015] Step S13: Label the vertical and horizontal observation points from 1 to n to generate permutation codes [001, 002, 003, ...]. Deploy six UAVs and fly according to the six modules allocated according to the bridge length. Each UAV flies to cover the positions of the observation points included in each module. Use the DQN algorithm to train and generate a UAV cooperative path.

[0016] Preferably, step S2 includes the following sub-steps:

[0017] Step S21: Using the observation point as the shooting point, the UAV flies and collects the current data of the bridge according to the UAV cooperative path and the preset overlap width to obtain the raw image data.

[0018] Step S22: Set up a visual inertial estimation mechanism. For each new frame of raw image data acquired, extract sparse feature points. The sparse feature points include steel plate welds, bolt edges, concrete spalling pits, expansion joint interfaces, crack edges, rust patches, structural joints, and component outlines.

[0019] Step S23: When sparse feature points of the next frame of the new image are collected, the distance function is used to evaluate the distance similarity between the current sparse feature point and the sparse feature point of the previous frame. The sparse feature point with the smallest distance function value in the new image is selected as the matching point with the previous frame image. All the obtained matching points are used as the query set. The exhaustive method is used to match the sparse feature points of each frame image with the matching points of the current query set one by one. The successfully matched sparse feature points are used as the query set and iterated continuously until the last frame image is matched and the bridge feature matching points are generated.

[0020] Step S24: The original image data is stitched together according to the positions of the bridge feature matching points to generate bridge image data.

[0021] Preferably, step S2 further includes:

[0022] The original image data is first processed to generate feature point coordinates and corresponding descriptors. The original image data under different cameras is pre-integrated and measured using an inertial measurement unit to obtain the relative change value of each image. The relative change value includes relative rotation change, relative velocity change and relative position change. The bridge feature matching points and the relative change value are second processed to output the optimal pose estimate and update the three-dimensional spatial view.

[0023] The first process is used to extract sparse feature points from the image using the SuperPoint feature extractor;

[0024] The second processing is used for visual-inertial joint optimization, to find the error value between the current bridge feature matching point and the current pose projection onto the three-dimensional spatial view, and to use the least squares method to calculate the minimum cost function value of the visual projection error and the integral error.

[0025] Preferably, step S3 includes the following sub-steps:

[0026] Step S31: Using the improved maximum inter-class variance method, the segmentation threshold of the bridge image data is calculated to obtain the grayscale image threshold.

[0027] Step S32: Traverse the bridge image data. If the candidate pixel value of each sample is lower than the grayscale image threshold, mark the candidate pixel as an undamaged area. If the candidate pixel value of each sample reaches or exceeds the grayscale image threshold, mark the candidate pixel as a damaged area.

[0028] Step S33: By reverse matching projection of point cloud, the damaged area and the undamaged area are pixel-fused. The damaged area is marked with a pixel of 1, and the undamaged area is marked with a pixel of 0. The damaged area is mapped to the current three-dimensional space view according to the pose of the bridge image data to generate abnormal coordinate data.

[0029] Preferably, the specific method for calculating the segmentation threshold of the bridge image data using the improved Otsu's method is as follows:

[0030] The bridge image data is preprocessed using the Laplacian of Gaussian thresholding algorithm, followed by first pixel processing and second pixel processing. The first pixel processing is used to pre-set an assumed threshold to initially classify the damaged areas of the bridge image data, obtaining damaged and undamaged pixel classes. The average grayscale value of the initial pixels in the damaged and undamaged pixel classes is calculated according to the class, yielding a first grayscale value and a second grayscale value. The second pixel processing is used to statistically analyze the probability of the first and second grayscale values ​​appearing in the current image, obtaining a first probability and a second probability. The average grayscale value M of all pixels in the current bridge image data is calculated, and then the variance of the damaged interval is calculated. The expression for the variance of the damaged interval is:

[0031] ;

[0032] in, The variance of the damaged interval. The first probability, The second probability, The first grayscale value, This is the second grayscale value. This represents the average grayscale value of all pixels.

[0033] The assumed threshold with the largest variance in the damaged interval of the bridge image data is selected as the grayscale image threshold.

[0034] Preferably, step S4 specifically includes:

[0035] The damaged area is traversed to calculate the total length of the damaged area, and the actual physical length of the damaged area is calculated based on the resolution of the bridge image data.

[0036] Along the damaged area, a normal line is drawn at regular intervals. The intersection length of the normal line with the contour of the damaged area is calculated and used as the local width of the current damaged area point. The average value of the local width is calculated to generate the width of the damaged interval. The actual physical width of the damaged area is calculated using the resolution of the bridge image data.

[0037] Based on the actual physical length and actual physical width, the damaged area is calculated, the damaged area percentage is obtained based on the percentage of the damaged area to the surface area of ​​the bridge component, a damage threshold is set, the damaged area percentage is used to determine the damage, and a damage level is generated.

[0038] The identified damage levels are marked in the three-dimensional spatial view, generating a damage level and evaluation report.

[0039] Preferably, the specific method for determining the damage based on the proportion of the damaged area by setting a damage threshold is as follows:

[0040] The damage determination includes a first determination, a second determination, and a third determination. When the proportion of the damaged area is less than the damage threshold, the damage level is set to minor.

[0041] When the proportion of the damaged area is within the damage threshold, the damage level is set to medium.

[0042] When the proportion of the damaged area is greater than the damage threshold, the damage level is set to severe.

[0043] Preferably, the damage level and evaluation report specifically includes: damage discovery time, discovery location, damage level, and name of the bridge section where the damage occurred;

[0044] The damage detection time is the time when the drone acquires the images;

[0045] The location of the discovery is the corresponding abnormal coordinate data in the three-dimensional spatial view;

[0046] The damage level is rendered as a visual color classification based on the severity of the damage.

[0047] Secondly, a bridge engineering anomaly identification system based on UAV image acquisition includes a UAV path planning module, an image acquisition and stitching module, an anomaly damage identification module, and a quantitative assessment module.

[0048] The path planning module is used to perform multi-UAV collaborative path planning for the observation points based on the bridge shape and preset observation points.

[0049] The image acquisition and stitching module is used to acquire raw image data in real time according to the UAV collaborative path generated by the path planning module, and set up a visual inertial estimation mechanism to synchronously modify the UAV's perspective to complete image overlay stitching, generate bridge image data and update the three-dimensional spatial view.

[0050] The abnormal damage identification module is used to construct a bridge surface crack segmentation model using the improved maximum inter-class variance method, identify abnormal bridge locations, and generate abnormal coordinate data.

[0051] The quantitative assessment module is used to divide the identified damaged areas into quantitative indicators, classify the damage levels according to the size of the abnormal areas, and obtain key damage evaluation indicators.

[0052] The beneficial effects of this invention are as follows: By utilizing the collaborative operation of multiple drones and synchronously flying in zones according to preset observation points, the detection task that traditionally requires several days or even weeks can be shortened to a few hours, greatly improving data acquisition efficiency. Drones can complete the detection from the air, far away from the bridge structure, with zero interference to the bridge structure. Drone data collection and detection can easily reach hidden parts that are difficult for humans to access, such as the top of high piers, the inside of box girders of long-span bridges, and cable systems, achieving detection without blind spots. In addition, through crack segmentation models and damage quantification algorithms, the geometric parameters such as the length, width, and area of ​​cracks can be accurately measured, generating key evaluation indicators such as the proportion of damaged area and cracking degree, eliminating the subjectivity and ambiguity of manual visual inspection. This embodiment promotes the digital and intelligent transformation and upgrading of drone bridge inspection, effectively ensuring bridge safety and extending its service life. Attached Figure Description

[0053] Figure 1 A flowchart illustrating the steps of a bridge engineering anomaly identification method based on UAV image acquisition, as provided in an embodiment of the present invention;

[0054] Figure 2 This is a basic flowchart of a bridge engineering anomaly identification system based on UAV image acquisition, provided as an embodiment of the present invention. Detailed Implementation

[0055] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0056] Example 1, referring to Figure 1 This paper presents a method for anomaly identification in bridge engineering based on UAV image acquisition, including the following steps:

[0057] Step S1: Based on the bridge shape, preset observation points and perform multi-UAV collaborative path planning for the observation points;

[0058] Step S2: Real-time acquisition of raw image data based on UAV collaborative path, setting up visual inertial estimation mechanism, synchronously modifying UAV viewpoint to complete image overlay stitching, generating bridge image data and updating 3D spatial view;

[0059] Step S3: Construct a segmentation model of cracks on the bridge surface, identify abnormal locations on the bridge, and generate abnormal coordinate data;

[0060] Step S4: Divide the identified damaged areas into damage quantification indicators to obtain key damage evaluation indicators.

[0061] By utilizing multi-drone collaborative operations and synchronous flight across pre-set observation points, inspection tasks that traditionally take days or even weeks can be completed within hours, significantly improving data acquisition efficiency. Drones can perform inspections from the air, far from the bridge structure, causing zero interference. Drone data collection and inspection can easily reach hidden areas that are difficult for humans to access, such as the tops of high piers, the interior of box girders in long-span bridges, and cable systems, achieving comprehensive inspection without blind spots. In addition, through crack segmentation models and damage quantification algorithms, the geometric parameters such as crack length, width, and area can be accurately measured, generating key evaluation indicators such as damage area ratio and cracking degree, eliminating the subjectivity and ambiguity of manual visual inspection. This embodiment promotes the digital and intelligent transformation and upgrading of drone bridge inspection, effectively ensuring bridge safety and extending its service life.

[0062] Step S1 includes the following sub-steps:

[0063] Step S11: Use a drone to take images of the bridge from four directions: top, bottom, left, and right, to obtain a preliminary understanding of the bridge's shape, which includes its length, width, height, span, number of piers, and geometric shape.

[0064] Step S12: Construct a three-dimensional spatial view with the bridge length as the x-axis, width as the y-axis, and height as the z-axis. Divide the three-dimensional spatial view into three layers horizontally. Mark three vertical observation points for each pier from high to low. The observation points are spaced at the same distance and are parallel to the z-axis. Mark several horizontal observation points according to the length of the bridge. The horizontal observation points are parallel to the x-axis.

[0065] Step S13: Label the vertical and horizontal observation points from 1 to n to generate permutation codes [001, 002, 003, ...]. Deploy six drones and fly them according to the six modules allocated according to the bridge length. Each drone flies to cover the positions of the observation points included in each module. Use the DQN algorithm to train and generate a drone cooperative path.

[0066] First, drones are used to take rapid aerial photos from four basic directions. Through techniques such as oblique photography, the macroscopic three-dimensional shape and key geometric parameters (such as length, span, and number of piers) of the bridge are quickly reconstructed, providing a basic digital model for subsequent detailed inspection. Based on the preliminary model, a complex three-dimensional bridge structure is meshed and divided into three layers in height (Z-axis) (such as upper bridge deck system, middle main beam, and lower pier). Observation points are regularly arranged on the piers (vertical) and bridge length (horizontal) to form a standardized three-dimensional point grid covering all key surfaces of the bridge. The bridge is divided into six sections along the length direction (X-axis). Each drone is responsible for all observation points in one section. The DQN algorithm is used to calculate the optimal flight path within its module for each drone.

[0067] In this embodiment, a large task is decomposed into multiple sub-tasks that can be executed in parallel by spatial partitioning, which are then executed by multiple drones simultaneously, greatly improving efficiency. Secondly, the DQN algorithm is introduced to dynamically find the shortest path covering all points through self-learning, ensuring efficient data collection for large-scale bridge inspection tasks.

[0068] Step S2 includes the following sub-steps:

[0069] Step S21: Using the observation point as the shooting point, the UAV flies and collects the current data of the bridge according to the UAV cooperative path and the preset overlap width to obtain the raw image data.

[0070] Step S22: Set up a visual inertial estimation mechanism. For each new frame of raw image data acquired, extract sparse feature points. The sparse feature points include steel plate welds, bolt edges, concrete spalling pits, expansion joint interfaces, crack edges, rust patches, structural joints, and component outlines.

[0071] Step S23: When sparse feature points of the next frame of the new image are collected, the distance function is used to evaluate the distance similarity between the current sparse feature point and the sparse feature point of the previous frame. The sparse feature point with the smallest distance function value in the new image is selected as the matching point with the previous frame image. All the obtained matching points are used as the query set. The exhaustive method is used to match the sparse feature points of each frame image with the matching points of the current query set one by one. The successfully matched sparse feature points are used as the query set and iterated continuously until the last frame image is matched and the bridge feature matching points are generated.

[0072] Step S24: The original image data is stitched together according to the location of the bridge feature matching points to generate bridge image data.

[0073] The drone follows the cooperative path generated by S1 and ensures a preset overlap during shooting. A visual-inertial estimation mechanism is introduced. For each new image frame, not only are sparse feature points representing the structural features of the bridge (such as welds, cracks, and corrosion) extracted, but IMU data is also fused to estimate the drone's precise position and attitude (pose) in real time and at high frequency. This provides accurate spatiotemporal correlation information for image stitching. Then, a distance function is used. Based on the principle of continuity between adjacent frames, the distance function quickly finds the most similar points in a small range near the feature points of the previous frame to complete the initial matching of adjacent frames. All matching points are then treated as a continuously growing query set. New image frames need to be matched one-to-one with this global set. New image information is continuously registered in the same global coordinate system, which effectively prevents error accumulation and ensures the consistency of the entire model.

[0074] Step S2 also includes:

[0075] The original image data is first processed to generate feature point coordinates and corresponding descriptors. The original image data under different cameras is pre-integrated and measured using an inertial measurement unit to obtain the relative change value of each image. The relative change value includes relative rotation change, relative velocity change and relative position change. The bridge feature matching points and relative change values ​​are second processed to output the optimal pose estimate and update the three-dimensional space view.

[0076] The first process is used to extract sparse feature points from the image using the SuperPoint feature extractor;

[0077] The second processing step is used for joint visual-inertial optimization. It finds the error value between the current bridge feature matching point and the current pose projection onto the 3D spatial view, and uses the least squares method to calculate the minimum cost function value of the visual projection error and the integral error.

[0078] The first processing step uses the SuperPoint neural network model to process the image, obtaining feature point coordinates and corresponding descriptors. Feature point coordinates are the pixel positions of a set of salient points in the image (e.g., [x, y]), and feature descriptors are high-dimensional vectors (e.g., 256-dimensional) corresponding to each feature point, used to uniquely describe the visual appearance around the point for matching in different images. Assuming a 640x480 image is input into SuperPoint, the output is, for example, 500 feature points and their corresponding 500 256-dimensional descriptor vectors. The inertial measurement unit integrates the IMU data in the two frames of the vector image to obtain a relative change value. The relative rotation change is the number of rotations of the drone between the two shots, the relative position change is the direction of movement of the drone between the two shots, and the relative speed change is the magnitude of the speed change of the drone between the two shots.

[0079] The second processing step is to perform joint visual-inertial optimization based on the relative change value of the pre-integrated output. A known 3D map point is projected onto the current image plane according to the currently estimated UAV pose to obtain a 2D point. The pixel distance between this 2D point and the actually observed 2D feature points is calculated. The optimization objective is to minimize the sum of this distance of all matching points. The optimized pose and the new map point are added to the system to update and maintain the global 3D map of the bridge in real time. This embodiment deeply fuses visual and inertial data to output the UAV's own precise position status in real time at a high frequency.

[0080] Step S3 includes the following sub-steps:

[0081] Step S31: Using the improved maximum inter-class variance method, the segmentation threshold of the bridge image data is calculated to obtain the grayscale image threshold.

[0082] Step S32: Traverse the bridge image data. If the candidate pixel value of each sample is lower than the grayscale image threshold, label the candidate pixel as an undamaged area; if the candidate pixel value of each sample reaches or exceeds the grayscale image threshold, label the candidate pixel as a damaged area.

[0083] Step S33: Through point cloud reverse matching projection, perform pixel fusion on the damaged area and the undamaged area. Label the pixels in the damaged area as 1 and the pixels in the undamaged area as 0, and map the damaged area to the current three-dimensional space view according to the pose of the bridge image data to generate abnormal coordinate data.

[0084] According to the bridge image data, use an improved Otsu method to calculate the global segmentation threshold. The improvement lies in being able to better handle the situation of uneven illumination or complex background in the bridge image, generating an optimal grayscale image threshold T. Traverse each pixel (candidate pixel) of the original image. If the pixel grayscale value < T, it is considered that the pixel belongs to the undamaged area (such as intact concrete); if the pixel grayscale value ≥ T, it is considered that the pixel belongs to the damaged area (such as cracks, rust spots, because they are usually darker). For each pixel point marked as 1 (damaged) in the binary mask image, through the camera imaging model, a ray in the three-dimensional space is calculated in reverse. This ray will intersect with the existing bridge three-dimensional point cloud model, and the intersection point is the real position (X, Y, Z) of the damaged pixel in the three-dimensional space; this embodiment replaces the traditional method that completely relies on manual visual recognition. The algorithm can automatically and quickly locate the suspected damaged areas from a large number of images, greatly improving the analysis efficiency.

[0085] The specific method for calculating the segmentation threshold of the bridge image data using the improved Otsu method is as follows:

[0086] Preprocess the bridge image data using the Laplacian of Gaussian threshold algorithm, and then perform the first pixel processing and the second pixel processing on the bridge image data. The first pixel processing is used to preset a hypothetical threshold to preliminarily classify the damaged areas of the bridge image data, obtaining the damaged pixel class and the undamaged pixel class, calculating the average grayscale values of the initial pixels of the damaged pixel class and the undamaged pixel class according to the category, obtaining the first grayscale value and the second grayscale value. The second pixel processing is used to count the occurrence probabilities of the first grayscale value and the second grayscale value in the current image, obtaining the first probability and the second probability, calculating the average grayscale value M of the global pixels of the current bridge image data, and then calculating the damaged interval variance. The calculation expression of the damaged interval variance is:

[0087] ;

[0088] Where is the damaged interval variance, is the first probability, The second probability, The first grayscale value, This is the second grayscale value. This represents the average grayscale value of all pixels.

[0089] The assumed threshold with the largest variance in the damaged interval of the bridge image data is selected as the grayscale image threshold.

[0090] In this embodiment, the first pixel processing is based on an initial classification with an assumed threshold to obtain a first gray value and a second gray value. The assumed threshold range is 0-255. The calculated variance reflects the degree of separation between the two categories. All possible assumed thresholds are scalarized, and steps S32 and S33 are repeated to filter out the assumed threshold corresponding to the maximum variance, which is the optimal grayscale image threshold. This method effectively suppresses interference from irrelevant backgrounds such as uneven lighting and surface stains, while enhancing the signal of real damage. The improved method has stronger robustness and adaptability to bridge images with complex backgrounds.

[0091] Step S4 specifically includes:

[0092] The damaged area is traversed to calculate the total length of the damaged area, and the actual physical length of the damaged area is calculated based on the resolution of the bridge image data.

[0093] Along the damaged area, a normal line is drawn at regular intervals. The intersection length of the normal line with the contour of the damaged area is calculated and used as the local width of the current damaged area point. The average value of the local width is calculated to generate the width of the damaged interval. The actual physical width of the damaged area is calculated using the resolution of the bridge image data.

[0094] The damaged area is calculated based on the actual physical length and actual physical width. The percentage of the damaged area is obtained based on the percentage of the damaged area to the surface area of ​​the bridge component. A damage threshold is set, and the damage is judged based on the percentage of the damaged area to generate a damage level.

[0095] The identified damage levels are marked in the 3D spatial view to generate a damage level and evaluation report.

[0096] Traverse all connected regions marked as damaged in the binary mask image, extract the central axis of the crack using a skeletonization algorithm and calculate its pixel length, then multiply it by the image resolution to obtain the actual physical length. Draw a normal line along the skeleton line at fixed intervals (e.g., 10 pixels), calculate the length of the line segment (i.e., local width) where the normal line intersects the damage contour, and calculate the average of all local widths, then multiply it by GSD to obtain the actual physical average width. For crack-type damage, the area can be simplified to the physical average width multiplied by the actual physical length. For planar damage (e.g., peeling), the total number of pixels in the damaged area can be directly counted and multiplied by GSD². Based on rule-based damage level determination, set a damage threshold and generate damage levels (slight, moderate, severe).

[0097] The specific method for determining damage based on the proportion of damaged area by setting a damage threshold is as follows:

[0098] Damage assessment includes a first assessment, a second assessment, and a third assessment. The first assessment is that when the proportion of the damaged area is less than the damage threshold, the damage level is set to minor.

[0099] The second criterion is that when the proportion of the damaged area is within the damage threshold, the damage level is set to medium.

[0100] The third criterion is that when the proportion of the damaged area is greater than the damage threshold, the damage level is set to severe.

[0101] In this embodiment, the first criterion is a minor defect that has no impact on structural safety; the second criterion is a significant defect that requires attention and planned repair; and the third criterion is a serious defect that may affect structural safety and requires immediate action. This embodiment enables maintenance management departments to identify the most serious and urgent defects in the entire bridge at a glance, thereby optimizing resource allocation, realizing the transformation from regular maintenance to on-demand maintenance, and improving the efficiency of maintenance funds.

[0102] The damage level and evaluation report specifically includes: the time of damage discovery, the location of discovery, the damage level, and the name of the bridge section where the damage occurred;

[0103] The time of damage detection is the time when the drone acquires the images;

[0104] The location was found to be abnormal coordinate data in the 3D spatial view;

[0105] The damage level is visually categorized by color based on the severity of the damage.

[0106] In this embodiment, the damage level information is divided into different colors in the three-dimensional spatial view, such as slight: green-level 1, moderate: yellow-level 2, severe: red-level 3, and marked at the corresponding position in the three-dimensional spatial view updated in step S2, so that the maintenance management department can quickly identify the defects in the bridges along the entire line.

[0107] Example 2, refer to Figure 2 It provides a bridge engineering anomaly identification system based on UAV image acquisition, including a UAV path planning module, an image acquisition and stitching module, an anomaly damage identification module, and a quantitative evaluation module;

[0108] The path planning module is used to pre-set observation points based on the bridge shape and perform multi-UAV collaborative path planning for the observation points;

[0109] The image acquisition and stitching module is used to acquire raw image data in real time based on the UAV collaborative path generated by the path planning module, and set up a visual inertial estimation mechanism to synchronously modify the UAV's perspective to complete image overlay stitching, generate bridge image data and update the three-dimensional spatial view.

[0110] The abnormal damage identification module is used to construct a bridge surface crack segmentation model using an improved maximum inter-class variance method, identify abnormal bridge locations, and generate abnormal coordinate data.

[0111] The quantitative assessment module is used to classify the identified damaged areas into quantitative indicators, classify the damage levels according to the size of the abnormal areas, and obtain key damage evaluation indicators.

[0112] In this embodiment, the entire implementation method is divided into four modules. The efficiency problem of large-scale bridge inspection is solved by multi-UAV collaborative path planning. Then, visual inertial estimation and real-time stitching are used to ensure the integrity and accuracy of data acquisition. Furthermore, the use of a deep learning model greatly improves the accuracy of crack identification. Finally, the quantitative evaluation provides a scientific basis for decision-making. This embodiment constructs a closed-loop system through multi-UAV collaboration, visual inertial estimation, AI recognition, and quantitative evaluation, realizing the automation and intelligence of the entire bridge anomaly inspection process.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0114] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for anomaly identification in bridge engineering based on UAV image acquisition, characterized in that, Includes the following steps: Step S1: Based on the bridge shape, preset observation points and perform multi-UAV collaborative path planning for the observation points; Step S2: Real-time acquisition of raw image data based on UAV collaborative path, setting up visual inertial estimation mechanism, synchronously modifying UAV viewpoint to complete image overlay stitching, generating bridge image data and updating 3D spatial view; Step S3: Construct a segmentation model of cracks on the bridge surface, identify abnormal locations on the bridge, and generate abnormal coordinate data; Step S4: Divide the identified damaged areas into damage quantification indicators to obtain key damage evaluation indicators. Step S1 includes the following sub-steps: Step S11: Use a drone to capture images of the bridge to obtain the bridge shape, which includes length, width, height, span, number of piers, and geometric shape. Step S12: Construct a three-dimensional spatial view with the bridge length as the x-axis, width as the y-axis, and height as the z-axis. Divide the three-dimensional spatial view into three horizontal layers. Mark three vertical observation points for each pier from high to low. Mark several horizontal observation points according to the bridge length. Step S13: Label the vertical and horizontal observation points from 1 to n to generate permutation codes [001, 002, 003, ...]. Deploy k drones and fly them according to the k modules allocated according to the bridge length. Each drone flies to cover the positions of the observation points included in each module. Use the DQN algorithm to train and generate a drone cooperative path. Step S2 includes the following sub-steps: Step S21: Using the observation point as the shooting point, the UAV flies and collects the current data of the bridge according to the UAV cooperative path and the preset overlap width to obtain the raw image data. Step S22: Set up a visual inertial estimation mechanism. For each new frame of raw image data acquired, extract sparse feature points. The sparse feature points include steel plate welds, bolt edges, concrete spalling pits, expansion joint interfaces, crack edges, rust patches, structural joints, and component outlines. Step S23: When sparse feature points of the next frame of the new image are collected, the distance function is used to evaluate the distance similarity between the current sparse feature point and the sparse feature point of the previous frame. The sparse feature point with the smallest distance function value in the new image is selected as the matching point with the previous frame image. All the obtained matching points are used as the query set. The exhaustive method is used to match the sparse feature points of each frame image with the matching points of the current query set one by one. The successfully matched sparse feature points are used as the query set and iterated continuously until the last frame image is matched and the bridge feature matching points are generated. Step S24: The original image data is stitched together according to the positions of the bridge feature matching points to generate bridge image data; Step S2 further includes: The original image data is first processed to generate feature point coordinates and corresponding descriptors. The original image data under different cameras is pre-integrated and measured using an inertial measurement unit to obtain the relative change value of each image. The relative change value includes relative rotation change, relative velocity change and relative position change. The bridge feature matching points and the relative change value are second processed to output the optimal pose estimate and update the three-dimensional spatial view. The first process is used to extract sparse feature points from the image using the SuperPoint feature extractor; The second processing is used for visual-inertial joint optimization, to find the error value between the current bridge feature matching point and the current pose projection onto the three-dimensional space view, and to use the least squares method to calculate the minimum cost function value of the visual projection error and the integral error. Step S3 includes the following sub-steps: Step S31: Using the improved maximum inter-class variance method, the segmentation threshold of the bridge image data is calculated to obtain the grayscale image threshold. Step S32: Traverse the bridge image data. If the candidate pixel value of each sample is lower than the grayscale image threshold, mark the candidate pixel as an undamaged area. If the candidate pixel value of each sample reaches or exceeds the grayscale image threshold, mark the candidate pixel as a damaged area. Step S33: By reverse matching projection of point cloud, the damaged area and the undamaged area are pixel-fused. The damaged area is marked with a pixel of 1, and the undamaged area is marked with a pixel of 0. The damaged area is mapped to the current three-dimensional space view according to the pose of the bridge image data to generate abnormal coordinate data.

2. The bridge engineering anomaly identification method based on UAV image acquisition as described in claim 1, characterized in that, The specific method for calculating the segmentation threshold of the bridge image data using the improved Otsu's method is as follows: The bridge image data is preprocessed using the Laplacian of Gaussian thresholding algorithm, followed by first pixel processing and second pixel processing. The first pixel processing is used to pre-set an assumed threshold to initially classify the damaged areas of the bridge image data, obtaining damaged and undamaged pixel classes. The average grayscale value of the initial pixels in the damaged and undamaged pixel classes is calculated according to the class, yielding a first grayscale value and a second grayscale value. The second pixel processing is used to statistically analyze the probability of the first and second grayscale values ​​appearing in the current image, obtaining a first probability and a second probability. The average grayscale value M of all pixels in the current bridge image data is calculated, and then the variance of the damaged interval is calculated. The expression for the variance of the damaged interval is: ； in, The variance of the damaged interval. The first probability, The second probability, The first grayscale value, Here, M is the second grayscale value, and M is the average grayscale value of all pixels. The assumed threshold with the largest variance in the damaged interval of the bridge image data is selected as the grayscale image threshold.

3. The bridge engineering anomaly identification method based on UAV image acquisition as described in claim 2, characterized in that, Step S4 specifically includes: The damaged area is traversed to calculate the total length of the damaged area, and the actual physical length of the damaged area is calculated based on the resolution of the bridge image data. Along the damaged area, a normal line is drawn at regular intervals. The intersection length of the normal line with the contour of the damaged area is calculated and used as the local width of the current damaged area point. The average value of the local width is calculated to generate the width of the damaged interval. The actual physical width of the damaged area is calculated using the resolution of the bridge image data. Based on the actual physical length and actual physical width, the damaged area is calculated, the damaged area percentage is obtained based on the percentage of the damaged area to the surface area of ​​the bridge component, a damage threshold is set, the damaged area percentage is used to determine the damage, and a damage level is generated. The identified damage levels are marked in the three-dimensional spatial view, generating a damage level and evaluation report.

4. The bridge engineering anomaly identification method based on UAV image acquisition as described in claim 3, characterized in that, The specific method for determining damage based on the proportion of the damaged area by setting a damage threshold is as follows: The damage determination includes a first determination, a second determination, and a third determination. When the proportion of the damaged area is less than the damage threshold, the damage level is set to minor. When the proportion of the damaged area is within the damage threshold, the damage level is set to medium. When the proportion of the damaged area is greater than the damage threshold, the damage level is set to severe.

5. The bridge engineering anomaly identification method based on UAV image acquisition as described in claim 4, characterized in that, The damage level and evaluation report specifically includes: the time of damage discovery, the location of discovery, the damage level, and the name of the bridge section where the damage occurred; The damage detection time is the time when the drone acquires the images; The location of the discovery is the corresponding abnormal coordinate data in the three-dimensional spatial view; The damage level is rendered as a visual color classification based on the severity of the damage.

6. A bridge engineering anomaly identification system based on UAV image acquisition, applied in the bridge engineering anomaly identification method based on UAV image acquisition as described in any one of claims 1-5, characterized in that, It includes a drone path planning module, an image acquisition and stitching module, an abnormal damage identification module, and a quantitative assessment module; The path planning module is used to perform multi-UAV collaborative path planning for the observation points based on the bridge shape and preset observation points. The image acquisition and stitching module is used to acquire raw image data in real time according to the UAV collaborative path generated by the path planning module, and set up a visual inertial estimation mechanism to synchronously modify the UAV's perspective to complete image overlay stitching, generate bridge image data and update the three-dimensional spatial view. The abnormal damage identification module is used to construct a bridge surface crack segmentation model using the improved maximum inter-class variance method, identify abnormal bridge locations, and generate abnormal coordinate data. The quantitative assessment module is used to divide the identified damaged areas into quantitative indicators, classify the damage levels according to the size of the abnormal areas, and obtain key damage evaluation indicators.

Images