Intelligent unmanned aerial vehicle bridge slope crack detection and early warning system

Abstract

The invention relates to the technical field of unmanned aerial vehicle inspection, in particular to an intelligent unmanned aerial vehicle bridge slope crack detection and early warning system, which comprises an unmanned aerial vehicle cruise unit and a detection and early warning management unit, and is characterized in that the unmanned aerial vehicle cruise unit is used for performing autonomous flight inspection on a bridge slope in a target area according to a preset period; the visible light image, the infrared thermal imaging and the three-dimensional laser point cloud data of the target area are synchronously obtained; the detection and early warning management unit is used for processing the image data acquired by the unmanned aerial vehicle cruise unit in real time through a pre-trained crack intelligent analysis algorithm, extracting geometric features and thermodynamic features of the crack, constructing a time sequence model to predict a crack expansion trend and a structure safety risk level, and outputting the crack expansion trend and the structure safety risk level. A multi-level early warning signal is generated based on the risk level and is pushed to the maintenance terminal; the geometrical characteristics comprise crack depth and strike angle, and the thermodynamic characteristics comprise abnormal temperature gradient distribution. The problems of long detection period, high cost and terrain limitation in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of drone inspection, and particularly to an intelligent drone bridge slope crack detection and early warning system. Background Art

[0002] A bridge slope refers to an artificial or natural slope in the abutment, approach road or adjacent area of a bridge project, which is mainly used to stabilize the soil body, prevent soil erosion, ensure the safety of the bridge structure and the surrounding environment, provide stable support for the bridge foundation, and ensure that the bridge remains stable under various loads, bearing the weight transmitted from the upper structure of the bridge and loads such as vehicles and pedestrians.

[0003] Due to factors such as poor geological conditions, rain erosion, and earthquakes, the soil or rock mass of the slope may lose stability. Under long-term load action or due to reasons such as uneven settlement of the foundation, cracks and deformations may occur in the slope, which will weaken the bearing capacity of the slope and may further lead to slope instability.

[0004] Existing crack detection technologies for bridge slopes mainly include methods such as manual inspection and ground monitoring equipment. However, these methods have disadvantages such as long detection cycles, high costs, and being restricted by terrain. Especially for large bridges and complex terrains, traditional detection methods are difficult to meet the requirements of efficient, accurate, and comprehensive detection. At the same time, existing detection methods mainly judge by determining the width of the cracks. For example, natural cracks on the concrete surface will generate a large amount of interference data, resulting in low accuracy and requiring manual verification repeatedly.

[0005] Therefore, the present invention provides an intelligent drone bridge slope crack detection and early warning system to solve the above problems. Summary of the Invention

[0006] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an intelligent drone bridge slope crack detection and early warning system, which solves the problems of long detection cycle, high cost, and terrain limitation existing in the prior art.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows: An intelligent unmanned aerial vehicle (UAV) bridge slope crack detection and early warning system, the system comprising a UAV cruise unit and a detection and early warning management unit; the UAV cruise unit is configured to autonomously fly and inspect the bridge slope within the target area at a preset period, and synchronously acquire visible light images, infrared thermal images and three-dimensional laser point cloud data of the target area; the detection and early warning management unit is configured to process the image data acquired by the UAV cruise unit in real time through a pre-trained crack intelligent analysis algorithm, extract the geometric features and thermodynamic features of the cracks, construct a time series model to predict the crack propagation trend and the structural safety risk level, and generate multi-level early warning signals based on the risk level and push them to the maintenance terminal; the geometric features include crack depth and strike angle, and the thermodynamic features include abnormal temperature gradient distribution.

[0008] A further improvement of the present application lies in that the UAV cruise unit includes: a multi-modal image acquisition module, configured to receive image data transmitted by a visible light camera, an infrared thermal imager and a lidar sensor, and perform spatio-temporal alignment of multi-source data through a time synchronization controller; an edge computing module, configured to compress and preprocess the spatio-temporally aligned multi-source data in real time; an autonomous navigation module, based on the SLAM algorithm, to generate a three-dimensional environment map in real time and plan an optimal inspection path.

[0009] A further improvement of the present application lies in that the detection and early warning management unit includes: a crack intelligent analysis algorithm module, configured to process image data in real time and extract the geometric features and thermodynamic features of the cracks; a threat assessment module, configured to predict the crack propagation trend, with the crack depth sequence as the network input and the future propagation curve as the output; a dynamic early warning module, configured to generate multi-level early warning signals according to the crack parameters and the instability probability and automatically push them to the maintenance management terminal; a data fusion module, configured to associate historical detection data with real-time monitoring data to generate a crack evolution map.

[0010] A further improvement of the present application lies in that the crack intelligent analysis algorithm includes:

[0011] S100. Based on the improved YOLOv5 model, perform initial screening of cracks on the visible light image to eliminate the interference of vegetation and stains. The model loss function is:

[0012] L = λ cls ·L cls + λ box ·L box + λ obj ·L obj (1),

[0013] In expression (1), L cls represents the classification loss, used to distinguish cracks from interfering objects, L boxDenotes the bounding box regression loss, which is used to optimize the crack location accuracy, L obj Denotes the confidence loss, which is used to screen high-confidence crack regions, λ cls , λ box , λ obj respectively represent the corresponding weight coefficients;

[0014] S200. Perform temperature difference analysis on the infrared thermal imaging data to extract the abnormal temperature gradient distribution in the crack region. The expression is:

[0015]

[0016] In expression (2), T(x, y) represents the real-time temperature value of the target point (x, y), which is directly measured by an infrared thermal imager. N represents the number of reference points in the neighborhood around the target point (x, y), and N = 8. T(x i , y i ) represents the temperature value of the i-th reference point in the neighborhood. ΔT(x, y) represents the difference between the target point temperature and the average temperature of the surrounding environment;

[0017] When ΔT(x, y) > T th , it is determined that there is a crack region. Among them, T th represents a preset temperature gradient threshold;

[0018] S300. Integrate the three-dimensional point cloud data, and use the NURBS surface fitting algorithm to fit the discrete crack point cloud into a continuous surface S(u, v), and calculate the crack depth. The expression of S(u, v) is:

[0019]

[0020] In expression (3), S(u, v) represents the coordinates of the surface points after fitting at the parameter space (u, v). By adjusting u and v, the entire surface can be traversed to describe the three-dimensional shape of the crack. The value ranges of u and v are both [0, 1]. R i,j represents the basis function, which is composed of B-spline basis functions and defines the influence weight of the crack control points on the surface shape. P i,j represents the crack control points, and the coordinate values are (x, y, z), representing the geometric characteristics of the crack surface.

[0021] A further improvement of this application is that in step S300, the crack depth d is calculated from the deviation of the surface in the z direction. The expression is:

[0022] d = |S z (u, v) - Z baseline (u, v)| (4),

[0023] In expression (4), Zbaseline Denote the reference surface elevation without cracks, which is obtained through the following steps:

[0024] T100. Extract lidar or photogrammetry data without cracks from the historical database of bridge construction or regular inspections;

[0025] T200. Perform coordinate transformation through the ICP algorithm to align the historical data with the coordinate system of the current detection area. The expression of the target optimization formula is:

[0026]

[0027] In expression (5), R represents a 3×3 orthogonal matrix used for three-dimensional rotation transformation of the historical data point cloud to match the pose of the current data. t represents a 3×1 vector representing the translation amounts in the X, Y, and Z directions, used to adjust the position of the historical data to align with the current data, and P 历史,i represents the three-dimensional point coordinates of the i-th point in the historical data, from the lidar or photogrammetry data without cracks, and P 当前,i represents the three-dimensional point coordinates of the i-th point in the current detection data, corresponding to the same physical position in the historical data, and ||·|| 2 represents the square of the Euclidean distance, used to measure the matching error between the historical point cloud after rotation and translation and the current point cloud;

[0028] T300. Perform surface fitting on the historical point cloud to generate the reference surface elevation in the crack-free state.

[0029] A further improvement of this application is that the evaluation process of the threat assessment module includes:

[0030] Input the crack depth d, the strike angle θ, and the geological parameters of the bridge slope into the finite element analysis model to calculate the structural instability probability P fail , and the expression is:

[0031]

[0032] In expression (6), μ R , μ S are respectively the means of the structural resistance and the load effect, and σ R , σ S are respectively the standard deviations corresponding to the structural resistance and the load effect;

[0033] When the instability probability P fail > the warning threshold, trigger the structural reinforcement suggestion and associate it with the maintenance work order system.

[0034] A further improvement of this application is that the finite element analysis model is constructed based on the ANSYS platform, and the input parameters include the elastic modulus E of concrete, Poisson's ratio v, and crack position coordinates (x, y, z), and the output is the stress distribution nephogram and the critical instability region.

[0035] A further improvement of this application is that the grading logic of the dynamic warning module is as follows:

[0036] Level 1 warning: The crack depth is less than the first warning value and the predicted depth change amount in the next 30 days is less than the first predicted depth change amount value, and it is pushed to the mobile terminal of the inspection personnel; Level 2 warning: The first warning value is less than or equal to the crack depth and the crack depth is less than the second warning value, or the predicted depth change amount in the next 30 days is greater than or equal to the first predicted depth change amount value and less than the second predicted depth change amount value, triggering an audible and visual alarm and generating a maintenance suggestion report; Level 3 warning: The crack depth is greater than or equal to the second warning value or the predicted depth change amount in the next 30 days is greater than or equal to the second predicted depth change amount value, immediately sending an emergency instruction to the supervision platform and starting a secondary review.

[0037] A further improvement of this application is that the pushing methods of the multi-level warning signals include: real-time transmitting the warning information to the supervision platform through the 5G communication module; synchronously generating a detection report, including crack parameters, prediction trends, and maintenance suggestions, and automatically sending it to the designated responsible person by email.

[0038] The beneficial effects of the present invention are as follows: Through the autonomous flight inspection of the drone cruise unit, combined with multi-modal image acquisition and edge computing technology, various data of the target area can be quickly obtained, providing a rich and accurate information basis for subsequent intelligent analysis. The detection and warning management unit can process image data in real time through a pre-trained crack intelligent analysis algorithm, extract the geometric and thermodynamic characteristics of cracks, construct a time series model to predict the crack propagation trend and the structural safety risk level, thereby realizing the accurate prediction and multi-level warning of crack development. It significantly improves the efficiency and accuracy of bridge slope crack detection, reduces the detection cost, breaks through the terrain limitation, and realizes the real-time monitoring and warning of bridge slope cracks. This intelligent warning mechanism not only improves the timeliness of maintenance management but also effectively prevents potential safety risks, providing strong technical support for the safety maintenance of bridge slopes. Description of the Drawings

[0039] Figure 1 It is a schematic structural diagram of an intelligent drone bridge slope crack detection and warning system of the present invention. Detailed Embodiments

[0040] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0041] Existing crack detection technologies for bridge slopes mainly include methods such as manual inspections and ground monitoring equipment. However, these methods have disadvantages such as long detection cycles, high costs, and being restricted by terrain. Especially for large bridges and complex terrains, traditional detection methods are difficult to meet the requirements of efficient, accurate, and comprehensive detection. At the same time, existing detection methods mainly judge by determining the width of cracks. For example, natural cracks on the concrete surface will generate a large amount of interference data, resulting in low accuracy and requiring manual verification repeatedly.

[0042] Therefore, the present invention provides an intelligent unmanned aerial vehicle (UAV) bridge slope crack detection and early warning system, which focuses on detecting the depth of cracks on the bridge slope to solve the above problems.

[0043] The technical solutions will be described in detail below in combination with specific embodiments.

[0044] Embodiment

[0045] Reference Figure 1 , an intelligent UAV bridge slope crack detection and early warning system, the system includes a UAV cruise unit and a detection and early warning management unit;

[0046] The UAV cruise unit is used to autonomously fly and inspect the bridge slope in the target area at a preset cycle, and simultaneously acquire visible light images, infrared thermal images, and three-dimensional laser point cloud data of the target area;

[0047] Specifically, the visible light camera acquires images at a resolution of 2000*2000 pixels and a frame rate of 1Hz; the sensitivity of the infrared thermal imager is ≤0.05°C, and the temperature measurement range is -20°C - 150°C; the point cloud density of the three-dimensional lidar is ≥1000 points per square meter, and the scanning frequency is greater than or equal to 10Hz.

[0048] The detection and early warning management unit is used to process the image data acquired by the UAV cruise unit in real time through a pre-trained crack intelligent analysis algorithm, extract the geometric features and thermodynamic features of the cracks, construct a time series model to predict the crack propagation trend and the structural safety risk level, and generate multi-level early warning signals based on the risk level and push them to the maintenance terminal; the geometric features include crack depth and strike angle, and the thermodynamic features include abnormal temperature gradient distribution.

[0049] In an embodiment of the present application, the UAV cruise unit includes:

[0050] The multi-modal image acquisition module is used to receive the image data transmitted by the visible light camera, the infrared thermal imager and the lidar sensor, and align the multi-source data in space and time through the time synchronization controller;

[0051] The edge computing module is used to compress and preprocess the multi-source data aligned in space and time in real time;

[0052] The autonomous navigation module generates a three-dimensional environmental map in real time based on the SLAM algorithm and plans the optimal inspection path. Its path planning formula is: where P represents the set of paths, d i represents the distance of the i-th section of the path, and t i represents the acquisition time weight, and w1, w2 represent the normalization coefficients.

[0053] In an embodiment of the present application, the detection and early warning management unit includes:

[0054] The crack intelligent analysis algorithm module is used to process the image data in real time and extract the geometric and thermodynamic characteristics of the cracks;

[0055] The threat assessment module is used to predict the crack propagation trend through the LSTM neural network. The network input is the crack depth sequence, and the output is the future propagation curve;

[0056] The dynamic early warning module is used to generate multi-level early warning signals according to the crack parameters and the instability probability and automatically push them to the maintenance management terminal;

[0057] The data fusion module is used to correlate the historical detection data and the real-time monitoring data to generate a crack evolution map.

[0058] Specifically, the crack intelligent analysis algorithm includes the following steps S100-S300:

[0059] S100. Based on the improved YOLOv5 model, perform a preliminary screening of cracks on the visible light image to eliminate the interference of vegetation and stains. The model loss function is:

[0060] L = λ cls ·L cls +λ box ·L box +λ obj ·L obj (1),

[0061] In expression (1), L cls represents the classification loss, which is used to distinguish cracks from interference objects, and L box represents the bounding box regression loss, which is used to optimize the crack positioning accuracy, and L obj represents the confidence loss, which is used to screen high-confidence crack regions, λ cls , λbox , λ obj respectively represent the corresponding weight coefficients;

[0062] S200. Perform temperature difference analysis on the infrared thermal imaging data to extract the abnormal temperature gradient distribution in the crack area. The expression is:

[0063]

[0064] In expression (2), T(x, y) represents the real-time temperature value of the target point (x, y), which is directly measured by an infrared thermal imager. N represents the number of reference points in the neighborhood around the target point (x, y), and N = 8 (i.e., eight neighborhoods). T(x i , y i ) represents the temperature value of the i-th reference point in the neighborhood, and ΔT(x, y) represents the difference between the target point temperature and the average temperature of the surrounding environment;

[0065] When ΔT(x, y) > T th , it is determined that there is a crack area. Among them, T th represents a preset temperature gradient threshold; it should be noted that the preset temperature gradient threshold is dynamically adjusted according to the following factors:

[0066] Historical data: By analyzing the temperature distribution under normal conditions of the bridge slope, the reference threshold is determined.

[0067] Environmental conditions: Consider the influence of external factors such as seasons and weather on the temperature field. For example, the threshold can be appropriately increased in summer.

[0068] Crack characteristics: Combine parameters such as crack width and depth to correct the threshold to improve detection sensitivity.

[0069] In step S200, the principle used is: The crack area causes a change in the thermal conductivity due to material fracture or water seepage, which is manifested as local temperature anomalies (such as sudden temperature rises or drops) in the infrared thermal imaging. Compared with the absolute temperature threshold method, the gradient threshold method can effectively eliminate environmental interference and highlight local anomalies, especially suitable for crack positioning in complex backgrounds.

[0070] S300. Integrate the three-dimensional point cloud data, use the NURBS surface fitting algorithm to fit the discrete crack point cloud into a continuous surface S(u, v), and calculate the crack depth. The expression of S(u, v) is:

[0071]

[0072] In expression (3), S(u, v) represents the coordinates of the surface points after fitting at the parameter space (u, v). By adjusting u and v, the entire surface can be traversed to describe the three-dimensional shape of the crack. The value ranges of both u and v are [0, 1], and R i,j represents the basis function, which is composed of B-spline basis functions and defines the influence weight of the crack control points on the surface shape. P i,j represents the crack control points, with coordinate values (x, y, z), representing the geometric features of the crack surface.

[0073] Using the NURBS surface fitting algorithm to fit the discrete crack point cloud into a continuous surface has the following advantages:

[0074] High precision: The NURBS surface can accurately describe the complex geometry of the crack (such as branches and tortuous trends); Noise resistance: By screening control points and optimizing weights, noise interference in the point cloud data is eliminated; Scalability:

[0075] Supports dynamic addition of control points to adapt to the morphological changes during the crack propagation process.

[0076] In step S300, the crack depth d is calculated from the deviation of the surface in the z direction, and the expression is:

[0077] d = |S z (u, v) - Z baseline (u, v)| (4),

[0078] In expression (4), Z baseline represents the elevation of the reference surface without cracks, which is obtained through the following steps T100 - T300:

[0079] T100: Extract lidar or photogrammetry data when there are no cracks from the historical database of bridge construction or regular inspections;

[0080] T200: Perform coordinate transformation through the ICP algorithm to align the historical data with the coordinate system of the current detection area. The expression of the target optimization formula is:

[0081]

[0082] In expression (5), R represents a 3×3 orthogonal matrix used for three-dimensional rotation transformation of the historical data point cloud to match the pose of the current data. t represents a 3×1 vector representing the translation amounts in the X, Y, and Z directions, used to adjust the position of the historical data to align with the current data. P 历史,i represents the three-dimensional point coordinates of the i-th point in the historical data, from the lidar or photogrammetry data when there are no cracks. P 当前,iRepresents the i-th three-dimensional point coordinate in the current detection data, corresponding to the same physical position in the historical data, ||·|| 2 Represents the square of the Euclidean distance, used to measure the matching error between the historical point cloud after rotation and translation and the current point cloud;

[0083] T300. Perform surface fitting on the historical point cloud to generate the reference surface elevation in the crack-free state.

[0084] In an embodiment of the present application, specifically, the evaluation process of the threat assessment module includes:

[0085] Input the crack depth d, the strike angle θ, and the geological parameters of the bridge slope into the finite element analysis model to calculate the structural instability probability P fail , and the expression is:

[0086]

[0087] In expression (6), μ R , μ S Are the mean values of the structural resistance and the load effect respectively, and σ R , σ S Are the standard deviations corresponding to the structural resistance and the load effect respectively;

[0088] When the instability probability P fail > the warning threshold (set to 15%), trigger a structural reinforcement suggestion and associate it with the maintenance work order system.

[0089] Among them, the finite element analysis model is constructed based on the ANSYS platform, and the input parameters include the concrete elastic modulus E, Poisson's ratio v, and the crack position coordinates (x, y, z), and the output is the stress distribution nephogram and the critical instability region.

[0090] In an embodiment of the present application, specifically, the classification logic of the dynamic warning module is:

[0091] Level 1 warning (blue): The crack depth is less than the first warning value and the predicted depth change amount in the next 30 days is less than the first predicted depth change amount value, and it is pushed to the mobile terminal of the inspection personnel;

[0092] Level 2 warning (orange): The first warning value is less than or equal to the crack depth and the crack depth is less than the second warning value, or the predicted depth change amount in the next 30 days is greater than or equal to the first predicted depth change amount value and less than the second predicted depth change amount value, trigger an audible and visual alarm and generate a maintenance suggestion report;

[0093] Level 3 warning (red): The crack depth is greater than or equal to the second warning value, or the predicted depth change amount in the next 30 days is greater than or equal to the second predicted depth change amount value, immediately send an emergency instruction to the supervision platform and initiate a secondary review.

[0094] In one embodiment of the present application, specifically, the pushing method of the multi-level warning signal includes:

[0095] Transmitting the warning information to the supervision platform in real time through the 5G communication module;

[0096] Synchronously generating a detection report in PDF format, including crack parameters, prediction trends, and maintenance suggestions, and automatically sending it to the designated responsible person via email.

[0097] To better understand the present application, the following is an example:

[0098] The slope of a concrete bridge spanning a valley in a certain city may be at risk of crack expansion due to long-term rain erosion and geological activities. The system starts a periodic inspection task to detect the structural safety of the bridge slope.

[0099] 1. The drone cruise unit performs inspections

[0100] Autonomous flight and data collection

[0101] The drone autonomously flies according to a preset route (such as covering key areas of the bridge slope), equipped with a visible light camera, an infrared thermal imager, and a lidar.

[0102] Multi-modal data synchronous collection:

[0103] Visible light image: Take high-definition images to capture the surface crack morphology.

[0104] Infrared thermal imaging: Detect abnormal temperature gradients in the crack area (such as local temperature changes caused by water seepage).

[0105] Three-dimensional laser point cloud: Generate a high-precision three-dimensional model of the slope surface.

[0106] Edge computing module: Real-time compress data and preprocess it (such as denoising, time synchronization), and transmit it back to the detection and warning management unit through the 5G network.

[0107] 2. The detection and warning management unit processes data

[0108] Intelligent crack analysis

[0109] Step S100 (visible light image analysis):

[0110] The improved YOLOv5 model performs a preliminary screening of cracks on the visible light image, excludes interference from vegetation or stains, and locates the crack bounding box.

[0111] Output the crack position and preliminary geometric features (such as the trend angle).

[0112] Step S200 (infrared thermal imaging analysis):

[0113] Calculate the temperature difference between the target point and the surrounding environment (Formula 2). If the temperature difference in a certain area exceeds the threshold (e.g., ΔT≥5°C), mark it as a potential crack area.

[0114] Step S300 (3D point cloud fusion):

[0115] Use the NURBS surface fitting algorithm to fit the crack point cloud into a continuous surface and calculate the crack depth (Formulas 3 and 4).

[0116] Compare with the historical reference surface elevation (align the historical and current point clouds through the ICP algorithm) to determine the deviation of the crack in the z direction.

[0117] Threat assessment and early warning generation

[0118] Finite element analysis: Input the crack depth, strike angle, and slope geological parameters (such as concrete elastic modulus) into the ANSYS model to calculate the probability of structural instability (Formula 6).

[0119] Dynamic early warning classification:

[0120] Level 1 early warning: The crack depth < 10mm and the predicted expansion amount in the next 30 days < 2mm, and it is pushed to the mobile APP of the inspection personnel.

[0121] Level 2 early warning: The crack depth ≥ 10mm and < 20mm, or the predicted expansion amount ≥ 2mm and < 5mm, trigger an audible and visual alarm and generate a maintenance work order.

[0122] Level 3 early warning: The crack depth ≥ 20mm or the predicted expansion amount ≥ 5mm, immediately send an emergency instruction to the supervision platform and dispatch a drone for secondary verification.

[0123] 3. Early warning push and subsequent response

[0124] Multi-level early warning push:

[0125] Transmit the early warning information to the supervision platform in real time through the 5G communication module, and synchronously generate a PDF report (including crack parameters, expansion trend, and reinforcement suggestions).

[0126] In the case of a level 3 early warning, the system automatically associates with the maintenance work order system and activates the emergency plan (such as closing the traffic and dispatching an engineering team).

[0127] Data fusion and historical comparison:

[0128] The data fusion module associates the current detection results with historical data to generate a crack evolution map, visually showing the change trend of the crack over time.

[0129] Example results

[0130] During a routine inspection, the system detected a crack with a depth of 15 mm in a certain slope area, the temperature difference ΔT = 7 °C, and the finite element model predicted that it might expand to 18 mm within 30 days. The system triggered a secondary warning:

[0131] The maintenance staff received an alarm on their mobile APP and checked the crack location and the 3D model.

[0132] The audible and visual alarm was activated on-site to alert the staff.

[0133] The maintenance work order system automatically generated a solution, suggesting "grouting reinforcement + surface sealing treatment".

[0134] Through periodic re-inspections, the system continuously monitored the crack changes until the risk was eliminated.

[0135] Compared with the prior art, the technical solution of the present application realizes high-precision and real-time detection and warning of bridge slope cracks by integrating multi-modal sensors and using advanced algorithms for data processing and feature extraction. By fusing visible light images, infrared thermal imaging, and 3D laser point cloud data, the system can effectively eliminate interference factors, accurately identify the crack location, depth, and morphology, providing a reliable basis for subsequent structural safety assessment and maintenance decision-making. In addition, the hierarchical logic of the dynamic warning module and the multi-level signal push mechanism ensure the timely transmission and effective response of warning information, helping to reduce the risk of structural instability and ensure the safe operation of the bridge.

[0136] The various embodiments of the systems and techniques described above in this article can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0138] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, electronics, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] In order to provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0140] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0141] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs that run on the respective computers and have a client-server relationship with each other. The server can be a cloud server, can also be a server of a distributed system, or a server that incorporates blockchain.

[0142] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. There is no limitation herein in this regard.

[0143] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. An intelligent unmanned aerial vehicle bridge slope crack detection and early warning system, characterized in that, The system includes a drone cruise unit and a detection and early warning management unit; The drone cruise unit is used to autonomously fly and inspect the bridge slopes in the target area at a preset cycle, and simultaneously obtain visible light images, infrared thermal images, and three-dimensional lidar point cloud data of the target area; The detection and early warning management unit is used to real-time process the image data obtained by the drone cruise unit through a pre-trained crack intelligent analysis algorithm, extract the geometric and thermodynamic characteristics of the cracks, construct a time series model to predict the crack propagation trend and the structural safety risk level, and generate multi-level early warning signals based on the risk level and push them to the maintenance terminal; the geometric characteristics include crack depth and strike angle, and the thermodynamic characteristics include abnormal temperature gradient distribution.

2. The intelligent unmanned aerial vehicle bridge slope crack detection and early warning system according to claim 1, characterized in that The drone cruise unit includes: A multi-modal image acquisition module, which is used to receive the image data transmitted by the visible light camera, infrared thermal imager, and lidar sensor, and perform spatio-temporal alignment of multi-source data through a time synchronization controller; An edge computing module, which is used to real-time compress and preprocess the multi-source data after spatio-temporal alignment; An autonomous navigation module, which is based on the SLAM algorithm to real-time generate a three-dimensional environment map and plan the optimal inspection path.

3. The intelligent UAV bridge slope crack detection and early warning system according to claim 1, wherein, The detection and early warning management unit includes: A crack intelligent analysis algorithm module, which is used to real-time process the image data and extract the geometric and thermodynamic characteristics of the cracks; A threat assessment module, which is used to predict the crack propagation trend, the network input is the crack depth sequence, and the output is the future expansion curve; A dynamic early warning module, which is used to generate multi-level early warning signals according to the crack parameters and the instability probability, and automatically push them to the maintenance management terminal; A data fusion module, which is used to associate historical detection data with real-time monitoring data to generate a crack evolution map.

4. The intelligent UAV bridge slope crack detection and warning system according to claim 3, characterized in that, The crack intelligent analysis algorithm includes: S100. Perform initial screening of cracks on the visible light image based on the improved YOLOv5 model to eliminate the interference of vegetation and stains. The model loss function is: L = λ cls ·L cls + λ box ·L box + λ obj ·L obj (1) In expression (1), L cls represents the classification loss, which is used to distinguish cracks from interference objects, and L box represents the bounding box regression loss, which is used to optimize the crack localization accuracy, and L obj represents the confidence loss, which is used to screen high-confidence crack regions, and λ cls , λ box , λ obj represent the corresponding weight coefficients respectively; S200. Perform temperature difference analysis on the infrared thermal imaging data to extract the abnormal temperature gradient distribution in the crack area. The expression is: In expression (2), T(x, y) represents the real-time temperature value of the target point (x, y), which is directly measured by an infrared thermal imager. N represents the number of reference points in the neighborhood around the target point (x, y), and N = 8. T(x i , y i ) represents the temperature value of the i-th reference point in the neighborhood, and ΔT(x, y) represents the difference between the temperature of the target point and the average temperature of the surrounding environment; When ΔT(x,y) > T th it is determined that there is a crack area, where T th represents a preset temperature gradient threshold value; S300. Integrate the three-dimensional point cloud data, and use the NURBS surface fitting algorithm to fit the discrete crack point cloud into a continuous surface S(u,v), and calculate the crack depth. The expression of S(u,v) is: In expression (3), S(u, v) represents the coordinates of the surface points after fitting at the parameter space (u, v). By adjusting u and v, the entire surface can be traversed to describe the three-dimensional shape of the crack. The value ranges of both u and v are [0, 1], and R i,j represents the basis function, which is composed of B-spline basis functions and defines the influence weight of the crack control points on the surface shape. P i,j represents the crack control points, with coordinate values of (x, y, z), representing the geometric features of the crack surface.

5. The intelligent UAV bridge slope crack detection and warning system according to claim 4, characterized in that, In step S300, the crack depth d is calculated by the deviation of the surface in the z direction. The expression is: d = |S z (u, v)-Z baseline (u, v)|(4), In expression (4), Z baseline represents the reference surface elevation without cracks and is obtained through the following steps: T100. Extract the lidar or photogrammetry data when there are no cracks from the historical database of bridge construction or regular inspections; T200. Perform coordinate transformation through the ICP algorithm to align the historical data with the coordinate system of the current detection area. The expression of the target optimization formula is: In expression (5), R represents a 3×3 orthogonal matrix used for performing a three-dimensional rotation transformation on the historical data point cloud to match the pose of the current data, t represents a 3×1 vector representing the translation amounts in the X, Y, and Z directions, used to adjust the position of the historical data to align with the current data, P 历史,i represents the 3D point coordinates of the i-th point in the historical data, from lidar or photogrammetry data when there is no crack, P 当前,i represents the 3D point coordinates of the i-th point in the current detected data, corresponding to the same physical position in the historical data, ||·|| 2 represents the square of the Euclidean distance, used to measure the matching error between the historical point cloud after rotation and translation and the current point cloud; T300. Perform surface fitting on the historical point cloud to generate the reference surface elevation in the crack-free state.

6. The intelligent unmanned aerial vehicle bridge slope crack detection and early warning system according to claim 5, characterized in that The evaluation process of the threat assessment module includes: Input the crack depth d, strike angle θ and geological parameters of the bridge slope into the finite element analysis model to calculate the structural instability probability P fail , and the expression is: In expression (6), μ R , μ S are the mean values of the structural resistance and the load effect respectively, and σ R , σ S are the standard deviations corresponding to the structural resistance and the load effect respectively; When the buckling probability P fail exceeds the warning threshold, trigger structural reinforcement suggestions and associate with the maintenance work order system.

7. The intelligent UAV bridge slope crack detection and warning system according to claim 6, characterized in that, The finite element analysis model is built based on the ANSYS platform. The input parameters include the concrete elastic modulus E, Poisson's ratio v, and crack position coordinates (x, y, z), and the output is the stress distribution cloud map and the critical instability area.

8. The intelligent UAV bridge slope crack detection and warning system according to claim 3, characterized in that, The grading logic of the dynamic early warning module is: Level 1 warning: The crack depth is less than the first warning value and the predicted depth change within the next 30 days is less than the first predicted depth change value, and it is pushed to the mobile terminal of the inspection personnel; Level 2 warning: The first warning value is less than or equal to the crack depth and the crack depth is less than the second warning value, or the predicted depth change within the next 30 days is greater than or equal to the first predicted depth change value and less than the second predicted depth change value, triggering an audible and visual alarm and generating a maintenance advice report; Level 3 warning: The crack depth is greater than or equal to the second warning value, or the predicted depth change within the next 30 days is greater than or equal to the second predicted depth change value, immediately sending an emergency instruction to the supervision platform and initiating a secondary review.

9. The intelligent UAV bridge slope crack detection and early warning system according to claim 8, characterized in that, The push methods of the multi-level warning signals include: Real-time transmission of warning information to the supervision platform through the 5G communication module; Simultaneously generating an inspection report, including crack parameters, prediction trends, and maintenance suggestions, and automatically sending it to the designated responsible person by email.

Images