A rapid scanning and diagnostic method and system for breakwater structures based on drones

Abstract

The present invention relates to the field of intelligent detection and discloses a method for rapid scanning and diagnosis of breakwater structures based on drones, comprising the following steps: Step 1: Collect multimodal data of the breakwater's above-water and underwater structures using sensors carried by drones; Step 2: Perform spatiotemporal alignment processing on the multimodal data to generate fused point cloud data; Step 3: Construct a multidimensional tensor representing damage characteristics based on the fused point cloud data; Step 4: Upload the damage classification results to the cloud; Step 5: Dynamically adjust the drone's scanning path planning based on the damage classification results and real-time environmental parameters; Step 6: Output a three-dimensional visualization result containing the damage location, type, and emergency plan. The present invention achieves expanded damage detection coverage and information dimensionality through cross-media collaborative acquisition using millimeter-wave radar, sonar, and multispectral sensors, providing reliable data for breakwater structure monitoring in complex marine environments.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology, and in particular to a method and system for rapid scanning and diagnosing breakwater structures based on an unmanned aerial vehicle (UAV). Background Art

[0002] As a key facility to resist erosion from the marine environment, the structural health inspection of breakwaters is crucial to ensuring the safety of coastal projects. Traditional inspection methods mostly rely on manual inspections or single sensor data collection, which makes it difficult to take into account the simultaneous monitoring of structures above and below the water. Optical sensors are limited by the shielding effect of the water body and cannot penetrate the water surface to obtain underwater damage information, while sonar equipment has insufficient resolution for subtle surface defects of water structures, resulting in significant blind spots in damage assessment. In addition, existing technologies usually adopt an offline processing mode, and data transmission and analysis lag behind actual detection, which cannot meet the real-time requirements of rapid response in complex marine environments. Especially under extreme conditions such as typhoons and waves, the problems of insufficient detection efficiency and timeliness are more prominent.

[0003] Existing drone inspection systems often rely on pre-set fixed scanning paths and lack the ability to dynamically adjust based on real-time damage feedback. This results in missed inspections in high-risk areas or repeated scanning of low-risk areas, resulting in low hardware resource utilization. Damage identification models are typically trained on static datasets specific to specific scenarios. Given the differences in materials, structures, and environments of breakwaters in different sea areas, model generalization is limited, and detection accuracy fluctuates significantly. Furthermore, the credible evidence storage mechanism for inspection results is weak, and the risk of data tampering and traceability difficulties hinder engineering decision-making, limiting the timeliness and scientific nature of maintenance plans.

[0004] The above technical defects make it difficult for existing methods to meet the needs of modern coastal engineering intelligent operation and maintenance in terms of comprehensive detection, real-time response, resource optimization and engineering credibility. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for rapid scanning and diagnosis of breakwater structures based on drones. The present invention solves the technical difficulties in existing breakwater detection, such as the inability to synchronously and accurately perceive above-water and underwater structures, and the existence of cross-media blind spots in damage identification.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for rapid scanning and diagnosis of breakwater structures based on drones, comprising the following steps:

[0007] Step 1: Collect multimodal data of the breakwater's above-water and underwater structures using sensors carried by drones;

[0008] Step 2: performing spatiotemporal alignment processing on the multimodal data to generate fused point cloud data;

[0009] Step 3: constructing a multi-dimensional tensor representing damage characteristics based on the fused point cloud data, and performing real-time damage classification on the tensor;

[0010] Step 4: Upload the damage classification results to the cloud, update the global model parameters through federated learning, and then send them to the drone end;

[0011] Step 5: Dynamically adjust the scanning path planning of the UAV based on the damage classification results and real-time environmental parameters;

[0012] Step 6: Output 3D visualization results including damage location, type and emergency plan.

[0013] Preferably, the step 1 includes the following sub-steps:

[0014] Before the drone takes off, it loads a millimeter-wave radar, a synthetic aperture sonar, and a multispectral laser-induced fluorescence sensor, and performs time synchronization calibration on the sensors based on the GPS clock signal.

[0015] The millimeter-wave radar emits electromagnetic waves in the 77 GHz frequency band to penetrate the water surface and scan point cloud data of the underwater structure of the breakwater;

[0016] emitting a 100kHz acoustic signal through the synthetic aperture sonar to generate an underwater three-dimensional acoustic image;

[0017] The multi-spectral laser-induced fluorescence sensor is used to excite the fluorescence of chemical substances on the surface of the breakwater and detect the chloride ion penetration corrosion characteristics.

[0018] Preferably, the time synchronization calibration comprises the following steps:

[0019] Generate a reference clock signal through the GPS module and send synchronization pulses to the millimeter-wave radar, synthetic aperture sonar, and multispectral laser-induced fluorescence sensor respectively;

[0020] Record the data acquisition timestamp of each sensor and calculate the maximum time deviation Δt:

[0021] ;

[0022] in, : The acquisition timestamp of millimeter wave radar; : Synthetic aperture sonar acquisition timestamp; : The acquisition timestamp of the multispectral laser-induced fluorescence sensor; if Δt>1ms, the current batch of data is discarded and the acquisition is retriggered.

[0023] Preferably, the step 2 includes the following sub-steps:

[0024] Water point cloud data collected by millimeter wave radar and underwater point cloud data collected by synthetic aperture sonar Perform denoising filtering to obtain a preprocessed point cloud set and ;

[0025] Calculate the mutual information value of the above-water and underwater point cloud data, and solve the rigid body transformation matrix that maximizes the mutual information ;

[0026] Based on the rigid body transformation matrix , underwater point cloud data Map to the radar coordinate system to generate a fused point cloud ;

[0027] The fused point cloud is voxel-downsampled, and the output spatial resolution is 1cm 3 uniform point cloud data.

[0028] Preferably, the step three includes the following steps:

[0029] Perform spatial gradient calculation on the fused point cloud data to extract geometric deformation feature vectors:

[0030] ;

[0031] in, : Represents the spatial height, width and depth coordinates of the point cloud respectively; :The coordinates in the fused point cloud are The point cloud position vector of

[0032] Obtaining surface chemical corrosion index using a multispectral laser-induced fluorescence sensor:

[0033] ;

[0034] in, :wavelength The fluorescence intensity; : Preset chloride ion corrosion characteristic spectrum weight.

[0035] Preferably, the step three further includes the following sub-steps:

[0036] Construct a four-dimensional damage characteristic tensor: ;

[0037] in, : spatial dimension index of the tensor; : feature channel dimension;

[0038] The four-dimensional damage feature tensor is input into the pre-trained lightweight convolutional neural network model to output the damage classification result:

[0039] ;

[0040] in, : Damage type labels, including cracks, peeling, and corrosion.

[0041] Preferably, updating the global model parameters through federated learning includes the following sub-steps:

[0042] Local model parameters on the drone side Perform dimensionality reduction compression to generate a 256-dimensional feature vector :

[0043] ;

[0044] in, : compression matrix; : Flatten the four-dimensional damage feature tensor into a one-dimensional vector;

[0045] The feature vectors of multiple drones Upload to the cloud server and update the global model parameters through weighted aggregation :

[0046] ;

[0047] in, : cloud learning rate; : No. The amount of data from each drone;

[0048] The updated global parameters Send it to each drone to constrain the local model training loss function:

[0049] ;

[0050] in, : cross entropy loss for damage classification; : Parameter consistency constraint coefficient.

[0051] Preferably, the step five includes the following sub-steps:

[0052] Generate a structured inspection report based on the damage classification results, and calculate the distribution parameters of each type of damage:

[0053] ;

[0054] in, : No. the number of instances of the class injury; : No. Total affected area of ​​the type of damage; : voxel volume; : indicator function, takes 1 when the condition is true, otherwise takes 0;

[0055] The damage classification results are mapped to the breakwater BIM model through a 3D rendering engine to generate a visual heat map:

[0056] ;

[0057] Among them, RGB color coding indicates different types of damage;

[0058] The test report and visualization data are encrypted and stored in the blockchain node to generate a data hash fingerprint.

[0059] Preferably, the step six includes the following sub-steps:

[0060] Based on damage statistics Calculate the sensor sensitivity adaptation coefficient:

[0061] ;

[0062] in, : total surface area of ​​the breakwater; : Sensitivity adjustment coefficient, used to control the transmission power of millimeter wave radar and sonar;

[0063] Adjust the weight of federated learning constraints based on historical detection errors:

[0064] ;

[0065] in, : historical iteration times; : updated regularization coefficient;

[0066] Generate maintenance priority matrix and output work orders:

[0067] ;

[0068] in, : Breakwater Section Maintenance priority score for the area; : weight coefficient of quantity and area; : Global maximum damage quantity and area.

[0069] The present invention also provides a breakwater structure rapid scanning and diagnosis system based on a drone, comprising the following modules:

[0070] A multimodal sensor group is used to synchronously collect multimodal data of the breakwater's above-water and underwater structures using a UAV equipped with millimeter-wave radar, synthetic aperture sonar, and multispectral laser-induced fluorescence sensors;

[0071] A spatiotemporal alignment processing module, configured to perform spatiotemporal calibration and fusion on the multimodal data to generate three-dimensional fused point cloud data;

[0072] A damage feature modeling and classification module, configured to construct a multi-dimensional damage feature tensor based on the fused point cloud data and implement real-time damage classification through a lightweight convolutional neural network;

[0073] A federated learning module is used to upload the damage classification results to a cloud server, update the global model parameters through weighted aggregation, and send them to the UAV end;

[0074] Adaptive path planning module, used to dynamically adjust the drone's scanning path and sensor operating mode based on damage classification results and real-time environmental parameters;

[0075] The 3D visualization and emergency decision-making module is used to generate 3D visualization results including damage spatial distribution, type classification and emergency response plans, and bind them to the breakwater BIM model.

[0076] In summary, the present invention includes at least one of the following beneficial technical effects:

[0077] 1. This invention overcomes the limitation of a single sensor that cannot detect both above- and underwater structures through cross-media collaborative acquisition of millimeter-wave radar, sonar, and multispectral sensors. It combines a spatiotemporal alignment algorithm to achieve high-precision fusion of multi-source heterogeneous data, significantly expanding the coverage and information dimension of damage detection, and providing a reliable data foundation for full-structure monitoring of breakwaters in complex marine environments.

[0078] 2. The damage modeling method of the present invention, based on multi-dimensional feature tensors and lightweight convolutional neural networks, can analyze geometric deformation and chemical corrosion characteristics in real time, realize automatic classification of damage types such as cracks and spalling, meet the low-latency processing requirements in drone edge computing scenarios, and improve detection efficiency and engineering response speed.

[0079] 3. Through the cloud-edge parameter compression aggregation and dynamic constraint mechanism, the present invention realizes multi-UAV collaborative training while protecting data privacy, effectively addresses the distribution differences of breakwater damage characteristics in different regions, improves the generalization of the global model and the scenario adaptability of the local model, and enhances the intelligence level of the detection system.

[0080] 4. The present invention combines blockchain-based detection data hashing with three-dimensional visualization results to ensure the integrity and non-tamperability of damage reports and maintenance decisions. Combined with the interactive display of BIM models, it provides intuitive and reliable decision-making support for breakwater maintenance, and comprehensively improves the digital level of engineering safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 is a flow chart of the method of the present invention;

[0082] Figure 2 It is the main framework diagram of the present invention. DETAILED DESCRIPTION

[0083] The following is combined with Figure 1 , the present invention is described in further detail.

[0084] The present invention provides a method for rapid scanning and diagnosis of breakwater structures based on an unmanned aerial vehicle, comprising the following steps:

[0085] Step 1: Collect multimodal data of the breakwater's above-water and underwater structures using sensors carried by drones;

[0086] Step 2: Perform spatiotemporal alignment processing on the multimodal data to generate fused point cloud data;

[0087] Step 3: Construct a multi-dimensional tensor representing damage characteristics based on the fused point cloud data, and perform real-time damage classification on the tensor;

[0088] Step 4: Upload the damage classification results to the cloud, update the global model parameters through federated learning, and then send them to the drone end;

[0089] Step 5: Dynamically adjust the UAV's scanning path planning based on the damage classification results and real-time environmental parameters;

[0090] Step 6: Output 3D visualization results including damage location, type and emergency plan.

[0091] Step 1 includes the following sub-steps:

[0092] Before the drone takes off, it loads millimeter-wave radar, synthetic aperture sonar, and multispectral laser-induced fluorescence sensors, and calibrates the sensors' time synchronization based on the GPS clock signal.

[0093] The millimeter-wave radar emits electromagnetic waves in the 77GHz frequency band, penetrating the water surface to scan the point cloud data of the underwater structure of the breakwater;

[0094] Synthetic aperture sonar emits 100kHz acoustic signals to generate underwater three-dimensional acoustic images;

[0095] A multispectral laser-induced fluorescence sensor is used to excite the fluorescence of chemical substances on the breakwater surface and detect the characteristics of chloride ion penetration corrosion.

[0096] Time synchronization calibration includes the following steps:

[0097] Generate a reference clock signal through the GPS module and send synchronization pulses to the millimeter-wave radar, synthetic aperture sonar, and multispectral laser-induced fluorescence sensor respectively;

[0098] Record the data acquisition timestamp of each sensor and calculate the maximum time deviation Δt:

[0099] ;

[0100] in, : The acquisition timestamp of millimeter wave radar; : Synthetic aperture sonar acquisition timestamp; : The acquisition timestamp of the multispectral laser-induced fluorescence sensor; if Δt>1ms, the current batch of data is discarded and the acquisition is retriggered.

[0101] Step 2 includes the following sub-steps:

[0102] Water point cloud data collected by millimeter wave radar and underwater point cloud data collected by synthetic aperture sonar Perform denoising filtering to obtain a preprocessed point cloud set and ;

[0103] Calculate the mutual information value of the above-water and underwater point cloud data, and solve the rigid body transformation matrix that maximizes the mutual information ;

[0104] Based on the rigid body transformation matrix , underwater point cloud data Map to the radar coordinate system to generate a fused point cloud ;

[0105] The fused point cloud is voxel-downsampled and the output spatial resolution is 1cm 3 uniform point cloud data.

[0106] Step three includes the following steps:

[0107] Perform spatial gradient calculation on the fused point cloud data and extract the geometric deformation feature vector:

[0108] ;

[0109] in, : Represents the spatial height, width and depth coordinates of the point cloud respectively; :The coordinates in the fused point cloud are The point cloud position vector of

[0110] Obtaining surface chemical corrosion index using a multispectral laser-induced fluorescence sensor:

[0111] ;

[0112] in, :wavelength The fluorescence intensity; : Preset chloride ion corrosion characteristic spectrum weight.

[0113] Step 3 further includes the following sub-steps:

[0114] Construct a four-dimensional damage characteristic tensor:

[0115] ;

[0116] in, : spatial dimension index of the tensor; : feature channel dimension;

[0117] Input the four-dimensional damage feature tensor into the pre-trained lightweight convolutional neural network model and output the damage classification result:

[0118] ;

[0119] in, : Damage type labels, including cracks, peeling, and corrosion.

[0120] Updating global model parameters through federated learning includes the following sub-steps:

[0121] Local model parameters on the drone side Perform dimensionality reduction compression to generate a 256-dimensional feature vector :

[0122] ;

[0123] in, : compression matrix; : Flatten the four-dimensional tensor into a one-dimensional vector;

[0124] The feature vectors of multiple drones Upload to the cloud server and update the global model parameters through weighted aggregation :

[0125] ;

[0126] in, : cloud learning rate; : No. The amount of data from each drone;

[0127] The updated global parameters Send it to each drone to constrain the local model training loss function:

[0128] ;

[0129] in, : cross entropy loss for damage classification; : Parameter consistency constraint coefficient.

[0130] Step 5 includes the following sub-steps:

[0131] Generate a structured inspection report based on the damage classification results, and calculate the distribution parameters of each type of damage:

[0132] ;

[0133] in, : No. the number of instances of the class injury; : No. Total affected area of ​​the type of damage; : voxel volume; : indicator function, takes 1 when the condition is true, otherwise takes 0;

[0134] The damage classification results are mapped to the breakwater BIM model through a 3D rendering engine to generate a visual heat map:

[0135] ;

[0136] Among them, RGB color coding indicates different types of damage;

[0137] The test report and visualization data are encrypted and stored in the blockchain node to generate a data hash fingerprint.

[0138] Step 6 includes the following sub-steps:

[0139] Based on damage statistics Calculate the sensor sensitivity adaptation coefficient:

[0140] ;

[0141] in, : total surface area of ​​the breakwater; : Sensitivity adjustment coefficient, used to control the transmission power of millimeter wave radar and sonar;

[0142] Adjust the weight of federated learning constraints based on historical detection errors:

[0143] ;

[0144] in, : historical iteration times; : updated regularization coefficient;

[0145] Generate maintenance priority matrix and output work orders:

[0146] ;

[0147] in, : Breakwater Section Maintenance priority score for the area; : weight coefficient of quantity and area; : Global maximum damage quantity and area.

[0148] In this embodiment, the drone is equipped with a multimodal sensor group, including a millimeter-wave radar, a synthetic aperture sonar, and a multispectral laser-induced fluorescence sensor. Preferably, the millimeter-wave radar adopts a frequency-modulated continuous wave system, which transmits wide-band electromagnetic waves to penetrate the water surface medium to achieve non-contact scanning of the underwater structure of the breakwater; the synthetic aperture sonar is based on the principle of acoustic reflection, and generates underwater three-dimensional point cloud data through array signal processing; the multispectral laser-induced fluorescence sensor is configured to stimulate the chemical substances on the surface of the breakwater to produce characteristic fluorescence, and quantify the degree of corrosion according to the spectral distribution. During the sensor initialization stage, each sensor is time-synchronized and calibrated based on the GPS clock signal. Specifically, the GPS module sends a synchronization pulse signal to the millimeter-wave radar, synthetic aperture sonar, and multispectral laser-induced fluorescence sensor, triggering each sensor to start data collection with a unified reference time. Each sensor records the GPS timestamp when collecting data, and calculates the maximum time deviation between adjacent sensors:

[0149] ;

[0150] in, is the acquisition timestamp of the millimeter wave radar, is the acquisition timestamp of synthetic aperture sonar, is the acquisition time stamp of the multi-spectral laser-induced fluorescence sensor. When the preset threshold is exceeded, the current batch of data is discarded and the collection is retriggered to ensure the temporal and spatial consistency of multi-source data.

[0151] During the data collection process, the millimeter wave radar transmits high-frequency electromagnetic waves and receives the reflected signals from the breakwater structure to generate a water point cloud data set. Preferably, the millimeter wave radar resolves the target position through a range-Doppler algorithm, where each point cloud data point contains three-dimensional coordinates and reflection intensity information.

[0152] For underwater structure detection, synthetic aperture sonar transmits broadband acoustic signals and generates high-resolution underwater three-dimensional acoustic images through synthetic aperture processing algorithms. It contains the geometric shape information of underwater structures and inverts the target depth through the acoustic wave propagation time.

[0153] At the same time, the multi-spectral laser-induced fluorescence sensor emits excitation light of a specific wavelength to the surface of the breakwater, inducing the fluorescence effect of corrosion products such as chloride ions in the concrete. The reflection spectrum is collected through the spectrometer and the fluorescence intensity of each wavelength channel is recorded. Preferably, the sensor covers the visible to near-infrared band to capture the characteristic spectrum of chloride ion corrosion.

[0154] Furthermore, time synchronization calibration is achieved through a combination of hardware triggering and software verification. Specifically, before the drone takes off, the driver parameters for each sensor are loaded, and synchronization start commands are sent to the sensors via the GPIO interface. During data acquisition, the clock drift of each sensor is monitored in real time. If the accumulated time deviation exceeds a threshold, a dynamic resynchronization mechanism is triggered to realign the acquisition timing.

[0155] The correlation of multimodal data is achieved through spatiotemporal tag binding. Each data point is appended with an acquisition timestamp and spatial coordinate information, providing a benchmark for subsequent spatiotemporal alignment. Preferably, the coordinate system for the millimeter-wave radar and sonar is the drone's body coordinate system, which is converted to a global coordinate system using calibration parameters.

[0156] In this embodiment, the spatiotemporal alignment of multimodal data includes data preprocessing, coordinate system calibration, rigid body transformation solution and point cloud fusion. Underwater point cloud data collected by synthetic aperture sonar ,First, noise filtering and outlier removal are performed. Preferably, a statistical outlier filtering algorithm is used to calculate the average distance between each data point and its neighboring points, and outliers exceeding the preset standard deviation threshold are removed to obtain the pre-processed water point cloud. With underwater point cloud In order to achieve spatial alignment of cross-medium data, a rigid body transformation relationship between the millimeter wave radar coordinate system and the sonar coordinate system is established. Specifically, the initial relative pose matrix of the two sensors is obtained by measuring the calibration plate. , and optimize the transformation matrix based on the mutual information maximization criterion. The mutual information is calculated as:

[0157] ;

[0158] in, and are the probability distributions of the above-water point cloud and the transformed underwater point cloud, is the joint probability distribution. By iteratively solving the gradient ascent method, Maximized rigid body transformation matrix , thus eliminating the spatial dislocation caused by the difference in sensor perspective. After completing the transformation matrix solution, the underwater point cloud data is mapped to the radar coordinate system to generate a fused point cloud . Preferably, the fusion process retains the reflection intensity and timestamp attributes of the original point cloud, and adds a source sensor identifier to each data point to support differentiated processing in the subsequent feature extraction stage. Furthermore, the fused point cloud is voxelized and downsampled to improve data uniformity. The three-dimensional space is divided into voxel grids of preset sizes, the centroid coordinates of all points in each voxel are calculated, and the data point closest to the centroid is retained as the representative point. Through this operation, the scale of point cloud data is reduced while retaining key geometric features, and the efficiency of subsequent processing is improved. The spatial resolution of the point cloud output by voxelization is Voxel volume consistency ensures accuracy during the damage signature modeling phase. Spatiotemporal alignment processing, through a multi-stage algorithm chain, enables precise fusion of millimeter-wave radar and sonar data, resolving registration challenges associated with media differences between above- and below-water structures. This ensures geometric consistency for the construction of multi-dimensional damage signature tensors.

[0159] In this embodiment, the damage feature modeling and classification process is based on the fused point cloud data after spatiotemporal alignment. First, the spatial gradient of the fused point cloud is calculated to quantify the geometric deformation characteristics. Specifically, the point cloud data is divided into a regular three-dimensional grid, and each grid cell corresponds to a point cloud subset at the spatial coordinate (h, w, d). The position gradient of each grid center point in the height, width, and depth directions is calculated using the central difference method:

[0160] ;

[0161] Among them, z(h, w, d) represents the point cloud position vector with coordinates (h, w, d), and Δh is the spacing in the grid height direction. Similarly, the width direction gradient is calculated and depth gradient , and synthesize the geometric deformation feature vector The gradient calculation reflects the deformation severity of the breakwater surface and internal structure, providing a geometric basis for damage classification. At the same time, the chemical corrosion index is calculated by the reflectance spectrum data obtained by the multi-spectral laser induced fluorescence sensor. Preferably, the predefined chloride ion corrosion characteristic spectrum weight is loaded , whose peak corresponds to the sensitive band of the chloride ion-related fluorescence spectrum. For each spatial position (h, w, d), the weighted summation of the fluorescence intensity I(λ) of each wavelength channel is performed:

[0162] ;

[0163] Corrosion Index Quantify the degree of chemical corrosion. The higher the value, the more significant the chloride ion penetration in the area. Construct a four-dimensional damage characteristic tensor based on geometric deformation and chemical corrosion characteristics. Among them, the spatial dimensions H, W, D correspond to the number of grid divisions of the breakwater height, width and depth, and the channel dimension c=1 stores the geometric deformation intensity , channel c=2 stores chemical corrosion index The tensor is defined by the following conditional expression:

[0164] ;

[0165] The four-dimensional damage feature tensor is fed into a pre-trained lightweight convolutional neural network for damage classification. The network architecture consists of a depthwise separable convolutional layer and a three-dimensional max pooling layer, extracting local and global damage features through layer-by-layer abstraction. The final output layer uses a softmax function to generate the probability distribution of each damage category:

[0166] ;

[0167] in, This represents a set of damage type labels, including categories such as cracks, spalling, and corrosion. The classification results annotate the damage location and type at the voxel level, providing a foundation for subsequent quantitative statistics and visualization. Preferably, the parameters of the lightweight convolutional neural network are dynamically updated via a federated learning framework. During the model inference phase, weight quantization and matrix decomposition techniques are used to reduce computational complexity to accommodate the resource constraints of the drone's edge computing terminal.

[0168] In this embodiment, the process of updating global model parameters by federated learning includes three stages: local parameter compression, cloud aggregation, and parameter distribution constraints. Perform dimensionality reduction compression to generate low-dimensional feature vectors to reduce communication overhead. Specifically, the compression matrix pre-trained by principal component analysis (PCA) is used. Perform a linear transformation on the flattened parameter vector:

[0169] ;

[0170] in, Operation converts a 4D tensor Expand to A one-dimensional vector of length, is the bias term. Preferably, the compression matrix Through offline training of historical model parameter datasets, more than 95% of the feature variance is retained to ensure that the information loss in the compression process is controllable.

[0171] Local feature vectors generated by multiple drones After uploading to the cloud server, weighted aggregation is performed to update the global model parameters The aggregation formula is:

[0172] ;

[0173] in, is the cloud learning rate, which controls the parameter update step size; Indicates the The amount of valid data (i.e., the number of non-noise voxels) of the current batch of drones, is the total global data volume. The weighting mechanism enables drones with abundant data to have a greater impact on the global model, alleviating the model bias caused by data distribution heterogeneity.

[0174] Updated global parameters After being sent to each drone, it is integrated into the local training objective function through regularization constraints:

[0175] ;

[0176] in, is the cross entropy classification loss, is the parameter consistency constraint coefficient that is dynamically adjusted. Preferably, the coefficient Adaptive adjustment based on the relative deviation of local and global parameters in historical training cycles:

[0177] ;

[0178] When the average relative deviation between the local model and the global model is large, reduce To enhance local training freedom; otherwise, constraints are strengthened to maintain model consistency.

[0179] The federated learning module realizes multi-UAV collaborative modeling and knowledge sharing through the closed-loop mechanism of compression-aggregation-constraint while reducing the communication bandwidth requirements. The dimensional definition of strictly matches the 256-dimensional feature vector, ensuring the feasibility of the technical solution. The cloud server uses differential privacy technology to add Gaussian noise to the aggregated global parameters to prevent privacy risks caused by model parameter leakage.

[0180] In this embodiment, the process of dynamically adjusting the drone scanning path planning is based on a multi-dimensional analysis of damage classification results and real-time environmental parameters. First, the damage classification results are statistically quantified to generate a structured inspection report. Specifically, for each damage type k (e.g., cracks, spalling, corrosion), its spatial distribution parameter is calculated:

[0181] ;

[0182] ;

[0183] in, Represents the number of instances of the kth type of damage, through the indicator function Count the number of voxels that meet the classification conditions; Represents the total affected area of ​​the kth type of damage, through the geometric deformation gradient The Euclidean norm and voxel volume of The statistical results quantify the spatial distribution density and severity of the damage, providing a decision basis for path planning.

[0184] Based on the statistical results, a maintenance priority matrix for the breakwater is constructed. The breakwater structure is divided into grid areas, and calculate the priority score of each area:

[0185] ;

[0186] in, and Respectively The number and area of ​​damage in the region, and is the global maximum value for normalization. Preferably, the weight coefficient and Dynamic configuration based on engineering needs, such as increasing the corrosion risk in high-risk scenarios The scoring matrix generates priority labels based on thresholds, driving drones to prioritize high-risk areas.

[0187] At the same time, the damage classification results are mapped to the breakwater BIM model through the 3D rendering engine to generate a visual heat map. Assign RGB color codes according to the classification results:

[0188] ;

[0189] The heat map is superimposed on the surface and internal structure of the BIM model, allowing engineers to interactively view the spatial distribution of damage. Control the visual saliency of local areas and highlight high deformation areas.

[0190] Furthermore, the test report and visualization data are encrypted and stored in the blockchain node. The SHA-256 hash algorithm is used to generate the data fingerprint:

[0191] ;

[0192] in, This represents a data concatenation operation, sequentially concatenating the structured report and the 3D model binary stream before calculating a hash value. The hash fingerprint is written to a blockchain smart contract, ensuring the immutability of the test results through timestamps and distributed ledgers.

[0193] The UAV-based breakwater structure rapid scanning and diagnosis system described below and the UAV-based breakwater structure rapid scanning and diagnosis method described above can refer to each other.

[0194] Please see the attached Figure 2 The present invention also provides a breakwater structure rapid scanning and diagnosis system based on a drone, comprising the following modules:

[0195] A multimodal sensor group is used to synchronously collect multimodal data of the breakwater's above-water and underwater structures using a UAV equipped with millimeter-wave radar, synthetic aperture sonar, and multispectral laser-induced fluorescence sensors;

[0196] The spatiotemporal alignment processing module is used to perform spatiotemporal calibration and fusion of multimodal data to generate three-dimensional fused point cloud data;

[0197] The damage feature modeling and classification module is used to construct a multi-dimensional damage feature tensor based on fused point cloud data and realize real-time damage classification through a lightweight convolutional neural network;

[0198] The federated learning module is used to upload the damage classification results to the cloud server, update the global model parameters through weighted aggregation, and send them to the drone end;

[0199] Adaptive path planning module, used to dynamically adjust the drone's scanning path and sensor operating mode based on damage classification results and real-time environmental parameters;

[0200] The 3D visualization and emergency decision-making module is used to generate 3D visualization results including damage spatial distribution, type classification and emergency response plans, and bind them to the breakwater BIM model.

[0201] The device of this embodiment can be used to execute the above method embodiment, and its principles and technical effects are similar, so they will not be repeated here.

[0202] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A rapid scanning and diagnostic method for breakwater structures based on drones, characterized in that: The following steps are involved: Step 1: Collect multimodal data of the breakwater's above-water and underwater structures using sensors carried by drones; Step 2: performing spatiotemporal alignment processing on the multimodal data to generate fused point cloud data; Step 3: constructing a multi-dimensional tensor representing damage characteristics based on the fused point cloud data, and performing real-time damage classification on the tensor; Step 4: Upload the damage classification results to the cloud, update the global model parameters through federated learning, and then send them to the drone end; Step 5: Dynamically adjust the scanning path planning of the UAV based on the damage classification results and real-time environmental parameters; Step 6: Output 3D visualization results including damage location, type, and emergency plan; The step three comprises the following steps: Perform spatial gradient calculation on the fused point cloud data to extract geometric deformation feature vectors: Where, h, w, d: represent the spatial height, width and depth coordinates of the point cloud respectively; z: the point cloud position vector with coordinates (h, w, d) in the fused point cloud; Obtaining surface chemical corrosion index using a multispectral laser-induced fluorescence sensor: Where, I(λ): fluorescence intensity at wavelength λ; χ(λ): preset chloride ion corrosion characteristic spectrum weight.

2. The method for rapid scanning and diagnosis of breakwater structures based on drones according to claim 1 is characterized in that: The step 1 includes the following sub-steps: Before the drone takes off, it loads a millimeter-wave radar, a synthetic aperture sonar, and a multispectral laser-induced fluorescence sensor, and performs time synchronization calibration on the sensors based on the GPS clock signal. The millimeter-wave radar emits electromagnetic waves in the 77 GHz frequency band to penetrate the water surface and scan point cloud data of the underwater structure of the breakwater; emitting a 100kHz acoustic signal through the synthetic aperture sonar to generate an underwater three-dimensional acoustic image; The multi-spectral laser-induced fluorescence sensor is used to excite the fluorescence of chemical substances on the surface of the breakwater and detect the chloride ion penetration corrosion characteristics.

3. The method for rapid scanning and diagnosis of breakwater structures based on drones according to claim 2 is characterized in that: The time synchronization calibration comprises the following steps: Generate a reference clock signal through the GPS module and send synchronization pulses to the millimeter-wave radar, synthetic aperture sonar, and multispectral laser-induced fluorescence sensor respectively; Record the data acquisition timestamp of each sensor and calculate the maximum time deviation Δt: Δt=max(|t radar -t sonar |,|t sonar -t LIF |,|t radar -t LIF |); Among them, t radar : millimeter wave radar acquisition timestamp; t sonar : Synthetic aperture sonar acquisition timestamp; t LIF : The acquisition timestamp of the multispectral laser-induced fluorescence sensor; if Δt>1ms, the current batch of data is discarded and the acquisition is retriggered.

4. The method for rapid scanning and diagnosis of breakwater structures based on drones according to claim 1 is characterized in that: The second step includes the following sub-steps: Water point cloud data collected by millimeter wave radar and underwater point cloud data collected by synthetic aperture sonar Perform denoising filtering to obtain the preprocessed point cloud set {x′ i } and {y′ j }; Calculate the mutual information value of the above-water and underwater point cloud data, and solve the rigid body transformation matrix that maximizes the mutual information Based on the rigid body transformation matrix T, the underwater point cloud data {y′ j } is mapped to the radar coordinate system to generate the fused point cloud {z k }={x′ i }∪T{y′ j }; The fused point cloud is voxel-downsampled, and the output spatial resolution is 1cm 3 uniform point cloud data.

5. The method for rapid scanning and diagnosis of breakwater structures based on drones according to claim 1 is characterized in that: The step three further includes the following sub-steps: Construct a four-dimensional damage characteristic tensor: Among them, h∈[1,H],w∈[1,W],d∈[1,D]: spatial dimension index of tensor; c: feature channel dimension; The four-dimensional damage feature tensor is input into the pre-trained lightweight convolutional neural network model to output the damage classification result: Where, k∈{1,...,K}: damage type label, including crack, spalling, and corrosion.

6. The method for rapid scanning and diagnosis of breakwater structures based on drones according to claim 1, characterized in that: Updating global model parameters through federated learning includes the following sub-steps: On the UAV side, the local model parameters θ edge Perform dimensionality reduction compression to generate a 256-dimensional feature vector in, Compression matrix; Flatten(·): Flatten the four-dimensional damage feature tensor into a one-dimensional vector; The feature vectors of multiple drones Upload to the cloud server and update the global model parameters θ through weighted aggregation cloud : Where, η: cloud learning rate; n i : the data volume of the i-th drone; The updated global parameters Send it to each drone to constrain the local model training loss function: in, Cross entropy loss for damage classification; λ≥0: parameter consistency constraint coefficient.

7. The method for rapid scanning and diagnosis of breakwater structures based on drones according to claim 1 is characterized in that: The step five includes the following sub-steps: Generate a structured inspection report based on the damage classification results, and calculate the distribution parameters of each type of damage: s k =S h，w，dek ‖g h，w，d ‖2·V voxel ; Among them, N k : the number of instances of the kth type of damage; S k : the total affected area of ​​the kth type of damage; V voxel =1cm 3 : voxel volume; Indicator function, takes 1 if the condition is true, otherwise takes 0; The damage classification results are mapped to the breakwater BIM model through a 3D rendering engine to generate a visual heat map: Among them, RGB color coding indicates different types of damage; The test report and visualization data are encrypted and stored in the blockchain node to generate a data hash fingerprint.

8. The method for rapid scanning and diagnosis of breakwater structures based on drones according to claim 1 is characterized in that: The step six includes the following sub-steps: Based on the damage statistics results {N k , S k }Calculate the sensor sensitivity adaptive coefficient: Among them, S total : total surface area of ​​the breakwater; γ: sensitivity adjustment coefficient, used to control the transmission power of millimeter-wave radar and sonar; adjust the weight of the federated learning constraint according to the historical detection error: Where, T: number of historical iterations; λ t+1 : updated regularization coefficient; Generate maintenance priority matrix and output work orders: Among them, P i，j : maintenance priority score of the jth area of ​​the i-th section of the breakwater; α, β: weight coefficients of quantity and area; N max , S max : Global maximum damage quantity and area.

9. A breakwater structure rapid scanning and diagnosis system based on a drone, using a breakwater structure rapid scanning and diagnosis method based on a drone as claimed in any one of claims 1 to 8, characterized in that: Includes the following modules: A multimodal sensor group is used to synchronously collect multimodal data of the breakwater's above-water and underwater structures using a UAV equipped with millimeter-wave radar, synthetic aperture sonar, and multispectral laser-induced fluorescence sensors; A spatiotemporal alignment processing module is used to perform spatiotemporal calibration and fusion on the multimodal data to generate three-dimensional fused point cloud data; a damage feature modeling and classification module is used to construct a multi-dimensional damage feature tensor based on the fused point cloud data and realize real-time damage classification through a lightweight convolutional neural network; A federated learning module is used to upload the damage classification results to a cloud server, update the global model parameters through weighted aggregation, and send them to the UAV end; Adaptive path planning module, used to dynamically adjust the drone's scanning path and sensor operating mode based on damage classification results and real-time environmental parameters; The 3D visualization and emergency decision-making module is used to generate 3D visualization results including damage spatial distribution, type classification and emergency response plans, and bind them to the breakwater BIM model.

Images