Underwater structure defect dynamic grading method based on multi-modal perception fusion

Abstract

The invention discloses an underwater structure defect dynamic grading method based on multi-modal perceptual fusion, which comprises the following steps of: acquiring multi-modal data by deploying a multi-sensor array with a self-adaptive collaborative deployment mechanism, and extracting features by using a double-backbone architecture deep learning network comprising a self-adaptive noise suppression module and a cross-modal attention fusion module; and dynamic grading of defect risks is realized by combining an optimized attention mechanism and a fuzzy logic reasoning system. Detection resource allocation is adjusted according to the grading result and the defect expansion rate, an LSTM prediction model containing an environment factor correction module is introduced to assist decision making, and the weight of the SHAP algorithm can be dynamically adjusted. According to the method, the real-time performance, the resource utilization rate and the grading accuracy of defect detection are remarkably improved, the interpretability and the prospective decision-making ability of the system are enhanced, and the method is suitable for health management of complex underwater structures such as ocean platforms and bridge pile foundations.

Description

Technical Field

[0001] This invention relates to the field of underwater structure health monitoring technology, and in particular to a dynamic classification method for underwater structure defects based on multimodal sensing fusion. Background Technology

[0002] Underwater structures, such as offshore oil platforms, bridge foundations across rivers and seas, submarine oil pipelines, and water conservancy hub gates, are in complex and harsh environments with high salinity, high pressure, water erosion, and biological attachment for a long time. Their structural integrity is directly related to the safety of the project and the operating cost.

[0003] Traditional underwater structural defect detection methods have the following problems: Data limitation: Relying on a single sensor, such as sonar or camera, to collect data makes it difficult to fully reflect the characteristics of defects; Assessment lag: Periodic inspections cannot respond to defect evolution in real time, leading to missed detections in high-risk areas; Inefficient resource allocation: Fixed detection strategies lead to redundant detection in low-risk areas and insufficient resources in high-risk areas; Lack of interpretability: The "black box" nature of deep learning models hinders transparency in maintenance decisions; Poor environmental adaptability: In complex water flow and water temperature changes, the quality of data collected by the sensor is unstable, affecting the detection accuracy; Insufficient prediction accuracy: Existing prediction models do not fully consider the impact of environmental factors on defect evolution, resulting in large deviations in prediction results. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a dynamic classification method for underwater structural defects based on multimodal perception fusion, which solves the significant deficiencies of existing technologies in multimodal data fusion, dynamic classification mechanisms, resource optimization and allocation, and environmental adaptive prediction.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A dynamic classification method for underwater structural defects based on multimodal sensing fusion includes the following steps: 1. Multimodal data acquisition: A multi-beam sonar (ResonSeaBatT50-P) with an adaptive collaborative deployment mechanism, an optical camera (GoPro Hero11), and a corrosion potential probe (Cu / CuSO4 electrode) were deployed to collect data on geometry, surface texture, and corrosion potential, respectively. The multi-sensor array dynamically adjusts according to the morphology of the underwater structure and the water flow velocity to ensure data acquisition quality.

[0006] 2. Feature fusion and risk assessment: A dual backbone network (depth separable convolution + CSPDarknet) with an adaptive noise suppression module is used to extract sonar and optical features. A threshold denoising algorithm based on wavelet transform is used for sonar images, and an improved nonlocal mean denoising algorithm is used for optical images. A coordinate attention (CA) mechanism is introduced to enhance spatial feature interaction, and a cross-modal attention fusion module is combined to deeply fuse the temporal features of sonar, optical features and corrosion potential. Using GRU to model the temporal characteristics of corrosion potential data; Generate a risk assessment vector that includes the percentage of defect area, corrosion rate, surface roughness, residual strength coefficient of the component, and defect propagation rate.

[0007] 3. Dynamic hierarchical classification and resource allocation: Based on the fuzzy logic reasoning system (FIS), four risk levels are divided, and the defect propagation rate is added as an input variable; The rules for each level are as follows: Level 1: Defect area ratio < 5%, corrosion rate < 0.1 mm / year, surface roughness < 10 μm, and defect propagation rate < 1% / month; Level 2: (5% ≤ defect area ratio < 20% or 0.1 mm / year ≤ corrosion rate < 0.5 mm / year or 10 μm ≤ surface roughness < 50 μm) and defect propagation rate < 3% / month; Level 3: (20% ≤ defect area ratio < 50% or 0.5 mm / year ≤ corrosion rate < 1.0 mm / year or 50 μm ≤ surface roughness < 100 μm) and defect propagation rate < 5% / month; Level 4: Defect area ratio ≥ 50% or corrosion rate ≥ 1.0 mm / year or surface roughness ≥ 100 μm or defect propagation rate ≥ 5% / month; The detection strategy is dynamically adjusted based on the level and defect expansion rate, such as frequency, accuracy, and coverage. When the defect expansion rate exceeds 80% of the upper limit of the corresponding level, the detection frequency is increased by one level.

[0008] 4. Enhanced interpretability: Visualize areas of significant defects using Grad-CAM heatmaps with enhanced color processing; The contribution of each modal data to risk assessment is quantified by combining the SHAP algorithm, and the weight ratio can be dynamically adjusted according to different underwater structure types.

[0009] 5. Defect evolution prediction: An LSTM neural network with an environmental factor correction module is introduced, using environmental parameters such as water flow velocity and salinity as correction factors to predict the probability of defect development in the next 3 months, thus assisting in dynamic hierarchical decision-making.

[0010] Compared with the prior art, the present invention has the following beneficial effects: 1. Multimodal complementarity and environmental adaptation: By integrating sonar, optical and potential data, combined with adaptive sensor deployment and noise suppression, the defect characteristics in complex environments can be fully captured. 2. Dynamic optimization and precise allocation: Detection resources are adaptively allocated based on risk level and defect expansion rate to improve resource utilization and avoid redundant detection and missed detection. 3. Enhanced interpretability: The optimized attention mechanism and dynamic weight SHAP algorithm make model decisions more transparent and easier to maintain. 4. Predictive accuracy and forward-looking decision-making: The LSTM prediction model with environmental factor correction improves the accuracy of defect evolution prediction and provides strong support for forward-looking decision-making. 5. Wide applicability: The algorithm parameters can be adjusted according to different underwater structure types, making it suitable for health management of various complex underwater structures. Detailed Implementation

[0011] The technical solutions of the present invention will be further described below with reference to the embodiments.

[0012] A dynamic classification method for underwater structural defects based on multimodal perception fusion includes: multimodal data acquisition, feature fusion network design, defect risk assessment vector calculation, construction of a dynamic classification model, and SHAP value and LSTM prediction.

[0013] in, 1. Multimodal data acquisition includes: Multibeam sonar: sampling frequency 20Hz, coverage area 300m×300m, resolution 0.1°, and angle adjustment of ±30° via an adjustable bracket; Optical camera: 15fps, 1920×1080 resolution, IP68 waterproof rating, lens with automatic cleaning device to prevent underwater debris from adhering; Corrosion potential probe: sampling frequency 10Hz, measurement range -1000mV to +1000mV, accuracy ±1mV, adopts wireless self-organizing network, and the sampling point spacing can be automatically adjusted within the range of 10-50cm.

[0014] 2. Feature fusion network design includes: (1) Depthwise separable convolution calculation: Depthwise separable convolution reduces computational complexity by decomposing standard convolution into depthwise convolution and pointwise convolution. Standard convolution computational cost: ; Computational cost of depthwise separable convolution: ; in, The kernel size is the convolution kernel size. Number of input / output channels This refers to the feature map size. In this embodiment, =3, reducing the computational cost to 1 / 3 of that of traditional convolution.

[0015] (2) Coordinate Attention Mechanism (CA): Channel attention calculation: ; Spatial attention is calculated as follows: ; in, This is the result of global average pooling. For MLP weights, Embed features into coordinate information, and finally output .

[0016] (3) Cross-modal attention fusion: Calculate intermodal weights using scaled dot product attention: ; in, Sonar ( =64), Optics ( =256), Potential ( =128) Query / key / value matrix of features, output fused features .

[0017] 3. Defect risk assessment vector calculation: (1) Corrosion rate conversion: Calculated using the Tafel equation based on corrosion potential data: ; in , Corrosion current density , For material density , This represents the charge transfer number. For steel structures... .

[0018] (2) Surface roughness (Ra): Calculation based on the gray-level co-occurrence matrix of optical images: ; in, The probability distribution is the gray-level co-occurrence matrix probability distribution. In this embodiment, the distance is taken as the probability distribution. =1, angle .

[0019] (3) Residual strength coefficient of the component: ; in Remaining / original strength, For service time, Surface roughness / 100.

[0020] 4. Dynamic hierarchical model (fuzzy logic): (1) Trapezoidal membership function: Based on the percentage of defective area ( For example: ; ; Similarly, the membership functions of variables such as corrosion rate and surface roughness are optimized using the PSO algorithm to optimize inflection point parameters.

[0021] (2) Fuzzy reasoning output: Risk level Calculated by weighted summation: ; in For the first The trigger strength of the rule, Output the rule levels (1-4). =27 represents the total number of rules.

[0022] 5. SHAP value and LSTM prediction: (1) Calculation of Shapley value: feature Contribution level: ; in, For feature set, These are the predicted values ​​for the feature subset.

[0023] In this embodiment =3 The basic weights are calculated iteratively using this formula.

[0024] (2) Environmental factor correction model: The prediction results were corrected using multiple linear regression: ; in, For water flow velocity, Salinity The correction coefficient for temperature changes was obtained by fitting measured data from the South China Sea.

[0025] 6. Verification of technical effectiveness: Through root mean square error and Score assessment: ; ; Actual test data show that this method can classify defects. The score reached 0.92, predicted. =0.043, which is better than the traditional method.

[0026] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for dynamic grading of underwater structure defects based on multi-modal perception fusion, characterized in that, The method comprises the following steps: deploying a multi-sensor array to collect multi-modal data of an underwater structure, the multi-modal data comprising sonar images, optical images and corrosion potential data; using a multi-modal feature extraction network to perform feature fusion on the multi-modal data to generate a defect risk assessment vector; constructing a dynamic grading model based on the defect risk assessment vector, and dynamically adjusting a detection resource allocation strategy, including detection frequency, detection accuracy and detection coverage, according to the defect risk level; performing visual analysis on the defect features through an attention mechanism to output an interpretable grading result; the multi-modal feature extraction network further comprises an adaptive noise suppression module for dynamically filtering underwater noise in the sonar images and the optical images, wherein a wavelet transform-based threshold denoising algorithm is used for the sonar images, and an improved non-local mean denoising algorithm is used for the optical images.

2. The method as claimed in claim 1, wherein the method of dynamic grading of underwater structural defects based on multi-modal perception fusion is characterized by, The multi-sensor array comprises a multi-beam sonar, a high-definition optical camera and a copper / copper sulfate corrosion potential probe, and the sampling frequencies of the respective sensors are: multi-beam sonar: 20 Hz; high-definition optical camera: 15 fps; corrosion potential probe: 10 Hz; The multi-sensor array adopts an adaptive cooperative deployment mechanism, and according to the morphology of the underwater structure and the flow velocity, the multi-beam sonar and the high-definition optical camera realize dynamic angle adjustment through an adjustable support, and the corrosion potential probe adopts a wireless ad hoc network mode, which can automatically adjust the sampling point distribution according to the real-time monitored corrosion situation.

3. The method as claimed in claim 1, wherein the method of dynamic grading of underwater structural defects based on multi-modal perception fusion is characterized by, The multi-modal feature extraction network is a double-main-stem architecture, comprising the following modules: a depth separable convolutional layer for extracting geometric features of the sonar images, the computational complexity of which satisfies: ； wherein, is a convolution kernel size, is an input / output channel number, is a feature map size; an improved CSPDarknet network for extracting texture features of the optical images; a coordinate attention mechanism CA for fusing spatial features of the sonar images and the optical images, the channel attention calculation of which is: ； the spatial attention calculation is: ； Final output ; a time series gated recurrent unit GRU for modeling time series features of the corrosion potential data; a cross-modal attention fusion module using scaled dot-product attention to calculate the inter-modal weight: ; wherein, are sonar, optical, electrical potential signature query / key / value matrices, respectively.

4. The method as claimed in claim 1, wherein The defect risk assessment vector comprises the following dimensions: defect area proportion; Corrosion rate, calculated by Tafel equation: ; wherein , is the corrosion current density, is the material density, is the number of charge transfers; surface roughness calculated based on a gray level co-occurrence matrix: ; wherein is a probability distribution; Component residual strength factor: ; wherein, is the service time, is the surface roughness / 100; defect propagation rate.

5. The method as claimed in claim 1, wherein The dynamic grading model is based on a fuzzy logic reasoning system, the input variables of which are the defect area proportion, the corrosion rate, the surface roughness and the defect propagation rate, and the output variable is the defect risk level, which is divided into four levels, and the rules corresponding to each level are as follows: level one: defect area proportion < 5% and corrosion rate < 0.1 mm / year and surface roughness < 10 μm and defect propagation rate < 1% / month; level two: (5% ≤ defect area proportion < 20% or 0.1 mm / year ≤ corrosion rate < 0.5 mm / year or 10 μm ≤ surface roughness < 50 μm) and defect propagation rate < 3% / month; level three: (20% ≤ defect area proportion < 50% or 0.5 mm / year ≤ corrosion rate < 1 mm / year or 50 μm ≤ surface roughness < 100 μm) and defect propagation rate < 5% / month; level four: defect area proportion ≥ 50% or corrosion rate ≥ 1 mm / year or surface roughness ≥ 100 μm or defect propagation rate ≥ 5% / month. Level 3: (20% ≤ defect area ratio < 50% or 0.5 mm / year ≤ corrosion rate < 1.0 mm / year or 50 μm ≤ surface roughness < 100 μm) and defect propagation rate < 5% / month; Level 4: defect area ratio ≥ 50% or corrosion rate ≥ 1.0 mm / year or surface roughness ≥ 100 μm or defect propagation rate ≥ 5% / month; The risk level is calculated by weighted summation: ； wherein, is the rule trigger strength, is the output level.

6. The method as claimed in claim 5, wherein the method of dynamic grading of underwater structural defects based on multi-modal perception fusion is characterized by, The dynamic adjustment of the detection resource allocation strategy includes: For level 1 defect area: low frequency detection once a month and laser range finder for close-range observation; For level 2 defect area: medium frequency detection once every two weeks and ROV camera system for medium-range observation; For level 3 defect area: high frequency detection once a week and ACFM detector for microscopic detection; For level 4 defect area: immediately start the intervention of human divers and close the relevant structure function; At the same time, according to the defect propagation rate, the detection frequency is dynamically adjusted, and when the defect propagation rate exceeds 80% of the upper limit of the corresponding level, the detection frequency is increased by one level.

7. The method as claimed in claim 1, wherein the method is based on multi-modal perception fusion based dynamic grading of underwater structural defects. The attention mechanism uses Grad-CAM algorithm, and the significant area of defect features is visualized through heat map, and the SHAP algorithm is used to quantify the contribution of each modal data to risk assessment; The heat map generated by the Grad-CAM algorithm is color enhanced, and by adjusting the color mapping range, the contrast between the significant area and the non-significant area of the defect is improved by more than 30%; The SHAP algorithm realizes by calculating Shapley value: ； wherein, is a feature set, is a feature subset prediction value.

8. The method as claimed in claim 7, wherein the method is based on multi-modal perception fusion based dynamic grading of underwater structural defects. The basic Shapley value weight of each input feature calculated by the SHAP algorithm is: sonar image feature accounts for 40%, optical image feature accounts for 35%, and corrosion potential feature accounts for 25%; According to different types of underwater structures, the weight proportion of Shapley value is dynamically adjusted. For concrete structure, the weight of optical image feature is increased by 5%, and the weight of sonar image feature is decreased by 5%; for steel structure, the weight of corrosion potential feature is increased by 5%, and the weight of optical image feature is decreased by 5%.

9. The method as claimed in claim 1, wherein the method is based on multi-modal perception fusion based dynamic grading of underwater structural defects. The method further includes establishing a defect evolution prediction model, based on LSTM neural network, for time series modeling of historical defect data, to predict the defect development probability in the next 3 months as an auxiliary decision basis for dynamic classification; The defect evolution prediction model introduces an environmental factor correction module, and the correction formula is: ； wherein, is the water flow velocity, is the salinity, is the temperature change.

10. The method as claimed in claim 9, wherein the method is based on multi-modal perception fusion based dynamic grading of underwater structural defects. The input layer of the LSTM neural network includes defect area, corrosion rate and environmental temperature, the hidden layer uses bidirectional LSTM structure, the output layer is defect development probability in the interval of 0-1, and the hyperparameters are optimized by Bayesian optimization algorithm; In the bidirectional LSTM structure, the unit number ratio of forward LSTM to backward LSTM is 3:2, to better capture the trend characteristics of defect development; The search space of the Bayesian optimization algorithm is expanded to 0.00001-0.01 learning rate, 16-128 batch size and 64-256 hidden layer unit number, to improve the accuracy of hyperparameter optimization.

Images