Intelligent characterization method for structural steel weld damage based on DIC and acoustic emission

Abstract

The application discloses a structure steel weld damage intelligent characterization method based on DIC and acoustic emission. First, a multi-modal acquisition system composed of a DIC visual acquisition system, an acoustic emission acquisition system and a universal testing machine is built, second, a multi-modal time sequence sample set is constructed, then a double-flow deep learning fusion model is constructed, and finally DIC and acoustic emission data of the structure steel weld sample to be measured are collected, and after preprocessing, the data are input into the trained fusion model, and the damage stage discrimination result and the continuous damage factor value are synchronously output. The application adopts a two-step alignment method of "time offset correction + frame rate down-sampling alignment", and realizes accurate time correspondence of multi-modal data without the need of high-precision synchronous equipment at the hardware level. Meanwhile, through 2DCNN, spatial strain distribution characteristics are excavated, 1DCNN is combined with GRU to excavate time sequence evolution characteristics, time and space information are complemented at the feature level, and the accuracy and comprehensiveness of damage identification are significantly improved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of nondestructive testing, structural health monitoring and artificial intelligence. Specifically, it relates to a method for qualitative stage identification and quantitative characterization of the damage evolution process of structural steel welds under load by using digital image correlation (DIC) technology and acoustic emission (AE) technology, combined with a dual-stream deep learning model. Background Technology

[0002] Structural steel welds are weak points in the mechanical properties of load-bearing structures such as wind turbines. Under cyclic loading, overload, or corrosive environments, microcracks easily initiate and propagate in the weld area, eventually leading to structural failure and causing serious safety accidents and economic losses. Therefore, accurate and real-time characterization of weld damage has significant engineering value.

[0003] Currently, monitoring technologies for weld damage are mainly divided into two categories: Localized point-based monitoring technology, represented by acoustic emission technology, captures transient elastic waves released by micro-fractures and dislocation movements within materials to achieve real-time monitoring of internal damage activity, excelling at capturing the temporal evolution of damage. However, this method has limited spatial positioning accuracy, and a single parameter is insufficient to distinguish signals from different damage mechanisms such as plastic deformation, microcrack initiation, and macroscopic fracture.

[0004] Full-field visual monitoring technology, represented by digital image correlation technology, can acquire the full-field displacement and strain distribution of the surface of the object under test non-contactly, intuitively reflecting the spatial evolution characteristics of damage. However, this method can only observe surface deformation and is not sensitive to early internal micro-damage that has not yet been transmitted to the surface.

[0005] The existing technology has the following significant shortcomings: Although patents such as CN118228136A proposed multi-sensor fusion, their fusion level remained at the data or decision level. They used traditional algorithms and failed to utilize deep learning to automatically mine high-dimensional and nonlinear damage features from the raw data.

[0006] Most AE-based patents rely on manually defining parameters such as ring count, energy, and RA value. These parameters are simplifications of waveform characteristics, and their discriminative power and robustness are insufficient in complex noise environments and under variable damage mechanisms.

[0007] Patents involving DIC typically use DIC data only for post-processing verification or simple strain threshold alarms, failing to mine the spatial pattern of its full-field strain distribution as a deep feature and to deeply couple it with the time-series AE signal.

[0008] Therefore, existing technologies cannot meet the requirements for high-precision, automated, and quantitative characterization of structural steel weld damage from "microscopic initiation" to "macroscopic expansion". Summary of the Invention

[0009] This invention aims to overcome the shortcomings of existing technologies and provide an intelligent characterization method for structural steel weld damage based on DIC and acoustic emission. This method constructs a dual-stream deep learning network to extract DIC spatial strain features and AE temporal damage features respectively. Through feature fusion, information complementarity is achieved, and finally, damage stage classification results and continuous damage factors are simultaneously output, realizing an integrated qualitative and quantitative characterization of weld damage.

[0010] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution.

[0011] A smart characterization method for structural steel weld damage based on DIC and acoustic emission includes: 1) Construct a multimodal acquisition system consisting of a DIC visual acquisition system, an acoustic emission acquisition system, and a universal testing machine to perform tensile loading on structural steel weld specimens and simultaneously acquire DIC strain field data, acoustic emission waveform and parameter data, and load-displacement curve data; 2) Perform parallel preprocessing and time alignment on DIC data and acoustic emission data respectively, generate damage stage classification labels and continuous damage factor regression labels based on load-displacement curves, and construct a multimodal time series sample set; 3) Construct a dual-stream deep learning fusion model, which includes a DIC spatiotemporal feature extraction branch, an acoustic emission temporal feature extraction branch, a feature fusion module, and a dual-task output head; use the sample set to train the model using a two-stage strategy: first, pre-train the two single-modal branches separately, and then freeze the branch parameters and fine-tune the fusion layer and output head; 4) Collect DIC and acoustic emission data from the welded steel specimens of the structure to be tested, input them into the trained fusion model after preprocessing, and output the damage stage discrimination results and continuous damage factor values ​​simultaneously.

[0012] Preferably, in step 1), the DIC acquisition system acquires speckle images of the sample surface at a frame rate of 5 frames / second, and obtains the principal strain scatter data by Vic-3D software; the acoustic emission system uses two broadband sensors symmetrically arranged on both sides of the weld to continuously acquire full waveform and event parameter data; the load-displacement curve output by the universal testing machine is used as the standard for label generation.

[0013] Preferably, step 2) of the DIC data preprocessing specifically includes: ①Time offset correction: Skip the initial empty sampling frame according to the early start time and align with the zero point of the test machine time; ② Grid interpolation: Define the gauge length region of the specimen, generate a 128×128 regular grid, and use the Delaunay triangulation linear interpolation method to convert the discrete scatter point strain into a regular strain contour map; ③ Frame rate alignment: Take the average pixel value of the strain contour map every 5 frames, downsample to 1 frame / second, and match the label time resolution; ④Standardization and time series window construction: The strain field is Z-score standardized, and time series samples are constructed with a window length of 10 frames and a step size of 5 frames.

[0014] Preferably, the acoustic emission data preprocessing in step 2) specifically includes: ① A 1-second time window was opened with the tag time point as the center, and 10-dimensional acoustic emission parameter statistical characteristics were obtained; ② Extract 10-dimensional statistical features in both the time and frequency domains from a single waveform and select the top 70% of high-energy waveforms; ③ The waveform features are aggregated to the label time points through interpolation and concatenated with the parameter features to form a 20-dimensional multi-feature vector; ④ After standardization, time series samples are constructed using window parameters consistent with those of the DIC branch.

[0015] Preferably, the label generation method in step 2) is as follows: Based on the mechanical inflection point of the load-displacement curve, the damage process is divided into four stages: elastic stage, yielding stage, strengthening stage, and necking fracture stage. The continuity damage factor D is calculated using the plastic dissipation energy normalization method. The calculation formula is as follows:

[0016] In the formula, F(t) for t The load output by the constant testing machine k The initial elastic stiffness is obtained by fitting the initial elastic segment of the load-displacement curve. δ(t) for t Loading displacement at any moment d e (t) for t The elastic displacement component at time t. d p (t) for t The plastic displacement component at time t is calculated by forcing the plastic displacement to be non-negative, i.e. d p (t) ≥0.

[0017] Preferably, the specific structure of the dual-stream deep learning fusion model described in step 3) is as follows: The DIC spatiotemporal feature extraction branch uses a 2DCNN combined with a two-layer GRU structure. The input is a temporal strain field sequence, and the output is a 64-dimensional spatial temporal feature vector. The acoustic emission temporal feature extraction branch adopts a 1DCNN combined with a two-layer GRU structure. The input is a temporal sequence of multiple acoustic emission features, and the output is a 128-dimensional temporal damage feature vector. The feature fusion module obtains 192-dimensional fused features by directly splicing features. After dimensionality reduction and feature interaction through two layers of fully connected interactive networks, the features are connected to the classification output head and the regression output head respectively, and the damage stage probability and damage factor value are output simultaneously.

[0018] Preferably, the loss function of the two-stage training strategy in step 3) is a multi-task weighted loss: L total =L cls +λ⋅L reg In the formula L cls For classification cross-entropy loss, L reg The mean squared error loss is used for regression, λ is the balancing weight, and the default value is 0.5. The optimizer used is the Adam optimizer, which is combined with Dropout and weight decay to suppress overfitting.

[0019] Preferably, after the model training is completed, the performance is evaluated using both classification and regression metrics. Classification metrics include accuracy, precision, recall, F1 score, and confusion matrix; regression metrics include mean absolute error, mean squared error, root mean squared error, and coefficient of determination R0. 2 .

[0020] The specific details of this invention are as follows: The core of this invention lies in constructing a complete technology chain of "synchronous data acquisition → data preprocessing → dual-stream deep learning and feature extraction → AE+DIC feature fusion → multi-task decision output". The specific steps are as follows: Step S1: Synchronous acquisition of multimodal data A synchronized experimental system for DIC and AE was established. A universal testing machine was used to perform tensile testing on standard tensile specimens of structural steel welds. At the start of the test, a unified start signal was sent to the DIC system, AE system, and universal testing machine to ensure time alignment of the three systems. Load-displacement-time data was recorded, including: DIC data acquisition: High-contrast artificial speckle patterns need to be created on the specimen observation surface. Two calibrated high-resolution CCD cameras are used to acquire digital images of the specimen surface at a fixed frequency. The (x,y) coordinates of the center of the calculated sub-region of the specimen surface at each time point, the horizontal strain εxx, the vertical strain εyy, and the shear strain εxy are calculated using Vic-3D software.

[0021] AE data acquisition: Two broadband acoustic emission sensors need to be symmetrically coupled to the base material on both sides of the weld using a coupling agent. Set the AE acquisition system parameters and continuously acquire the raw waveform signals.

[0022] Step S2: Data Preprocessing To address the data characteristics of the two modalities, preprocessing and time alignment are performed separately to ensure a one-to-one correspondence between the two in the time dimension.

[0023] (1) DIC data stream preprocessing: Input: Discrete scatter plot DIC calculation results at each time step.

[0024] Time offset correction: Based on the early start time (5 seconds) and sampling frame rate (5 frames / second) of the DIC system, calculate and skip the first 25 empty sampling frames before the start of the experiment to achieve initial alignment with the time reference of the test machine.

[0025] Mesh interpolation: Determine the rectangular area covering the gauge length of the specimen, set a uniform mesh size (128×128), and use a linear interpolation method based on Delaunay triangulation to interpolate the strain values ​​of discrete scattered points onto a regular mesh to generate a single-channel strain field cloud map.

[0026] Frame rate downsampling alignment: The high frame rate data of DIC5 frames / second is downsampled to 1 frame / second by taking the pixel-level average value every 5 frames, which is consistent with the time resolution of the tag data, and achieves time granularity alignment while suppressing measurement noise.

[0027] Data standardization: Calculate the mean and standard deviation of the strain field on all samples in the training set, and perform Z-score standardization on all strain fields.

[0028] Temporal sliding window construction: The continuous strain field sequence is slicing with a fixed window length (10 frames, corresponding to 10 seconds) and a fixed step size (5 frames) to construct a temporal sample with the shape [window length, 1, grid height, grid width].

[0029] (2) AE data stream preprocessing Input: Acoustic emission parameter file and raw waveform file.

[0030] Parameter time window aggregation: Centered on the time point of the label, a 1-second time window is set, and the acoustic emission parameter statistical characteristics within each time window are statistically analyzed, including 10-dimensional parameter characteristics such as number of events, total energy, average amplitude, maximum amplitude, average duration, total ring count, total rise count, average rise time, average RMS, and average ASL.

[0031] Waveform feature extraction: Extract time-domain and frequency-domain statistical features for each acoustic emission waveform file, including 10-dimensional features such as peak voltage, RMS value, waveform energy, kurtosis, skewness, peak factor, impulse factor, peak frequency, centroid frequency, and spectral energy.

[0032] High-energy waveform screening: Sort by waveform energy, retain the top 70% of high-energy waveforms, and filter out low-energy noise waveforms; then use linear interpolation to aggregate waveform features to a time point consistent with the label.

[0033] Feature fusion and standardization: The 10-dimensional parametric features and 10-dimensional waveform features are concatenated into a 20-dimensional multimodal feature vector, which is then standardized using StandardScaler. Temporal sliding window construction: Using the same window length and step size as the DIC branch, acoustic emission time-series samples with the shape [window length, 20] are constructed.

[0034] (3) Tag data generation Based on the load-displacement curves collected by the universal testing machine, labels are generated using mechanical methods: Damage stage classification: Based on the inflection point of the curve's mechanical characteristics, the tensile process is divided into 4 stages: elastic stage, yielding stage, strengthening stage, and necking fracture stage, corresponding to category labels 0~3.

[0035] Damage Factor Calculation: The true value of the damage factor is based on Lemaitre's continuous damage mechanics theory and is calculated using the plastic dissipation energy normalization method. First, the material's lossless stiffness is obtained by fitting the initial elastic segment. Then, the plastic displacement components at each time step are decomposed, and the cumulative plastic dissipation energy is obtained through integration. Finally, the total plastic dissipation energy at fracture is used as a benchmark for normalization, resulting in a continuous damage factor D with a value range of [0,1], representing the current relative damage level of the material. The specific calculation steps are as follows: Initial elastic stiffness calibration: The initial elastic stiffness of the load-displacement curve (the first 5% of data points) is extracted, and a first-order linear fit is used to obtain the initial elastic stiffness of the material under undamaged conditions.

[0036] In the formula, F For the output load of the testing machine, d To load displacement, ΔF This represents the change in load within the initial elastic segment. Dd This corresponds to the change in displacement. k This represents the initial elastic stiffness.

[0037] Elastoplastic displacement decomposition: For the load and displacement at each moment, the elastic displacement component corresponding to the current load is calculated based on the initial stiffness, and the total displacement is decomposed into two parts: reversible elastic displacement and irreversible plastic displacement.

[0038] In the formula, F(t) for t The load output by the constant testing machine k The initial elastic stiffness is obtained by fitting the initial elastic segment of the load-displacement curve. δ(t) for t Loading displacement at any moment d e (t) for t The elastic displacement component at time t. d p (t) for t The plastic displacement component at time t is calculated by forcing the plastic displacement to be non-negative, i.e. d p (t) ≥0.

[0039] Cumulative plastic dissipated energy integral: During plastic deformation, the work done by external forces on the plastic displacement component is entirely irreversible dissipated energy, which is the energy source for damage evolution. Integrating the product of the differential of the plastic displacement and the load yields the cumulative plastic dissipated energy W from the start of loading to time t. p (t):

[0040] In the formula, F For load, W p (t) To accumulate plastic dissipation energy, d p (t) This is a plastic displacement.

[0041] In numerical calculations, gradient difference and accumulation are used to achieve discrete integration, while the cumulative dissipation energy is forced to be non-negative.

[0042] Damage factor normalization calculation: using the total plastic dissipation energy at the moment of complete fracture of the specimen. W p,total Using a normalized baseline, calculate the continuous damage factor D(t) at each time step:

[0043] In the formula, W p (t) To accumulate plastic dissipation energy, W p,total This represents the total plastic dissipation energy at fracture. D(t) The damage factor is calculated within the interval [0,1]. D A higher value indicates more severe internal damage to the material.

[0044] The calculation results are constrained within the range of [0,1]. The larger the D value, the more severe the internal damage of the material.

[0045] Step S3: Construct a two-stream deep learning fusion model This invention employs a network architecture of "dual-stream feature extraction + feature-level fusion + multi-task output," where two branches process DIC and AE data respectively, and then fuse them at the feature layer to output a joint result. The model uses a two-stage training strategy of "single-modal pre-training + frozen transfer learning + fusion layer fine-tuning."

[0046] (1) DIC branch: 2DCNN+GRU spatiotemporal feature extraction network Input: A DIC strain field time series with shape [batch, window length, 1, 128, 128].

[0047] Network structure: The 2DCNN spatial feature extraction module contains three sets of convolutional blocks, each consisting of a Conv2d convolutional layer, a BatchNorm2d batch normalization layer, a ReLU activation layer, and a MaxPool2d pooling layer. It extracts local features such as spatial texture and strain concentration areas of the strain field through layer-by-layer convolution, and finally maps the strain field of each frame into a 128-dimensional spatial feature vector through a fully connected layer.

[0048] GRU temporal modeling module: Input the spatial feature vectors of 10 consecutive frames within the window into the two-layer GRU network in chronological order to capture the dynamic evolution of the strain field, and take the hidden state of the last time step of GRU as the DIC global feature vector F_dic (dimension 64) of the temporal sample.

[0049] (2) AE branch: 1DCNN+GRU temporal feature extraction network Input: A time-series sequence of acoustic emission features with shape [batch, window length, 20].

[0050] Network structure: The 1DCNN local feature extraction module contains three groups of one-dimensional convolutional blocks, each consisting of a Conv1d convolutional layer, a BatchNorm1d batch normalization layer, a ReLU activation layer, and a MaxPool1d pooling layer, which extract local mutation patterns and short-term dependencies in time series sequences.

[0051] GRU temporal modeling module: Input the feature sequence output by convolution into a two-layer GRU network to capture the long-term damage evolution trend, and take the hidden state of the last time step as the global feature vector F_ae (dimension 128) of the temporal sample.

[0052] (3) Feature fusion and multi-task output Feature fusion: The F_dic output from the DIC branch and the F_ae output from the AE branch are directly concatenated in the channel dimension to obtain a 192-dimensional fused feature vector F_fusion.

[0053] Fusion Interaction Network: The fused features are processed through two fully connected layers (including BatchNorm, ReLU activation and Dropout regularization) to perform feature interaction and dimensionality reduction, and to explore the coupling relationship between the two modal features.

[0054] Multi-task output head: Classification task head: The fully connected layer outputs the probability distribution of 4 categories, corresponding to 4 damage stages, and is trained using the cross-entropy loss function.

[0055] Regression task head: The fully connected layer outputs a continuous value, corresponding to the damage factor D, and is trained using the mean squared error loss function.

[0056] (4) Two-stage model training strategy Phase 1: Separate pre-training for each mode: Train the AE branch model separately using all acoustic emission data, and train the DIC branch model separately using all DIC data, so that the two branches have independent feature extraction capabilities. After training, save the optimal weight file.

[0057] Phase 2: Fine-tuning the fusion model: Load two pre-trained branch networks, freeze all parameters of each branch, and train only the fusion interaction network and the two output heads. Employ the Adam optimizer, introduce a learning rate decay strategy, and combine it with Dropout and weight decay to prevent overfitting.

[0058] Loss function: Multi-task weighted total loss is used. L total =L cls +λ⋅L reg Where L cls For classification cross-entropy loss, L reg The regression mean squared error loss is represented by λ, which is the balancing weight and has a default value of 0.5. Step S4: Characterization and Output For the new sample to be tested, follow the process in step S2 to complete the preprocessing and time alignment of DIC and acoustic emission data, input the trained dual-stream fusion model, and directly output the qualitative results (the damage stage at the current moment) and the quantitative results (the value of the continuous damage factor D at the current moment).

[0059] The present invention has the following beneficial effects.

[0060] 1. This invention employs a two-step alignment method of "time offset correction + frame rate downsampling alignment," achieving accurate time correspondence of multimodal data without requiring high-precision hardware synchronization equipment. Simultaneously, it mines spatial strain distribution features through 2DCNN and temporal evolution features through 1DCNN combined with GRU, achieving deep complementarity of spatiotemporal information at the feature level. Furthermore, it adopts a multi-task learning architecture to synchronously output damage stage classification results and continuous damage factors, possessing both qualitative discrimination and quantitative evaluation capabilities, significantly improving the accuracy and comprehensiveness of damage identification.

[0061] 2. This invention adopts a transfer training strategy of single-modal pre-training and frozen fine-tuning, which makes full use of all single-modal data and significantly reduces the training cost and overfitting risk of the fusion model. It can still achieve excellent results in small-sample multimodal scenarios. Moreover, the entire process from raw data input to damage result output is automated, reducing the dependence on human experience. It can be widely applied to tensile, fatigue and other damage monitoring scenarios of various structural steel welds, and has strong engineering applicability. Attached Figure Description

[0062] Figure 1 This is a technical roadmap of the method described in this invention.

[0063] Figure 2 (a) shows the damage change curves during training and testing.

[0064] Figure 2 (b) is the curve showing the change in accuracy of the model on the test set.

[0065] Figure 3 Confusion matrix diagram for classifying damage stages.

[0066] Figure 4 This is a comparison chart of predicted and actual values ​​of the damage factor. Detailed Implementation

[0067] The technical solutions of the present invention will be described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments.

[0068] See Figure 1 The diagram below illustrates the technical roadmap of this invention. The experiment uses multiple sets of Q355 structural steel weld tensile specimens as the research object. Monotonic tensile tests are conducted using a universal testing machine with a loading rate set to 1.2 mm / min. Throughout the entire process from initial loading to complete fracture, DIC full-field strain data and acoustic emission signals are simultaneously acquired. The dual-stream deep learning method of this invention is used to achieve intelligent characterization of weld damage. The specific implementation steps are as follows: Step 1: Specimen Design and Multimodal Experimental System Setup Design standard tensile specimens for butt welds of various structural steels, with the gauge length of the specimens covering the core area of ​​the weld. Construct a multimodal collaborative testing system, comprising three parts: Universal testing machine: used to provide quasi-static tensile loads, and employs displacement control mode to precisely regulate the loading rate; Binocular DIC vision acquisition system: High-contrast black and white artificial speckle patterns are created on the observation surface of the specimen. Two calibrated CCD cameras are positioned directly in front of the gauge length of the specimen to acquire surface deformation images throughout the tensile process. Acoustic emission acquisition system: Two broadband acoustic emission sensors are symmetrically attached to the surface of the base material on both sides of the weld seam with a coupling agent to capture the elastic wave signal released by the internal damage evolution of the material.

[0069] Step 2: Monotonic tensile test and multimodal data acquisition The specimen was clamped in the universal testing machine, and a constant tensile rate of 1.2 mm / min was set for monotonic loading until the specimen completely fractured. During the test, three systems worked in concert: The DIC system starts acquisition 5 seconds in advance, continuously capturing speckle images of the sample surface at a frame rate of 5 frames per second, ensuring that the camera exposure and acquisition status are stable when the testing machine is started. The acoustic emission system has a preset acquisition threshold and sampling frequency, and continuously acquires raw waveform data and event parameter data throughout the stretching process. The universal testing machine simultaneously records load, displacement, and time data, which serve as the gold standard for subsequent damage label generation.

[0070] After the experiment was completed, the DIC sequence images were solved using Vic-3D software, and the coordinates and principal strain data of the discrete calculation points at each time point were exported.

[0071] Step 3: Multimodal data preprocessing and sample set construction Import all raw data into the data processing program, and perform preprocessing, time alignment, and label generation in parallel using a dual-branch approach: DIC data preprocessing: Time offset correction is performed, skipping the empty sampling frames before the test starts, and aligning with the zero point of the testing machine time; the center gauge length area of ​​the weld is defined as the target calculation area, and a 128×128 regular grid is generated. A single-channel strain contour map is generated by linear interpolation through Delaunay triangulation; downsampling is performed to 1 frame / second by taking the average pixel value every 5 frames to match the time resolution of the label; after Z-score standardization is completed, time series samples are constructed with a window length of 10 frames and a step size of 5 frames.

[0072] Acoustic emission data preprocessing: A 1-second time window is opened with the tag time point as the center, and 10-dimensional parameter statistical features such as the number of events, total energy, and average amplitude are obtained; 10-dimensional features in the time domain and frequency domain are extracted from a single waveform, and the high-energy effective waveforms in the top 70% are selected; the waveform features are aggregated to the corresponding time point through interpolation, and concatenated with the parameter features to form a 20-dimensional multi-feature vector. After standardization, time series samples are constructed according to the same window parameters.

[0073] Tag generation: Based on the load-displacement curve output by the universal testing machine, the damage process is divided into four categories according to the inflection point of mechanical characteristics: elastic stage, yielding stage, strengthening stage, and necking fracture stage, and classification tags are generated; the point-by-point continuous damage factor D is calculated using the plastic dissipation energy normalization method, and regression tags are generated.

[0074] Finally, the time of the multimodal samples was matched one-to-one to construct a complete time-series sample dataset, which was then randomly divided into training and test sets in a 7:3 ratio.

[0075] Step 4: Training the dual-stream deep learning fusion model A dual-stream fusion network is constructed, comprising a DIC spatiotemporal feature branch, an acoustic emission temporal feature branch, a feature fusion module, and a dual-task output head. A two-stage strategy is employed to complete model training. The first stage is single-modal pre-training: the corresponding branch networks are trained independently using DIC single-modal data and acoustic emission single-modal data, and the optimal weights are saved after iterative optimization until convergence. The second stage of fusion fine-tuning involves loading the pre-trained weights of both branches, freezing all feature extraction layer parameters, and training only the fusion interaction network and the dual-task output head. The Adam optimizer is used with a batch size of 32 and an initial learning rate of 0.0005. The weighted multi-task loss function is used for optimization, along with Dropout and weight decay to suppress overfitting.

[0076] Step 5: Model Performance Validation and Damage Characterization The trained fusion model was quantitatively evaluated on an independent test set; the results are shown in [link to results]. Figure 2 3 and 4, as detailed below: In the classification task, the model achieved an overall recognition accuracy of 96.46% for the four damage stages, with a weighted average precision of 0.9654, a weighted average recall of 0.9646, and a weighted average F1 score of 0.9648, demonstrating excellent discrimination for each damage stage. In the regression task, the prediction determination coefficient R² of the continuous damage factor D reached 0.8792, the mean absolute error (MAE) was 0.0712, the mean square error (MSE) was 0.0092, and the root mean square error (RMSE) was 0.0959, indicating a high degree of agreement between the predicted and actual values.

[0077] Compared with single DIC detection models and single acoustic emission detection models, the multimodal fusion method of this invention significantly improves both classification accuracy and quantitative prediction accuracy, verifying the effectiveness of the dual-stream deep fusion architecture.

[0078] The model can quickly output the current damage stage and damage factor values ​​when the DIC and acoustic emission data are collected in real time and input to the test sample, thus realizing online intelligent characterization of weld damage.

[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent characterization of structural steel weld damage based on DIC and acoustic emission, characterized in that, include: 1) Construct a multimodal acquisition system consisting of a DIC visual acquisition system, an acoustic emission acquisition system, and a universal testing machine to perform tensile loading on structural steel weld specimens and simultaneously acquire DIC strain field data, acoustic emission waveform and parameter data, and load-displacement curve data; 2) Perform parallel preprocessing and time alignment on DIC data and acoustic emission data respectively, generate damage stage classification labels and continuous damage factor regression labels based on load-displacement curves, and construct a multimodal time series sample set; 3) Construct a dual-stream deep learning fusion model, which includes a DIC spatiotemporal feature extraction branch, an acoustic emission temporal feature extraction branch, a feature fusion module, and a dual-task output head; use the sample set to train the model using a two-stage strategy: first, pre-train the two single-modal branches separately, and then freeze the branch parameters and fine-tune the fusion layer and output head; 4) Collect DIC and acoustic emission data from the welded steel specimens of the structure to be tested, input them into the trained fusion model after preprocessing, and output the damage stage discrimination results and continuous damage factor values ​​simultaneously.

2. The intelligent characterization method for structural steel weld damage based on DIC and acoustic emission according to claim 1, characterized in that: In step 1), the DIC acquisition system acquires speckle images of the sample surface at a frame rate of 5 frames / second, and obtains the principal strain scatter data by Vic-3D software; the acoustic emission system uses two broadband sensors symmetrically arranged on both sides of the weld to continuously acquire full waveform and event parameter data; the load-displacement curve output by the universal testing machine is used as the standard for label generation.

3. The intelligent characterization method for structural steel weld damage based on DIC and acoustic emission according to claim 1, characterized in that, Step 2) DIC data preprocessing specifically includes: ①Time offset correction: Skip the initial empty sampling frame according to the early start time and align with the zero point of the test machine time; ② Grid interpolation: Define the gauge length region of the specimen, generate a 128×128 regular grid, and use the Delaunay triangulation linear interpolation method to convert the discrete scatter point strain into a regular strain contour map; ③ Frame rate alignment: Take the average pixel value of the strain contour map every 5 frames, downsample to 1 frame / second, and match the label time resolution; ④Standardization and time series window construction: The strain field is Z-score standardized, and time series samples are constructed with a window length of 10 frames and a step size of 5 frames.

4. The intelligent characterization method for structural steel weld damage based on DIC and acoustic emission according to claim 1, characterized in that, Step 2) Acoustic emission data preprocessing specifically includes: ① A 1-second time window was opened with the tag time point as the center, and 10-dimensional acoustic emission parameter statistical characteristics were obtained; ② Extract 10-dimensional statistical features in both the time and frequency domains from a single waveform and select the top 70% of high-energy waveforms; ③ The waveform features are aggregated to the labeled time points through interpolation and concatenated with the parameter features to form a 20-dimensional multi-feature vector; ④ After standardization, time series samples are constructed using window parameters consistent with those of the DIC branch.

5. The intelligent characterization method for structural steel weld damage based on DIC and acoustic emission according to claim 1, characterized in that, The tag generation method in step 2) is as follows: Based on the mechanical inflection point of the load-displacement curve, the damage process is divided into four stages: elastic stage, yielding stage, strengthening stage, and necking fracture stage. The continuity damage factor D is calculated using the plastic dissipation energy normalization method. The calculation formula is as follows: In the formula, F(t) for t The load output by the constant testing machine k The initial elastic stiffness is obtained by fitting the initial elastic segment of the load-displacement curve. δ(t) for t Loading displacement at any moment δ e (t) for t The elastic displacement component at time t. δ p (t) for t The plastic displacement component at time t is calculated by forcing the plastic displacement to be non-negative, i.e. δ p (t) ≥0.

6. The intelligent characterization method for structural steel weld damage based on DIC and acoustic emission according to claim 1, characterized in that, The specific structure of the dual-stream deep learning fusion model described in step 3) is as follows: The DIC spatiotemporal feature extraction branch uses a 2DCNN combined with a two-layer GRU structure. The input is a temporal strain field sequence, and the output is a 64-dimensional spatial temporal feature vector. The acoustic emission temporal feature extraction branch adopts a 1DCNN combined with a two-layer GRU structure. The input is a temporal sequence of multiple acoustic emission features, and the output is a 128-dimensional temporal damage feature vector. The feature fusion module obtains 192-dimensional fused features by directly splicing features. After dimensionality reduction and feature interaction through two layers of fully connected interactive networks, the features are connected to the classification output head and the regression output head respectively, and the damage stage probability and damage factor value are output simultaneously.

7. The intelligent characterization method for structural steel weld damage based on DIC and acoustic emission according to claim 1, characterized in that, In step 3), the loss function of the two-stage training strategy is a multi-task weighted loss: L total =L cls +λ⋅L reg In the formula L cls For classification cross-entropy loss, L reg The mean squared error loss is used for regression, λ is the balancing weight, and the default value is 0.

5. The optimizer used is the Adam optimizer, which is combined with Dropout and weight decay to suppress overfitting.

8. The intelligent characterization method for structural steel weld damage based on DIC and acoustic emission according to claim 1, characterized in that: After model training, performance is evaluated using both classification and regression metrics. Classification metrics include accuracy, precision, recall, F1 score, and confusion matrix; regression metrics include mean absolute error, mean squared error, root mean squared error, and coefficient of determination R². 2 .

Images