Structural damage identification method and device based on transfer learning, medium and product

Abstract

The invention discloses a structure damage identification method and device based on transfer learning, a medium and a product. The method comprises the steps of obtaining a source domain data set and a target domain data set; constructing a first damage identification model and a second damage identification model; training the first damage identification model by using the source domain data set to obtain a source domain model; migrating hyper-parameters of the source domain model to a second damage identification model; freezing all layers of the second damage identification model except a full connection layer, and performing fine adjustment on the full connection layer of the second damage identification model by using the first data set to obtain a target domain model; performing damage identification on the second data set by using the target domain model to obtain a damage type and a confidence coefficient thereof; when the confidence of the damage type is greater than or equal to a confidence threshold, updating the first data set; and performing fine tuning on the target domain model by using the updated first data set. The damage identification capability of the model is improved under the condition of a small number of labeled samples.

Description

Technical Field

[0001] The present invention belongs to the technical field of structural health monitoring, and in particular relates to a structural damage identification method, equipment, medium and product based on transfer learning. Background Art

[0002] In recent years, deep learning technology has made significant progress in image recognition, speech processing and other fields, and has also provided new ideas for the field of structural health monitoring. However, effective training of deep learning models usually requires a large amount of labeled data, and in bridge engineering practice, the process of obtaining labeled data is expensive and time-consuming, especially in actual monitoring environments, where manually annotated damage data is even more scarce. In addition, directly using simulation data generated by physical models or finite element models to train deep learning models may result in insufficient generalization capabilities of the model in practical applications due to differences in data distribution.

[0003] With the development of sensing technology and data acquisition technology, structural health monitoring systems have been applied to bridge engineering. Through sensors arranged on bridges, dynamic response data of structures can be obtained in real time. However, how to effectively identify and locate structural damage from massive monitoring data remains a challenging issue. At present, data-driven machine learning methods have begun to be introduced into the field of bridge damage identification. Among them, deep learning methods have gradually become a research hotspot due to their advantages in feature extraction and pattern recognition. However, in actual engineering problems, due to the high cost of manually annotating samples and the difficulty in obtaining labeled samples, the CNN model (Convolutional Neural Network, CNN) requires a large amount of labeled sample data for training to achieve high recognition accuracy. Under the condition of insufficient number of labeled samples, the efficiency of cable damage recognition is low. Therefore, how to improve the recognition accuracy of the model under limited sample conditions has become an urgent problem to be solved.

[0004] Transfer learning is an effective technology for solving the problem of small sample learning. It reduces the reliance on a large amount of labeled data by migrating models and knowledge from other fields (source domains) to the target domain. However, there are certain differences between the target domain and the source domain, and simple model migration may lead to a decrease in model performance. Therefore, how to combine transfer learning and other technical means to further improve the adaptability and accuracy of the model in the target domain has become an important research direction in this field. Summary of the invention

[0005] The purpose of the present invention is to provide a structural damage identification method, device, medium and product based on transfer learning to solve at least one of the problems that traditional technology relies on a large amount of labeled sample data, resulting in high cost and difficulty in obtaining labeled samples, the problem of insufficient labeled sample data resulting in reduced model recognition performance, and simple transfer learning resulting in reduced model recognition performance.

[0006] The present invention solves the above technical problems through the following technical solutions: a structural damage identification method based on transfer learning, comprising:

[0007] Step 1: Acquire a source domain dataset and a target domain dataset; wherein the source domain dataset includes a first time-frequency image and its damage type; the target domain dataset includes a first dataset and a second dataset, the first dataset includes a second time-frequency image and its damage type, and the second dataset includes a third time-frequency image; the first time-frequency image, the second time-frequency image and the third time-frequency image are obtained by converting the acceleration response signal of the structure;

[0008] Step 2: construct a first damage identification model and a second damage identification model with the same architecture;

[0009] Step 3: Using the source domain data set to train the first damage identification model to obtain a source domain model;

[0010] Step 4: Migrating the hyperparameters of the source domain model to the second damage identification model;

[0011] Step 5: Freeze all layers of the second damage recognition model except the fully connected layer, and use the first data set to fine-tune the fully connected layer of the second damage recognition model to obtain a target domain model;

[0012] Step 6: Use the target domain model to perform damage identification on the third time-frequency image in the second data set to obtain the damage type and its confidence level;

[0013] Step 7: Determine whether the confidence of each damage type is greater than or equal to the confidence threshold. If so, add the damage type and the third time-frequency image corresponding to the damage type to the first data set; if not, retain the damage type and the third time-frequency image corresponding to the damage type in the second data set;

[0014] Step 8: Determine whether there is a damage type and a third time-frequency image corresponding to the damage type added to the first data set; if so, proceed to step 9; otherwise, output the final target domain model;

[0015] Step 9: Use the updated first data set to fine-tune the target domain model, and then go to step 6.

[0016] Furthermore, the first time-frequency image is obtained by simulating the first acceleration response signal of the structure under different damage types using a finite element method and converting the first acceleration response signal;

[0017] The second time-frequency image is obtained by converting the second acceleration response signal by simulating the second acceleration response signal of the structure under different damage types using a field test method;

[0018] The third time-frequency image is obtained by real-time monitoring of the actual structure to obtain a third acceleration response signal, and then converting the third acceleration response signal.

[0019] Furthermore, the acceleration response signal of the structure is converted using the Gram angle field, specifically including:

[0020] Normalizing the acceleration response signal to obtain a normalized signal;

[0021] Converting the normalized signal from a Cartesian coordinate system to a polar coordinate system to obtain a signal in the polar coordinate system;

[0022] Calculate the angle difference of the signal in the polar coordinate system at different time intervals to generate the Gram angular field matrix;

[0023] Normalizing the Gram angular field matrix to obtain a normalized Gram angular field matrix;

[0024] The normalized Gram angular field matrix is ​​resized and converted to obtain a time-frequency image.

[0025] Furthermore, the first damage identification model and the second damage identification model adopt a GoogLeNet network structure, which includes an input layer, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a third convolutional layer, 9 Inception modules, a global average pooling layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a classification layer and an output layer connected in sequence.

[0026] Furthermore, after fine-tuning the target domain model using the first data set, and before performing damage recognition on the third time-frequency image in the second data set using the target domain model, the recognition method further includes:

[0027] The target domain model is verified using a verification set; and the confidence threshold is adjusted according to the verification result.

[0028] Furthermore, the confidence threshold gradually decreases according to the fine-tuning rounds; the initial value of the confidence threshold is 0.9.

[0029] Furthermore, during the fine-tuning process, the cross entropy loss function and the Adam optimizer are used to update the parameters of the second damage identification model or the target domain model.

[0030] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and computer program instructions stored in the memory, wherein the processor executes the computer program / instructions to implement the structural damage identification method based on transfer learning as described above.

[0031] Based on the same concept, the present invention also provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program / instructions are executed by a processor, the structural damage identification method based on transfer learning as described above is implemented.

[0032] Based on the same concept, the present invention also provides a computer program product, including computer program instructions, which implement the structural damage identification method based on transfer learning as described above when the computer program / instructions are executed by a processor.

[0033] Beneficial Effects

[0034] Compared with the prior art, the advantages of the present invention are:

[0035] The present invention, based on the source domain model, combines a small amount of labeled samples (i.e., the first data set) of the target domain data set and a large number of unlabeled samples for iterative training, thereby gradually improving the recognition performance and robustness of the target domain model; uses the target domain model to recognize the unlabeled samples and generate pseudo-labeled samples; uses the labeled samples and pseudo-labeled samples of the target domain data set to self-train the target domain model, thereby significantly improving the damage recognition accuracy, solving the problem that the traditional method has a strong dependence on a large number of labeled samples and is difficult to obtain labeled samples, and avoiding the problem that the recognition performance is reduced due to insufficient number of labeled samples or simple transfer learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 is a flow chart of a method for identifying structural damage in an embodiment of the present invention;

[0038] Figure 2 is a physical model of a cable-stayed bridge in an embodiment of the present invention;

[0039] Figure 3is a finite element model of a cable-stayed bridge in an embodiment of the present invention;

[0040] Figure 4 is a schematic diagram of GADF conversion (or encoding) in an embodiment of the present invention;

[0041] Figure 5 It is a GoogLeNet network structure diagram in an embodiment of the present invention;

[0042] Figure 6 is a schematic diagram of an Inception module in an embodiment of the present invention;

[0043] Figure 7 is a schematic diagram of hyperparameter migration in an embodiment of the present invention;

[0044] Figure 8 is the damage identification confusion matrix of model 1 in the embodiment of the present invention;

[0045] Fig. 9 is the damage identification confusion matrix of model 2 in the embodiment of the present invention;

[0046] Fig.10 is the damage identification confusion matrix of model 3 in the embodiment of the present invention;

[0047] Fig.11 is the damage identification confusion matrix of model 4 in the embodiment of the present invention;

[0048] Fig.12 is a t-sne visualization diagram of the characteristic distribution of different damage types of model 1 in an embodiment of the present invention;

[0049] Fig.13 This is a t-SNE visualization diagram of the characteristic distribution of different damage types in model 2 according to an embodiment of the present invention.

[0050] Fig.14 This is a t-SNE visualization diagram of the characteristic distribution of different damage types of model 3 in the embodiment of the present invention.

[0051] Fig.15 It is a t-SNE visualization diagram of the characteristic distribution of different damage types of model four in the embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following is a clear and complete description of the technical solutions in the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0054] Example 1

[0055] like Figure 1 As shown, a structural damage identification method based on transfer learning provided by an embodiment of the present invention includes the following steps:

[0056] Step 1: Obtain source domain dataset and target domain dataset.

[0057] The source domain data set includes the first time-frequency image and the damage type of each first time-frequency image. The first time-frequency image is obtained by converting the first acceleration response signal of the structure under different damage types using the finite element method to simulate the first acceleration response signal. Taking the cable-stayed bridge as an example, the specific process of obtaining the source domain data set includes:

[0058] Step 1.1: According to the actual design parameters of the cable-stayed bridge, a physical model of the cable-stayed bridge is constructed at a scale ratio (e.g., 1:50).

[0059] like Figure 2 As shown in Figure 1, the physical model includes key structural components such as the main beam, cables and bridge towers. R1~R8 and L1~L8 represent the 8 cables on the right side of the main beam and the 8 cables on the left side of the main beam respectively.

[0060] Step 1.2: Based on the geometric parameters and material parameters of the physical model, construct the finite element model of the cable-stayed bridge and mesh the finite element model.

[0061] In this example, the grid size is 0.1 m to ensure accurate details; Shell units are used to simulate the bridge deck, and beam units are used to simulate the cables; a linear elastic material model is used to describe the mechanical behavior of the bridge structure, including an elastic modulus of 2.1×10 5 N / mm 2 , density is 7.9×10 -6 kg / mm 3 , Poisson's ratio is 0.3.

[0062] Step 1.3: Apply boundary conditions and loads.

[0063] The bottom of the bridge tower is fixed and the simply supported boundary condition is applied. The load is applied to the main beam through the dload subroutine to simulate the vehicle load. The vehicle load can be selected with different weights and positions according to the actual situation.

[0064] Step 1.4: Use the implicit dynamic analysis method to calculate the first acceleration response signal of each measuring point of the finite element model of the cable-stayed bridge under different damage types.

[0065] like Figure 3 As shown, four measuring points are set on the main beam of the finite element model, and the acceleration response signals of the four measuring points are spliced ​​into a complete timing signal (i.e., the first acceleration response signal) in chronological order to ensure that the signal contains more structural features. At the same time, considering the complexity of the data in the actual project, the number of samples is expanded by loading different loads and 20dB Gaussian white noise. The first acceleration response signal of the main beam under different damage types is obtained, and the signal sampling frequency is 500Hz. In this embodiment, there are 10 types of damage types, as shown in Table 1, and the degree of damage of the cable is simulated by changing the elastic modulus of the cable.

[0066] Table 1 Ten types of damage of cable-stayed bridges

[0067] Damage Type (Label) Damage location Damage level Vehicle weight (load) S0 Lossless / 5kg~30kg S1 L1 40％ 5kg~30kg S2 L3 40％ 5kg~30kg S3 L5 40％ 5kg~30kg S4 L7 40％ 5kg~30kg S5 L1, L4 40％ 5kg~30kg S6 L2, L5 40％ 5kg~30kg S7 L3, L6 40％ 5kg~30kg S8 R3, R4 40％ 5kg~30kg S9 R4, R6 40％ 5kg~30kg

[0068] Step 1.5: Convert each first acceleration response signal to obtain a plurality of first time-frequency images.

[0069] In a specific embodiment of the present invention, a one-dimensional time series signal is converted into a two-dimensional image using Gramian Angular Difference Fields (GADF), retaining the time dependence and nonlinear characteristics of the signal. The first acceleration response signal is converted using GADF, specifically including:

[0070] Step 1.51: Normalize the first acceleration response signal and scale each sample value of the first acceleration response signal to the interval [-1,1]. The specific formula is:

[0071]

[0072] Among them, x i represents the i-th sampling value of the first acceleration response signal, x i ' represents the normalized value of the i-th sampling value, X represents the first acceleration response signal, max(X) represents the maximum sampling value in the first acceleration response signal, and min(X) represents the minimum sampling value in the first acceleration response signal. The normalized signal is formed by the normalized value of each sampling value in time sequence.

[0073] Step 1.52: Convert the normalized signal from the Cartesian coordinate system to the polar coordinate system to obtain a signal in the polar coordinate system.

[0074] The normalized value in each normalized signal is converted into an angle. The specific formula is:

[0075] φ i =arccos(xi ')(2)

[0076] Among them, φ i represents the angle corresponding to the i-th normalized value (i.e., the angle corresponding to sampling point i), arccos represents the inverse cosine function. All angles form a signal in the polar coordinate system.

[0077] Step 1.53: Calculate the angle difference of the signal in the polar coordinate system at different time intervals to generate the Gram angular field matrix.

[0078] The element g in the i-th row and j-th column of the Gram angular field matrix G is ij The calculation formula is:

[0079] g ij = cos(φ i -φ j )(3)

[0080] Wherein, i and j are different sampling points of the first acceleration response signal.

[0081] Step 1.54: Normalize the Gram angular field matrix, such as Figure 4 shown.

[0082] In order to map the Gram angular field matrix to the range of image pixel values ​​(usually 0 to 255), the Gram angular field matrix needs to be normalized. The specific formula is:

[0083]

[0084] Among them, g' ij represents the normalized value of the elements in the Gram angular field matrix, min(G) represents the minimum element of the Gram angular field matrix, and max(G) represents the maximum element of the Gram angular field matrix.

[0085] The normalized Gram angular field matrix is ​​not the size required by the first damage identification model and the second damage identification model. In order to adapt to the input layer size of the first damage identification model and the second damage identification model (for example, 224×224), the normalized Gram angular field matrix needs to be interpolated and adjusted to the target size; then the resized matrix is ​​converted into an image format to obtain a time-frequency image. If a grayscale image is used, the image size is 224×224, and the single-channel image is copied to three channels to generate a time-frequency image of size 224×224×3. Each acceleration response signal corresponds to a two-dimensional time-frequency image, which represents the time-frequency characteristics of the acceleration response signal.

[0086] In this embodiment, a bilinear interpolation method is used to adjust the normalized Gram angular field matrix to a target size.

[0087] For each damage type, 200 first acceleration response signals are taken, for a total of 2000 first acceleration response signals, that is, the source domain data set has 2000 labeled sample data, each labeled sample data includes the damage type and its first time-frequency image. The source domain data set is generated according to the first time-frequency images under different damage types, and the source domain data set is divided into training set, validation set and test set in a ratio of 3:1:1. 1200 training samples and 400 validation samples are used to train the first damage recognition model, and 400 test samples are used to verify the damage recognition accuracy of the source domain model.

[0088] The target domain data set includes a first data set and a second data set. The first data set includes a second time-frequency image and its damage type, and the second data set includes a third time-frequency image. The second time-frequency image is obtained by converting the second acceleration response signal of the structure under different damage types by simulating the second acceleration response signal using a field test method. Taking a cable-stayed bridge as an example, the second time-frequency image is obtained by using a field test method, which specifically includes:

[0089] Construct a field test model of a cable-stayed bridge; the field test model includes the main beam, cables and bridge towers; arrange acceleration sensors on the main beam of the field test model, and use carts with different weights and travel speeds to simulate the load on the main beam; simulate different damage types on the field test model, and use acceleration sensors to collect the second acceleration response signal of the main beam under different damage types; use the Gram angle field to convert each second acceleration response signal to obtain multiple second time-frequency images, and the conversion process is the same as step 1.5. In this way, the second time-frequency images of the structure under different damage types are obtained to form the first data set of the target domain data set. The damage types of the field test are shown in Table 1. Each damage type is tested 15 times, and a total of 150 labeled sample data are obtained.

[0090] The third time-frequency image is obtained by real-time monitoring of the actual structure, obtaining the third acceleration response signal, and then converting the third acceleration response signal. Taking the cable-stayed bridge as an example, an acceleration sensor is set on the main beam of the actual cable-stayed bridge, and the third acceleration response signal of the main beam is collected by the acceleration sensor. Then, each third acceleration response signal is converted using the Gram angle field to obtain multiple third time-frequency images. The second data set of the target domain data set is constructed based on the multiple third time-frequency images. In this embodiment, the number of samples in the second data set is 350. The third acceleration response signal can also be obtained by real-time monitoring of the field test model of the cable-stayed bridge. The samples in the first data set of the target domain data set are labeled samples, and the samples in the second data set are unlabeled samples.

[0091] Step 2: Construct a first damage identification model and a second damage identification model with the same architecture.

[0092] In a specific embodiment of the present invention, the first damage identification model and the second damage identification model adopt the GoogLeNet network structure. Figure 5 As shown in the figure, the GoogLeNet network structure includes the sequentially connected input layer, the first convolution layer, the maximum pooling layer, the second convolution layer, the third convolution layer, 9 Inception modules, the global average pooling layer, the first fully connected layer, the second fully connected layer, the third fully connected layer, the classification layer and the output layer. The convolution kernel size of the first convolution layer is 7×7, the kernel size of the maximum pooling layer is 3×3, the convolution kernel size of the second and third convolution layers is 3×3, and the kernel size of the global average pooling layer is 7×7. Figure 6 As shown in the figure, each Inception module contains convolution kernels of various scales, such as 1×1, 3×3, and 5×5, and a 3×3 pooling layer. In the image recognition process, no matter how large the area of ​​the target object is, the characteristics can be captured by the appropriate convolution kernel, which improves the network's ability to express features of different scales. At the same time, the 3×3 pooling layer and two 1×1 convolution kernels operate in parallel, which accelerates the network training and reasoning process while expanding the channel. The introduction of the Inception module makes the GoogleNet network structure deeper than VGG and AlexNet, but the network model parameters are fewer, and the requirements for memory and computing resources are lower.

[0093] Step 3: Use the source domain dataset to train the first damage identification model to obtain the source domain model.

[0094] The first time-frequency image of the source domain data set is used as the input of the first damage recognition model, and the damage type corresponding to the first time-frequency image is used as the expected output (or label). The first damage recognition model is trained, verified and tested to obtain the source domain model.

[0095] During the training process, the cross entropy loss function is used, and the optimization algorithm is the Adam optimizer. The specific training parameters are set as follows: the learning rate is 0.001, the training batch is 100 times, and the number of samples per batch is 32. After the training is completed, the hyperparameter file of the trained source domain model is saved.

[0096] Step 4: Transfer the hyperparameters of the source domain model to the second damage identification model.

[0097] The hyperparameters of the source domain model are transferred to the second damage identification model. Transfer learning is to adapt the source domain model that has been trained on the source domain dataset to the specific task of the target domain. In this embodiment, there are certain differences between the target domain dataset and the source domain dataset, such as the difference between the physical model and the finite element model, but their basic features are similar. Therefore, the sample requirement of the target domain dataset can be reduced through transfer learning.

[0098] Step 5: Freeze all layers of the second damage recognition model except the fully connected layer, and use the first data set to fine-tune the fully connected layer of the second damage recognition model to obtain the target domain model.

[0099] For the convolutional neural network (CNN) model, a large number of experiments have shown that low-level features have strong transfer capabilities, while the features extracted by high-level convolutional layers are highly relevant to specific tasks. That is, the higher the similarity between the two tasks, the fewer convolutional layers need to be retrained. Therefore, when the source domain dataset and the target domain dataset are very close, only the fully connected layer (Fc) of the network model needs to be replaced, and the parameters of the remaining layers remain unchanged to retain the feature extraction capability of the source domain. That is, when training the second damage recognition model, the convolutional layer and pooling layer of the second damage recognition model are frozen, and only the fully connected layer of the second damage recognition model needs to be trained, such as Figure 7 shown.

[0100] The samples in the first data set are labeled samples. The first data set is used to fine-tune only the fully connected layer of the second damage recognition model to obtain the target domain model. When fine-tuning or training, the cross entropy loss function and Adam optimizer are used. In the first round of fine-tuning (i.e., no third time-frequency image is added), the learning rate is set to a low value (such as 0.0001) to ensure that the model parameters do not change significantly; the batch size is 32, and the number of training iterations is 50. After completing the first round of fine-tuning, the parameters and structure of the target domain model are saved to provide a basis for subsequent pseudo-label generation and model updating. The expression of the cross entropy loss function is:

[0101]

[0102] Among them, L represents the loss value, m represents the number of samples, and y i represents the true label of the i-th sample, represents the predicted probability of the i-th sample.

[0103] After each round of fine-tuning, the target domain model is verified using the constructed target domain verification set, and the parameters in the fine-tuning process are dynamically adjusted according to the verification results to prevent overfitting.

[0104] Step 6: Use the target domain model to perform damage identification on the third time-frequency image in the second data set to obtain the damage type and its confidence.

[0105] The target domain model trained with the first data set is used to perform damage identification on the unlabeled samples in the second data set (i.e., the third time-frequency image). The classification layer of the target domain model uses the Softmax function to limit multiple outputs between [0,1], and the sum of probabilities is 1. That is, the probability of each damage type is obtained through the classification layer. Taking the 10 damage types of cable-stayed bridges as an example, the classification layer obtains the probabilities of the 10 damage types. The damage type corresponding to the maximum probability is used as the final damage type of the unlabeled sample, and the probability of the final damage type is used as its confidence. The second data set has 350 unlabeled samples, so 350 damage identification results can be obtained.

[0106] Step 7: Determine whether the confidence of each damage type is greater than or equal to the confidence threshold; if so, add the damage type and the third time-frequency image corresponding to the damage type to the first data set (i.e., update the first data set), and delete the corresponding third time-frequency image from the second data set; if not, retain the damage type and the third time-frequency image corresponding to the damage type in the second data set (update the second data set).

[0107] In order to expand the first data set, the confidence (i.e., the probability of the damage type) in each damage identification result obtained in step 6 is compared with the confidence threshold. When the confidence in the damage identification result is greater than or equal to the confidence threshold, the corresponding third time-frequency image and its pseudo-label (i.e., the damage type identified by the target domain model) are added to the first data set to serve as labeled samples to participate in the next round of fine-tuning of the target domain model; when the confidence in the damage identification result is less than the confidence threshold, the corresponding third time-frequency image and its pseudo-label are retained in the second data set for the next round of identification and judgment.

[0108] Before using the target domain model to identify damage on the third time-frequency image in the second data set in each round, the confidence threshold is dynamically adjusted according to the verification result of the target domain model using the verification set. The confidence threshold gradually decreases with the increase of the fine-tuning rounds, avoiding the loss of important information while reducing the introduction of noise samples. That is, after a round of fine-tuning of the target domain model using the first data set, the target domain model is verified using the verification set, and then the confidence threshold is adjusted according to the verification result, and then the target domain model is used to identify damage on the third time-frequency image in the second data set.

[0109] The target domain model is fine-tuned once using the first dataset, and the number of fine-tuning rounds is increased by 1; each time the first dataset (i.e., the first dataset formed by merging the pseudo-labeled samples and the original labeled samples of the first dataset) is updated according to the damage identification results of the second dataset, the target domain model is fine-tuned once using the first dataset. In the first round of fine-tuning, the confidence threshold is set to 0.9, that is, the initial value of the confidence threshold is 0.9.

[0110] Step 8: Determine whether there is a damage type and a third time-frequency image corresponding to the damage type added to the first data set (i.e., determine whether the first data set is updated); if so, proceed to step 9; otherwise, output the final target domain model.

[0111] The third time-frequency image corresponding to no loss type and damage type is added to the first data set, that is, the first data set is no longer expanded, and the performance of the target domain model is no longer improved, so the fine-tuning process is performed to obtain the final target domain model. The final target domain model is used to identify the structural damage type.

[0112] Step 9: Fine-tune the target domain model using the updated first dataset, and then go to step 6.

[0113] The target domain model is fine-tuned for the next round using the updated first dataset. When pseudo-label samples are added to the first dataset, the convolutional layers and pooling layers of the target domain model are no longer frozen, allowing the parameters of the entire target domain model to be adjusted in the updated first dataset.

[0114] The samples in the first data set are labeled samples, and the samples in the second data set are unlabeled samples. A small number of labeled samples in the first data set are used to preliminarily fine-tune the second damage identification model to obtain the target domain model. Then, the high-confidence unlabeled samples (with pseudo-labels) in the second data set and the labeled samples in the first data set are used to self-train the target domain model, so that the final target domain model is more suitable for engineering practice and improves the damage identification ability of the model.

[0115] It is difficult to obtain a large number of labeled samples in the target domain dataset. Therefore, the present invention adopts a semi-supervised self-training learning method to gradually improve the damage recognition ability of the target domain model by generating pseudo labels and adding them to the first dataset through multiple rounds of fine-tuning.

[0116] In order to verify the effectiveness of the present invention, the damage identification accuracy of the final target domain model (i.e., Model 4) obtained by the present invention is compared with that of the other three models. Model 1: The untrained GoogLeNet network structure is trained using the initial first data set of the target domain; Model 2: The untrained GoogLeNet network structure is trained using the first data set of the target domain, and then the trained GoogLeNet network structure is used to identify damage to the second data set of the target domain, the first data set is updated according to the damage identification result, and the GoogLeNet network structure is trained using the updated first data set until the first data set is no longer updated; Model 3: The untrained GoogLeNet network structure is trained using the source domain data set, and then the fully connected layer of the trained GoogLeNet network structure is fine-tuned using the initial first data set.

[0117] The test data set is used to evaluate the damage identification accuracy of Model 1, Model 2, Model 3 and Model 4. The damage identification accuracy of Model 1, Model 2, Model 3 and Model 4 is 73.6%, 79.4%, 84.6% and 89.8% respectively. Figure 8 to Figure 11 The damage identification confusion matrix of the four models on the cable-stayed bridge test data set is shown. The ordinate of the confusion matrix represents the actual damage type, the abscissa represents the damage type predicted by the model, and the color-coded numbers represent the number of samples. The number of samples on the diagonal of the damage identification confusion matrix is ​​the number of samples that are correctly predicted, and the others are the number of samples that are incorrectly predicted. Figure 8 to Figure 11 It can be seen that due to the small number of samples, Model 1 failed to fully learn the features, resulting in the lowest damage recognition accuracy on the test data set; Model 2 and Model 3 adopted self-training learning and transfer learning strategies respectively, which were improved compared with Model 1, but Model 2 may have selected more erroneous pseudo-labels, and the single-condition recognition accuracy of some working conditions (such as S1 and S2) decreased; Model 4 showed the highest damage recognition accuracy. Although there were more misjudgments in the S3 and S9 working conditions, it was basically able to achieve accurate recognition in other working conditions.

[0118] T-distributed random neighbor embedding is a visualization tool that can be used to obtain a clear intuition of the data feature space. In order to visualize the damage type recognition results of models 1 to 4, t-distributed random neighbor embedding is used to display the damage type recognition results of the four models, such as Figure 12 to Figure 15 As shown in the figure, solid circles of different colors mark different types of damage. Figure 12 to Figure 15 It can be seen that in model 1, except for damage types S6 and S8, the spatial distribution of data of different damage types is relatively close or even overlapping; in model 2 and model 3, except for the almost overlapping of damage types S3 and S9 and the misidentification of some damage type S8 as S7, other damage types can clearly identify clusters from the same damage type; in contrast, in model 4, except for the slight overlap of damage types S3 and S9, the clusters of the remaining damage types are clearly displayed, which verifies that the model of the present invention has superior damage identification performance and can effectively distinguish various types of data.

[0119] The present invention realizes accurate damage identification of structures under the condition of limited labeled samples. The present invention not only improves the accuracy and robustness of damage identification, but also significantly reduces the dependence on a large number of labeled samples, and is suitable for the fields of structural monitoring and fault diagnosis.

[0120] Example 2

[0121] An embodiment of the present invention also provides an electronic device, which includes: a memory, a processor, and a computer program / instructions stored in the memory, and the processor executes the computer program / instructions to implement the structural damage identification method based on transfer learning in the embodiment of the present application.

[0122] Although not shown, the electronic device includes a processor, which can perform various appropriate operations and processes according to the program and / or data stored in the read-only memory (ROM) or the program and / or data loaded from the storage portion into the random access memory (RAM). The processor can be a multi-core processor, or it can include multiple processors. In some embodiments, the processor can include a general main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0123] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0124] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the structural damage identification method based on transfer learning in an embodiment of the present application.

[0125] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be implemented by any method or technology to store information. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0126] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0127] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the structural damage identification method based on transfer learning in the embodiment of the present application.

[0128] What is disclosed above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A structural damage identification method based on transfer learning, characterized in that: include: Step 1: Acquire a source domain dataset and a target domain dataset; wherein the source domain dataset includes a first time-frequency image and its damage type; the target domain dataset includes a first dataset and a second dataset, the first dataset includes a second time-frequency image and its damage type, and the second dataset includes a third time-frequency image; the first time-frequency image, the second time-frequency image and the third time-frequency image are obtained by converting the acceleration response signal of the structure; Step 2: construct a first damage identification model and a second damage identification model with the same architecture; Step 3: Using the source domain data set to train the first damage identification model to obtain a source domain model; Step 4: Migrating the hyperparameters of the source domain model to the second damage identification model; Step 5: Freeze all layers of the second damage recognition model except the fully connected layer, and use the first data set to fine-tune the fully connected layer of the second damage recognition model to obtain a target domain model; Step 6: Use the target domain model to perform damage identification on the third time-frequency image in the second data set to obtain the damage type and its confidence level; Step 7: Determine whether the confidence of each damage type is greater than or equal to the confidence threshold. If so, add the damage type and the third time-frequency image corresponding to the damage type to the first data set; if not, retain the damage type and the third time-frequency image corresponding to the damage type in the second data set; Step 8: Determine whether there is a damage type and a third time-frequency image corresponding to the damage type added to the first data set; if so, proceed to step 9; otherwise, output the final target domain model; Step 9: Use the updated first data set to fine-tune the target domain model, and then go to step 6.

2. The structural damage identification method based on transfer learning according to claim 1 is characterized in that: The first time-frequency image is obtained by converting the first acceleration response signal by simulating the first acceleration response signal of the structure under different damage types using the finite element method; The second time-frequency image is obtained by converting the second acceleration response signal by simulating the second acceleration response signal of the structure under different damage types using a field test method; The third time-frequency image is obtained by real-time monitoring of the actual structure to obtain a third acceleration response signal, and then converting the third acceleration response signal.

3. The structural damage identification method based on transfer learning according to claim 1 is characterized in that: The Gram angle field is used to convert the acceleration response signal of the structure, including: Normalizing the acceleration response signal to obtain a normalized signal; Converting the normalized signal from a Cartesian coordinate system to a polar coordinate system to obtain a signal in the polar coordinate system; Calculate the angle difference of the signal in the polar coordinate system at different time intervals to generate the Gram angular field matrix; Normalizing the Gram angular field matrix to obtain a normalized Gram angular field matrix; The normalized Gram angular field matrix is ​​resized and converted to obtain a time-frequency image.

4. The structural damage identification method based on transfer learning according to claim 1 is characterized in that: The first damage identification model and the second damage identification model adopt a GoogLeNet network structure, which includes an input layer, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a third convolutional layer, 9 Inception modules, a global average pooling layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a classification layer and an output layer connected in sequence.

5. The structural damage identification method based on transfer learning according to any one of claim 1, characterized in that: After fine-tuning the target domain model using the first data set, and before performing damage recognition on the third time-frequency image in the second data set using the target domain model, the recognition method further includes: The target domain model is verified using a verification set; and the confidence threshold is adjusted according to the verification result.

6. The structural damage identification method based on transfer learning according to claim 1 is characterized in that: The confidence threshold gradually decreases according to the fine-tuning rounds; the initial value of the confidence threshold is 0.

9.

7. The structural damage identification method based on transfer learning according to any one of claims 1 to 6, characterized in that: During the fine-tuning process, the cross entropy loss function and Adam optimizer are used to update the parameters of the second damage identification model or the target domain model.

8. An electronic device comprising a memory, a processor, and computer program instructions stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the structural damage identification method as described in any one of 1 to 7.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the structural damage identification method described in any one of 1 to 7 is implemented.

10. A computer program product comprising computer program instructions, characterized in that When the computer program / instructions are executed by a processor, the structural damage identification method described in any one of 1 to 7 is implemented.

Images