Structural damage identification method, device, medium and product based on transfer learning

Abstract

The application discloses a structure damage identification method and device based on transfer learning, a medium and a product, and relates to the technical field of structure damage identification. The method comprises the following steps: acquiring a source domain data set and a target domain data set; constructing a first damage identification model and a second damage identification model; training the first damage identification model by using the source domain data set to obtain a source domain model; migrating the hyperparameters of the source domain model to the second damage identification model; freezing all layers of the second damage identification model except the fully connected layer, fine-tuning the fully connected layer of the second damage identification model by using the first data set, and obtaining a target domain model; identifying damage of the second data set by using the target domain model to obtain a damage type and a confidence; when the confidence of the damage type is greater than or equal to a confidence threshold, updating the first data set; and fine-tuning the target domain model by using the updated first data set. The application improves the damage identification capability of the model under a small amount of labeled samples.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of structural health monitoring, and particularly relates to a structural damage identification method, device, medium and product based on transfer learning. BACKGROUND

[0002] In recent years, deep learning technology has made significant progress in image recognition, speech processing and other fields, and also provides new ideas for the field of structural health monitoring. However, the effective training of deep learning models usually requires a large amount of labeled data, and in bridge engineering practice, the process of obtaining labeled data is expensive and time-consuming, especially in actual monitoring environment, the manually annotated damage data is even scarce. In addition, directly using simulation data generated by physical model or finite element model to train deep learning model may lead to insufficient generalization ability of the model in actual application due to the difference in data distribution.

[0003] With the development of sensing technology and data acquisition technology, structural health monitoring systems have been applied to bridge engineering, and through sensors arranged on the bridge, real-time dynamic response data of the structure can be obtained. However, how to effectively identify and locate structural damage from massive monitoring data is still a challenging problem. At present, data-driven machine learning methods have been introduced into the field of bridge damage identification, among which deep learning methods have gradually become a research hotspot due to their advantages in feature extraction and pattern recognition. However, in actual engineering problems, due to the high cost of manually annotated samples and the difficulty of obtaining labeled samples, CNN model (Convolutional Neural Network, CNN) requires a large amount of labeled sample data for training to achieve high recognition accuracy. Under the condition of insufficient labeled samples, the efficiency of cable stay damage identification is low, so how to improve the recognition accuracy of the model under the condition of limited samples has become a problem to be solved.

[0004] Transfer learning as an effective solution to the problem of small sample learning, by transferring models and knowledge from other fields (source domain) to the target field, thereby reducing the dependence on a large amount of labeled data. However, there is a certain difference between the target domain and the source domain, and simple model transfer may lead to a decline in model performance. Therefore, how to combine transfer learning and other technical means to further improve the adaptability and accuracy of the model in the target domain has become an important direction of research in this field. SUMMARY

[0005] The present application aims to provide a structure damage identification method, device, medium and product based on transfer learning, to solve at least one of the problems that traditional technology relies on a large number of labeled sample data, resulting in high cost and difficulty in obtaining labeled samples, that there is not enough labeled sample data, resulting in reduced model recognition performance, and that simple transfer learning results in reduced model recognition performance.

[0006] The present application solves the above technical problems by the following technical solutions: a structure damage identification method based on transfer learning, comprising:

[0007] Step 1: Obtain a source domain dataset and a target domain dataset; wherein the source domain dataset includes a first time-frequency image and its damage type; the target domain dataset includes a first dataset and a second dataset, the first dataset includes a second time-frequency image and its damage type, and the second dataset includes a third time-frequency image; the first time-frequency image, the second time-frequency image and the third time-frequency image are obtained by converting the acceleration response signal of the structure;

[0008] Step 2: Construct a first damage identification model and a second damage identification model with the same architecture;

[0009] Step 3: Train the first damage identification model using the source domain dataset to obtain a source domain model;

[0010] Step 4: Migrate the hyperparameters of the source domain model to the second damage identification model;

[0011] Step 5: Freeze all layers of the second damage identification model except the fully connected layer, fine-tune the fully connected layer of the second damage identification model using the first dataset, and obtain a target domain model;

[0012] Step 6: Use the target domain model to identify the damage of the third time-frequency image in the second dataset, and obtain the damage type and its confidence;

[0013] Step 7: Determine whether the confidence of each damage type is greater than or equal to the confidence threshold, if yes, add the damage type and the third time-frequency image corresponding to the damage type to the first dataset; if not, keep the damage type and the third time-frequency image corresponding to the damage type in the second dataset;

[0014] Step 8: Determine whether there is a damage type and a third time-frequency image corresponding to the damage type added to the first dataset; if yes, go to step 9; otherwise, output the final target domain model;

[0015] Step 9: Fine-tune the target domain model using the updated first dataset, and then go to step 6.

[0016] Further, the first time-frequency image is a first acceleration response signal of the structure under different damage types simulated by a finite element method, and is obtained by converting the first acceleration response signal;

[0017] The second time-frequency image is a second acceleration response signal of the structure under different damage types simulated by a field test method, and is obtained by converting the second acceleration response signal;

[0018] The third time-frequency image is obtained by converting a third acceleration response signal obtained by real-time monitoring of an actual structure.

[0019] Further, the acceleration response signal of the structure is converted by using a Gram angle field, specifically including:

[0020] The acceleration response signal is normalized to obtain a normalized signal;

[0021] The normalized signal is converted from a Cartesian coordinate system to a polar coordinate system to obtain a signal in the polar coordinate system;

[0022] The angle difference of the signal in the polar coordinate system at different time intervals is calculated to generate a Gram angle field matrix;

[0023] The Gram angle field matrix is normalized to obtain a normalized Gram angle field matrix;

[0024] The normalized Gram angle field matrix is size-adjusted and format-converted to obtain a time-frequency image.

[0025] Further, the first damage identification model and the second damage identification model use a GoogLeNet network structure, and the GoogLeNet network structure includes an input layer, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a third convolutional layer, nine Inception modules, a global average pooling layer, a first full connection layer, a second full connection layer, a third full connection layer, a classification layer and an output layer connected in sequence.

[0026] Further, after the target domain model is fine-tuned for one round by using the first data set, before the target domain model is used to identify the damage of the third time-frequency image in the second data set, the identification method further includes:

[0027] The target domain model is verified by using a verification set; and the confidence threshold is adjusted according to the verification result.

[0028] Further, the confidence threshold gradually decreases according to the fine-tuning round; and the initial value of the confidence threshold is 0.9.

[0029] Further, in the fine-tuning process, the parameters of the second damage identification model or the target domain model are updated using a cross-entropy loss function and an Adam optimizer.

[0030] Based on the same concept, the present application also provides an electronic device comprising a memory, a processor, and computer program instructions stored on the memory, wherein the processor executes the computer program / instructions to implement the structure damage identification method based on transfer learning as described above.

[0031] Based on the same concept, the present application also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program / instructions are executed by a processor to implement the structure damage identification method based on transfer learning as described above.

[0032] Based on the same concept, the present application also provides a computer program product comprising computer program instructions, wherein the computer program / instructions are executed by a processor to implement the structure damage identification method based on transfer learning as described above.

[0033] Advantages

[0034] Compared with the prior art, the present application has the following advantages:

[0035] On the basis of the source domain model, the present application iteratively trains the target domain model in combination with a small amount of labeled samples (i.e., the first data set) and a large amount of unlabeled samples of the target domain data set, gradually improving the recognition performance and robustness of the target domain model; the target domain model is used to recognize the unlabeled samples to generate pseudo-labeled samples; the target domain model is self-trained using the labeled samples and the pseudo-labeled samples of the target domain data set, significantly improving the damage identification accuracy, solving the problem that the traditional method is strongly dependent on a large amount of labeled samples and it is difficult to obtain labeled samples, and avoiding the problem of insufficient number of labeled samples or reduced recognition performance caused by simple transfer learning. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only one embodiment of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0037] Figure 1 is the flow chart of the structure damage identification method in the embodiment of the present application;

[0038] Figure 2 is the physical model of the cable-stayed bridge in the embodiment of the present application;

[0039] Figure 3is a finite element model of a cable-stayed bridge in an embodiment of the present application;

[0040] Figure 4 is a GADF conversion (or encoding) schematic diagram in an embodiment of the present application;

[0041] Figure 5 is a GoogLeNet network structure diagram in an embodiment of the present application;

[0042] Figure 6 is an Inception module schematic diagram in an embodiment of the present application;

[0043] Figure 7 is a hyperparameter transfer schematic diagram in an embodiment of the present application;

[0044] Figure 8 is a damage identification confusion matrix of model one in an embodiment of the present application;

[0045] Figure 9 is a damage identification confusion matrix of model two in an embodiment of the present application;

[0046] Figure 10 is a damage identification confusion matrix of model three in an embodiment of the present application;

[0047] Figure 11 is a damage identification confusion matrix of model four in an embodiment of the present application;

[0048] Figure 12 is a t-sne visualization diagram of feature distribution of different damage types of model one in an embodiment of the present application;

[0049] Figure 13 is a t-sne visualization diagram of feature distribution of different damage types of model two in an embodiment of the present application

[0050] Figure 14 is a t-sne visualization diagram of feature distribution of different damage types of model three in an embodiment of the present application

[0051] Figure 15 is a t-sne visualization diagram of feature distribution of different damage types of model four in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0053] The technical solutions of the present application are described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described in detail in some examples.

[0054] Embodiment 1

[0055] As shown in the following, Figure 1 a method for structural damage identification based on transfer learning provided by an embodiment of the present application includes the following steps:

[0056] Step 1: Obtain a source domain data set and a target domain data set.

[0057] The source domain data set includes first time-frequency images and damage types of each first time-frequency image. The first time-frequency image is obtained by converting the first acceleration response signal of the structure under different damage types simulated by the finite element method. Taking a cable-stayed bridge as an example, the specific process of obtaining the source domain data set includes:

[0058] Step 1.1: According to the actual design parameters of the cable-stayed bridge, a physical model of the cable-stayed bridge is constructed according to the scale ratio (for example, 1:50).

[0059] As shown in the following, Figure 2 the physical model includes key structural components such as the main girder, the cable and the tower.

[0060] Step 1.2: Based on the geometric parameters and material parameters of the physical model, a finite element model of the cable-stayed bridge is constructed, and the finite element model is meshed.

[0061] In this embodiment, the grid size is 0.1m to ensure accuracy of details; Shell element is used to simulate the deck slab, and beam element is used to simulate the cable; linear elastic material model is used to describe the mechanical behavior of the bridge structure, including elastic modulus of 2.1×10 5 N / mm 2 , density of 7.9×10 -6 kg / mm 3 , and Poisson's ratio of 0.3.

[0062] Step 1.3: Apply boundary conditions and loads.

[0063] The bottom of the tower is fixed, and the simply supported boundary condition is applied. The load is applied to the main girder by the dload subroutine to simulate the vehicle load, and the vehicle load can be selected according to the actual situation.

[0064] Step 1.4: The implicit dynamic analysis method is used to calculate the first acceleration response signal of each measuring point of the finite element model of the cable-stayed bridge under different damage types.

[0065] like Figure 3 As shown, four measuring points were set on the main beam of the finite element model. The acceleration response signals of the four measuring points were spliced ​​together in time sequence to form a complete time-series signal (i.e., the first acceleration response signal) to ensure that the signal contains more structural features. Meanwhile, considering the complexity of data in actual engineering, the sample size was expanded by loading different loads and 20dB Gaussian white noise. Thus, the first acceleration response signals of the main beam under different damage types were obtained, with a signal sampling frequency of 500Hz. In this embodiment, 10 damage types were included, as shown in Table 1. The degree of damage to the cables was simulated by changing the elastic modulus of the cables.

[0066] Table 1. Ten types of damage to cable-stayed bridges

[0067] Damage type (label) Damage location Damage degree Vehicle weight (load) S0 No damage / 5kg~30kg S1 L1 40％ 5kg~30kg S2 L3 40％ 5kg~30kg S3 L5 40％ 5kg~30kg S4 L7 40％ 5kg~30kg S5 L1, L4 40％ 5kg~30kg S6 L2, L5 40％ 5kg~30kg S7 L3, L6 40％ 5kg~30kg S8 R3, R4 40％ 5kg~30kg S9 R4, R6 40％ 5kg~30kg

[0068] Step 1.5: Convert each first acceleration response signal to obtain multiple first time-frequency images.

[0069] In a specific embodiment of the present invention, Gramian Angular Difference Fields (GADF) are used to convert a one-dimensional time series signal into a two-dimensional image, preserving the time dependence and nonlinear characteristics of the signal. The conversion of the first acceleration response signal using GADF specifically includes:

[0070] Step 1.51: Normalize the first acceleration response signal by scaling each sample value of the first acceleration response signal to the interval [-1, 1]. The specific formula is as follows:

[0071]

[0072] Where, x i Let x represent the i-th sampled value of the first acceleration response signal. i Let ' represent the normalized value of the i-th sample, X represent the first acceleration response signal, max(X) represent the maximum sample value in the first acceleration response signal, and min(X) represent the minimum sample value in the first acceleration response signal. The normalized signal is formed by the normalized values ​​of each sample in chronological order.

[0073] Step 1.52: Transform the normalized signal from the Cartesian coordinate system to the polar coordinate system to obtain the signal in the polar coordinate system.

[0074] The normalized values ​​in each normalized signal are converted into angles using the following formula:

[0075] φ i =arccos(xi ')(2)

[0076] where, φ i represents the angle corresponding to the i-th normalized value (i.e. the angle corresponding to the i-th sampling point), arccos represents the inverse cosine function. The signal in the polar coordinate system is formed by all angles.

[0077] Step 1.53: Calculate the angle difference of the signal in the polar coordinate system at different time intervals to generate the Gram angle field matrix.

[0078] The calculation formula of the i-th row and j-th column element g ij of the Gram angle field matrix G is:

[0079] g ij = cos(φ i - φ j )(3)

[0080] where i and j are different sampling points of the first acceleration response signal.

[0081] Step 1.54: Normalize the Gram angle field matrix as shown in Figure 4 .

[0082] In order to map the Gram angle field matrix into the range of image pixel values (usually 0 to 255), the Gram angle field matrix needs to be normalized, and the specific formula is:

[0083]

[0084] where g' ij represents the normalized value of the element in the Gram angle field matrix, min(G) represents the minimum element of the Gram angle field matrix, and max(G) represents the maximum element of the Gram angle field matrix.

[0085] The normalized Gram angle field matrix is not the size required by the first damage identification model and the second damage identification model. In order to adapt to the input layer size (such as 224x224) of the first damage identification model and the second damage identification model, the normalized Gram angle field matrix needs to be adjusted by interpolation to adjust the normalized Gram angle field matrix to the target size; then the size-adjusted matrix is converted to an image format to obtain a time-frequency image. If a grayscale image is used, the image size is 224x224, and the single-channel image is copied to three channels to generate a time-frequency image with a size of 224x224x3. Each acceleration response signal corresponds to a two-dimensional time-frequency image representing the time-frequency characteristics of the acceleration response signal.

[0086] In this embodiment, the normalized Gram angle field matrix is adjusted to the target size using the bilinear interpolation method.

[0087] Take 200 first acceleration response signals for each damage type, a total of 2000 first acceleration response signals, that is, the source domain data set has 2000 labeled sample data, each labeled sample data includes a damage type and its first time-frequency image. The source domain data set is generated according to the first time-frequency image under different damage types, and the source domain data set is divided into a training set, a validation set and a test set according to a ratio of 3:1:1. 1200 training samples and 400 validation samples are used to train the first damage identification model, and 400 test samples are used to verify the damage identification accuracy of the source domain model.

[0088] The target domain data set includes a first data set and a second data set, the first data set includes a second time-frequency image and its damage type, and the second data set includes a third time-frequency image. The second time-frequency image is obtained by converting the second acceleration response signal simulated by the field test method under different damage types of the structure. Taking a cable-stayed bridge as an example, the second time-frequency image is obtained by the field test method, specifically including:

[0089] A field test model of the cable-stayed bridge is constructed; wherein the field test model includes a main beam, a cable and a tower, etc.; an acceleration sensor is arranged on the main beam of the field test model, and a trolley with different weights and driving speeds is used to simulate the load action on the main beam; different damage types are simulated on the field test model, and the second acceleration response signal of the main beam under different damage types is collected by the acceleration sensor; a plurality of second time-frequency images are obtained by converting each second acceleration response signal using a gram angle field, and the conversion process and step 1.5 are the same. The second time-frequency image under different damage types of the structure is obtained, and the first data set of the target domain data set is formed. The damage types of the field test are shown in Table 1, and 15 tests are performed for each damage type, a total of 150 labeled sample data are obtained.

[0090] The third time-frequency image is obtained by converting the third acceleration response signal obtained by real-time monitoring of the actual structure. Taking a cable-stayed bridge as an example, an acceleration sensor is arranged on the main beam of the actual cable-stayed bridge, the third acceleration response signal of the main beam is collected by the acceleration sensor, and a plurality of third time-frequency images are obtained by converting each third acceleration response signal using a gram angle field. The second data set of the target domain data set is constructed according to the plurality of third time-frequency images, and in this embodiment, the number of samples in the second data set is 350. The third acceleration response signal can also be obtained by real-time monitoring of the field test model of the cable-stayed bridge. The samples in the first data set of the target domain data set are labeled samples, and the samples in the second data set are unlabeled samples.

[0091] Step 2: Construct a first damage identification model and a second damage identification model with the same architecture.

[0092] In the specific embodiments of the present application, the first damage identification model and the second damage identification model adopt a GoogLeNet network structure. As shown in Figure 5 the GoogLeNet network structure includes an input layer, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a third convolutional layer, nine Inception modules, a global average pooling layer, a first full connection layer, a second full connection layer, a third full connection layer, a classification layer, and an output layer connected in sequence. The convolutional kernel size of the first convolutional layer is 7x7, the kernel size of the maximum pooling layer is 3x3, the convolutional kernel size of the second convolutional layer and the third convolutional layer is 3x3, and the kernel size of the global average pooling layer is 7x7. As shown in Figure 6 each Inception module contains 1x1, 3x3, and 5x5 multi-scale convolutional kernels and a 3x3 pooling layer, which is used to capture the characteristics of the target object regardless of the size of the target object in the image recognition process, thereby improving the expression ability of the network for different scale features. At the same time, the 3x3 pooling layer and the two 1x1 convolutional kernels operate in parallel, which expands the channel while speeding up the network training and inference process. The introduction of the Inception module makes the GoogleNet network structure deeper than VGG and AlexNet, but the network model parameters are fewer, and the requirements for memory and computing resources are lower.

[0093] Step 3: training the first damage identification model using the source domain dataset to obtain a source domain model.

[0094] The first time-frequency image of the source domain dataset is used as the input of the first damage identification model, and the damage type corresponding to the first time-frequency image is used as the expected output (or label). The first damage identification model is trained, verified, and tested to obtain a source domain model.

[0095] During the training process, a cross-entropy loss function is used, and the optimization algorithm is an Adam optimizer. The training parameters are specifically set as follows: the learning rate is 0.001, the training batch is 100 times, and the number of samples per batch is 32. After training, the hyperparameter file of the trained source domain model is saved.

[0096] Step 4: migrating the hyperparameters of the source domain model to the second damage identification model.

[0097] The hyperparameters of the source domain model are migrated to the second damage identification model. Migration learning is to adapt the source domain model trained on the source domain dataset to the specific task of the target domain. In this embodiment, the target domain dataset and the source domain dataset have certain differences, such as the difference between the physical model and the finite element model, but the basic features are similar. Therefore, through migration learning, the sample demand of the target domain dataset can be reduced.

[0098] Step 5: Freeze all layers of the second damage identification model except the fully connected layer, fine-tune the fully connected layer of the second damage identification model using the first data set, and obtain the target domain model.

[0099] For a convolutional neural network (CNN) model, a large number of experiments show that low-level features have strong migration ability, while high-level convolutional layers extract features that are highly related to specific tasks. That is, the higher the similarity between two tasks, the fewer the convolutional layers that need to be retrained. Therefore, when the source domain data set and the target domain data set are very similar, only the fully connected layer (Fc) of the network model needs to be replaced, and the parameters of the remaining layers remain unchanged to retain the feature extraction ability of the source domain. That is, when training the second damage identification model, the convolutional layers and the pooling layers of the second damage identification model are frozen, and only the fully connected layers of the second damage identification model need to be trained, as shown in Figure 7 .

[0100] The samples in the first data set are labeled samples. The first data set is used to fine-tune only the fully connected layer of the second damage identification model to obtain the target domain model. During fine-tuning or training, the cross-entropy loss function and the Adam optimizer are used. During the first round of fine-tuning (i.e., without adding the third time-frequency image), the learning rate is set to a low value (such as 0.0001) to ensure that the model parameters do not change significantly; the batch size is 32, and the number of training iterations is 50. After completing the first round of fine-tuning, the parameters and structure of the target domain model are saved to provide a basis for subsequent pseudo-label generation and model updating. The expression of the cross-entropy loss function is:

[0101]

[0102] where L represents the loss value, m represents the number of samples, y i represents the true label of the i-th sample, represents the predicted probability of the i-th sample.

[0103] After each round of fine-tuning, the target domain model is verified using the constructed target domain validation set. The parameters in the fine-tuning process are dynamically adjusted based on the verification results to prevent overfitting.

[0104] Step 6: Use the target domain model to identify the damage of the third time-frequency image in the second data set, and obtain the damage type and its confidence.

[0105] The target domain model trained by the first data set is used for damage identification of the unlabeled samples (i.e. the third time-frequency images) in the second data set. The classification layer of the target domain model limits multiple outputs to [0, 1] through the Softmax function, and the sum of the probabilities is 1. That is, the probability of each damage type is obtained through the classification layer. Taking the 10 damage types of the cable-stayed bridge as an example, the classification layer obtains the probabilities of the 10 damage types. The damage type corresponding to the maximum probability is taken as the final damage type of the unlabeled sample, and the probability of the final damage type is taken as the confidence thereof. There are 350 unlabeled samples in the second data set, and 350 damage identification results can be obtained.

[0106] Step 7: Determine whether the confidence of each damage type is greater than or equal to the confidence threshold; if yes, add the damage type and the third time-frequency image corresponding to the damage type to the first data set (i.e. update the first data set), and delete the corresponding third time-frequency image from the second data set; if no, keep the damage type and the third time-frequency image corresponding to the damage type in the second data set (update the second data set).

[0107] In order to expand the first data set, the confidence (i.e. the probability of the damage type) in each damage identification result obtained in step 6 is compared with the confidence threshold. When the confidence in the damage identification result is greater than or equal to the confidence threshold, the corresponding third time-frequency image and its pseudo label (i.e. the damage type identified by the target domain model) are added to the first data set to serve as a labeled sample for the next round of fine-tuning of the target domain model; when the confidence in the damage identification result is less than the confidence threshold, the corresponding third time-frequency image and its pseudo label are kept in the second data set for the next round of identification and judgment.

[0108] Before each round of damage identification of the third time-frequency images in the second data set by the target domain model, the confidence threshold is dynamically adjusted according to the verification result of the target domain model on the verification set. The confidence threshold gradually decreases with the increase of the fine-tuning rounds, which avoids the loss of important information and reduces the introduction of noise samples. That is, after one round of fine-tuning of the target domain model by the first data set, the target domain model is verified by the verification set, then the confidence threshold is adjusted according to the verification result, and then the third time-frequency images in the second data set are identified by the target domain model.

[0109] After one round of fine-tuning of the target domain model by the first data set, the fine-tuning round is increased by 1; after updating the first data set once according to the damage identification result of the second data set (i.e. the first data set formed by combining the pseudo label samples and the original labeled samples of the first data set), the target domain model is fine-tuned by the first data set. In the first round of fine-tuning, the confidence threshold is set to 0.9, i.e. the initial value of the confidence threshold is 0.9.

[0110] Step 8: judging whether the damage type and the third time-frequency image corresponding to the damage type are added to the first data set (i.e. judging whether the first data set is updated); if yes, turning to step 9; otherwise, outputting the final target domain model.

[0111] Without the damage type and the third time-frequency image corresponding to the damage type being added to the first data set, i.e. the first data set is no longer expanded, the performance of the target domain model is no longer improved, so the fine-tuning process is performed to obtain the final target domain model. The final target domain model is used to recognize the structural damage type.

[0112] Step 9: using the updated first data set to fine-tune the target domain model, and then turning to step 6.

[0113] The updated first data set is used to fine-tune the target domain model in the next round, and when the pseudo-labeled sample is added to the first data set, the convolutional layer, the pooling layer and the like of the target domain model are no longer frozen, and the parameters of the entire target domain model are allowed to be adjusted in the updated first data set.

[0114] The samples in the first data set are labeled samples, and the samples in the second data set are unlabeled samples. The first data set is used to preliminarily fine-tune the second damage recognition model to obtain the target domain model, and then the high-confidence unlabeled samples (with pseudo-labels) in the second data set and the labeled samples in the first data set are used to self-train the target domain model, so that the final target domain model is more suitable for engineering practice, and the damage recognition ability of the model is improved.

[0115] In the target domain data set, it is difficult to obtain a large number of labeled samples, so the present application adopts a semi-supervised self-training learning method, generates pseudo-labels and adds them to the first data set, and gradually improves the damage recognition ability of the target domain model through multiple rounds of fine-tuning.

[0116] In order to verify the effectiveness of the present application, the damage recognition accuracy of the final target domain model (i.e. model four) obtained by the present application is compared with that of other three models. Model one: using the initial first data set of the target domain to train an untrained GoogLeNet network structure; model two: using the first data set of the target domain to train an untrained GoogLeNet network structure, and then using the trained GoogLeNet network structure to recognize the damage of the second data set of the target domain, updating the first data set according to the damage recognition result, and then using the updated first data set to train the GoogLeNet network structure until the first data set is no longer updated; model three: using the source domain data set to train an untrained GoogLeNet network structure, and then fine-tuning the fully connected layer of the trained GoogLeNet network structure using the initial first data set.

[0117] The damage identification accuracy of the model one, the model two, the model three and the model four is evaluated by using the test data set. The damage identification accuracy of the model one, the model two, the model three and the model four is 73.6%, 79.4%, 84.6% and 89.8% respectively. Figure 8~Figure 11 The damage identification confusion matrix of the four models on the test data set of the cable-stayed bridge is shown, the vertical coordinate of the confusion matrix represents the true damage type, the horizontal coordinate represents the damage type predicted by the model, the number of the color scale represents the sample number, the sample number on the diagonal line of the damage identification confusion matrix is the number of the correctly predicted samples, and the others are the number of the incorrectly predicted samples. It can be seen from Figure 8~Figure 11 that, due to the small sample number, the model one fails to learn the features sufficiently, resulting in the lowest damage identification accuracy on the test data set; the model two and the model three respectively adopt the self-training learning and the transfer learning strategy, which are improved compared with the model one, but the model two may be selected with more false pseudo-labels, so that the single working condition identification accuracy in some working conditions (such as S1 and S2) decreases; the model four exhibits the highest damage identification accuracy, although there are more misjudgments in the S3 and S9 working conditions, but it can basically achieve accurate identification in other working conditions.

[0118] The t-distributed stochastic neighbor embedding is a visualization tool, which can be used to obtain a clear intuition of the feature space of the data. In order to realize the visualization of the damage type identification results of the model one to the model four, the t-distributed stochastic neighbor embedding is used to realize the damage type identification result display of the four models, as shown in Figure 12~Figure 15 , different damage types are marked by different color solid circles. It can be seen from Figure 12~Figure 15 that, except for the damage types S6 and S8, the data of different damage types in the model one are relatively close or even overlap in the space; except for the damage types S3 and S9 which are almost overlapped and part of the damage type S8 which is misidentified as S7, the other damage types of the model two and the model three can be clearly identified as the clustering from the same damage type; compared with the model two and the model three, the model four clearly shows the clustering of the remaining damage types except for the slight overlap of the damage types S3 and S9, which verifies that the model has superior damage identification performance and can effectively distinguish various data.

[0119] The present application realizes accurate damage identification of the structure under the condition of limited labeled samples. The present application not only improves the accuracy and robustness of damage identification, but also significantly reduces the dependence on a large number of labeled samples, and is suitable for the field of structure monitoring and fault diagnosis.

[0120] Example 2

[0121] The electronic device includes a memory, a processor, and a computer program / instruction stored on the memory, and the processor executes the computer program / instruction to implement the method for structural damage identification based on transfer learning in the embodiments of the present application.

[0122] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to programs and / or data stored in a read-only memory (ROM) or programs and / or data loaded from a storage section into a random access memory (RAM). The processor can be a multi-core processor or can include a plurality of processors. In some embodiments, the processor can include a general-purpose main processor and one or more special-purpose coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), and the like. In the RAM, various programs and data required for device operation are also stored. The processor, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0123] The processor and the memory are used together to execute programs / instructions stored in the memory, and the programs / instructions, when executed by a computer, can implement the methods, steps, or functions described in the above embodiments.

[0124] Although not shown, the embodiments of the present application also provide a computer-readable storage medium having a computer program / instruction stored thereon, and the computer program / instruction, when executed by a processor, implements the method for structural damage identification based on transfer learning in the embodiments of the present application.

[0125] The storage medium of the embodiments of the present application includes permanent and non-permanent, removable and non-removable information storage articles that can be realized by any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0126] The readable storage medium includes permanent and non-permanent, movable and non-movable media, which can realize information storage by any method or technology. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, the computer readable medium does not include transitory media such as modulated data signals and carriers.

[0127] Although not shown, the embodiments of the present application also provide a computer program product, comprising: computer programs / instructions, which, when executed by a processor, implement the structure damage identification method based on transfer learning in the embodiments of the present application.

[0128] The above only discloses specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or modifications within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for structural damage identification based on transfer learning, characterized in that, The method comprises the following steps: Step 1: obtaining a source domain data set and a target domain data set; wherein the source domain data set comprises a first time-frequency image and a damage type thereof; the target domain data set comprises a first data set and a second data set, the first data set comprises a second time-frequency image and a damage type thereof, and the second data set comprises a third time-frequency image; the first time-frequency image, the second time-frequency image and the third time-frequency image are obtained by performing gram angle field conversion on an acceleration response signal of a structure; Step 2: constructing a first damage identification model and a second damage identification model with the same architecture; Step 3: training the first damage identification model by using the source domain data set to obtain a source domain model; Step 4: migrating the hyperparameters of the source domain model to the second damage identification model; Step 5: freezing all layers of the second damage identification model except the fully connected layer, fine-tuning the fully connected layer of the second damage identification model by using the first data set to obtain a target domain model; Step 6: verifying the target domain model by using a verification set, and adjusting a confidence threshold gradually in a decreasing manner according to a verification result and a fine-tuning round; Step 7: identifying damage of the third time-frequency image in the second data set by using the target domain model to obtain a damage type and a confidence thereof; Step 8: determining whether the confidence of each damage type is greater than or equal to the confidence threshold, if yes, adding the damage type and the third time-frequency image corresponding to the damage type to the first data set; if not, retaining the damage type and the third time-frequency image corresponding to the damage type in the second data set; Step 9: determining whether there is a damage type and a third time-frequency image corresponding to the damage type added to the first data set; if yes, turning to step 10; otherwise, outputting a final target domain model; Step 10: fine-tuning the target domain model by using the updated first data set, and then turning to step 6. 2.The method of claim 1, wherein, The first time-frequency image is obtained by simulating a first acceleration response signal of a structure under different damage types by using a finite element method, and the first acceleration response signal is converted to obtain the first time-frequency image; The second time-frequency image is obtained by simulating a second acceleration response signal of a structure under different damage types by using a field test method, and the second acceleration response signal is converted to obtain the second time-frequency image; The third time-frequency image is obtained by monitoring an actual structure in real time to obtain a third acceleration response signal, and the third acceleration response signal is converted to obtain the third time-frequency image. 3.The method of claim 1, wherein, The gram angle field conversion of the acceleration response signal of the structure comprises the following steps: normalizing the acceleration response signal to obtain a normalized signal; converting the normalized signal from a Cartesian coordinate system to a polar coordinate system to obtain a signal in the polar coordinate system; calculating the angle difference of the signal in the polar coordinate system at different time intervals to generate a gram angle field matrix; normalizing the gram angle field matrix to obtain a normalized gram angle field matrix; adjusting the size and format of the normalized gram angle field matrix to obtain a time-frequency image. 4.The method of claim 1, wherein, The first damage identification model and the second damage identification model adopt a GoogLeNet network structure, and the GoogLeNet network structure comprises an input layer, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a third convolutional layer, nine Inception modules, a global average pooling layer, a first full connection layer, a second full connection layer, a third full connection layer, a classification layer and an output layer connected in sequence. 5.The method of claim 1, wherein, The initial value of the confidence threshold is 0.

9. 6.The method of any one of claims 1-5, wherein, In the fine-tuning process, the parameters of the second damage identification model or the target domain model are updated by using a cross-entropy loss function and an Adam optimizer.

7. An electronic device comprising a memory, a processor, and computer program instructions stored on the memory, wherein the computer program instructions, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-6. The processor executes the computer program / instruction to implement the structural damage identification method according to any one of 1-6.

8. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program / instruction is executed by the processor to implement the structural damage identification method according to any one of 1-6.

9. A computer program product comprising computer program instructions, characterised in that, The computer program / instruction is executed by the processor to implement the structural damage identification method according to any one of 1-6.

Images