A transfer learning method and system for microseismic signal recognition model

Abstract

The present invention relates to a transfer learning method and system for a microseismic signal recognition model, wherein the method comprises: respectively initializing a feature extractor in a base model and a classifier connected to the feature extractor; respectively configuring a first discriminator and a second discriminator at the bottom layers of the feature extractor and the classifier to obtain a two-level adversarial domain adaptive model; respectively inputting source domain data and target domain data into the two-level adversarial domain adaptive model for training to obtain multiple loss functions; back-propagating each loss function along the two-level adversarial domain adaptive model to update the network parameters of the two-level adversarial domain adaptive model; the updated two-level adversarial domain adaptive model is a migrated recognition model. The present invention separately adapts a feature extractor composed of a convolutional layer and a classifier composed of a fully connected layer, thereby enabling efficient domain migration.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring of rock mass engineering disasters, and in particular to a transfer learning method and system for a microseismic signal recognition model. Background Art

[0002] Although the widespread use of deep learning models in microseismic signal recognition has greatly improved the processing efficiency of microseismic monitoring data, how to reuse a trained model is still a problem, and there is currently a lack of corresponding research in this area. It is well known that a large amount of unstructured data is crucial for training models, and the performance of the network also depends to a large extent on the quality of these labeled data. When the labeled data is insufficient, the performance of the deep learning model will inevitably degrade. In real projects, in most cases it is impossible to obtain so much labeled data, because collecting and labeling a lot of high-quality data is very labor-intensive and time-consuming. In addition, in some projects that have just started construction, there is no monitoring data available at all.

[0003] In addition, because the signal characteristics in different projects are different and the data distribution of the signals varies greatly, a model trained on a certain project cannot usually be directly deployed to other projects for use. The geological conditions, construction methods, sensor embedding methods and monitoring settings between different projects will affect the characteristics of the corresponding monitoring signals. For example, the monitoring signals stimulated by the crushing of marble and granite must be different, and the noise caused by the drilling and blasting method and the shield machine construction is also different. In addition, the sensor signal cutoff threshold and the setting of the corresponding frequency will directly affect the length and resolution of the data. Therefore, there must be domain differences between the data sets of two different projects. This is why it is impossible to directly deploy a trained model to a new project.

[0004] In summary, the challenges in model migration are:

[0005] 1) High labeling cost and lack of data. When the project is just starting, there is not much data in the monitoring database to provide model training. Even if there is enough project data, it takes a lot of time to carefully select and label the data.

[0006] 2) Model migration. Deep learning models are very sensitive to data, so the domain differences between engineering datasets lead to difficulties in model migration. Summary of the invention

[0007] In order to overcome the deficiencies of the prior art, the object of the present invention is to provide a transfer learning method and system for a microseismic signal recognition model, which can efficiently realize domain migration.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A transfer learning method for a microseismic signal recognition model, comprising:

[0010] Initializing the feature extractor in the base model and the classifier connected to the feature extractor respectively;

[0011] A first discriminator and a second discriminator are respectively configured at the bottom layers of the feature extractor and the classifier to obtain a two-level adversarial domain adaptive model;

[0012] Inputting the source domain data and the target domain data into the two-stage adversarial domain adaptive model for training respectively to obtain multiple loss functions;

[0013] Each of the loss functions is back-propagated along the two-stage adversarial domain adaptive model to update the network parameters of the two-stage adversarial domain adaptive model; the updated two-stage adversarial domain adaptive model is the migrated recognition model.

[0014] Preferably, after back-propagating the two-stage adversarial domain adaptive model according to the first adversarial loss, the second adversarial loss and the cross entropy loss and updating the network parameters of the two-stage adversarial domain adaptive model, the method further includes:

[0015] Intermittently extract unlabeled data from unlabeled engineering datasets;

[0016] Determine whether the uncertainty score of the extracted unlabeled data is greater than a preset threshold, and if so, label the unlabeled data by an expert to obtain labeled data;

[0017] The labeled data is added to a labeled data set; the labeled data set is used to train the recognition model.

[0018] Preferably, the base model is constructed based on a convolutional neural network, and the base model is used to identify microseismic signals.

[0019] Preferably, the initializing the feature extractor in the base model and the classifier connected to the feature extractor respectively comprises:

[0020] Replacing the parameters of the first convolutional layer of the feature extractor with the parameters of the base model;

[0021] Freezing parameters of the first convolutional layer so that the parameters of the first convolutional layer do not participate in training;

[0022] Replacing the parameters of the second residual block of the feature extractor with the parameters of the base model; the parameters of the second residual block are in an activated state and participate in training;

[0023] The He Uniform method is used to initialize all other convolutional layers of the feature extractor except the first convolutional layer and the second residual block.

[0024] Preferably, the initializing the feature extractor in the base model and the classifier connected to the feature extractor respectively comprises:

[0025] Replacing the parameters of the fully connected layer of the classifier with the parameters of the fully connected layer of the base model;

[0026] An adaptive layer is connected to the output end of the fully connected layer of the classifier; the adaptive layer is a fully connected layer;

[0027] The adaptive layer is initialized using the He normal method.

[0028] Preferably, the first discriminator and the second discriminator are respectively configured at the bottom layers of the feature extractor and the classifier to obtain a two-level adversarial domain adaptive model, including:

[0029] Connecting the first discriminator to the output of the feature extractor;

[0030] The second discriminator is connected to the output end of the adaptive layer of the classifier; the two-stage adversarial domain adaptive model includes the feature extractor, the classifier, the first discriminator and the second discriminator.

[0031] Preferably, the source domain data and the target domain data are respectively input into the two-stage adversarial domain adaptive model for training to obtain multiple loss functions, including:

[0032] When the training data is the source domain data, obtaining first output data of the feature extractor in the two-stage adversarial domain adaptive model, second output data of the classifier output, and a cross entropy loss;

[0033] When the training data is the target domain data, obtaining third output data of the feature extractor in the two-stage adversarial domain adaptive model and fourth output data of the classifier output;

[0034] Inputting the first output data and the third output data into the first discriminator to generate a first adversarial loss;

[0035] The second output data and the fourth output data are input into the second discriminator to generate a second adversarial loss; the loss function is any one of a cross entropy loss, a first adversarial loss or a second adversarial loss.

[0036] A transfer learning system for a microseismic signal recognition model, comprising:

[0037] An initialization module, used to respectively initialize the feature extractor in the base model and the classifier connected to the feature extractor;

[0038] A configuration module, configured to configure a first discriminator and a second discriminator at the lowest layers of the feature extractor and the classifier, respectively, to obtain a two-level adversarial domain adaptive model;

[0039] A loss function determination module, used to input source domain data and target domain data into the two-stage adversarial domain adaptive model for training to obtain multiple loss functions;

[0040] A migration module is used to back-propagate each of the loss functions along the two-level adversarial domain adaptive model to update the network parameters of the two-level adversarial domain adaptive model; the updated two-level adversarial domain adaptive model is the migrated recognition model.

[0041] Preferably, the migration module further includes:

[0042] An extraction module, used for intermittently extracting unlabeled data from an unlabeled engineering dataset;

[0043] A judgment module, used to judge whether the uncertainty score of the extracted unlabeled data is greater than a preset threshold, and if so, the unlabeled data is labeled by an expert to obtain labeled data;

[0044] The adding module is used to add the labeled data to the labeled data set; the labeled data set is used to train the recognition model.

[0045] Preferably, the initialization module specifically includes:

[0046] A first copy unit, used to replace the parameters of the first convolutional layer of the feature extractor with the parameters of the base model;

[0047] A freezing unit, used to freeze the parameters of the first convolutional layer so that the parameters of the first convolutional layer do not participate in training;

[0048] A second copy unit, used to replace the parameters of the second residual block of the feature extractor with the parameters of the base model; the parameters of the second residual block are in an activated state and participate in the training;

[0049] An initialization unit is used to initialize all other convolutional layers of the feature extractor except the first convolutional layer and the second residual block by using a He Uniform method.

[0050] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0051] The present invention provides a transfer learning method and system for a microseismic signal recognition model, wherein the method includes: respectively initializing a feature extractor in a base model and a classifier connected to the feature extractor; respectively configuring a first discriminator and a second discriminator at the bottom layers of the feature extractor and the classifier to obtain a two-level adversarial domain adaptive model; respectively inputting source domain data and target domain data into the two-level adversarial domain adaptive model for training to obtain multiple loss functions; back-propagating each of the loss functions along the two-level adversarial domain adaptive model to update the network parameters of the two-level adversarial domain adaptive model; the updated two-level adversarial domain adaptive model is the recognition model after migration. The present invention separately adapts the feature extractor composed of convolutional layers and the classifier composed of fully connected layers, thereby being able to efficiently achieve domain migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 A flow chart of the migration method in the embodiment provided by the present invention;

[0054] Figure 2 A schematic diagram of a base model in an embodiment provided by the present invention;

[0055] Figure 3 A schematic diagram of the complete structure of TADA in the embodiment provided by the present invention;

[0056] Figure 4 A structural diagram of a discriminator in an embodiment of the present invention;

[0057] Figure 5 A conceptual diagram of active learning in an embodiment of the present invention;

[0058] Figure 6 The EADA structure diagram in the embodiment provided by the present invention;

[0059] Figure 7 This is a CADA structure diagram in an embodiment provided by the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The purpose of the present invention is to provide a transfer learning method and system for a microseismic signal recognition model, which can efficiently realize domain migration.

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] Figure 1 A flow chart of the migration method in the embodiment provided by the present invention, such as Figure 1 As shown, a transfer learning method for a microseismic signal recognition model in this embodiment includes:

[0064] Step 100: Initializing the feature extractor in the base model and the classifier connected to the feature extractor respectively;

[0065] Step 200: configuring a first discriminator and a second discriminator at the bottom layers of the feature extractor and the classifier respectively to obtain a two-level adversarial domain adaptive model;

[0066] Step 300: inputting the source domain data and the target domain data into the two-stage adversarial domain adaptive model for training respectively to obtain multiple loss functions;

[0067] Step 400: back-propagate each of the loss functions along the two-stage adversarial domain adaptive model to update the network parameters of the two-stage adversarial domain adaptive model; the updated two-stage adversarial domain adaptive model is the migrated recognition model.

[0068] Figure 2 A schematic diagram of a base model in an embodiment of the present invention, such as Figure 2 As shown, the base model consists of an input layer, a feature extractor, a classifier, and an output layer connected in sequence. The model is used to complete the task of microseismic signal recognition in a specific project. It is worth mentioning that the transfer learning framework involved in the present invention is independent of the specific base model selected, as long as the base model is based on a convolutional neural network for microseismic signal recognition.

[0069] Preferably, the base model is constructed based on a convolutional neural network, and the base model is used to identify microseismic signals.

[0070] Optionally, the basic residual block consists of two convolutional layers, and the bottleneck residual block consists of three convolutional layers. In this embodiment, a special initialization strategy is mainly used for the first two layers, namely the first convolutional layer and the first basic residual block, and the remaining layers are uniformly initialized regardless of how many layers there are.

[0071] Specifically, this base model, like other convolution-based models, consists of two parts. The first part is a feature extractor mainly composed of convolutional layers, which is used to extract high-order features from the original input data. The second part is a simple single-layer fully connected layer, which is used to map the extracted high-order features into the classification space to complete the waveform classification.

[0072] Preferably, the initializing the feature extractor in the base model and the classifier connected to the feature extractor respectively comprises:

[0073] Replacing the parameters of the first convolutional layer of the feature extractor with the parameters of the base model;

[0074] Freezing parameters of the first convolutional layer so that the parameters of the first convolutional layer do not participate in training;

[0075] Replacing the parameters of the second residual block of the feature extractor with the parameters of the base model; the parameters of the second residual block are in an activated state and participate in training;

[0076] The He Uniform method is used to initialize all other convolutional layers of the feature extractor except the first convolutional layer and the second residual block.

[0077] Preferably, the initializing the feature extractor in the base model and the classifier connected to the feature extractor respectively comprises:

[0078] Replacing the parameters of the fully connected layer of the classifier with the parameters of the fully connected layer of the base model;

[0079] An adaptive layer is connected to the output end of the fully connected layer of the classifier; the adaptive layer is a fully connected layer;

[0080] The adaptive layer is initialized using the He normal method.

[0081] Specifically, the present invention proposes an unsupervised transfer learning strategy, named as the two-level adversarial domain adaptation (TADA) framework, that is, the two-level adversarial domain adaptation model in this embodiment, such as Figure 3 As shown, Figure 3In the example, we can directly think of it as a set of networks, because the parameters of each layer are exactly the same, that is, this network receives and processes the data of the source domain and the target domain respectively. We first process a batch of source domain data, then process a batch of target domain data, save the results of these two processes, and then generate losses, and back-propagate to update the network parameters.

[0082] In this embodiment, the first operation step of the migration framework is initialization. The base model is divided into two parts, the first part is the feature extractor, and the second part is the classifier. First, each layer must be initialized differently. The complete migration learning framework is as follows Figure 1 In general, the final migration framework is obtained by copying some parameters of the base model and making some modifications to the base model. The migration framework receives samples from the source domain and the target domain respectively during training, and all layers that process the source domain and the target domain share parameters. This is because the signal data is not very different and the number of shared features is large. Even if the two signals are completely different, there will be many shared features for discrimination.

[0083] In the specific application process, the feature extractor is initialized first: the first convolutional layer directly copies the base model parameters and freezes the parameters so that they do not participate in training. The second residual block also directly copies the base model parameters, but does not freeze the parameters so that they participate in training. All other convolutional layers of the feature extractor are initialized with He Uniform because we hope to make the training in the source domain less smooth through random initialization, thereby preventing the optimal solution in the source domain from falling into.

[0084] Secondly, for the classifier, in addition to copying the fully connected layer parameters, a new fully connected layer needs to be added after this layer as an adaptive layer and initialized using the He normal method. This layer is to strengthen the domain alignment in the classifier.

[0085] Optionally, one of the biggest features of the present invention is that the feature extractor composed of the convolutional layer and the classifier composed of the fully connected layer are separately adapted. Figure 3 As can be seen in the figure, there is a discriminator (discriminator 1) behind the feature extractor and another discriminator (discriminator 2) behind the classifier. These two discriminators are responsible for providing gradients to the feature extractor and classifier respectively. In order to prevent the new network from falling into the local optimal solution of the source domain data, not every layer can be copied from the original model. Therefore, for the classifier, only the first fully connected layer is copied from the original model. Then a new fully connected layer called the adaptation layer is added after this layer. This layer is to enhance the nonlinear ability of the classifier, so that it can better project the source domain data into the public space. Since this layer is a newly added layer, it is natural to use the common He Normal for initialization.

[0086] Preferably, the first discriminator and the second discriminator are respectively configured at the bottom layers of the feature extractor and the classifier to obtain a two-level adversarial domain adaptive model, including:

[0087] Connecting the first discriminator to the output of the feature extractor;

[0088] The second discriminator is connected to the output end of the adaptive layer of the classifier; the two-stage adversarial domain adaptive model includes the feature extractor, the classifier, the first discriminator and the second discriminator.

[0089] Furthermore, the second operation step of the migration framework in this embodiment is to configure the discriminator. After initialization, a neural network needs to be configured after the feature extractor and after the adaptive layer in the classifier as a discriminator to guide the training of the backbone neural network. The structures of the two discriminators are as follows: Figure 4 As shown in . Both discriminators are small neural networks with two hidden layers. Except for the input layer, the dimensions of the other layers are the same. The purpose of the discriminator is to distinguish whether the output data comes from the source domain or the target domain, while the purpose of the backbone neural network is to confuse the discriminator so that it cannot distinguish which domain the data comes from. In this way, the backbone network is encouraged to extract domain-invariant features for the two domains. Among them, the loss function of the discriminator is a common binary classification loss function. Specifically, the binary classification loss function is The loss function used by the backbone network is Among them, E is the expectation, D is the discriminator, and G is the feature generator or classifier. The specific one can be confirmed by looking at the installation position of the discriminator. s is the source domain data, X T The target domain data.

[0090] like Figure 3 As shown in the figure, in addition to back-propagating a gradient at discriminators 1 and 2 respectively, a gradient is also back-propagated after the final source domain classification layer. This gradient is the classification result of training on the source domain, using the ordinary cross entropy loss function. Therefore, the training of the backbone network is driven by three loss functions at the same time, namely, those provided by discriminator 1, discriminator 2, and classification training on the source domain. The optimization process is β and γ are regularization coefficients (hyperparameters) used to scale the three loss functions. advG1 is the discriminator behind the feature extractor (i.e. Figure 2 The loss provided by the discriminator 1) in the formula is as follows: advG2 The loss provided by discriminator 2 is also in the form of formula 2. clfIt is a common binary classification loss. Since the source domain data is labeled, when the network processes the source domain data, it will also perform signal classification training on the source domain data.

[0091] Furthermore, in this embodiment, experiments have shown that better effects can be obtained when β and γ are 1.8 and 1.6 respectively.

[0092] Preferably, the source domain data and the target domain data are respectively input into the two-stage adversarial domain adaptive model for training to obtain multiple loss functions, including:

[0093] When the training data is the source domain data, obtaining first output data of the feature extractor in the two-stage adversarial domain adaptive model, second output data of the classifier output, and a cross entropy loss;

[0094] When the training data is the target domain data, obtaining third output data of the feature extractor in the two-stage adversarial domain adaptive model and fourth output data of the classifier output;

[0095] Inputting the first output data and the third output data into the first discriminator to generate a first adversarial loss;

[0096] The second output data and the fourth output data are input into the second discriminator to generate a second adversarial loss; the loss function is any one of a cross entropy loss, a first adversarial loss or a second adversarial loss.

[0097] Preferably, after back-propagating the two-stage adversarial domain adaptive model according to the first adversarial loss, the second adversarial loss and the cross entropy loss and updating the network parameters of the two-stage adversarial domain adaptive model, the method further includes:

[0098] Intermittently extract unlabeled data from unlabeled engineering datasets;

[0099] Determine whether the uncertainty score of the extracted unlabeled data is greater than a preset threshold, and if so, label the unlabeled data by an expert to obtain labeled data;

[0100] The labeled data is added to a labeled data set; the labeled data set is used to train the recognition model.

[0101] Optionally, the above-mentioned unsupervised domain adaptation ensures that the cross-project deployment of the model can be completed without the need for any labeled data on the target project. After being deployed to a new project, as the project progresses, it is urgently necessary to further fine-tune the model by intermittently obtaining a small amount of labeled data, thereby further improving the performance of the model. Therefore, the present invention integrates and implements an active learning mechanism in the framework, which is used to further improve the model after transfer learning with the smallest possible sample annotation cost.

[0102] Specifically, active learning is a machine learning concept in which the learning algorithm can interactively select the data that really needs to be labeled with human engineers. The motivation for active learning is that we have a large amount of unlabeled data. Consider the problem of training a model to distinguish microseismic events. There are many signals monitored and stored in the data pool every day, but training a good model may not require labeling all signals. Some signals may provide more information than other signals. For example, Figure 5 As shown in Figure 1, the algorithm repeatedly selects the most informative examples based on some value metric and sends those unlabeled examples to the on-call operator, who returns the true labels of those queried examples to the algorithm. Clearly, the metric function determines which data points are the most informative.

[0103] In this task, there are only two candidate categories (microseismic signal or noise signal), so the metric function is defined as 1 minus the difference in the predicted probabilities of the two categories. In actual implementation, we set a threshold in advance. Once the uncertainty score is greater than the given threshold, the corresponding example will be annotated by the expert and added to the training set for the next round of learning. In this study, the threshold is set to 0.75. After selecting the sample, the model is trained according to x * = argmax x (1-(P θ (y 1 |x)-P θ (y 2 |x))) for training. Among them, argmax is the operation of taking the maximum point, and P is the predicted probability of the neural network output.

[0104] Furthermore, by iteratively increasing the size of the labeled training set, we can achieve performance similar to that of fully supervised training, while greatly reducing the labeling and time costs. By combining this approach with the two-level unsupervised domain adaptation method proposed in this invention, the model can be deployed quickly at the beginning of construction and fine-tuned during the construction process without building any large-scale datasets.

[0105] During the application process, this embodiment can efficiently realize domain migration and greatly save the annotation cost required to train a microseismic signal recognition model. First, the base model disclosed in this embodiment was trained on an engineering training set called HW and achieved an accuracy of 97%, but the trained model could not achieve good results when directly used in other engineering data sets, as shown in Table 1, which shows the performance of the initial model on different data sets. Among them, SJK, JP-2, and JP-1 are all data sets of different projects. Since the similarity between these new projects and the source project data is different, the model trained on the source data set will have different performances in the new projects. When the similarity is high, the model is easy to achieve better results, such as JP-1; when the similarity between the two projects is low, the performance of the model will be very poor (such as SJK).

[0106] Table 1

[0107]

[0108] All experimental accuracies are given in Table 2, which shows the performance comparison of different transfer methods. These methods have the same training configuration, such as learning rate, optimizer, batch size, and discriminator structure. It is worth noting that our goal is not to maximize absolute performance, but to study the transfer strategy in a standard environment, so there is no very fine tuning of hyperparameters. The learning rate is set to 1e-5, and the Adam optimization algorithm is used.

[0109] Table 2

[0110]

[0111] Except for the last three rows in Table 2, each item is a commonly used transfer learning method in computer vision. From this experiment, we can see that they cannot achieve significant results in image processing when used for signal recognition problems. In addition, EADA and CADA are two variants of TADA, such as Figure 6 and Figure 7 As shown, they correspond to the case where only one discriminator is used. These two cases are not as complicated as TADA, but are simpler to train. As can be seen from Table 2, EADA is quite effective for all tasks, and it always achieves the second highest accuracy. It provides basically the same results as TADA on JP-1 and JP-2. CADA only provides domain alignment loss after the classifier. Obviously, CADA performs worse than EADA, which shows that the adaptation of the classifier cannot propagate enough gradients to update the randomly initialized extractor layer. Therefore, domain alignment of the feature extractor should be prioritized, that is, EADA should be performed first.

[0112] Based on the above analysis, this embodiment obtains an unsupervised transfer learning strategy for performing tasks with different difficulties. That is, TADA can be performed in parts and situations. If the two projects are similar in geological conditions and monitoring configurations, the domain difference will be relatively small. In this case, it is only necessary to properly initialize the extractor and retrain the source classifier in the new space, that is, to perform EADA. When the two projects are not very similar, the entire TADA should be performed, and the extractor and classifier should be domain aligned to obtain a suboptimal decision boundary.

[0113] After unsupervised adaptation, the accuracy of the model can be further improved by continuously acquiring data for training as the project progresses. As more and more data is acquired, the microseismic experts can select a small portion of the recorded data for annotation through active learning strategies to recalibrate the migrated model. We simulate a common application scenario in the early stages of construction and evaluate how much sample annotation costs can be reduced using active learning techniques. We randomly selected 500–1000 instances from the monitoring data of the first two months of each project database for SJK and JP-2 respectively (for JP-1, the effect achieved by unsupervised domain adaptation alone is sufficient, and fine-tuning can be performed with some arbitrarily labeled instances). The experimental results are shown in Table 3, which shows the results of further training using active learning. Compared with random sampling, the active learning algorithm selects those instances with the “most information” and then asks human experts what their true labels are. From the results, we can see that for both projects, the active learning strategy significantly reduces the annotation cost to achieve the same accuracy. One difference is that the higher the initial accuracy, the more significant the reduction in annotation cost. This shows that it is very necessary to first perform unsupervised training to improve the basic accuracy. The reason for this phenomenon may be that for more difficult tasks, the model needs to learn some more general basic representations, thus requiring more samples. In summary, from the two proposed algorithms and techniques, a generalized microseismic event recognition model only needs dozens or hundreds of labeled examples to effectively complete the migration. The proposed method successfully solves the problem of model portability and reusability in microseismic monitoring.

[0114] Table 3

[0115]

[0116] This embodiment further provides a transfer learning system for a microseismic signal recognition model, which is applied to the above transfer learning method. The system includes:

[0117] An initialization module, used to respectively initialize the feature extractor in the base model and the classifier connected to the feature extractor;

[0118] A configuration module, configured to configure a first discriminator and a second discriminator at the lowest layers of the feature extractor and the classifier, respectively, to obtain a two-level adversarial domain adaptive model;

[0119] A loss function determination module, used to input source domain data and target domain data into the two-stage adversarial domain adaptive model for training to obtain multiple loss functions;

[0120] A migration module is used to back-propagate each of the loss functions along the two-level adversarial domain adaptive model to update the network parameters of the two-level adversarial domain adaptive model; the updated two-level adversarial domain adaptive model is the migrated recognition model.

[0121] Preferably, the migration module further includes:

[0122] An extraction module, used for intermittently extracting unlabeled data from an unlabeled engineering dataset;

[0123] A judgment module, used to judge whether the uncertainty score of the extracted unlabeled data is greater than a preset threshold, and if so, the unlabeled data is labeled by an expert to obtain labeled data;

[0124] The adding module is used to add the labeled data to the labeled data set; the labeled data set is used to train the recognition model.

[0125] Preferably, the initialization module specifically includes:

[0126] A first copy unit, used to replace the parameters of the first convolutional layer of the feature extractor with the parameters of the base model;

[0127] A freezing unit, used to freeze the parameters of the first convolutional layer so that the parameters of the first convolutional layer do not participate in training;

[0128] A second copy unit, used to replace the parameters of the second residual block of the feature extractor with the parameters of the base model; the parameters of the second residual block are in an activated state and participate in the training;

[0129] An initialization unit is used to initialize all other convolutional layers of the feature extractor except the first convolutional layer and the second residual block by using a He Uniform method.

[0130] The beneficial effects of the present invention are as follows:

[0131] (1) By iteratively increasing the size of the labeled training set, the present invention can achieve performance similar to that of fully supervised training, while greatly reducing the labeling cost and time cost. By combining this approach with the two-level unsupervised domain adaptation method proposed in the present invention, the model can be quickly deployed at the beginning of construction and fine-tuned during the construction process without building any large-scale datasets.

[0132] (2) The present invention reduces the difficulty of model migration by separately adapting the feature extractor composed of convolutional layers and the classifier composed of fully connected layers, thereby enabling efficient domain migration.

[0133] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0134] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A transfer learning method for microseismic signal recognition model, characterized in that: include: Initializing a feature extractor in a base model and a classifier connected to the feature extractor respectively; wherein the base model is constructed based on a convolutional neural network, and the base model is used to identify microseismic signals; the initializing a feature extractor in the base model and a classifier connected to the feature extractor respectively specifically includes: Replacing the parameters of the first convolutional layer of the feature extractor with the parameters of the base model; Freezing parameters of the first convolutional layer so that the parameters of the first convolutional layer do not participate in training; Replacing the parameters of the second residual block of the feature extractor with the parameters of the base model; the parameters of the second residual block are in an activated state and participate in training; Initializing all other convolutional layers of the feature extractor except the first convolutional layer and the second residual block using the He Uniform method; Replacing the parameters of the fully connected layer of the classifier with the parameters of the fully connected layer of the base model; An adaptive layer is connected to the output end of the fully connected layer of the classifier; the adaptive layer is a fully connected layer; Initializing the adaptive layer using the He normal method; A first discriminator and a second discriminator are respectively configured at the bottom layers of the feature extractor and the classifier to obtain a two-level adversarial domain adaptive model, which specifically includes: Connecting the first discriminator to the output of the feature extractor; The second discriminator is connected to the output end of the adaptive layer of the classifier; the two-stage adversarial domain adaptive model includes the feature extractor, the classifier, the first discriminator and the second discriminator; Inputting the source domain data and the target domain data into the two-stage adversarial domain adaptive model for training respectively to obtain multiple loss functions; Each of the loss functions is back-propagated along the two-stage adversarial domain adaptive model to update the network parameters of the two-stage adversarial domain adaptive model; the updated two-stage adversarial domain adaptive model is the migrated recognition model.

2. The transfer learning method according to claim 1, characterized in that: The method further comprises: Intermittently extract unlabeled data from unlabeled engineering datasets; Determine whether the uncertainty score of the extracted unlabeled data is greater than a preset threshold, and if so, label the unlabeled data by an expert to obtain labeled data; The labeled data is added to a labeled data set; the labeled data set is used to train the recognition model.

3. The transfer learning method according to claim 1, characterized in that: The source domain data and the target domain data are respectively input into the two-stage adversarial domain adaptive model for training to obtain multiple loss functions, including: When the training data is the source domain data, obtaining first output data of the feature extractor in the two-stage adversarial domain adaptive model, second output data of the classifier output, and a cross entropy loss; When the training data is the target domain data, obtaining third output data of the feature extractor in the two-stage adversarial domain adaptive model and fourth output data of the classifier output; Inputting the first output data and the third output data into the first discriminator to generate a first adversarial loss; The second output data and the fourth output data are input into the second discriminator to generate a second adversarial loss; the loss function is any one of a cross entropy loss, a first adversarial loss or a second adversarial loss.

4. A transfer learning system for microseismic signal recognition model, characterized in that: include: An initialization module is used to respectively initialize a feature extractor in a base model and a classifier connected to the feature extractor; wherein the base model is constructed based on a convolutional neural network, and the base model is used to identify microseismic signals; the feature extractor in the base model and the classifier connected to the feature extractor are respectively initialized, specifically including: A first copy unit, used to replace the parameters of the first convolutional layer of the feature extractor with the parameters of the base model; A freezing unit, used to freeze the parameters of the first convolutional layer so that the parameters of the first convolutional layer do not participate in training; A second copy unit, used to replace the parameters of the second residual block of the feature extractor with the parameters of the base model; the parameters of the second residual block are in an activated state and participate in the training; An initialization unit, used for initializing all other convolutional layers of the feature extractor except the first convolutional layer and the second residual block by using a He Uniform method; A configuration module is used to configure a first discriminator and a second discriminator at the bottom layers of the feature extractor and the classifier, respectively, to obtain a two-level adversarial domain adaptive model, specifically comprising: Connecting the first discriminator to the output of the feature extractor; The second discriminator is connected to the output end of the adaptive layer of the classifier; the two-stage adversarial domain adaptive model includes the feature extractor, the classifier, the first discriminator and the second discriminator; A loss function determination module, used to input source domain data and target domain data into the two-stage adversarial domain adaptive model for training to obtain multiple loss functions; A migration module is used to back-propagate each of the loss functions along the two-level adversarial domain adaptive model to update the network parameters of the two-level adversarial domain adaptive model; the updated two-level adversarial domain adaptive model is the migrated recognition model.

5. The transfer learning system according to claim 4, characterized in that: The migration module then further includes: An extraction module, used for intermittently extracting unlabeled data from an unlabeled engineering dataset; A judgment module, used to judge whether the uncertainty score of the extracted unlabeled data is greater than a preset threshold, and if so, the unlabeled data is labeled by an expert to obtain labeled data; The adding module is used to add the labeled data to the labeled data set; the labeled data set is used to train the recognition model.

Images