High-speed rail intelligent operation-oriented cycle domain adaptive knowledge acquisition method

Abstract

The application discloses a cyclic field self-adaptive knowledge acquisition method for high-speed rail intelligent operation and maintenance, and is used for field knowledge self-adaptive acquisition, comprising the following steps: constructing a field self-adaptive learning framework; converting entity recognition and relationship extraction processes in the field self-adaptive learning framework into classification processes; inputting training text into the field self-adaptive learning framework for training to obtain an optimal field self-adaptive learning framework; wherein, in order to complete the training process of the optimal field self-adaptive learning framework, a loss function of components in the field self-adaptive learning framework is defined; inputting actual data text into the optimal field self-adaptive learning framework to obtain target knowledge; wherein, the field self-adaptive learning framework process is: encoding of input text, shared decoding, first classification and second classification. In the case that the number of labeled data is insufficient, massive high-speed train fault text data can be automatically subjected to knowledge extraction, and three-tuples with knowledge, such as fault sources, fault categories and fault features, are constructed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge acquisition, in particular to a cyclic domain adaptive knowledge acquisition method for high-speed rail intelligent operation and maintenance. BACKGROUND

[0002] With the increase of high-speed rail mileage, the service life of high-speed trains is also gradually increasing. The technical support and operation and maintenance of high-speed trains become more and more important. As one of the effective solutions to improve the reliability and safety of system operation, domestic and foreign researchers have carried out a large number of researches on real-time fault diagnosis and detection of high-speed train information control system and its subsystems. However, most of the researches focus on real-time fault classification problems, and the relevant knowledge and rules contained in these data have not been fully utilized. In the actual system operation process, a large amount of fault data and fault description text are accumulated, which makes it necessary to extract operation and maintenance knowledge from related text, and also makes it possible to further manage and use such knowledge.

[0003] With the development of deep learning technology, most machine learning algorithms are often criticized for being difficult to apply to new tasks or data sets without enough labeled information, which also brings the rise of small sample or zero sample learning methods. The domain adaptation (DA) method solves this problem by establishing knowledge transfer from the source domain with labels to the target domain without labels or with sparse labels. The mainstream DA methods are generally divided into three types: supervised domain adaptation (SDA), unsupervised domain adaptation (UDA) and semi-supervised domain adaptation (SSDA). Supervised domain adaptation generally assumes that there are some labeled data in the target domain, and only uses labeled data to perform transfer learning when executing the DA algorithm; in contrast, unsupervised domain adaptation does not require any target data labels, but it needs a large amount of unlabeled target data to align the distribution between different domains; semi-supervised domain adaptation refers to the use of less labeled target data, and the use of unlabeled and labeled target domain data to complete the knowledge transfer from the source domain to the target domain.

[0004] A typical DA method is to embed the data of the source domain and the target domain into a common embedding representation space and align the marginal probability distribution between the two domains. There are generally two strategies to achieve this alignment: one is to use adversarial training; the other is to directly minimize the distance between the two distributions. Both methods can generate feature representations with domain invariance for input data, and use the feature representations of the source domain to train a classifier, and then extend the classifier to the target domain.

[0005] The former is based on the idea and theory of adversarial learning, mainly training a domain classifier to distinguish whether the input features are from the source domain or the target domain, while also training a feature encoder to generate feature representations that cannot be distinguished to deceive the domain classifier. Such methods generally use gradient reversal to maximize the loss of domain discrimination; some methods enforce these adversarial losses in a shared feature space while learning a private feature space for each domain to avoid interference from the learned shared representation; further improvements are to align the marginal distribution and conditional distribution between the source domain and the target domain at the same time.

[0006] The latter mainly measures the distance between the source distribution and the target distribution according to the distance metric. A commonly used distance metric is the Maximum Mean Discrepancy (MMD), which can calculate the norm of the mean difference of two domains in the Reproducing Kernel Hilbert Space (RKHS). There are also deep adaptive networks (DAN) or variants based on them that apply MMD to the last fully connected layer to match the high-order statistics of the two distributions. However, since the conditional probability distributions between the source domain and the target domain may be different, adjusting the marginal distribution alone is not enough to ensure the success of DA. In addition, this strategy only considers the differences between different domains, but does not consider the class differences between different domains.

[0007] At the same time, most domain adaptation algorithms are designed based on the transfer from one source domain to another target domain, while in reality there are often multiple source domains, but there is a target domain with significant differences. For example, in sentiment analysis tasks, we have evaluation sentiment category labeled data of books, video tapes, DVDs, etc. as source domains, and target domains are fresh food or baby products, etc. These items have significant differences in attributes and categories. We need to transfer classification knowledge obtained from different domains to the target domain, so we need to learn the feature representation of different source domains while transferring these features to the target domain.

[0008] Therefore, those skilled in the art urgently need to design a new cycle domain adaptation knowledge acquisition method for high-speed rail intelligent operation and maintenance to improve the existing adaptive method. SUMMARY

[0009] The purpose of the present application is to provide a cycle domain adaptation knowledge acquisition method for high-speed rail intelligent operation and maintenance, which solves the problems existing in the prior art.

[0010] In order to achieve the above purpose, the present application provides the following technical scheme:

[0011] The method comprises the following steps:

[0012] Constructing a domain adaptive learning framework;

[0013] Converting the entity recognition and relationship extraction process in the domain adaptive learning framework into a classification process;

[0014] Obtaining training text input into the domain adaptive learning framework for training to obtain an optimal domain adaptive learning framework; wherein, to complete the training process of the optimal domain adaptive learning framework, the loss function of the components in the domain adaptive learning framework is defined;

[0015] Inputting actual data text into the optimal domain adaptive learning framework to obtain target knowledge;

[0016] The domain adaptive learning framework process is: encoding input text-shared decoding-first classification-second classification.

[0017] Preferably, the input text is encoded by using a pre-constructed encoder;

[0018] The encoder comprises an encoder E c , an encoder E d .

[0019] The encoder E c and the encoder E d are constructed by using a multi-layer perception or a long short-term memory network;

[0020] The input text is encoded by using the encoder E c to obtain a general representation code Rc;

[0021] The input text is encoded by using the encoder E d to obtain a related representation code Rd.

[0022] Preferably, the first classification and the second classification are both classified by using a pre-constructed classifier;

[0023] The classifier comprises a classifier C d , a classifier C c .

[0024] The domain category of the input sample is classified by using the classifier C d ;

[0025] The category of the input sample data itself is judged by using the classifier C c .

[0026] Preferably, the shared decoding process employs a shared decoder D cd to perform decoding processing;

[0027] the shared decoder D cd is simultaneously constructed by a domain-independent representation z c and a domain-dependent representation z d to build a new joint data representation z c +z d , which is defined as follows:

[0028]

[0029] where θ cd is the relevant parameter of the decoder D cd .

[0030] where the loss function of the shared encoder is defined by the mean square error, which is expressed as follows:

[0031]

[0032] where C is the dimension of the input X, N is the total number of input samples of the source domain and the target domain, x i represents the i-th sample, λ cd is the hyperparameter of the loss function, and ‖‖ is the square of the two-norm.

[0033] Preferably, it further comprises: in order to control the redundant representation between the domain-independent feature representation and the domain-dependent feature representation, an orthogonal constraint mechanism is introduced to encourage the two encoders to encode the same input in different directions, and the loss function is defined as follows:

[0034]

[0035] where H c represents an independent matrix, each row of which is the domain-independent feature representation of a sample; H d represents a dependent matrix, each row of which is the domain-dependent feature representation of a sample; and λd represents a proportionality factor.

[0036] Preferably, the loss function of the classifier C d is defined as follows:

[0037] L domain = L d + L c

[0038]

[0039] where d iis the one-hot encoding of the domain category label of the i-th instance, and class represents a scaling factor, class is the relevant parameter of the classifier C d .

[0040] Preferably, the loss function of the classifier Cc is defined as:

[0041]

[0042]

[0043] L class = -λ label logp(y=k|x);

[0044] where Rc is the domain-independent feature representation, n k is the number of instances belonging to class k, and d() represents the distance metric function, and λ label represents a scaling factor.

[0045] Preferably, the definition expression of the domain-independent representation zc and the domain-dependent representation z d is:

[0046]

[0047] where θ c represents the relevant parameter in the encoder E c , and θ d represents the relevant parameter in the encoder E d .

[0048] Preferably, the conversion of the entity recognition and relation extraction process in the domain adaptation learning framework into a classification process is that the classification algorithm of the entity recognition and relation extraction is replaced by the classifier C c .

[0049] From the above content, it can be known that, compared with the prior art, the application has the beneficial effects in that:

[0050] (1) By using the related technologies of entity recognition and relation extraction, a large amount of high-speed train operation and maintenance knowledge can be automatically obtained from massive text data, which provides a technical platform for the construction of the domain knowledge system, and further can continuously accumulate the related technical experience in the domain;

[0051] (2) Only a small amount of professional annotation data and a related technical term dictionary provided by a professional can complete the training and learning process of the entity recognition and relation extraction algorithm, which saves a large amount of labor cost;

[0052] (3) A domain adaptation learning framework is implemented to migrate from multiple domains to a special domain, and the learned knowledge representation from other professional domains helps the knowledge representation learning in the high-speed train operation and maintenance domain by using a cycle learning process and an adversarial learning principle, so as to accelerate the domain knowledge acquisition process. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0054] Figure 1 A schematic diagram of the domain adaptation framework of the present application is shown in Figure 1.

[0055] Figure 2 A pseudo-code algorithm implementation process diagram of the present application is shown in Figure 2. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0057] Embodiment 1

[0058] The present embodiment discloses a knowledge acquisition method based on domain adaptation technology applied in a low-resource environment: a cycle domain adaptation knowledge acquisition method for high-speed rail intelligent operation and maintenance, which enables the learning process between different domains to be mutually referenced and migrated, so that the knowledge acquisition algorithm learned from the domain in a high-resource environment can also run in other domains in a low-resource environment. Since the high-speed train operation and maintenance domain exactly meets the characteristics of the low-resource environment, and the professional knowledge in this domain is required to be high, and the labor cost of professional personnel is also high, the high-speed train operation and maintenance domain is selected in the present embodiment.

[0059] The steps include:

[0060] Constructing a domain adaptation learning framework;

[0061] Converting the entity recognition and relationship extraction process in the domain adaptation learning framework into a classification process;

[0062] The training text is input into the field adaptive learning framework to obtain an optimal field adaptive learning framework; wherein, to complete the training process of the optimal field adaptive learning framework, a loss function of a component in the field adaptive learning framework is defined;

[0063] The actual data text is input into the optimal field adaptive learning framework to obtain target knowledge;

[0064] The field adaptive learning framework process is: encoding of the input text, shared decoding, first classification, and second classification.

[0065] Specifically,

[0066] In this embodiment, the data set of book, DVD, electronic product, and daily commodity description and evaluation from the e-commerce Amazon is taken as the training text. In the first stage of the training, the e-commerce data set is divided into source domain and target domain according to different categories to perform preliminary training, so that the representation of these products is constantly fused in the training process and achieves good entity recognition and relation extraction effect. This process can pre-train the corresponding encoder, decoder, and classifier. Then, the e-commerce data is regarded as a source domain data set with a large amount of annotation, and the high-speed rail vehicle fault description data set containing a small amount of annotation information is regarded as a low-resource target domain data set. The training is started according to the foregoing algorithm: the labeled data is input into the encoder as shown in Figure 1 , and the second training of the encoder, decoder, and classifier is completed according to the target training process. This is actually to adjust the parameters in the model again to realize the recognition of entities and relations in the high-speed rail vehicle fault text. The test process is to recognize the entities and relations in the unlabeled high-speed rail vehicle fault description, and then the recognition effect is determined by manual operation.

[0067] As shown in Table 1, the text annotation format of the training set and the test set in this embodiment is shown in Table 1. The text annotation format of the training set is performed according to the standard mode of BIO, and the test set also obtains a classification label sequence according to this.

[0068] Word Number Word Entity Tag Relation Tag Relation Position 1 Mar B-per N 1 2 Smith I-per Live-in, works-for 6,12 3 lives O N 3 4 in O N 4 5 New B-loc N 5 6 Orleans I-loc N 6 7 and O N 7 8 is O N 8 9 hired O N 9 10 by O N 10 11 the O N 11 12 government B-org N 12 13 . O N 13

[0069] In this embodiment, the field adaptive learning framework is constructed based on the principle of adversarial learning; wherein, two different encoders E c and E d are designed in this embodiment; wherein, the encoder E c is used to learn the general representation coding R c from the input from different fields or the field-unrelated general representation coding, which does not distinguish the source field of the text and adopts a unified coding mode for all input texts; the encoder E d is used to encode the input text or to learn the field-related input representation coding Rd This encoding distinguishes the domain to which the text belongs and uses different encoding methods for input text from different domains.

[0070] Encoder E c and encoder E d It is constructed using multilayer perceptrons or long short-term memory networks;

[0071] For both encoders, the domain-independent representation z they learn c Domain-related representation z d The definition is as follows:

[0072] z c =E c (x,θ c )

[0073] z d =E d (x,θ d )

[0074] Where, θ c Encoder E c The relevant parameters in; θ d Encoder E d The relevant parameters in.

[0075] In addition, a shared decoder D is designed in this embodiment. cd Classifier C d With classifier C c ;

[0076] By training decoder D cd It can generate reconstructed representations of input samples; while classifier C d This is used to classify and identify the domain category of the input samples; classifier C c It is used to determine the category of the data itself, or to classify entities and relationships during the knowledge acquisition process. The so-called adversarial learning process is the decoder D. cd By fusing two representations, one domain-relevant and one domain-unrelevant, an adversarial learning process is generated, thereby achieving cross-domain knowledge transfer and enabling classifier C to... d It cannot distinguish the domain to which the sample belongs, and classifier C d If we need to train the classifier to identify the domain of the examples as accurately as possible, then the classifier C... c This requires identifying the category label of the sample, i.e., the category of the entity or relationship. This is done through the shared decoder D. cd The adversarial process between C and classifier Cd ultimately led to C c It can correctly classify the sample categories in both the source and target domains.

[0077] where the shared decoder D cd is simultaneously constructed with the domain-independent representation z c and the domain-dependent representation z d to build a new joint data representation z c +z d defined as follows:

[0078] where θ cd is the relevant parameter of the decoder D cd .

[0079] where the loss function of the shared encoder is defined by the mean square error, and its expression is as follows:

[0080] where C is the dimension of the input X, N is the total number of input samples of the source domain and the target domain, x i represents the i-th sample, λ cd is the hyperparameter of the loss function, and ‖‖ is the square of the two-norm.

[0081] where the loss function of the classifier C d is defined as:

[0082] L domain = L d + L c

[0083]

[0084] where d i is the one-hot encoding of the domain class label of the i-th sample, λ class represents a proportion factor, and θ class is the relevant parameter of the classifier C d .

[0085] where the loss function of the classifier Cc is defined as:

[0086]

[0087]

[0088] L class = -λ label log p(y=k|x);

[0089] where R c is the domain-independent feature representation, n k is the number of samples belonging to the class k, d() represents the distance metric function, and λ labelrepresents a scale factor.

[0090] In addition to the above, in order to control the redundant representation between the field-independent feature representation and the field-dependent feature representation, in the embodiment, an orthogonal constraint mechanism is introduced to encourage two encoders to encode the same input in different directions, and the loss function is defined as follows:

[0091]

[0092] wherein H c represents an independent matrix, each row is a field-independent feature representation of a sample; H d represents a dependent matrix, each row is a field-dependent feature representation of a sample; λ d represents a scale factor.

[0093] It should be noted that since knowledge acquisition is mainly the extraction of entities and relations, it is essentially a classification problem. Therefore, in the embodiment, the entity recognition and relation extraction process in the field adaptation learning framework is converted into a classification process; as shown in Figure 2 , the classification algorithm for entity recognition and relation extraction is replaced by a classifier C c .

[0094] wherein in the 12th row in Figure 2 , a cyclic comparison strategy between the source field and the target field is adopted when selecting samples, at this time, the correlation between entities and relations in different fields needs to be measured before deciding whether to select the corresponding sample. Generally speaking, a higher correlation will be a prerequisite for selection. This processing can enhance the migration efficiency of feature representation between different fields, and is one of the strategies adopted by many image-related field adaptation methods. The embodiment extends this technology to text-related entity and relation classification, so that the temperature attribute description class text will be trained against the temperature-related fault text in high-speed rail fault to complete the migration learning process.

[0095] It should be noted that the data described in Embodiment 1 is not limited in the present application; it can also be applied to the following data types, etc.

[0096] (1) The traction transformer oil flow detection auxiliary relay on the train No. *** to Guangzhou South detected a short circuit phenomenon when the oil flow was detected, and it is recommended to take *** measures.

[0097] Extracted entities: Guangzhou South, train No. ***, traction transformer oil flow detection auxiliary relay, oil flow, short circuit, *** measures

[0098] Relationship: cause-of, belong-to, result, ID-of-cause.

[0099] (2) In the temperature auxiliary relay detection of the traction transformer on the train No. *** to Zhengzhou, the temperature rises when short circuit is found, and it is suggested to take *** measures;

[0100] Extract entities: Zhengzhou, *** train, traction transformer temperature auxiliary relay, temperature, short circuit, *** measures.

[0101] The above description of disclosed embodiments allows those skilled in the art to make or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to these embodiments shown herein but is to accord with the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cyclic domain adaptive knowledge acquisition method for intelligent operation and maintenance of high-speed railways, characterized in that, Includes the following steps: Construct a domain-adaptive learning framework; The entity recognition and relation extraction process in the domain adaptive learning framework is transformed into a classification process. The domain adaptive learning framework is trained by inputting training text to obtain the optimal domain adaptive learning framework; wherein, in order to complete the training process of the optimal domain adaptive learning framework, the loss function of the components in the domain adaptive learning framework is defined. The target knowledge is obtained by inputting actual data text into the optimal domain adaptive learning framework. The domain adaptive learning framework process is as follows: input text encoding - shared decoding - first classification - second classification; Both the first and second classifications are performed using pre-built classifiers; The classifier includes: Classifier C d Classifier C c ; Using the classifier C d Classify the domain categories of the input samples; Using the classifier C c The category of the input sample data itself is determined; The shared decoding process employs a shared decoder D. cd Perform decoding processing; The shared decoder D cd It simultaneously utilizes the domain-independent representation z c Domain-related representation z d To jointly construct a new joint data representation z c +z d Its definition is as follows: ； Where, θ cd It is decoder D cd The relevant parameters; The loss function of the shared decoder is defined using mean squared error, and its expression is as follows: ； Where C is the dimension of the input X, N is the total number of input samples in the source and target domains, and x i λ represents the i-th sample. cd Let be the hyperparameter of the loss function, and ||| be the square of the L2 norm.

2. The cyclic domain adaptive knowledge acquisition method for intelligent operation and maintenance of high-speed railways according to claim 1, characterized in that, The input text is encoded using a pre-built encoder; The encoder includes: encoder E c Encoder E d ; The encoder E c and the encoder E d It is constructed using multilayer perceptrons or long short-term memory networks; Using the encoder E c The input text is encoded to obtain the general representation code Rc; Using the encoder E d The input text is encoded to obtain the relevant representation code Rd.

3. The cyclic domain adaptive knowledge acquisition method for intelligent operation and maintenance of high-speed railways according to claim 1, characterized in that, It also includes: In order to control the redundancy between domain-independent and domain-dependent feature representations, an orthogonal constraint mechanism is introduced to incentivize two encoders to encode the same input along different directions, and its loss function is defined as follows: ； Among them, H c H represents a domain-independent matrix, where each row represents the domain-independent feature representation of a sample; d λ represents a relevance matrix, where each row represents the domain-related features of a single example; d This represents a scaling factor.

4. The cyclic domain adaptive knowledge acquisition method for intelligent operation and maintenance of high-speed railways according to claim 1, characterized in that, The classifier C d The loss function is defined as follows: ； Where, d i λ is the one-hot encoding of the domain category label of the i-th sample. class θ represents a scaling factor. class For classifier C d The relevant parameters.

5. The cyclic domain adaptive knowledge acquisition method for intelligent operation and maintenance of high-speed railways according to claim 1, characterized in that, The classifier C c The loss function is defined as follows: ； ； ； Among them, R c For domain-independent feature representation, n k Let d() represent the number of samples belonging to class k, and let λ be the distance metric function. label This represents a scaling factor.

6. The cyclic domain adaptive knowledge acquisition method for intelligent operation and maintenance of high-speed railways according to claim 1, characterized in that, The domain-independent representation z c The domain-related representation z d The definition expression is: ； Where, θ c Indicates encoder E c The relevant parameters in; θ d Indicates encoder E d The relevant parameters in the file.

7. The cyclic domain adaptive knowledge acquisition method for intelligent operation and maintenance of high-speed railways according to claim 1, characterized in that, Transforming the entity recognition and relation extraction processes in the aforementioned domain adaptive learning framework into a classification process involves replacing the entity recognition and relation extraction classification algorithms with the classifier C. c accomplish.

Images