Self-adaptive knowledge acquisition model for train safety risk identification

Abstract

Train safety risk identification in a safety supervision scene is a core for constructing driving safety risk knowledge engineering and knowledge-driven railway intelligent safety risk management. How to automatically acquire train safety risk knowledge from massive unstructured data is a bottleneck of quick conversion from data to knowledge in a safety supervision scene. In order to solve the problem, firstly, a train safety risk knowledge ontology structure is formalized, and a complete knowledge system is constructed; the method comprises the following steps of: firstly, extracting a domain-adaptive knowledge extraction (DAKA), and then, proposing a domain-adaptive knowledge extraction (DAKA) method. According to the method, a double-flow self-attention mechanism is used, universal semantic embedding of safety supervision check text data is learned, then field features are continuously learned in a large-scale railway field text, and the characterization capacity of railway field semantics is improved; on the basis of understanding semantic features of domain data, a segmented convolutional neural network is used to deeply mine train safety risk features in a fine-grained manner, and finally, a train safety risk knowledge automatic acquisition task is realized.

Description

[0001] Technical space

[0002] The present invention relates to the fields of machine learning and natural language processing, and in particular to a method for an area adaptation knowledge acquisition model to identify train safety risks. Technical background

[0003] Railways are important national infrastructures and core means of transportation. Applying artificial intelligence to the railway informatization system can improve the reliability and safety of rail transit systems. The train operation safety monitoring subsystem is an important guarantee for the safe operation of trains. It continuously monitors the train safety status through intelligent sensors, video surveillance, the Internet of Things, etc., and identifies safety risks as early as possible to reduce the occurrence probability of risk events. Safety risks are gradually formed by factors such as people, machines, the environment, and management during the train operation process. Relying on traditional manual methods for identification cannot meet the growth scale of massive heterogeneous data. How to use a data-driven approach to mine train safety risks is the primary prerequisite for moving the train operation safety checkpoint forward and changing the ex-post management to a pre-prevention strategy. Therefore, data-driven intelligent safety risk identification is an urgent problem to be solved for the continuous and stable train operation safety.

[0004] The train safety supervision and inspection texts contain rich safety risk information, but directly mining information in a data-driven manner cannot effectively support the construction of the train safety risk intelligent prevention and control system. The combination of deep learning and knowledge graphs provides strong technical support for the transformation of information-knowledge-intelligence in intelligent prevention and control. The core of knowledge graph construction is to obtain knowledge with accurate semantics, mainly relying on the effective representation of text abstract semantic information. The current mainstream text representation method is to perform unsupervised training on a large-scale open corpus through a deep neural network, project the text semantic information into discrete distributed vectors. After obtaining the semantic embedding of the text data in the general language framework, use RNN and CNN to extract the structural semantic features of knowledge and obtain the triple knowledge in the knowledge graph. However, the adaptive and self-learning capabilities of neural networks are closely related to the training corpus, and the language representation capabilities obtained on different corpora are not universal. And the open corpus is mainly composed of Wikipedia, financial news, biomedical information, etc., and the text information related to the railway field is minimal, and it cannot effectively represent the abstract domain semantic information in railway text data. Summary of the invention

[0005] To address the above problems, the present invention discloses a domain - adaptive knowledge acquisition model for train safety risk identification, which includes four modules: a domain semantic feature module, an enhanced semantic feature module, a knowledge structure semantic feature module, and a safety risk relationship classification module. For the knowledge acquisition of train safety risks, first, after pre - processing the safety supervision and inspection text by word segmentation, the processed train safety risk knowledge ontology structure is formalized to construct a complete knowledge system, and then it is input into the DAKA neural network. In the DAKA neural network, the train safety risk knowledge system first undergoes training on open - source corpora through a dual - stream self - attention mechanism to obtain the semantic understanding ability of the general language framework, and then continues to learn domain features on large - scale railway domain corpora to improve the representation ability of railway domain semantics, enhance the understanding of professional knowledge, and thus effectively express railway domain semantic information. Based on understanding the semantic features of domain data, PCNN is used to deeply mine train safety risk features at a fine - grained level, and finally, the task of automatic acquisition of train safety risk knowledge is achieved. Compared with the prior art, the effective effects of the present invention are:

[0006] (1) For the first time in this paper, the train safety risks in the safety supervision scenario are formally defined to describe the risk semantic relationship between the supervision object and its unsafe state, which is the core of intelligent train safety risk prevention and control.

[0007] (2) A domain - adaptive knowledge acquisition method DAKA is proposed. This method gradually learns the abstract semantic features of text data from the levels of general semantics, professional semantics, and knowledge structure semantics to achieve the automatic acquisition of train safety risk knowledge.

[0008] (3) A large number of experiments on the real - world dataset VSST show that the DAKA method can effectively extract train safety risk knowledge, and the generalization of this method lays a foundation for constructing a railway safety risk knowledge project. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] To more clearly illustrate the technical solutions of the present invention, the drawings required for use in the description of the embodiments or the prior art will be briefly introduced below.

[0010] Figure 1 It is a schematic framework diagram for acquiring train safety risk knowledge implemented by the present invention;

[0011] Figure 2 It is a schematic diagram of the DAKA model of the domain - adaptive knowledge acquisition method implemented by the present invention;

[0012] Figure 3 It is a schematic diagram for comparing the performance of the DAKA model of the present invention and different classical models;

[0013] Figure 4It is a comparison chart of the knowledge acquisition precision-recall rate of the present invention and the comparison algorithm under different pre-training models;

[0014] Figure 5 This is a comparison chart of the experimental precision-recall rate of the present invention, DBERT, and DGPT under the same knowledge structure features;

[0015] Figure 6 It is a comparison chart of the precision-recall rate of the knowledge structure feature analysis of the DAKA model implemented by the present invention and other models under the same semantic information;

[0016] Figure 7 This is a semantic visualization comparison chart of train safety risks.

[0017] Specific implementation

[0018] Train safety risk identification is an important guarantee for the safe operation of trains. The key is how to adaptively acquire knowledge of train safety risk identification from multiple scenarios such as people, machines, environment, and management. Build a train safety risk knowledge map, monitor the closed-loop evolution of safety risks, or trace and analyze safety risk events, so as to achieve the goal of moving the safety risk checkpoint forward and changing post-event management to pre-event prevention.

[0019] Formalizing the definition of train safety risks in safety supervision scenarios and describing the risk semantic relationship between the monitored object and its unsafe state are the core of intelligent train safety risk prevention and control. Figure 2 The DAKA neural network in the paper simulates the human way of understanding the world, learns from open corpus and railway data sets in a progressive manner, automatically recognizes unstructured safety inspection text data, and represents the semantic information in the text data through discrete vectors. Figure 1 In the train safety monitoring scenario shown, the operator "Zhang XX" did not perform the "hand, mouth, and call" operations when crossing the track. Through the DAKA neural network, the factors that lead to personnel safety risks are learned from the context information of the text data, and the triple knowledge [Zhang XX, R1, no hand, mouth, and call] is obtained more accurately and effectively, and the representation ability of the triple knowledge [Zhang XX, R1, no hand, mouth, and call] is improved. Then, the train safety risk semantic network is constructed in the form of knowledge interconnection, that is, the train safety risk knowledge graph. Finally, PCNN is used to deeply mine the safety risk characteristics in the train risk knowledge graph from a fine-grained level, and finally the task of automatically acquiring train safety risk knowledge is realized.

[0020] 1. Definition of train safety risk knowledge

[0021] In the safety supervision scenario, train safety risks are the initial states of risks or faults in train operation, often showing minor fault states. Train safety risks mainly stem from unsafe human behaviors, unsafe physical states, environmental impacts of climate or geology, and management deficiencies. When they spread in a closed loop in the train operation system, due to the lack of strict management constraints on the changes of the states of people, objects, and the environment, it will lead to environmental changes that induce the transformation of unsafe states of people and objects. When the unsafe states of people and objects overlap, it will directly trigger railway safety risk events, endangering the safety of train operation. In this paper, classes, instances, and relationships in geometry are used to describe the subject-predicate structure that induces train safety risks, and the semantic relationships between safety risk objects and their unsafe states are defined from the perspectives of people, machines, the environment, and management. The definition of train safety risk knowledge is as follows:

[0022] Definition 1: The elements of train safety risk knowledge are a triple, where:

[0023] 1) E1 = {e i |e i is an entity object of train safety risk factors, i ∈ N +}, which is a set of objective entity objects that have perceptions or perform action operations in train operation. A = {A1, A2, A3, A4} is a partition of E1, where: A1 = {e p |e p is a personnel entity object}, representing the operating personnel in train operation, including train inspectors, workers, protectors, repairmen, etc.; A2 = {e f |e f is a vehicle equipment entity object}, referring to the key monitoring equipment objects of vehicles to ensure the safe operation of trains, including 5T monitoring equipment, bogies, air cylinders, and other vehicle equipment objects prone to safety hazards; A3 = {e v |e v is an environmental entity object}, referring to extreme natural environments, such as harsh weather environments like strong winds, heavy rains, snow disasters, and debris flows; A4 = {e m |e m is a management entity object}, referring to the rules and regulations or documents issued by the China National Railway Group Co., Ltd., various railway bureaus, stations, and sections to ensure the safe operation of trains.

[0024] 2) E2 = {e j |e j is an entity object of train safety hazard states, j ∈ N +}, which is the unsafe state shown by safety risk objects during train operation. A = {A'1, A'2, A'3, A'4} is a partition of E2, where: A'1 = {e' p |e' pis the unsafe state shown by personnel entities, which refers to potential safety hazards caused by the behavior of personnel participating in the daily activities of train operation and maintenance, such as dozing off on duty, playing with mobile phones, not wearing seat belts, walking on the rail center, etc.; A'2 = {e' f |e' f is the unsafe state caused by the aging of vehicle equipment}, mainly referring to the initial aging phenomenon of vehicle parts and equipment due to wear over time, such as poor operation quality of 5T monitoring equipment, wear of flat pins of control rods of bogies, cracks in the suspension seats of air cylinders, etc.; A'3 = {e' v |e' v is the unsafe state induced by environmental factors}, which refers to the unsafe state of people or objects caused by extreme environmental impacts, such as icing of catenary equipment caused by snow accumulation, and waterlogging in culverts caused by heavy rain leading to potential train operation safety hazards, etc.; A'4 = {e' m |e' m is the object that violates the regulations}, indicating that there are situations where the regulations issued by the superior department are not fully implemented, resulting in potential safety hazard problems.

[0025] 3) R = {R1, R2, R3, R4} is the set of train safety risk relationships, expressing the types of safety risk semantic relationships between the main and object of train safety risks. Among them, R1 represents the personnel safety risk semantic relationship, R2 represents the equipment safety risk semantic relationship, R3 represents the environmental safety risk semantic relationship, and R4 represents the management safety risk semantic relationship.

[0026] Definition 2 Train safety risk knowledge TK = {(e i ,r i ,e' i )|e i ∈A i ,r i ∈R i ,e i '∈A i ',i∈N +}, which links the subject and object causing safety hazards through the train safety risk semantic relationship and expresses the train safety risk semantics in a triple-structured form.

[0027] 2. Train safety risk knowledge acquisition method

[0028] Such as Figure 2As shown in the figure, after the safety supervision inspection text is preprocessed by word segmentation, it is input into the DAKA neural network. First, it is trained on the open corpus through the dual-stream self-attention mechanism to obtain the semantic understanding ability of the general language framework, and then it continues to learn on the domain corpus to enhance the understanding of professional knowledge so as to effectively express the semantic information in the railway field; finally, the PCNN is used to obtain the train safety risk knowledge from the fine-grained level. This method transforms the feature training from data fitting into a knowledge-driven mode, realizes the true understanding of the railway safety risk semantics, and aims to ensure the correctness and consistency of the train safety risk relationship recognition to the greatest extent.

[0029] 2.1 Domain Adaptive Feature Learning

[0030] The mining of vertical domain data by artificial intelligence is similar to the human thinking of understanding the world, and it needs to rise from the perception of common sense knowledge to the cognition of professional knowledge. The safety supervision inspection text is professional railway domain data, which is different from the common sense data in the open domain. This paper adopts the dual-stream self-attention mechanism to learn from the open corpus and the railway domain data set in a progressive manner, automatically recognize the unstructured safety supervision inspection text data, and represent the semantic information in the text data through discrete vectors. The dual-stream attention mechanism combines the advantages of the autoencoder and the autoregressive language model, and integrates the bidirectional autoencoder mechanism into the autoregressive framework. This mechanism removes the independent hypothesis of high-order and long dependencies in real language, performs a full permutation of factors on the input sequence without destroying the original data structure, and then uses the autoencoder to perform bidirectional dependency learning, and predicts the probability distribution of the text under the autoregressive framework.

[0031] The dual-stream self-attention mechanism mainly includes the content stream attention and the query stream attention. Figure 2 As shown in the figure. Let Ω represent the set of all possible permutations of the input sequence s of length n, a t represent the t-th element, and a ＜t represent the first t - 1 elements of a permutation a ∈ Ω. The dual-stream self-attention mechanism can be expressed as:

[0032]

[0033] Among them, encodes the content and itself, which is the representation of the content in the hidden layer; is the query representation, which only encodes the context information a ＜t and the position a t , excluding itself

[0034] 2.2 Knowledge Structure Feature Learning

[0035] The two-stream self-attention mechanism learns the semantic representation results as the input of the downstream knowledge acquisition task. The segmented convolutional neural network extracts fine-grained features of the input word vector from the syntactic structure, including semantic features, dependency features, and sentence structure features.

[0036] The semantic feature is to concatenate the word vector representation learned by the two-stream self-attention mechanism with the relative distance [22,23] to increase the syntactic features that represent the safety inspection text. It is expressed as:

[0037]

[0038] in represents the word vector (d c represents the word vector space dimension); Representation word x i The relative distance from the subject-object structure (d l is the position vector space dimension), the word vector space dimension d * =d c +d l The input text is represented as s = [x1, x2, ..., x n ].

[0039] Dependency features use convolution operations to obtain contextual semantic information and dependency information between words. On the input sequence s, a sliding window of len=3 is selected for convolution operation, and the convolution kernel matrix is Where d' is the dimension of the sentence feature vector, the input vector in the i-th sliding window is:

[0040] q i =s i-len+1:i (1≤i≤n+len-1)

[0041] Perform a dot product operation on the input sequence and the convolution kernel matrix to obtain the i-th convolution output result:

[0042] C i =[Wq+b] i

[0043] Where b is the bias vector.

[0044] The syntactic structure feature is to divide the output of the convolutional layer into [C i1 ,C i2 ,C i3 ], perform pooling operations on each of them and concatenate their results as the final pooling representation, thereby mining richer security risk semantic information. As shown below:

[0045] p i =(p i1 ,p i2 ,p i3 )

[0046] where p ij =max(C ij ), 1≤j≤3, and then perform the following nonlinear operation on the concatenated result to obtain the final representation of the sentence:

[0047] s=tanh(p i )

[0048] 2.3 Acquisition of train safety risk knowledge

[0049] After the two-stream self-attention mechanism and the segmented convolutional neural network deeply learn the features of the text, the obtained text sequence representation is input into the softmax classifier to calculate the conditional probability p(r|S, θ) of the knowledge type contained in the subject-object structure in the sequence, as shown below:

[0050]

[0051] Where |R| is the total number of train safety risk relations in Definition 1, and O is the final output vector of the neural network model, which represents the score of all railway risk relations predictions, and is defined as follows:

[0052] O=MS+d

[0053] The cross entropy is used as the objective function as follows:

[0054]

[0055] Where n' is the number of train safety risk statements, and θ is the parameter that needs to be learned in the model. During the entire training process, the stochastic gradient descent algorithm (SGD) is used to minimize the objective function.

[0056] Figure 3 It is a schematic diagram of the performance comparison between the DAKA model implemented in the present invention and different classical models, in which two indicators and some comparison results are marked to better demonstrate the performance of the DAKA model. Figure 4 The precision-recall rate of knowledge acquisition of the present invention and the comparison algorithm under different pre-training models under the same knowledge structure feature extraction module is shown, and the precision / recall curve is used for a more intuitive representation. Figure 5Show the experimental precision-recall rate of the present invention, DBERT, and DGPT under the same knowledge structure features, indicating that the present invention can effectively represent the semantic features of data and promote the improvement of the performance of downstream tasks. Three classic neural networks were compared based on the same semantics to verify the effectiveness of fine-grained feature extraction. Figure 6 Show the precision-recall rate comparison of the knowledge structure feature analysis of the DAKA model implemented by the present invention and other models under the same semantic information, indicating that the present invention can mine the structural features of train safety risk knowledge triples from the fine-grained level and promote the improvement of knowledge acquisition performance. Figure 7 Show the semantic relevance in the test samples through heat maps, indicating that the DAKA method can stretch the depth of understanding of domain knowledge and more effectively obtain correct train safety risk knowledge.

[0057] Those skilled in the art can easily understand the embodiments of the present invention. The above are only examples of the basic implementation of the present invention and are not intended to limit the present invention.

Claims

1. A domain adaptive knowledge acquisition model for train safety risks, characterized in that, (1) Formalize the definition of train safety risks in the safety supervision scenario for the first time, introduce the triple concept in the knowledge graph, and describe the risk semantic relationship between the supervised object and its unsafe state. (2) Propose a domain-adaptive knowledge acquisition method DAKA, which gradually learns the abstract semantic features of text data from the levels of general semantics, professional semantics, and knowledge structure semantics through a dual-stream self-attention mechanism, improving the representational ability of railway domain semantics. (3) Introduce a segmented convolutional neural network to deeply mine train safety risk features in a fine-grained manner and realize the automatic acquisition of train safety risk knowledge.

2. The domain adaptive knowledge acquisition model for train safety risks according to claim 1, wherein Describe the subject-object structure that induces train safety risks using classes, instances, and relationships in geometry, define the semantic relationship between the safety risk object and its unsafe state from the perspectives of people, machines, the environment, and management respectively, and model the train safety risk knowledge elements as triples: ∑(E1,E2,R).

3. The domain adaptive knowledge acquisition model for train safety risks according to claim 1, wherein Adopt a dual-stream self-attention mechanism, combine the advantages of autoencoding and autoregressive language models, remove the independent assumption of high-order and long dependencies in real languages, and perform a full permutation of factors on the input sequence without breaking the original data structure, improve the feature ability of domain semantics. Let Ω denote the set of all possible permutations of the input sequence s of length n, and a i denote the t-th element, and a ＜t denote the first t- elements of a sorted a ∈ Ω. The dual-stream self-attention mechanism can be expressed as: Among them, the encoded content and itself are the representations of the content in the hidden layer; is the query representation, encoding only the context information a ＜t and the position a t , excluding itself 4. The domain adaptive knowledge acquisition model for train safety risks according to claim 1, wherein Perform relation extraction on train safety supervision texts. On the basis of enhancing semantic features, use a segmented convolutional neural network for relation extraction. The triple structure divides the sentence into three segments, and a segmented max-pooling operation is introduced in each segment instead of implementing a single max-pooling over the entire sentence to capture these fine-grained features. Formally, define the i-th convolutional operation as: Then, input the obtained text sequence representation into a softmax classifier to calculate the conditional probability p(r|S,θ) of the knowledge type contained in the subject-object structure in this sequence: where |R| is the total number of train safety risk relations in Definition 1.

5. The domain adaptive knowledge acquisition model for train safety risks according to claims 1 and 5, characterized in that, Automatically enhance the feature representation of domain semantics and capture fine-grained features to extract relations. O is the final output vector of the neural network model, representing the scores for predicting all railway risk relations, and is defined as follows: O = MS + d Use cross-entropy as the objective function, as follows: where n' is the number of train safety risk statements, and θ is the parameter to be learned in the model. During the entire training process, use the stochastic gradient descent algorithm (SGD) to minimize the objective function.

Images