Traffic incident knowledge graph construction method based on multi-layer semantic graph convolutional neural network

Abstract

The present invention proposes a method for constructing a knowledge graph of traffic incidents based on a multi-layer semantic graph convolutional neural network, the steps of which are as follows: using a top-down approach to define the hierarchical system of the event ontology and design an ontology model; downloading Chinese and English datasets from the official website of the public dataset, collecting and organizing text and image data of real university traffic risk incidents, and constructing a knowledge graph dataset; using the OneRel model as the basic model, constructing an event extraction model; using a multi-layer semantic graph convolutional neural network to learn the global semantic and syntactic graph embedding representation information of the knowledge graph dataset, and training the event extraction model; constructing a knowledge graph of university traffic safety incidents and visually displaying it. The present invention integrates the global semantic dependency analysis graph and the syntactic analysis graph to improve the accurate recognition of distant entities; constructing a multi-layer semantic graph convolutional neural network to capture deeper entity relationship semantic hidden information, and constructing a more complete knowledge graph of university traffic safety incidents.

Description

Technical Field

[0001] The present invention relates to the technical field of event knowledge graph construction methods, and in particular to a traffic event knowledge graph construction method based on a multi-layer semantic graph convolutional neural network. Background Art

[0002] With the rapid expansion of the number of universities and enrollments in China, the number of faculty and students on campus has increased, leading to frequent traffic accidents on university campuses. Consequently, a large amount of data on university traffic safety incidents has been accumulated. Utilizing this data to analyze and manage these issues is of great significance for protecting the lives of university faculty and students. Existing research on methods for constructing knowledge graphs for university traffic safety incidents is limited, and research on the construction of ontology models for university traffic incident knowledge graphs is lacking. Furthermore, existing event information extraction methods have limited ability to handle triple overlap and fuzzy entity boundaries in relationship extraction, leading to extraction errors and knowledge redundancy. Intelligent processing and analysis of university traffic incident resource data is both a pressing need for online public opinion monitoring and a key challenge in constructing event knowledge graphs for public opinion analysis.

[0003] As the demand for online public opinion analysis becomes increasingly diversified, the research on the construction technology of university traffic incident knowledge graphs can no longer meet the new demands for the supervision of the current online public opinion field. The knowledge described in the traditional knowledge graph construction model is static and definite facts, focusing more on knowledge questions and answers, entity portraits and other issues. However, it seems a bit weak in terms of public opinion monitoring and text intelligent analysis and reasoning. In actual network supervision scenarios, the use of crawler and event knowledge graph technology to construct a university traffic risk event knowledge graph to achieve visual display and intelligent analysis of risk event data is more valuable for reference. The problems that still exist in the existing technology are as follows:

[0004] (1) Currently, there is little research on the construction technology of knowledge graphs for on-campus traffic safety incidents. In addition, existing methods lack research on the ontology model construction of on-campus traffic incident knowledge graphs. Secondly, the non-standardized representation of information about on-campus traffic incidents in online social media and the low-quality data of multi-source knowledge information further make it difficult to construct a knowledge graph for on-campus traffic incidents.

[0005] (2) The core content of the knowledge graph construction of traffic safety incidents on university campuses is entity-relationship triples. Existing methods focus on event triple extraction in general fields, but are not effective in extracting event triples from real Chinese incident cases. In addition, existing entity-relationship joint extraction methods have limited ability to handle triple overlap and fuzzy entity boundaries in relationship extraction, which can lead to problems such as extraction errors and knowledge redundancy.

[0006] (3) Most existing studies focus on text data from online social media, and do not utilize image information data from on-campus traffic incidents to assist in building a rich semantic event knowledge graph. Secondly, most existing model methods are tested and validated on public English datasets, while few studies consider conducting experiments and evaluations on Chinese datasets.

[0007] The invention patent with application number 202310131194.8 discloses a method for constructing a traffic knowledge graph based on multi-source data fusion, including: step S1: obtaining traffic image modal data based on traffic image big data, and the traffic image modal data includes vehicle color information, vehicle model information and vehicle speed information; step S2: obtaining traffic text modal data based on road network data, and the traffic text modal data includes traffic intersection passing data, traffic intersection data, traffic section data and lane data; step S3: defining entities based on traffic image modal data and traffic text modal data, and matching entity attributes with the relationship between entities to obtain a knowledge graph data layer; step S4: classifying the knowledge graph data layer according to the vertical field knowledge graph to obtain structured data and semi-structured data, and establishing a traffic knowledge graph based on the structured data and semi-structured data. The above invention greatly reduces the workload of data modeling. However, traffic accidents occur frequently on college campuses. The above invention has not been used for monitoring online public opinion in colleges and universities, nor has it considered building a traffic event ontology model, resulting in the inability to structure the scattered knowledge of on-campus traffic events. In addition, the recognition rate of text modal data obtained based on road network data is low, and the effect of extracting event information from Chinese text is not ideal. Summary of the Invention

[0008] In view of the technical problems that existing knowledge graph construction methods have limited ability to handle triple overlap problems and fuzzy entity boundaries in relationship extraction, which will cause extraction errors and knowledge redundancy, the present invention proposes a traffic event knowledge graph construction method based on a multi-layer semantic graph convolutional neural network. The multi-layer semantic graph convolutional neural network is used to capture deeper entity relationship semantic hidden information, and the image and text data about the event are fused together as model input and trained. Better results are achieved in knowledge graph construction, prediction and analysis in the scenario of online public opinion analysis, which can effectively assist public opinion supervision departments in monitoring sensitive information.

[0009] In order to achieve the above-mentioned purpose, the technical solution of the present invention is implemented as follows: a method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network, the steps of which are as follows:

[0010] Step 1: Define the hierarchical system of event ontology in a top-down manner and design the ontology model of the knowledge graph of university traffic safety events;

[0011] Step 2: Download the Chinese and English datasets from the official website of the public dataset, collect and organize the text and image data of real university traffic risk events, and build a knowledge graph dataset of real cases;

[0012] Step 3: Using the OneRel model as the base model, we build an event extraction model for the knowledge graph of on-campus traffic safety events in universities. We use a multi-layer semantic graph convolutional neural network to learn the global semantic and syntactic graph embedding representation information of the knowledge graph dataset and input it into the event extraction model for training.

[0013] Step 4: Build a knowledge graph of traffic safety incidents in colleges and universities and present it visually.

[0014] Preferably, the ontology of the university traffic safety incident knowledge graph includes entity type O(E), attribute type O(S) and relationship type O(R), H KG ={O(E),O(R),O(S)} represents a set consisting of entity types, attribute types, and relationship types, which is reflected as a hierarchical relationship between entity types and attribute types and is displayed in the form of knowledge graph triples;

[0015] The ontology model of the knowledge graph of traffic safety incidents in colleges and universities is based on the core elements of basic attributes of incidents, accident types, model methods, and incident handling measures, and constructs semantic knowledge associations between the core elements; the domains are divided according to the conceptual level of each ontology class: the basic attributes of incidents include the location of the incident, the time of the incident, the casualties of the incident, and the information of the incident personnel; the accident types include two-wheeled vehicle accidents, three-wheeled vehicle accidents, and automobile accidents; the model methods include statistical analysis and machine learning; the accident handling measures include the accident handling department, the accident handling results, and the cause of the accident.

[0016] Preferably, the Chinese and English datasets include NYT datasets, WebNLG datasets, and DUIE datasets; based on risk keywords, texts and images of real university on-campus traffic risk events are collected and organized from official websites, Weibo, and Baidu News webpages as part of the knowledge graph event extraction dataset; the data types are structured or unstructured text data and image information; the text data is annotated using the BIO tagging method, and the image data is annotated using the Vott annotation software to establish a Chinese dataset of real cases;

[0017] Crawler technology is used to process the text of the Chinese dataset, and the PP-OCRV3 model is used to extract text information from the images of the Chinese dataset as training data for the event extraction model; the process of the PP-OCRV3 model to extract text information from images is as follows: the PP-OCRV3 model first preprocesses the input image, the text detection module in the PP-OCRV3 model marks the area to be detected in the form of coordinates, and then the text recognition module recognizes the text information in the marked coordinate area and outputs the recognized text information, and finally saves the text information.

[0018] Preferably, the implementation method of the event extraction model in step 3 is: based on the OneRel model, the global dependency semantics and syntactic graph embedding representation of the sentence are incorporated into the initial vector generation stage; the BERT model and Bi-LSTM network are used to obtain the semantic information of the text as the input vector H e ; Construct a multi-layer semantic graph convolutional neural network to learn global dependency semantics and syntactic graph embedding representation information, capture deeper entity relationship semantic hidden information; embed the learned graph into the semantic vector G e and the input vector H e Splicing to get a new sequence vector V n , the new sequence vector V n The global semantic information is captured through the graph mixing pooling layer; the label of each character of the sentence is obtained through the model output layer, and the final result is output.

[0019] Preferably, a multi-feature fusion attention mechanism is designed to enhance the accuracy of triple classification in the event extraction model, giving candidate entities a higher weight in the entity extraction stage;

[0020] The multi-feature fusion attention mechanism uses an attention mechanism guided by word-level features, text and syntactic dependency fusion features to assign different weight coefficients to the corpus text. The feature vectors obtained at the kth layer are expressed as:

[0021] r=V i (softmax(ω T tanh(V i ))) T ;

[0022] Z k =tanh(r);

[0023] Among them, the tanh function is used to concatenate the vector V i Transformed to [-1,1], r represents the feature vector after the softmax function, Z k The feature vector after the two-sided tangent activation function is represented by V i Represents the splicing vector V nAny feature vector in , ω represents the trained parameter vector; the softmax() function performs normalization processing;

[0024] And the vector H after the mixed pooling of the graph g k and the concatenated vector V i Perform nonlinear mapping learning to obtain the feature vector: L i k =tanh(w k [V i ,H g k ]+b k );

[0025] For the eigenvector L i k Normalize it to get the vector:

[0026] The feature vector input to the k layer is

[0027] Among them, w k 、b k They represent the model parameters learned by the k-1th layer attention mechanism; H g k represents the concatenated feature output after the k-1th layer fusion; exp is the exponential operation;

[0028] Drawing on the gating mechanism, the feature vector Z k and the eigenvector M k Fusion means:

[0029] C=σ(W l 1 tanh(W l 2 Z k +W l 3 M k ));

[0030] D k =C·Z k +(1-C)·M k ;

[0031] Among them, σ represents the Sigmoid activation function, W l 1 、W l 2 、W l 3 Represents the weight parameter of the model self-training learning, C represents the feature vector Z k and Mk The vector to be normalized.

[0032] Preferably, the method for obtaining the global dependency semantics and syntactic graph embedding representation is as follows: using an LTP tool to preprocess the input sentence to obtain part-of-speech tagging information and syntactic dependency information of the preprocessed sentence sequence, and performing syntactic dependency analysis to determine the syntactic structure of the sentence or the dependency semantic relationship between words in the sentence; using a pre-trained model to obtain the global dependency semantics and syntactic graph embedding representation;

[0033] The global dependency semantic and syntactic graph embedding is represented as G g = {V1, V2, E1, E2}, where V1 and V2 represent the set of semantic and syntactic nodes in a sentence, E1 and E2 represent the set of edges in the semantic and syntactic graphs in a sentence, and G g Represents the global dependency semantics and syntactic graph embedding representation in a sentence;

[0034] The Bi-LSTM network uses two independent LSTM networks to capture the contextual information of words and sentences and mine deep semantic information; it also learns the grammatical structure features of spanned sentences to improve the performance of relation extraction.

[0035] Preferably, the implementation method of the graph mixing pooling layer is: using the graph mixing pooling operation to concatenate the sequence vector V n Capture global scope information and get vector representation:

[0036]

[0037] Among them, V1, V2, ..., V n The concatenated sequence representation is embedded in the vector representation, and MaxPooling() represents the maximum pooling.

[0038] Preferably, the target triple set Y of statement S is:

[0039] Y={(h1,r1,t1),...,(h n ,r n ,t n )|h n ,r n ∈E,r n ∈R};

[0040] Among them, h n ,t n Represents the nth head entity and tail entity, r n Represents the nth relationship between entity pairs, E and R represent the entity set and relationship set respectively; statement S = {w1,w2,w3,...,w L}, where L represents the number of sentence sequences in a sentence, w1,w2,w3,...,w L Represent the sequence of each sentence separately;

[0041] The BERT model performs well on the sentence S={w1,w2,w3,...,w L} is encoded to obtain the input vector of the word corresponding to the sentence:

[0042] H e =[h1,h2,h3,...,h L ]=BERT[w1,w2,w3,...,w L ]{h L ∈R d};

[0043] Where d represents the embedding dimension, BERT represents the pre-training model; h1,h2,h3,...,h L Represented as a sequence vector representation of the training sentence S;

[0044] Construct a multi-layer semantic graph convolutional neural network to learn the topological structure in the global semantic and syntactic dependency graph and obtain the embedded vector representation G of the text e =MultiGCN(W S (h M +h N )+b S );

[0045] Among them, W S 、b S Represent the weight parameter matrix and the trainable parameter matrix respectively. MultiGCN() represents a multi-layer semantic graph convolutional neural network that captures high-order neighborhood information between word nodes. M 、h N Embedding representation vectors representing the semantic dependency analysis graph and syntactic analysis graph respectively;

[0046] The obtained text embedding vector is represented by G e With the input vector H e Perform splicing to obtain the spliced sequence vector

[0047] V n =[G e ;H e ].

[0048] Preferably, the cross entropy loss function is used to calculate the loss Loss during the training process in the global semantic and syntactic dependency graph embedding representation layer and the multi-feature attention mechanism layer. total =Loss g +αLoss a, and then assign weights to the losses of each layer; the weights assigned to the losses of each layer are implemented by the back propagation algorithm; among them, Loss g Represents the loss of the global semantic and syntactic dependency graph embedding representation layer, α represents the weight of the multi-feature attention mechanism layer loss, Loss a represents the multi-feature attention mechanism layer loss;

[0049] The Adam optimization algorithm is used to adjust different learning rates for each different parameter.

[0050] Preferably, the implementation method of step 4 is: using the trained event extraction model to extract triples from multi-source heterogeneous data, constructing a knowledge graph of university traffic events, and using the Neo4J graph database to visualize the stored triples.

[0051] Compared with the existing technology, the beneficial effects of the present invention are as follows: the method for constructing a knowledge graph of campus traffic safety incidents based on an improved multi-layer semantic graph convolutional neural network first designs a traffic incident knowledge graph ontology model based on multi-source heterogeneous data to construct multiple key elements ("people-vehicles-roads-time-space environment"); secondly, in response to the shortcomings of existing event information extraction methods, a campus traffic safety incident extraction model MGRel based on a graph convolutional neural network is proposed. The present invention first integrates the global semantic dependency analysis graph embedding information and the syntactic analysis graph embedding information to further improve the accurate recognition of distant entities; secondly, it constructs an improved multi-layer semantic graph convolutional neural network to capture deeper entity relationship semantic hidden information, and finally constructs a relatively complete knowledge graph of university traffic safety incidents.

[0052] This invention achieves the construction of a knowledge graph for on-campus traffic incidents in universities, building a universal model applicable to both image and text data. To improve the accuracy and standardization of information extracted from online social media about on-campus traffic safety incidents, this invention proposes a universal ontology model for the knowledge graph of on-campus traffic safety incidents. This model can provide a standardized definition for the construction of the knowledge graph of on-campus traffic safety incidents and reconstruct the multi-source knowledge structure. Furthermore, it fully considers the dynamic spatiotemporal information between events and constructs an on-campus traffic incident knowledge graph ontology model based on multi-source heterogeneous data, with multiple key elements ("people-vehicles-roads-spatiotemporal environment").

[0053] To meet the needs of online public opinion scenarios, this paper proposes a joint entity relationship extraction method based on graph convolutional neural networks. First, it fuses global semantic dependency analysis graph embedding information with syntactic analysis graph embedding information to further improve the accurate recognition of distant entities. Second, it constructs a multi-layer semantic graph convolutional neural network to capture deeper semantic hidden information about entity relationships. Finally, it designs a multi-feature fusion attention mechanism to enhance the accuracy of the model's triple classification. Ben Amin effectively improves the problem of entity boundary ambiguity in triple overlap and relationship extraction.

[0054] To effectively utilize image data from traffic incident cases, this paper employs the latest PP-OCRV3 technology to identify information within images. This data, along with the source text, is pre-processed and fed into a pre-trained model. In light of the specific needs of the online public opinion laboratory, the existing model is improved and a comparative experiment with a Chinese dataset is added, enhancing the model's generalization capabilities. Finally, a knowledge graph for on-campus traffic safety incidents, tailored to the online public opinion context, is constructed to provide decision support for emergency response departments.

[0055] In response to the problems of non-standardized representation of information about on-campus traffic incidents in online social media and low-quality data in multi-source knowledge information, this paper proposes a universal ontology model for on-campus traffic safety incident knowledge graphs, which can provide standardized definitions for the construction of on-campus traffic safety incident knowledge graphs and reconstruct the multi-source knowledge structure. The present invention utilizes multi-source heterogeneous data features to represent information, and combines PP-OCRV3 technology, graph convolutional neural network technology, and entity relationship extraction models to effectively extract text information from images or texts. This overcomes the limited ability of existing methods to handle triple overlap and the fuzzy entity boundaries in relationship extraction, which can lead to extraction errors, knowledge redundancy, and other problems. At the same time, a Chinese text-based event knowledge graph method is designed, and the constructed results are applied to the decision-making of relevant regulatory departments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 This is the event extraction model framework diagram of the present invention.

[0058] Figure 2 Schematic diagram of the division of the knowledge graph ontology layer of the present invention.

[0059] Figure 3This is a structural diagram of the multi-layer semantic graph convolutional neural network of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0061] like Figure 1 As shown in the figure, a method for constructing a traffic safety event knowledge graph using a multi-layer semantic graph convolutional neural network is described. The steps are as follows:

[0062] Step 1: Define the hierarchical system of event ontology in a top-down manner and design the ontology model of the knowledge graph of university traffic safety events.

[0063] The original ontology model is suitable for representing static concepts, but traffic incidents on college campuses occur dynamically and are in a state of continuous evolution. In addition, existing methods have relatively little research on the construction of knowledge graphs for traffic incidents on college campuses, and the research on the construction of rule bases in the ontology knowledge base is not in-depth enough. The present invention combines the characteristics of the knowledge of traffic safety incidents on college campuses, applies the ontology model to the construction of the knowledge graph of traffic safety incidents on college campuses, considers the various elements related to traffic incidents as an organic and dynamic whole, and constructs the conceptual model of the knowledge graph of traffic safety incidents on college campuses in a top-down manner. In addition, it also includes the definition of entities and attributes in the event knowledge graph, which can structure the scattered knowledge of traffic incidents on campus.

[0064] This paper combines the characteristics of the knowledge of traffic safety incidents on college campuses and applies the ontology model to the construction of the knowledge graph of traffic safety incidents on college campuses. It considers the various elements related to traffic incidents as an organic and dynamic whole and constructs the conceptual model of the knowledge graph of traffic safety incidents on college campuses in a top-down manner. The knowledge graph of traffic safety incidents on college campuses includes entity type O(E), attribute type O(S) and relationship type O(R). KG It represents a set of entity types, attribute types, and relationship types, which is reflected in the hierarchical relationship between entity types and attribute types, and is ultimately displayed in the form of knowledge graph triples. Specifically, it is shown in formula (1).

[0065] H KG ={O(E),O(R),O(S)} (1)

[0066] Ontology construction of campus traffic safety incidents Figure 2 shown. Figure 2 The knowledge graph ontology model for on-campus traffic safety incidents is based on four core elements: basic event attributes: accident types, modeling methods, and incident handling measures. It then constructs semantic knowledge associations between these core elements. First, these four categories are constructed at the ontology level. Next, specific domains are divided based on the conceptual hierarchy of each ontology class. Basic event attributes include the location, time, casualties, and personnel information. Accident types include two-wheeled vehicle accidents, three-wheeled vehicle accidents, and automobile accidents. Modeling methods include statistical analysis and machine learning. Accident handling measures include the department responsible for handling the incident, the results of the accident handling, and the cause of the accident. This model uses university events as cognitive units to acquire, describe, and represent relevant domain knowledge about sudden traffic incidents in universities, explore the relationships between university traffic incidents, and assist relevant departments in timely monitoring and handling university public opinion incidents. The specific process for constructing the ontology model is as follows: 1. Determine the professional domain of the event ontology; 2. Define the hierarchical structure of the event ontology using a top-down approach; 3. Define the category attributes contained in the event; and 4. Create instances. After the ontology model construction process, the organizational structure of the university traffic event ontology becomes more hierarchical, and then it is used as the skeleton for storing and displaying event triple knowledge data, and finally a clearer visualization effect of the university traffic knowledge graph can be obtained.

[0067] Step 2: Data preparation phase: Download Chinese and English datasets from the official website of the public dataset, collect and organize text and image data of real university traffic risk events, and build a knowledge graph event extraction dataset of real cases.

[0068] First, download relevant Chinese and English datasets from the official website of public datasets, including NYT, WebNLG, and DUIE datasets. The present invention uses the NYT dataset. The training set of the NYT dataset contains 56,195 sentences, the validation set contains 5,000 sentences, and the test set contains 5,000 sentences and 24 predefined relationships. The training set of the WebNLG dataset contains 5,019 sentences, the validation set contains 500 sentences, and the test set contains 703 sentences and 171 predefined relationships. In the WebNLG dataset, an instance includes a set of triples and several standard sentences, and each standard sentence contains all triples of this instance. The training set of the DUIE dataset contains 173,108 sentences, the validation set contains 21,639 sentences, and the test set contains 19,992 sentences and 49 predefined relationships.

[0069] Secondly, based on risk keywords (such as university traffic accidents), using universities as an example, we collected and organized text and image information about real on-campus traffic risk incidents from official websites, Weibo, and Baidu News. This not only validated the performance of our improved extraction model, but also constructed a knowledge graph of on-campus traffic safety incidents to assist relevant departments in oversight. This data included 3,689 pieces of text data and 10,258 knowledge triples.

[0070] First, based on the designed ontology model of traffic safety incidents on college campuses, we access public multi-source heterogeneous data information. The access data sources are official public accounts, Weibo and other social media, and the data types are mainly structured / unstructured text data and image information; and the data are annotated for model training. The BIO (B-begin, I-inside, O-outside) notation method is used to annotate text data, and the Vott annotation software is used to annotate images, and a Chinese data set of real cases is built. Secondly, the present invention uses public Chinese and English data sets to evaluate the constructed model to verify the effectiveness of the technology proposed in this invention.

[0071] Step 3: Model construction and training phase: Based on the OneRel model, an event extraction model for the knowledge graph of traffic safety incidents on university campuses is proposed. A multi-layer semantic graph convolutional neural network is used to learn the global semantic and syntactic graph embedding representation information of the knowledge graph dataset, and the information is input into the event extraction model for training.

[0072] Existing knowledge graph triple extraction mostly processes text in a character-based manner. However, compared with words, the semantics of a single character contains different semantic information in a character-based manner, which can easily lead to ambiguity and overlapping of extracted triples. In reality, texts generally have more than one triple, and these triples overlap with each other, making the extraction task difficult. If the model's ability to handle overlapping triples is limited, then the model will not be able to adapt to many data sets, resulting in serious limitations and extraction errors. In addition, the extraction effect of event triples is highly correlated with the distance between entities. The correlation between entities that are farther away is weaker, which will lead to fuzzy entity boundaries and low recall rates in relationship extraction. Therefore, the present invention takes the OneRel model as the basic model, and proposes a knowledge graph event extraction model for traffic safety incidents on college campuses---the MGRel model. The model design is as follows: Figure 1 shown. Figure 1 The syntactic dependency graph embedding representation and semantic dependency graph embedding representation in the

[15] are obtained by preprocessing the input sentences using the Harbin Institute of Technology LTP tool. The preprocessed sentence sequence is then given part-of-speech tagging information and syntactic dependency information. Syntactic dependency analysis can determine the syntactic structure of a sentence or the dependency semantic relationships between words in a sentence. The pre-trained model is then used to obtain the embedding vector representation.

[0073] In the multi-source heterogeneous data processing stage, first, crawler technology is used to process text data, and the PP-OCRV3 model is used to extract text information from images and public Chinese and English data sets as inputs for the MG Rel model. Crawler technology is used to obtain relevant information from media such as Weibo and Zhihu. The operation process is as follows: first, based on keywords such as university traffic incidents, a preset URL link is obtained, and then a browser access is simulated to obtain the HTML text of the web page. Then, the URL link contained therein is parsed to obtain the HTML text in the URL link. This process is repeated to obtain a large amount of web page text content, and finally the text data is saved. After the text data is annotated, it is used as training data for the MG Rel model. The source of the image data is the image information posted by users on media such as Weibo and Zhihu. The present invention uses crawler technology to download it to a local server for storage, and after the image data is annotated, it is used as training data for the PP-OCRV3 model. The process of the PP-OCRV3 model extracting text information from an image is as follows: first, a test image is input, and the PP-OCRV3 model first preprocesses the image. Then, the text detection module in the PP-OCRV3 model marks the area to be detected in the form of coordinates. Then, the text recognition module recognizes the text information in the marked coordinate area and outputs the recognized text information. Finally, the text information is saved. After the text data is annotated, it is used as training data for the MGrel model. Secondly, based on the OneRel model, the global dependency semantics and syntactic graph embedding representation of the sentence are incorporated into the initial vector generation stage. The Harbin Institute of Technology LTP tool is used to preprocess the input sentence, and then the part-of-speech tagging information and syntactic dependency information of the preprocessed sentence sequence are obtained. The syntactic dependency analysis can determine the syntactic structure of the sentence or the dependency semantic relationship between words in the sentence. By constructing a global semantic dependency syntactic graph, more text feature information can be captured. The semantic syntactic dependency graph is defined as G g = {V1, V2, E1, E2}, where V1 and V2 represent the set of semantic and syntactic nodes in a sentence, E1 and E2 represent the set of edges in the semantic and syntactic graphs in a sentence, and G g Represents the global dependency semantics and syntactic graph embedding representation in a sentence. The BERT model and Bi-LSTM network are used to obtain the semantic information of the original text (input vector H e , Figure 1 T1-T n , where H e ={T1,T2,...,T n}), Bi-LSTM network uses two independent LSTM networks to process forward and backward sequential data. In this invention, it is mainly used to capture the contextual information of words and sentences and mine deep semantic information. By learning the grammatical structure features of the span of sentences, the performance of relation extraction can be improved. Construct a multi-layer semantic graph convolutional neural network MultiGCN() to learn global semantics and syntactic graph embedding representation information, and capture deeper entity relationship semantic hidden information. Then the learned graph is embedded into the semantic vector G e and the original text vector H e Splicing to get a new sequence vector V n , the new sequence vector V n The graph hybrid pooling layer captures global semantic information. A multi-feature fusion attention mechanism is then designed to enhance triple classification accuracy in the MGRel model. This assigns higher weights to candidate entities during entity extraction, improving the precise recognition of distant entities. Finally, a softmax() layer improves triple recognition accuracy, reduces overlapping triplets, and reduces blurred entity boundaries during extraction.

[0074] First, given the training sentence S = {w1,w2,w3,...,w L}, where L represents the number of sentence sequences, w1,w2,w3,...,w L Represent the sequence symbols of each sentence respectively. Assume that the target triple set Y of sentence S is:

[0075] Y={(h1,r1,t1),...,(h n ,r n ,t n )|h n ,r n ∈E,r n ∈R} (2)

[0076] Among them, h n ,t n Represents the nth head entity and tail entity, r n represents the nth relationship between entity pairs, E and R represent the entity set and relationship set respectively.

[0077] Input sequence encoding: This paper first uses the BERT model to encode the training sentence S and obtain the word input vector representation corresponding to the sentence:

[0078] H e =[h1,h2,h3,...,h L ]=BERT[w1,w2,w3,...,w L ]{h L ∈Rd} (3)

[0079] Where L represents the number of characters in the sentence (training sentence S), d represents the embedding dimension, and BERT represents the pre-training model. L It is represented as a sequence vector representation of the training sentence S. The BERT model contains 12 hidden layers, each with a size of 768. The training sentence S is encoded to obtain the input vector H e .

[0080] Global semantic and syntactic dependency analysis graph embedding representation: First, the input training sentence S is preprocessed using the Harbin Institute of Technology LTP tool. The part-of-speech tagging information and syntactic dependency information of the preprocessed sentence sequence are then obtained. The part-of-speech tagging information and syntactic dependencies express the entire sentence structure through the dependency relationships between words. Syntactic dependency analysis can determine the syntactic structure of a sentence or the dependency relationships between words in a sentence. These dependencies can express the semantic dependencies between the various components of a sentence, helping the model understand the meaning of the entire sentence or certain components within it.

[0081] By constructing a global semantic dependency syntactic graph, more text feature information can be captured. The global semantic and syntactic dependency graph is defined as G g = {V1, V2, E1, E2}, where V1 and V2 represent the sets of semantic and syntactic nodes in the graph, respectively, and E1 and E2 represent the sets of edges in the semantic and syntactic graphs, respectively. A pre-trained model is used to obtain the global semantic-syntactic dependency graph embedding vector representation. The global semantic-syntactic dependency information includes part-of-speech tagging information and syntactic dependency information.

[0082] Multi-layer semantic graph convolutional neural network layer and mixed pooling: Construct a multi-layer semantic graph convolutional neural network to learn the topological structure in the global semantic and syntactic dependency graph, and then obtain the embedded vector representation G of the text e , the calculation process is shown in formula (4):

[0083] G e =MultiGCN(W S (h M +h N )+b S ) (4)

[0084] Among them, W S 、b S Represents the weight parameter matrix and the trainable parameter matrix respectively. The weight parameter matrix is initialized by the model pre-training method. The pre-training process can be achieved by methods such as autoencoders. MultiGCN() represents a multi-layer semantic graph convolutional neural network that can capture high-order neighborhood information between word nodes. M 、hN The embedded representation vectors of the semantic dependency analysis graph and the syntactic analysis graph are respectively represented. The image information obtained by the Harbin Institute of Technology LTP tool is converted into the embedded representation vectors of the semantic dependency analysis graph and the syntactic analysis graph using the pre-trained model. The multi-layer semantic graph convolutional neural network structure is as follows Figure 3 As shown, Figure 3 G1, G n , G e They respectively represent the sequence vector representations of the first layer, the nth layer, and the eth layer after the multi-layer semantic graph convolutional neural network. The multi-layer perceptron (MLP) consists of an input layer, multiple hidden layers, and an output layer. The MLP of the present invention is designed to have 5 layers, of which the input layer and the output layer are each one layer, and the hidden layer is 3 layers.

[0085] Then the obtained global semantic and syntactic dependency embedding vector representation G e The input vector H obtained by formula (3) e Perform splicing to obtain the spliced sequence representation vector V n ={h1,h2,...,h n}, as shown in formula (5).

[0086] V n =[G e ;H e ] (5)

[0087] However, under different relationships, the correlation between words and triples in a sentence is relatively large. Therefore, a graph hybrid pooling operation is used to concatenate the sequence vector V n Capture global scope information and get vector representation H g :

[0088]

[0089] Among them, V1, V2, ..., V n The concatenated sequence representation is embedded in the vector representation, MaxPooling() represents the maximum pooling, and the second half of the formula represents the mean pooling.

[0090] The multi-feature fusion attention mechanism layer includes word-level feature attention, semantic dependency attention, and syntactic dependency attention. At the same time, in order to discard noise data and improve the accuracy of the classifier, the present invention uses the attention mechanism guided by word-level features, text and syntactic dependency fusion features to assign different weight coefficients to the corpus text. Taking the kth layer as an example, the feature representations are Z k and M k :

[0091] r=V i (softmax(ω T tanh(Vi ))) T (7)

[0092] Z k =tanh(r) (8)

[0093] Among them, the tanh function is used to transform the concatenated vector to [-1,1], r, Z k 、V i , ω represent the feature vector representation after the softmax function, the feature vector representation after the two-sided tangent activation function, and the concatenated vector V n Any feature vector in represents the trained parameter vector, the weights initialized by ω are randomly extracted from Gaussian, and the softmax() function is used for normalization.

[0094] L i k =tanh(w k [V i ,H g k ]+b k ) (9)

[0095]

[0096]

[0097] Among them, w k 、b k Represent the model parameters learned by the k-1th layer attention mechanism. i represents the input of the k-1th layer attention mechanism and the i-th fused feature, H g k L represents the concatenated feature output after the k-1th layer fusion. i k The vector H after the mixed pooling layer of the graph g k and the concatenated vector V i Perform nonlinear mapping learning, α i k On behalf of L i k The eigenvector is the vector to be normalized, exp is the exponential operation, M k Represents the feature vector representation input to the k-th layer. Finally, referring to the gating mechanism, the fusion of multiple feature vectors is represented as D k , to achieve the purpose of complementary advantages.

[0098] C=σ(W l 1 tanh(W l2 Z k +W l 3 M k )) (12)

[0099] D k =C·Z k +(1-C)·M k (13)

[0100] Among them, σ represents the Sigmoid activation function, W l Represents the weight parameters of the model self-training learning. The initialized weight parameters are random values in the self-training learning. The dimension of vector C is the same as that of Z. k and M k are the same, this vector can dynamically assign weights to different features, thereby avoiding information redundancy. CPRF1 represents the feature vector Z k and M k Normalized vector, W l 1 、W l 2 、W l 3 They respectively represent the first weight parameter obtained by the model self-training learning, the second weight parameter obtained by the model self-training learning, and the third weight parameter obtained by the model self-training learning.

[0101] The overall training process of the MGRel model is:

[0102] First, in the data preprocessing stage, training data, primarily text, must be prepared. This method uses crawler technology to capture specific text and image data. Then, PP-OCRV3 technology is used to extract text from images. The captured specific text and the text extracted from the images are then cleaned, aligned, and fused to produce processed text data. The BIO data annotation method is then used to perform sequence annotation on the text data, resulting in a self-built dataset.

[0103] Model construction stage: This paper improves the existing algorithm strategy and proposes a new entity relationship joint extraction model MGRel.

[0104] Model training and evaluation phase: The MGRel model is trained using preprocessed annotated data (NYT dataset, WebNLG dataset, DUIE dataset, and a self-built Chinese dataset of real-world examples). During model training, the MGRel model needs to be evaluated to understand its performance and effectiveness. Evaluation metrics include precision (P), recall (R), and F1 score.

[0105] Step 4: The model output layer obtains the label of each character in the sentence and outputs the final result.

[0106] The output layer uses softmax to obtain the label of each character in the sentence and output the final result. However, due to the sparsity problem of the training dataset, it is necessary to use the cross-entropy loss function in the global semantic and syntactic dependency graph embedding representation layer and the multi-feature attention mechanism layer to calculate the loss during the training process, and then assign weights to the loss of each layer. The weight assignment of the loss of each layer is implemented by the backpropagation algorithm. The backpropagation algorithm calculates the error of each layer of neurons and backpropagates these errors to the previous layer to calculate the loss of each layer. Then, according to the loss of each layer, the corresponding weight can be assigned. The model loss is then added, and the calculation method is shown in formula (14):

[0107] Loss total =Loss g +αLoss a (14)

[0108] Among them, the Adam optimization algorithm is used to adjust different learning rates for each different parameter. This algorithm can adapt to the problem of sparse gradient or large gradient noise. g Represents the loss of the global semantic syntactic dependency graph embedding representation layer, α represents the weight of the multi-feature attention mechanism loss layer (artificially set parameters), Loss a Represents the multi-feature attention mechanism layer loss.

[0109] Step 5: Event Knowledge Graph Visualization

[0110] The model output layer represents the end of model training. After the training, the model can be used in business data to extract knowledge graph triples from text data and store them in the Neo4j graph database. Then, the constructed knowledge graph of university traffic incidents can be displayed. After the model output layer in step 4, the MGRel model can be used to extract triples from multi-source heterogeneous data, and then a knowledge graph of university traffic incidents can be constructed. The stored triples can be visualized using the Neo4J graph database. The knowledge graph can improve traffic control efficiency, enhance emergency response capabilities, and improve campus safety. The present invention proposes an entity-relationship joint extraction model---MGRel model, which extracts pre-defined entity relationships from unstructured text and formalizes the relationships into triples <s, r, o>, where s is the subject, o is the object, and r belongs to the target relationship set R{r1, r2, r3...}.

[0111] Evaluation indicators: The experiment of this invention uses the commonly used entity relationship joint extraction evaluation indicators to evaluate the performance of the models involved. The evaluation indicators mainly include precision P, recall R, and F1 measurement value. The calculation formulas are shown in (15), (16), and (17):

[0112]

[0113]

[0114]

[0115] In formulas (15) to (17), TP represents the number of correctly predicted triplets, FP represents the number of incorrectly predicted triplets, and FN represents the number of true triplets that were incorrectly predicted. P represents the accuracy of the prediction for the positive sample results. R represents the probability of a sample being predicted as a positive sample when it is actually positive. The F1 value represents the harmonic mean between the precision rate P and the recall rate R. In the present invention, only when a triple is formed will it be predicted as correct.

[0116] The present invention improves the F1 value by 1.3% and 0.4% on the NYT English dataset and the WebNLG English dataset, respectively, and improves the F1 value by 3.2% on the DuIE Chinese dataset.

[0117] Finally, a knowledge graph of university traffic safety incidents was constructed and visualized. The effectiveness of this method was verified on the laboratory's Tesla-P100 server platform. This method integrates PP-OCRV3 technology, the Scrapy crawler, the BERT pre-trained model, a graph convolutional neural network, and a multi-feature fusion attention mechanism to construct a knowledge graph of university traffic safety incidents. This knowledge graph can improve traffic control efficiency, enhance emergency response capabilities, and enhance the safety of teachers and students on campus.

[0118] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network, characterized in that: The steps are as follows: Step 1: Define the hierarchical system of event ontology in a top-down manner and design the ontology model of the knowledge graph of university traffic safety events; Step 2: Download the Chinese and English datasets from the official website of the public dataset, collect and organize the text and image data of real university traffic risk events, and build a knowledge graph dataset of real cases; Step 3: Build an event extraction model for the knowledge graph of on-campus traffic safety incidents in universities; use a multi-layer semantic graph convolutional neural network to learn the global semantic and syntactic graph embedding representation information of the knowledge graph dataset, and input it into the event extraction model for training; The implementation method of the event extraction model in step 3 is as follows: based on the OneRel model, the global dependency semantics and syntactic graph embedding representation of the sentence are incorporated into the initial vector generation stage; the BERT model and Bi-LSTM network are used to obtain the semantic information of the text as the input vector H e ; Construct a multi-layer semantic graph convolutional neural network to learn global dependency semantics and syntactic graph embedding information; embed the learned graph into the semantic vector G e and the input vector H e Splicing to get a new sequence vector V m , the new sequence vector V m The global semantic information is captured through the graph hybrid pooling layer; the label of each character of the sentence is obtained through the model output layer, and the final result is output; The attention mechanism guided by word-level features and text and syntactic dependency fusion features is used to assign different weight coefficients to the corpus text. The feature vectors obtained at the kth layer are expressed as: r=V i (softmax(ω Τ tanh(V i ))) Τ ; Z k =tanh(r); Among them, the tanh function is used to concatenate the vector V i Transformed to [-1,1], r represents the feature vector after the softmax function, Z k The feature vector after the two-sided tangent activation function is represented by V i Represents the concatenated sequence vector V m Any feature vector in , ω represents the trained parameter vector; the softmax() function performs normalization processing; And the vector H after the mixed pooling of the graph g k and the concatenated vector V i Perform nonlinear mapping learning to obtain the feature vector: L i k =tanh(w k [V i ,H g k ]+b k ); For the eigenvector L i k Normalize it to get the vector: The feature vector input to the k layer is Among them, w k 、b k They represent the model parameters learned by the k-1th layer attention mechanism; H g k represents the concatenated feature output after the k-1th layer fusion; exp is the exponential operation; Drawing on the gating mechanism, the feature vector represents Z k and the eigenvector M k Fusion means: C=σ(W l 1 tanh(W l 2 Z k +W l 3 M k )); D k =C·Z k +(1-C)·M k ; Among them, σ represents the Sigmoid activation function, W l 1 、W l 2 、W l 3 Represents the weight parameter of the model self-training learning, C represents the feature vector Z k and M k Normalized vector; Step 4: Build a knowledge graph of traffic safety incidents in colleges and universities and present it visually.

2. The method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network according to claim 1 is characterized in that: The ontology of the university traffic safety incident knowledge graph includes entity type O(E), attribute type O(S) and relationship type O(R). KG ={O(E),O(R),O(S)} represents a set consisting of entity types, attribute types, and relationship types, which is reflected as a hierarchical relationship between entity types and attribute types and is displayed in the form of knowledge graph triples; The ontology model of the university traffic safety incident knowledge graph takes the basic attributes of the incident, accident types, model methods, and incident handling measures as core elements, and constructs semantic knowledge associations between the core elements; The domains are divided according to the conceptual level of each ontology class: the basic attributes of an event include the location of the event, the time of the event, the casualties of the event, and the information of the personnel involved in the event; the types of accidents include two-wheeled vehicle accidents, three-wheeled vehicle accidents, and automobile accidents; the model methods include statistical analysis and machine learning; and the accident handling measures include the accident handling department, the accident handling results, and the cause of the accident.

3. The method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network according to claim 1 is characterized in that: The Chinese and English datasets include the NYT dataset, the WebNLG dataset, and the DUIE dataset. Based on risk keywords, the text and images of real university on-campus traffic risk events are collected and organized from official websites, Weibo, and Baidu News pages as part of the knowledge graph event extraction dataset. The data types include structured or unstructured text data and image information. The text data is annotated using the BIO notation method, and the image data is annotated using the Vott annotation software to establish a Chinese data set of real cases. We used crawler technology to process the text of the Chinese dataset and used the PP-OCRV3 model to extract text information from the images of the Chinese dataset as training data for the event extraction model. The process of the PP-OCRV3 model to extract text information from an image is as follows: the PP-OCRV3 model first preprocesses the input image, the text detection module in the PP-OCRV3 model marks the area to be detected in the form of coordinates, then the text recognition module identifies the text information in the marked coordinate area and outputs the recognized text information, and finally saves the text information.

4. The method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network according to any one of claims 1 to 3, characterized in that: The method for obtaining the global dependency semantics and syntactic graph embedding representation is as follows: using an LTP tool to preprocess an input sentence to obtain part-of-speech tagging information and syntactic dependency information of the preprocessed sentence sequence; performing syntactic dependency analysis to determine the syntactic structure of the sentence or the dependency semantic relationship between words in the sentence; and using a pre-trained model to obtain the global dependency semantics and syntactic graph embedding representation; The global dependency semantic and syntactic graph embedding is represented as G g = {V1, V2, E1, E2}, where V1 and V2 represent the set of semantic and syntactic nodes in a sentence, E1 and E2 represent the set of edges in the semantic and syntactic graphs in a sentence, and G g Represents the global dependency semantics and syntactic graph embedding representation in a sentence.

5. The method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network according to claim 4 is characterized in that: The implementation method of the graph mixing pooling layer is as follows: the concatenated sequence vector V is subjected to the graph mixing pooling operation. m Capture global scope information and get vector representation: Among them, V1, V2, ..., V m The concatenated sequence represents the embedding vector, and MaxPooling() represents the maximum pooling.

6. The method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network according to claim 5 is characterized in that: The target triple set Y of statement S is: Y={(h1,r1,t1),...,(h n ,r n ,t n )|h n ,r n ∈E,r n ∈R}; Among them, h n ,t n Represents the nth head entity and tail entity, r n Represents the nth relationship between entity pairs, E and R represent the entity set and relationship set respectively; statement S = {w1,w2,w3,...,w L }, where L represents the number of sentence sequences in a sentence, w1,w2,w3,...,w L Represent the sequence of each sentence separately; The BERT model performs well on the sentence S={w1,w2,w3,...,w L } is encoded to obtain the input vector of the word corresponding to the sentence: H e =[h1,h2,h3,...,h L ]=BERT[w1,w2,w3,...,w L ]{h L ∈R d }; Where d represents the embedding dimension, BERT represents the pre-training model; h1,h2,h3,...,h L Represented as a sequence vector representation of the training sentence S; Construct a multi-layer semantic graph convolutional neural network to learn the topological structure in the global semantic and syntactic dependency graph and obtain the embedded vector representation G of the text e =MultiGCN(W S (h M +h N )+b S ); Among them, W S 、b S Represent the weight parameter matrix and the trainable parameter matrix respectively. MultiGCN() represents a multi-layer semantic graph convolutional neural network that captures high-order neighborhood information between word nodes. M 、h N Embedding representation vectors representing the semantic dependency analysis graph and syntactic analysis graph respectively; The obtained text embedding vector is represented by G e With the input vector H e Perform splicing to obtain the spliced sequence vector V m =[G e ;H e ]。 7. The method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network according to claim 1 is characterized in that: The cross entropy loss function is used to calculate the loss during training in the global semantic and syntactic dependency graph embedding representation layer and the multi-feature attention mechanism layer. total =Loss g +αLoss a , and then assign weights to the losses of each layer; The loss distribution weight of each layer is implemented by the back-propagation algorithm; Among them, Loss g Represents the loss of the global semantic and syntactic dependency graph embedding representation layer, α represents the weight of the multi-feature attention mechanism layer loss, Loss a represents the multi-feature attention mechanism layer loss; The Adam optimization algorithm is used to adjust different learning rates for each different parameter.

8. The method for constructing a traffic event knowledge graph based on a multi-layer semantic graph convolutional neural network according to claim 1 is characterized in that: The implementation method of step 4 is: using the trained event extraction model to extract triples from multi-source heterogeneous data, constructing a knowledge graph of university traffic events, and using the Neo4J graph database to visualize the stored triples.

Images