Railway bridge design standard knowledge graph rapid construction method

Abstract

The invention discloses a rapid construction method for a standard knowledge graph of railway bridge design. The method comprises the steps that S1, data are collected and preprocessed; s2, a BERT pre-training model and a Bi-LSTM model are adopted to convert a text sequence in the processed data into a labeled tag sequence {y1, y2... yn}; s3, identifying and labeling entities, attributes and relationships thereof by using a conditional random field (CRF) and a graph neural network (GNN) technology, forming nodes and edges of the knowledge graph, and obtaining a complete knowledge graph (KG); and S4, adjusting the complete knowledge graph KG structure in real time by using a dynamic topological optimization algorithm. According to the method, the standard data is rapidly integrated and optimally designed, data redundancy is reduced, and newly added and changed data are effectively processed, so that the construction speed and the maintenance efficiency of the knowledge graph are improved.

Description

Technical Field

[0001] The present invention belongs to the field of engineering design information management, and specifically relates to a method for quickly constructing a railway bridge design standard knowledge graph. Background Art

[0002] With the continuous development of modern railway engineering, railway bridge design standards and specifications have become increasingly complex. These design standards involve a variety of technical requirements and specification documents, including design parameters, structural requirements, material properties, construction technology and other aspects. Since these design standards come from various specifications and technical documents, and the data formats and sources are different, data integration and management in the field of railway bridge design face significant challenges. Although the current knowledge graph construction method can handle complex data to a certain extent, it still has many shortcomings in quickly constructing and maintaining the knowledge graph of railway bridge design standards.

[0003] Traditional knowledge graph construction methods usually require a lot of manual intervention to process and integrate data, which makes the construction and update process of knowledge graphs inefficient and difficult to adapt to rapidly changing design standards. Secondly, in the existing knowledge graph construction methods, there are often duplicate records of standards and specifications and redundant information, which not only increases the complexity of data management, but also affects the query efficiency of knowledge graphs. Furthermore, the continuous updating of railway bridge design standards makes knowledge graphs face challenges in processing new additions and changes to data. Existing knowledge graph construction methods often require the reconstruction of knowledge graphs or complex adjustments, which makes it difficult to efficiently respond to changes in new standards.

[0004] Therefore, there is an urgent need for an efficient method to quickly construct a knowledge graph of railway bridge design standards to solve the problems existing in the existing technology. Summary of the invention

[0005] In order to solve the problems existing in the above-mentioned prior art, the present invention provides a method for quickly constructing a railway bridge design standard knowledge graph. The method quickly integrates and optimizes design standard data, reduces data redundancy, and effectively handles the addition and change of data, thereby improving the construction speed and maintenance efficiency of the knowledge graph.

[0006] The technical solution of the present invention is as follows:

[0007] A method for quickly constructing a railway bridge design standard knowledge graph includes the following steps:

[0008] S1, data collection and preprocessing: collect data from standard documents related to railway bridge design; remove redundant data and noise data from the collected data through data cleaning technology to obtain processed data, where:

[0009] The processed data includes multiple text sequences, the first text sequence S = {w1, w2, ..., w n},w i is the i-th word in the corresponding text sequence S, i = 1, 2, 3…n;

[0010] S2, using the BERT pre-training model and the Bi-LSTM model to convert the text sequence S into structured data with high information density, including:

[0011] S2-1, extracting embedding vectors: using the BERT pre-trained model to embed the text sequence S, and obtaining the embedding vector X of the text sequence S = {x1, x2, ..., x n};in, is the i-th word w in the text sequence S i The word vector of is a word embedding vector used to represent word w i The basic semantic information of s generate; is the sentence embedding vector used to represent word w i The role of the text sequence S, generated by the multi-layer Transformer encoder in the BERT pre-trained model; is the position embedding vector used to represent word w i The position in the text sequence S, represented by the position embedding matrix W in the BERT pre-trained model p generate;

[0012] S2-2, extract the context features corresponding to each word in the text sequence S according to the embedding vector X:

[0013] S2-2-1, traverse the embedding vector X = {x1, x2, ..., x n}, taking the word vector of each word as the input of the mapping function, the initial context features corresponding to each word in the text sequence S are calculated, and the initial context feature set H = {h0(w1),h0(w2),…,h0(w n )};

[0014] Among them, calculate the i-th word w in the text sequence S i The initial context feature h0(w i ) is calculated as:

[0015] h0(w i )=f(x i )

[0016] Where f is the mapping function used to process word w i The embedding vector xi , i∈[1,n];

[0017] S2-2-2, traverse the initial context feature set H = {h0(w1), h0(w2), ..., h0(w n )}, the initial context features of each word in the set H are input into the pre-trained BERT model respectively, and the N-layer Transformer encoder of the BERT model outputs the context features of each word and forms the context feature set H of each word S ={h N (w1),h N (w2),…,h N (w n )};

[0018] S2-3, extract the final context features based on the Bi-LSTM model:

[0019] Traverse the set H S ={h N (w1),h N (w2),…,h N (w n )}, the set H S Each context feature in is used as the input of the Bi-LSTM model to capture the bidirectional context dependency in the text sequence S, obtain the final context feature corresponding to each word in the text sequence S, and form the final context feature sequence C S ={c1,c2,…,c n};

[0020] S2-4, calculate the final context feature sequence C S The average of the vector s g ,s g is the overall semantic feature of the text sequence S: s g =avg{c1,c2,…,c n},in, 2×d he is the mean vector s g Dimensions;

[0021] S2-5, construct the relationship set R:

[0022] In the process of processing text sequences with the pre-trained BERT model, entities are identified from the text sequence, and the semantic relationship between them is inferred based on the contextual information between different entities. The semantic relationship is embedded in the relationship vector space to obtain the relationship vector, and then added to the relationship set R to obtain the relationship set R = {r1, r2, …, r k ,…r t}, where t is the total number of relationship vectors in the relationship set R, and the relationship vector r k Represents a specific relationship type or semantic connection in the text sequence S. represents the spatial dimension, d r is the dimension;

[0023] S2-6, according to the final context feature sequence C S and the mean vector s g , the attention mechanism is used to calculate the relationship vector r of each word in the text sequence S through linear mapping and nonlinear transformation k The attention alignment scores under , and form the attention alignment score set {e 1k ,e 2k ,…,e nk}, attention alignment score e ik Used to measure the final context feature c i In the relationship vector r k The weight under

[0024] S2-7, use the Softmax function to adjust the attention alignment score set {e 1k ,e 2k ,…,e nk} to normalize and obtain the relationship r of each word in the current target k The attention weights under the 1k ,α 2k ,…,α nk};

[0025] S2-8, through the attention weight set {α 1k ,α 2k ,…,α nk} for the final context feature sequence C S ={c1,c2,…,c n} Perform weighted summation to obtain the relationship vector r k The feature vector s of the text sequence S is k :

[0026]

[0027] Among them, the text sequence S is in the relation vector r k The specific semantic information under will be integrated into the feature vector s k middle, n is the total number of words in the text sequence S, and d is the feature dimension of each word;

[0028] S2-9, the feature vector s k Input to the relation gating unit, the relation gating unit controls the feature vector sk And the activation function is used to calculate the gating vector G; the gating vector G is used to control the feature vector s k Filter and obtain the filtered feature vector Among them, the gate vector G filters out the relationship vector r k Irrelevant feature information is retained and the most beneficial features for entity annotation are retained. The activation function is Sigmoid;

[0029] S2-10, the filtered feature vector Each element in is input into the hidden layer of the Bi-LSTM network, and the filtered feature vector is output The hidden state corresponding to each element in the text sequence is the hidden state corresponding to each word in the text sequence, and the hidden state set is obtained.

[0030] S2-11, the hidden state set Each hidden state in the text is input into the entity annotation layer of the Bi-LSTM network. The output of the entity annotation layer is the label of all words in the text sequence S, and the label sequence {y1, y2…y n};

[0031] S2-12, according to the label sequence {y1,y2…y n}, determine the text sequence S with the relationship vector r k The head entity and tail entity are related, and the label sequence {y1,y2…y n} for annotation, where the method for identifying the head entity and the tail entity is:

[0032] When identifying the head entity, we start from the label sequence {y1,y2…y n} find the first B-Head position as the starting position of the head entity, and end when encountering a non-entity O or the starting position B-Tail of the tail entity. This process includes the subsequent I-Head. When identifying the tail entity, start from the label sequence {y1, y2…y n}, find the position of the first B-Tail as the starting position of the tail entity, until encountering a non-entity O or B-Head, which includes the subsequent I-Tail;

[0033] S3 uses conditional random field (CRF) and graph neural network (GNN) technology to identify and annotate entities, attributes and their relationships in the design standards, forming nodes and edges of the knowledge graph to build a detailed knowledge graph. The specific steps are as follows:

[0034] S3-1, based on the labeled head entity and tail entity label sequence {y1,y2…y n}, add all identified head entities and tail entities to the entity set, which includes all entities with relationship vector r k Related entities;

[0035] S3-2, traverse the relation set R = {r1, r2, ..., r k ,…r t}, each time a relation vector in the relation set R is selected, and S2-6 to S3-1 are repeated to obtain the entity set E={E1,E2,…,E u}, E i is an entity, and u is the total number of entities;

[0036] S3-3, analyze the attributes of each entity in the entity set E, obtain the attribute set of each entity, traverse the entity set E, and obtain the attribute set A of all entities = A1, A2, ..., A u , where A i For entity E i The attribute set A i =a1,a2,…,a m ;

[0037] Among them, get entity E i The attribute set A i =a1,a2,…,a m The method is:

[0038] (1) Extract entity E from the text sequence S i The two words before and after constitute the context word set N i , where the context word set N i ={w i-2 ,w i-1 ,E i ,w i+1 ,w i+2}, where w i-2 ,w i-1 ,w i+1 ,w 1+2 Represents entity E i Two words before and after;

[0039] (2) Use the conditional random field CRF model to calculate the context word set N i Lower Entity E i The attribute set A i The conditional probability distribution of i The attribute set A i =a1,a2,…,a m , attribute set A iEach attribute in entity E i A specific feature of , where the conditional random field model formula is as follows:

[0040]

[0041] Among them, Z(N i ) is a normalization factor used to make the sum of the probabilities of all attributes equal to 1, ψ m (A i ,N i ) is a feature function used to capture the context N i With attribute set A i The mutual relationship between Indicates the multiplication of all characteristic functions;

[0042] S3-4, according to the entity set E = {E1, E2, ..., E u} and attribute set A=A1,A2,…,A u The node set N that forms the knowledge graph is {E1, E2, …, E u ,A1,A2,…,A u};

[0043] S3-5, traverse the node set N, add feature information to each node of the knowledge graph, use the vectorization technology Word2Vec to convert each node in the node set N into a vector, and obtain the vector set V(N), which is convenient for subsequent efficient storage and retrieval in the graph database;

[0044] Among them, the vector set V(N)={v(E1),v(E2),…,v(E u ),v(A1),v(A2),…,v(A u )},v(E i ) and v(A i ) are entities E i and attribute A i Convert to vector representation;

[0045] S3-6, according to the entity set E = {E1, E2, ..., E u}, construct the edges between nodes, the edges between nodes describe the relationship between entities and attributes and the relationship between entities, and convert the relationship set R = {r1, r2, ..., r t} as the representation of the edge, and each edge is vectorized to obtain the set V(R) = {v(r1), v(r2), …, v(r t )}, the vector set V(N) and the set V(R) form a complete knowledge graph KG = (V(N), V(R));

[0046] S3-7, execute S2 to S3-6 on all text sequences in the processed data described in S1 to obtain a complete knowledge graph KG;

[0047] S4, using a dynamic topology optimization algorithm to adjust the complete knowledge graph KG structure in real time, the specific steps are as follows:

[0048] S4-1, performing a preliminary analysis on the complete knowledge graph KG, evaluating the importance of each node in the complete knowledge graph KG by calculating the degree of each node, and calculating the weight of each edge according to the degree of the node, where the weight of the edge reflects the strength of the relationship between the nodes;

[0049] Among them, get node E i Degree d i The calculation formula is as follows:

[0050]

[0051] In the formula, A i,j is an element in the adjacency matrix, A i,j Represents node E i and node E j The connection between

[0052] Among them, get node E i →E j The edge weight w ij The calculation formula is as follows:

[0053]

[0054] Weight w ij Reflects node E i and node E j The strength of the relationship between i ,d j ) is the weight calculation function;

[0055] S4-2, according to the degree of each node and the weight of each edge in the complete knowledge graph KG, formulate a dynamic topology adjustment strategy to optimize the structure of the knowledge graph:

[0056] The nodes with the same or similar attributes in the complete knowledge graph KG have a degree less than d minThe nodes of are taken as redundant nodes, the redundant nodes in the complete knowledge graph KG are merged, the redundant nodes are deleted, and the connection relationship of the redundant nodes is transferred to the nodes with higher degrees to obtain the entity set E′; according to the calculation formula of the node degree, the degree of the nodes in the complete knowledge graph KG at this time is calculated, and the nodes with zero degree are taken as isolated nodes to obtain the isolated node set I, and the isolated nodes are removed from the knowledge graph according to the isolated node set I;

[0057] S4-3, according to the weight of the edge in the complete knowledge graph KG, optimize the edge, traverse the set of all edges in the complete knowledge graph KG V(R) = {v(r1), v(r2), ..., v(r k )}, delete the edges whose weights are less than or equal to the threshold in the complete knowledge graph KG. If the deleted edges disconnect some nodes in the complete knowledge graph KG and the degrees of these nodes become zero, these nodes will be regarded as isolated nodes and deleted. After the edge deletion operation in the complete knowledge graph KG, the connection relationship between the remaining nodes will change, then S4 will be re-executed to optimize the knowledge graph KG.

[0058] In S1, the standard documents related to railway bridge design include structural design specifications, material standards, construction specifications, inspection and maintenance standards, and environmental and sustainability standards. Data cleaning techniques include deleting duplicate records, processing missing values, correcting erroneous information, and standardizing data formats.

[0059] In S2-2-2, get the i-th word w in the text sequence S i The context feature h N (w i ) is as follows: the initial context feature h0(w i ) is input into the first layer of the N-layer Transformer encoder. Each layer of Transformer encoder performs self-attention calculation and nonlinear transformation on the output of the previous layer of Transformer encoder to obtain the context features output by the current layer of Transformer encoder. The context features output by the N-th layer of Transformer encoder are recorded as the following features h N (w i ), where the context feature output by the l-th layer Transformer encoder is h l (w i ):

[0060] h l (w i )=Trans(h l-1 (w i )),l∈[1,N]

[0061] Where Trans() represents the Transformer encoding operation, including self-attention calculation and nonlinear transformation, which is used to update the context features; h l-1 (w i ) is the context feature output by the l-1th layer Transformer encoder.

[0062] In S2-3, the Bi-LSTM model extracts word w i The corresponding final context feature c i The method is: N (w i ) is the input of the Bi-LSTM model. The forward LSTM and backward LSTM of the Bi-LSTM model are used to calculate h respectively. N (w i ) is the context dependency of the position in the text sequence S, and the hidden state of the forward LSTM output is obtained and the hidden state of the backward LSTM output Will and Perform concatenation and output word w i The corresponding final context feature c i , the calculation formula is:

[0063]

[0064] Where i∈[1,n], the final context feature is a two-dimensional real matrix, is the set of all real numbers, 2×d he is the final context feature c i Dimension.

[0065] In S2-6, the attention alignment score e is obtained ik The method is:

[0066] e ik =v T tanh(W r r k +W g s g +W h c i )

[0067] Where v is the attention vector; the linear mapping matrix W r Used to transform the relationship vector r k Mapped to the attention space; W g Used to average the vector s g Mapped to the attention space; W hUsed to convert the final context feature c i Mapping to the attention space.

[0068] In S2-7, get the word w in the text sequence S i The corresponding attention weight α ik The method is:

[0069]

[0070] Among them, the attention weight set {α 1k ,α 2k ,…,α nk}satisfy

[0071] S2-9 includes the following steps:

[0072] S2-9-1, obtain the gate vector G, the method is as follows:

[0073] G=σ(W g s k +b g )

[0074] Among them, W g is the weight matrix of the gating mechanism, the weight matrix W g Used to transform the feature vector s k Mapped to the gated space, b g is the bias term of the gating mechanism, and the bias term b g The offset used to control the gating mechanism, Weight matrix W g and the bias term b g It is obtained by linear transformation through the gating mechanism; σ is the activation function Sigmoid, which is used to map the input value to the range of [0,1], and the gating vector G is used to determine the feature vector s k The passing degree of each element in The value of each element in the gate vector G is between [0,1]. The closer it is to 1, the higher the importance of the element feature is, and the closer it is to 0, the smaller the contribution of the element feature is.

[0075] S2-9-2, through the gate vector G to the feature vector s k Filter, that is, multiply element by element to obtain the filtered feature vector

[0076]

[0077] in, Each element G in the gate vector G i Controls the eigenvector sk Each element s in ki The retention or suppression of G and s; ⊙ represents element-by-element multiplication. k Multiply the corresponding elements in the filter, and the feature vector Contained in the relation vector r k Provide meaningful information.

[0078] In S2-10, get the word w in the text sequence S i The corresponding hidden state The method is:

[0079]

[0080] In the formula, Contains the word w i The relation vector r in the text sequence S k The following context information, is the hidden state space, hidden is the hidden state The length of is determined by the parameters of the hidden layer of the Bi-LSTM network.

[0081] In S2-11, get the i-th word w in the text sequence S i The label method is:

[0082] In the entity annotation layer, the Softmax function is used to convert the hidden state Transformed into the probability distribution of each entity category [p1,p2,…,p q ], the calculation formula is as follows:

[0083]

[0084] In the formula, q is the number of entity categories, For the hidden state Perform a linear transformation to obtain the original score of each entity category. The softmax function converts the original score of each entity category into a probability distribution [p1, p2, ..., p q ], each element in the probability distribution is word w i is the probability of an entity category; W e is the weight matrix of the entity annotation layer; b e is the bias term;

[0085] The word w i The entity category corresponding to the maximum probability in the probability distribution of is taken as the word w i The label y of the entity category i , the calculation formula is as follows:

[0086] yi =argmax(p1,p2,…,p q )

[0087] In the formula, argmax(p1,p2,…,p q ) refers to the label of the entity category corresponding to the maximum output probability.

[0088] Preferably, in S4-2, d min =5; in S4-3, the threshold is 0.2.

[0089] The present invention has the following beneficial effects:

[0090] 1. The present invention integrates railway bridge design data from different sources, including technical specifications, industry standards, historical documents, etc., to form a comprehensive knowledge graph. The graph construction method realizes the seamless integration of data from different sources, making the graph information coverage more comprehensive and the association relationships richer, thereby ensuring the comprehensiveness of the knowledge graph.

[0091] 2. The present invention uses a dynamic topology optimization algorithm to adjust the structure of the knowledge graph, optimizes the layout of nodes and edges, effectively reduces data redundancy, improves the query efficiency of the graph, and improves the overall quality of data management.

[0092] 3. The present invention automatically identifies and deletes redundant nodes and duplicate information through topology optimization, selectively retains nodes with higher degrees, and ensures that the knowledge graph only contains information with practical significance and query value, thereby increasing the reliability of the data and ensuring the simplicity and readability of the graph; by automatically adjusting the weight and direction of the edges, the query path of the knowledge graph is more concise and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 It is a schematic diagram of the process of the present invention;

[0094] Figure 2 is a schematic flow chart of step S2 of the present invention;

[0095] Figure 3 It is a schematic diagram of the knowledge graph KG of the present invention. DETAILED DESCRIPTION

[0096] The construction method of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0097] See also Figure 1-Figure 3 The present invention provides a method for quickly constructing a railway bridge design standard knowledge graph, which specifically includes the following steps:

[0098] S1, data collection and preprocessing: collect data from standard documents related to railway bridge design; remove redundant data and noise data from the collected data through data cleaning technology to obtain processed data. Among them:

[0099] The processed data includes multiple text sequences, a text sequence S = {w1, w2, ..., w n},w i It is the i-th word in the text sequence S, i = 1, 2, 3...n. Data cleaning technology improves the quality of raw data and ensures the accuracy and consistency of raw data.

[0100] The standard documents related to railway bridge design include structural design specifications, material standards, construction specifications, inspection and maintenance standards, and environmental and sustainability standards. Data cleaning techniques include deleting duplicate records, processing missing values, correcting erroneous information, and standardizing data formats.

[0101] S2, using the BERT pre-trained model and the Bi-LSTM model to perform deep semantic analysis on the processed data, extract the key elements, technical parameters and relationships of the text sequence S in the processed data (design standards), and convert the processed data into structured data with high information density. It includes the following steps:

[0102] S2-1, extracting embedding vectors: using the BERT pre-trained model to embed a text sequence S in the processed data, and obtaining an embedding vector X = {x1, x2, ..., x n};

[0103] in, is the i-th word w in the text sequence S i The word vector of is a word embedding vector used to represent word w i The basic semantic information of s generate; is the sentence embedding vector used to represent word w i The role of the text sequence S, generated by the multi-layer Transformer encoder in the BERT pre-trained model; is the position embedding vector used to represent word w i The position in the text sequence S, represented by the position embedding matrix W in the BERT pre-trained model p generate;

[0104] S2-2, extract the context features corresponding to each word in the text sequence S according to the embedding vector X:

[0105] S2-2-1, traverse the embedding vector X = {x1, x2, ..., x n}, taking the word vector of each word as the input of the mapping function, the initial context features corresponding to each word in the text sequence S are calculated, and the initial context feature set H = {h0(w1),h0(w2),…,h0(w n )};

[0106] Among them, calculate the i-th word w in the text sequence S i The initial context feature h0(w i ) is calculated as:

[0107] h0(w i )=f(x i )

[0108] Where f is the mapping function used to process word w i The embedding vector x i , i∈[1,n].

[0109] S2-2-2, traverse the initial context feature set H = {h0(w1), h0(w2), ..., h0(w n )}, the initial context features of each word in the set H are input into the pre-trained BERT model respectively, and the N-layer Transformer encoder of the BERT model outputs the context features of each word and forms the context feature set H of each word S ={h N (w1),h N (w2),…,h N (w n )}.

[0110] Get the i-th word w in the text sequence S i The context feature h N (w i ) is as follows: the initial context feature h0(w i ) is input into the first layer of the N-layer Transformer encoder. Each layer of Transformer encoder performs self-attention calculation and nonlinear transformation on the output of the previous layer of Transformer encoder to obtain the context features output by the current layer of Transformer encoder. The context features output by the N-th layer of Transformer encoder are recorded as the following features h N (w i ), where the context feature output by the l-th layer Transformer encoder is h l (w i ):

[0111] hl (w i )=Trans(h l-1 (w i )),l∈[1,N]

[0112] Where Trans() represents the Transformer encoding operation, including self-attention calculation and nonlinear transformation, which is used to update the context features; h l-1 (w i ) is the context feature output by the l-1th layer Transformer encoder.

[0113] S2-3, extract the final context features based on the Bi-LSTM model:

[0114] Traverse the set H S ={h N (w1),h N (w2),…,h N (w n )}, the set H S Each context feature in is used as the input of the Bi-LSTM model to capture the bidirectional context dependency in the text sequence S, obtain the final context feature corresponding to each word in the text sequence S, and form the final context feature sequence C S ={c1,c2,…,c n}.

[0115] Among them, the Bi-LSTM model extracts the word w i The corresponding final context feature c i The method is: N (w i ) is the input of the Bi-LSTM model. The forward LSTM and backward LSTM of the Bi-LSTM model are used to calculate h respectively. N (w i ) is the context dependency of the position in the text sequence S, and the hidden state of the forward LSTM output is obtained and the hidden state of the backward LSTM output Will and Perform concatenation and output word w i The corresponding final context feature c i , the calculation formula is:

[0116]

[0117] Where i∈[1,n], the final context feature is a two-dimensional real matrix, is the set of all real numbers, 2×dhe is the final context feature c i Dimension.

[0118] S2-4, calculate the final context feature sequence G S The average of the vector s g ,s g is the overall semantic feature of the text sequence S: s g =avg{c1,c2,…,c n},in, 2×d he is the mean vector s g Dimension.

[0119] S2-5, construct the relationship set R:

[0120] In the process of processing text sequences with the pre-trained BERT model, entities are identified from the text sequence, and the semantic relationship between them is inferred based on the contextual information between different entities. The semantic relationship is embedded in the relationship vector space to obtain the relationship vector, and then added to the relationship set R to obtain the relationship set R = {r1, r2, …, r k ,…r t}, where t is the total number of relationship vectors in the relationship set R, and the relationship vector r k Represents a specific relationship type or semantic connection in the text sequence S. represents the spatial dimension, d r Is the dimension.

[0121] The relationship set R will continue to expand as new relationship vectors are discovered. Whenever the pre-trained BERT model discovers an unrecognized relationship vector when processing text, it will add this new relationship vector to the relationship set R, thereby maintaining the dynamic expansion of the knowledge graph.

[0122] S2-6, according to the final context feature sequence C s and the mean vector s g , the attention mechanism is used to calculate the relationship vector r of each word in the text sequence S through linear mapping and nonlinear transformation k The attention alignment scores under , and form the attention alignment score set {e 1k ,e 2k ,…,e nk}, attention alignment score e ik Used to measure the final context feature c i In the relationship vector r k The weight under which the attention alignment score e is obtained ik The method is:

[0123] eik =v T tanh(W r r k +W g s g +W h c i )

[0124] Where v is the attention vector, Linear mapping matrix W r Used to transform the relationship vector r k Mapped to the attention space, W g Used to average the vector s g Mapped to the attention space, W h Used to convert the final context feature c i Mapped to the attention space, is the spatial dimension of the attention vector v, d att is the dimension of the attention space, W g and W h The spatial dimension of 2×d he is the mean vector s g The dimension of .

[0125] S2-7, use the Softmax function to adjust the attention alignment score set {e 1k ,e 2k ,…,e nk} to normalize and obtain the relationship r of each word in the current target k The attention weights under the 1k ,α 2k ,…,α nk};

[0126] Get word w in text sequence S i The corresponding attention weight α ik The method is:

[0127]

[0128] Among them, the attention weight set {α 1k ,α 2k ,…,α nk}satisfy

[0129] S2-8, through the attention weight set {α 1k ,α 2k ,…,α nk} for the final context feature sequence C S={c1,c2,…,c n} Perform weighted summation to obtain the relationship vector r k The feature vector s of the text sequence S is k :

[0130]

[0131] Among them, the text sequence S is in the relation vector r k The specific semantic information under will be integrated into the feature vector s k middle, n is the total number of words in the text sequence S, and d is the feature dimension of each word;

[0132] S2-9, the feature vector s k Input to the relation gating unit, the relation gating unit controls the feature vector s k And the activation function is used to calculate the gating vector G; the gating vector G is used to control the feature vector s k Filter and obtain the filtered feature vector Among them, the gate vector G filters out the relationship vector r k Irrelevant feature information is removed and the most beneficial features for entity annotation are retained. The activation function is Sigmoid. The following steps are included:

[0133] S2-9-1, obtain the gate vector G, the method is as follows:

[0134] G=σ(W g s k +b g )

[0135] Among them, W g is the weight matrix of the gating mechanism, the weight matrix W g Used to transform the feature vector s k Mapped to the gated space, b g is the bias term of the gating mechanism, and the bias term b g The offset used to control the gating mechanism, Weight matrix W g and the bias term b g It is obtained by linear transformation through the gating mechanism; σ is the activation function Sigmoid, which is used to map the input value to the range of [0,1], and the gating vector G is used to determine the feature vector s k The passing degree of each element in The value of each element in the gate vector G is between [0, 1]. The closer it is to 1, the higher the importance of the element feature is, and the closer it is to 0, the smaller the contribution of the element feature is.

[0136] S2-9-2, through the gate vector G to the feature vector s k Filter, that is, multiply element by element to obtain the filtered feature vector

[0137]

[0138] in, Each element G in the gate vector G i Controls the eigenvector s k Each element s in ki The retention or suppression of G and s; ⊙ represents element-by-element multiplication. k Multiply the corresponding elements in the filter, and the feature vector Contained in the relation vector r k Provide meaningful information.

[0139] S2-10, the filtered feature vector Each element in is input into the hidden layer of the Bi-LSTM network, and the filtered feature vector is output The hidden state corresponding to each element in the text sequence is the hidden state corresponding to each word in the text sequence, and the hidden state set is obtained.

[0140] Among them, get the word w in the text sequence S i The corresponding hidden state The method is:

[0141]

[0142] In the formula, Contains the word w i The relation vector r in the text sequence S k The following context information, is the hidden state space, hidden is the hidden state The length of is determined by the parameters of the hidden layer of the Bi-LSTM network.

[0143] S2-11, the hidden state set Each hidden state in the text is input into the entity annotation layer of the Bi-LSTM network. The output of the entity annotation layer is the label of all words in the text sequence S, and the label sequence {y1, y2…y n}.

[0144] Among them, get the i-th word w in the text sequence S i The label method is:

[0145] In the entity annotation layer, the Softmax function is used to convert the hidden state Transformed into the probability distribution of each entity category [p1,p2,…,p q ], the calculation formula is as follows:

[0146]

[0147] In the formula, q is the number of entity categories, For the hidden state Perform a linear transformation to obtain the original score of each entity category. The softmax function converts the original score of each entity category into a probability distribution [p1, p2, ..., p q ], each element in the probability distribution is word w i is the probability of an entity category; W e is the weight matrix of the entity annotation layer; b e is the bias term.

[0148] The word w i The entity category corresponding to the maximum probability in the probability distribution of is taken as the word w i The label y of the entity category i , the calculation formula is as follows:

[0149] y i =argmax(p1,p2,…,p q )

[0150] In the formula, argmax(p1,p2,…,p q ) refers to the label of the entity category corresponding to the maximum output probability.

[0151] S2-12, according to the label sequence {y1,y2…y n}, determine the text sequence S with the relationship vector r k The head entity and tail entity are related, and the label sequence {y1,y2…y n} for annotation, where the method for identifying the head entity and the tail entity is:

[0152] When identifying the head entity, we start from the label sequence {y1,y2…y n} find the first B-Head position as the starting position of the head entity, and end when encountering O (non-entity) or the starting position B-Tail of the tail entity. This process includes the subsequent I-Head (continuation of the head entity); when identifying the tail entity, start from the label sequence {y1, y2…y n} finds the position of the first B-Tail as the starting position of the tail entity until it encounters O (non-entity) or B-Head, which includes the subsequent I-Tail (continuation of the tail entity).

[0153] S3, using conditional random field (CRF) and graph neural network (GNN) technology, identifies and annotates entities, attributes and their relationships in the design standards to form nodes and edges of the knowledge graph to build a detailed knowledge graph. The specific steps are as follows:

[0154] S3-1, based on the labeled head entity and tail entity label sequence {y1,y2…y n}, add all identified head entities and tail entities to the entity set, which includes all entities with relationship vector r k Related entities.

[0155] S3-2, traverse the relation set R = {r1, r2, ..., r k ,…r t}, each time a relation vector in the relation set R is selected, and S2-6 to S3-1 are repeated to obtain the entity set E={E1,E2,…,E u}, E i is an entity, and u is the total number of entities.

[0156] S3-3, analyze the attributes of each entity in the entity set E, obtain the attribute set of each entity, traverse the entity set E, and obtain the attribute set A of all entities = A1, A2, ..., A u , where A i For entity E i The attribute set A i =a1,a2,…,a m .

[0157] Among them, get entity E i The attribute set A i =a1,a2,…,a m The method is:

[0158] (1) Extract entity E from the text sequence S i The two words before and after constitute the context word set N i , where the context word set N i ={w i-2 ,w i-1 ,E i ,w i+1 ,w i+2}, where w i-2 ,w i-1 ,w i+1 ,w i+2 Represents entity E i Two words before and after;

[0159] (2) Use the conditional random field (CRF) model to calculate the context word set N i Lower Entity E i The attribute set A i The conditional probability distribution of i The attribute set A i =a1,a2,…,a m , attribute set A i Each attribute in entity E i A specific feature of , where the conditional random field model formula is as follows:

[0160]

[0161] Among them, Z(N i ) is a normalization factor used to make the sum of the probabilities of all attributes equal to 1, ψ m (A i ,N i ) is a feature function used to capture the context N i With attribute set A i The mutual relationship between It means to multiply all characteristic functions.

[0162] S3-4, according to the entity set E = {E1, E2, ..., E u} and attribute set A=A1,A2,…,A u The node set N that forms the knowledge graph is {E1, E2, …, E u ,A1,A2,…,A u}.

[0163] S3-5, traverse the node set N, add feature information (including node ID, type, description) to each node of the knowledge graph, so as to quickly locate and identify the node in the query of the knowledge graph. Use the vectorization technology Word2Vec to convert each node in the node set N into a vector, and obtain the vector set V(N), which is convenient for subsequent efficient storage and retrieval in the graph database.

[0164] Among them, the vector set V(N)={v(E1),v(E2),…,v(E u ),v(A1),v(A2),…,v(A u )},v(E i ) and v(A i ) are entities E i and attribute A i Convert to a vector representation.

[0165] S3-6, according to the entity set E = {E1, E2, ..., E u} constructs the edges between nodes. The edges between nodes describe the relationship between entities and attributes and between entities. t} as the representation of the edge, and each edge is vectorized to obtain the set V(R) = {v(r1), v(r2), …, v(r t )}, the vector set V(N) and the set V(R) form a complete knowledge graph KG = (V(N), V(R)).

[0166] S3-7, execute S2 to S3-6 on all text sequences in the processed data described in S1 to obtain a complete knowledge graph KG.

[0167] The construction of nodes in the complete knowledge graph KG ensures that all extracted information is included, while the construction of edges effectively connects this information, making the knowledge graph efficient and usable in query and reasoning.

[0168] S4, using a dynamic topology optimization algorithm to adjust the complete knowledge graph KG structure in real time, the dynamic topology optimization algorithm optimizes the layout of nodes and edges, reduces data redundancy, and automatically processes the addition and change of nodes in the complete knowledge graph KG to ensure efficient query and maintenance of the knowledge graph. The specific steps are as follows:

[0169] S4-1, perform preliminary analysis on the complete knowledge graph KG, evaluate the importance of each node in the complete knowledge graph KG by calculating the number of connections (degree) of each node, and identify potential structural problems; calculate the weight of each edge according to the degree of the node, the weight of the edge reflects the strength of the relationship between the nodes, the lower the weight of the edge, the lower the association strength between the nodes in the complete knowledge graph KG.

[0170] Among them, get node E i Degree d i The calculation formula is as follows:

[0171]

[0172] In the formula, A i,j is an element in the adjacency matrix, A i,j Represents node E i and node E j The connection between.

[0173] Among them, get node E i →E j The edge weight w ij The calculation formula is as follows:

[0174]

[0175] Weight w ij Reflects node E i and node E j The strength of the relationship between i ,d j ) is the weight calculation function.

[0176] S4-2, according to the degree of each node and the weight of each edge in the complete knowledge graph KG, formulate a dynamic topology adjustment strategy to optimize the structure of the knowledge graph:

[0177] The nodes with the same or similar attributes in the complete knowledge graph KG have a degree less than d min =5 as redundant nodes, merge the redundant nodes in the complete knowledge graph KG, delete the redundant nodes, and transfer the connection relationship of the redundant nodes to the nodes with higher degrees to obtain the entity set E′; according to the calculation formula of the node degree, calculate the degree of the nodes in the complete knowledge graph KG at this time, take the nodes with zero degree as isolated nodes, and obtain the isolated node set I; remove the isolated nodes from the knowledge graph according to the isolated node set I, so as to reduce the complexity of the complete knowledge graph KG.

[0178] Among them, redundant nodes are nodes with the same or similar attributes in the knowledge graph. Redundant nodes are entities of the same type or have repeated information.

[0179] S4-3, according to the weight of the edge in the complete knowledge graph KG, optimize the edge, traverse the set of all edges in the complete knowledge graph KG V(R) = {v(r1), v(r2), ..., v(r k )}, delete the edges with weights less than or equal to the threshold in the complete knowledge graph KG, where the threshold = 0.2. If the deleted edges disconnect some nodes in the complete knowledge graph KG and the degree of these nodes becomes zero, these nodes are regarded as isolated nodes and deleted. After deleting the edges in the complete knowledge graph KG, the connection relationship between the remaining nodes will change, and S4 will be re-executed to optimize the knowledge graph KG. Among them, deleting edges with lower weights and redundant nodes helps to reduce noise and the complexity of the complete knowledge graph KG, thereby improving query efficiency.

[0180] The performance of the complete knowledge graph KG is verified by evaluating the query performance, data redundancy and information aggregation of the optimized complete knowledge graph KG:

[0181] According to the benchmark query, the query response time of the complete knowledge graph KG is calculated to verify its query efficiency and maintenance cost, where the total time T to execute all benchmark queries is calculated. q The method is:

[0182]

[0183] Among them, t a is the response time of the ath query, h is the total number of queries, query performance: the response time of simple queries should be within 1 second, the response time of complex queries should not exceed 5 seconds, the query throughput should reach more than 1000 times / second, and the query response time should be stable with a standard deviation of less than 0.5 seconds.

[0184] The data redundancy of the complete knowledge graph KG: node redundancy and relationship redundancy are controlled below 5%, and attribute redundancy is less than 10%, thereby reducing storage waste and improving query efficiency.

[0185] The information aggregation degree of the complete knowledge graph KG: the average degree of each node reaches more than 10 times, the edge aggregation and information coverage are high, the clustering coefficient is close to 1, and the information of the complete knowledge graph KG is closely connected and effectively covered.

[0186] In summary, the complete knowledge graph KG meets the set quality standards and supports efficient query and maintenance operations.

Claims

1. A method for quickly constructing a railway bridge design standard knowledge graph, characterized in that: The following steps are involved: S1, data collection and preprocessing: collect data from standard documents related to railway bridge design; remove redundant data and noise data from the collected data through data cleaning technology to obtain processed data, where: The processed data includes multiple text sequences, the first text sequence S = {w1, w2, ..., w n },w i is the i-th word in the corresponding text sequence S, i = 1, 2, 3…n; S2, using the BERT pre-training model and the Bi-LSTM model to convert the text sequence S into structured data with high information density, including: S2-1, extracting embedding vectors: using the BERT pre-trained model to embed the text sequence S, and obtaining the embedding vector X of the text sequence S = {x1, x2, ..., x n }; where x i is the i-th word w in the text sequence S i The word vector of S2-2, extract the context features corresponding to each word in the text sequence S according to the embedding vector X: S2-2-1, traverse the embedding vector X = {x1, x2, ..., x n }, taking the word vector of each word as the input of the mapping function, the initial context features corresponding to each word in the text sequence S are calculated, and the initial context feature set H = {h0(w1),h0(w2),…,h0(w n )}; S2-2-2, traverse the initial context feature set H = {h0(w1), h0(w2), ..., h0(w n )}, the initial context features of each word in the set H are input into the pre-trained BERT model respectively, and the N-layer Transformer encoder of the BERT model outputs the context features of each word and forms the context feature set H of each word S ={h N (w1),h N (w2),…,h N (w n )}; S2-3, extract the final context features based on the Bi-LSTM model: Traverse the set H S ={h N (w1),h N (w2),…,h N (w n )}, the set H S Each context feature in is used as the input of the Bi-LSTM model to capture the bidirectional context dependency in the text sequence S, obtain the final context feature corresponding to each word in the text sequence S, and form the final context feature sequence C S ={c1,c2,…,c n }; S2-4, calculate the final context feature sequence C S The average of the vector s g ,s g is the overall semantic feature of the text sequence S: g =avg{c1,c2,…,c n }; S2-5, construct the relationship set R: In the process of processing text sequences with the pre-trained BERT model, entities are identified from the text sequence, and the semantic relationship between them is inferred based on the contextual information between different entities. The semantic relationship is embedded in the relationship vector space to obtain the relationship vector, and then added to the relationship set R to obtain the relationship set R = {r1, r2, …, r k ,…r t }, where t is the total number of relation vectors in the relation set R, and the relation vector r k Represents a specific relationship type or semantic connection in the text sequence S; S2-6, according to the final context feature sequence C S and the mean vector s g , the attention mechanism is used to calculate the relationship vector r of each word in the text sequence S through linear mapping and nonlinear transformation k The attention alignment scores under , and form the attention alignment score set {e 1k ,e 2k ,…,e nk }, attention alignment score e ik Used to measure the final context feature c i In the relationship vector r k The weight under S2-7, use the Softmax function to adjust the attention alignment score set {e 1k ,e 2k ,…,e nk } to normalize and obtain the relationship r of each word in the current target k The attention weights under , and form the attention weight set {α 1k ,α 2k ,…,α nk }; S2-8, through the attention weight set {α 1k ,α 2k ,…,α nk } for the final context feature sequence C S ={c1,c2,…,c n } Perform weighted summation to obtain the relationship vector r k The feature vector s of the text sequence S is k : Among them, the text sequence S is in the relation vector r k The specific semantic information under will be integrated into the feature vector s k In it, n is the total number of words in the text sequence S, and d is the feature dimension of each word; S2-9, the feature vector s k Input to the relation gating unit, the relation gating unit controls the feature vector s k And the activation function is used to calculate the gating vector G; the gating vector G is used to control the feature vector s k Filter and obtain the filtered feature vector Among them, the gate vector G filters out the relationship vector r k Irrelevant feature information is retained and the most beneficial features for entity annotation are retained. The activation function is Sigmoid; S2-10, the filtered feature vector Each element in is input into the hidden layer of the Bi-LSTM network, and the filtered feature vector is output The hidden state corresponding to each element in the text sequence is the hidden state corresponding to each word in the text sequence, and the hidden state set is obtained. S2-11, the hidden state set Each hidden state in the text is input into the entity annotation layer of the Bi-LSTM network. The output of the entity annotation layer is the label of all words in the text sequence S, and the label sequence {y1, y2…y n }; S2-12, according to the label sequence {y1,y2…y n }, determine the text sequence S with the relationship vector r k The head entity and tail entity are related, and the label sequence {y1,y2…y n } for marking; S3, using conditional random field (CRF) and graph neural network (GNN) technology, identifies and annotates entities, attributes and their relationships in the design standards to form nodes and edges of the knowledge graph to build a complete knowledge graph KG; S4, using a dynamic topology optimization algorithm to adjust the complete knowledge graph KG structure in real time, the specific steps are as follows: S4-1, performing a preliminary analysis on the complete knowledge graph KG, evaluating the importance of each node in the complete knowledge graph KG by calculating the degree of each node, and calculating the weight of each edge according to the degree of the node, where the weight of the edge reflects the strength of the relationship between the nodes; S4-2, according to the degree of each node and the weight of each edge in the complete knowledge graph KG, formulate a dynamic topology adjustment strategy to optimize the structure of the knowledge graph: The nodes with the same or similar attributes in the complete knowledge graph KG have a degree less than d min The nodes of are taken as redundant nodes, the redundant nodes in the complete knowledge graph KG are merged, the redundant nodes are deleted, and the connection relationship of the redundant nodes is transferred to the nodes with higher degrees to obtain the entity set E′; according to the calculation formula of the node degree, the degree of the nodes in the complete knowledge graph KG at this time is calculated, and the nodes with zero degree are taken as isolated nodes to obtain the isolated node set I, and the isolated nodes are removed from the knowledge graph according to the isolated node set I; S4-3, according to the weight of the edge in the complete knowledge graph KG, optimize the edge, traverse the set of all edges in the complete knowledge graph KG V(R) = {v(r1), v(r2), ..., v(r k )}, delete the edges whose weights are less than or equal to the threshold in the complete knowledge graph KG. If the deleted edges disconnect some nodes in the complete knowledge graph KG and the degrees of these nodes become zero, these nodes will be regarded as isolated nodes and deleted. After the edge deletion operation in the complete knowledge graph KG, the connection relationship between the remaining nodes will change, then S4 will be re-executed to optimize the knowledge graph KG.

2. A method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1, characterized in that: In S1, the standard documents related to railway bridge design include structural design specifications, material standards, construction specifications, inspection and maintenance standards, and environmental and sustainability standards. Data cleaning techniques include deleting duplicate records, processing missing values, correcting erroneous information, and standardizing data formats.

3. The method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1 is characterized by: In S2-2-2, get the i-th word w in the text sequence S i The context feature h N (w i ) is as follows: the initial context feature h0(w i ) is input into the first layer of the N-layer Transformer encoder. Each layer of Transformer encoder performs self-attention calculation and nonlinear transformation on the output of the previous layer of Transformer encoder to obtain the context features output by the current layer of Transformer encoder. The context features output by the N-th layer of Transformer encoder are recorded as the following features h N (w i ), where the context feature output by the l-th layer Transformer encoder is h l (w i ): h l (w i )=Trans(h l-1 (w i )),l∈[1,N] Where Trans() represents the Transformer encoding operation, including self-attention calculation and nonlinear transformation, which is used to update the context features; h l-1 (w i ) is the context feature output by the l-1th layer Transformer encoder.

4. The method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1 is characterized by: In S2-3, the Bi-LSTM model extracts word w i The corresponding final context feature c i The method is: N (w i ) is the input of the Bi-LSTM model. The forward LSTM and backward LSTM of the Bi-LSTM model are used to calculate h respectively. N (w i ) is the context dependency of the position in the text sequence S, and the hidden state of the forward LSTM output is obtained and the hidden state of the backward LSTM output Will and Perform concatenation and output word w i The corresponding final context feature c i , the calculation formula is: Where i∈[1,n], the final context feature is a two-dimensional real matrix, is the set of all real numbers, 2×d he is the final context feature c i Dimension.

5. The method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1 is characterized in that: In S2-6, the attention alignment score e is obtained ik The method is: e ik =v T tanh(W r r k +W g s g +W h c i ) Where v is the attention vector; the linear mapping matrix W r Used to transform the relationship vector r k Mapped to the attention space; W g Used to average the vector s g Mapped to the attention space; W h Used to convert the final context feature c i Mapping to the attention space.

6. The method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1 is characterized by: In S2-7, get the word w in the text sequence S i The corresponding attention weight α ik The method is: Among them, the attention weight set {α 1k ,α 2k ,…,α nk }satisfy 7. A method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1, characterized in that S2-9 includes the following steps: S2-9-1, obtain the gate vector G, the method is as follows: G=σ(W g s k +b g ) Among them, W g is the weight matrix of the gating mechanism, the weight matrix W g Used to transform the feature vector s k Mapped to the gated space, b g is the bias term of the gating mechanism, and the bias term b g The offset used to control the gating mechanism, Weight matrix W g and the bias term b g Obtained by linear transformation through the gating mechanism; σ is the activation function Sigmoid, which is used to map the input value to the range of [0,1], and the gating vector G is used to determine the feature vector s k The passing degree of each element in The value of each element in the gate vector G is between [0,1]. The closer it is to 1, the higher the importance of the element feature is, and the closer it is to 0, the smaller the contribution of the element feature is. S2-9-2, through the gate vector G to the feature vector s k Filter, that is, multiply element by element to obtain the filtered feature vector in, Each element G in the gate vector G i Controls the eigenvector s k Each element s in ki The retention or suppression of G and s; ⊙ represents element-by-element multiplication. k Multiply the corresponding elements in the filter, and the feature vector Contained in the relation vector r k Provide meaningful information.

8. The method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1 is characterized by: In S2-10, get the word w in the text sequence S i The corresponding hidden state The method is: In the formula, Contains the word w i The relation vector r in the text sequence S k The following context information, is the hidden state space, hidden is the hidden state The length of is determined by the parameters of the hidden layer of the Bi-LSTM network.

9. The method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1 is characterized by: In S2-11, get the i-th word w in the text sequence S i The label method is: In the entity annotation layer, the Softmax function is used to convert the hidden state Transformed into the probability distribution of each entity category [p1,p2,…,p q ], the calculation formula is as follows: In the formula, q is the number of entity categories, For the hidden state Perform a linear transformation to obtain the original score of each entity category. The softmax function converts the original score of each entity category into a probability distribution [p1, p2, ..., p q ], each element in the probability distribution is word w i is the probability of an entity category; W e is the weight matrix of the entity annotation layer; b e is the bias term; The word w i The entity category corresponding to the maximum probability in the probability distribution of is taken as the word w i The label y of the entity category i , the calculation formula is as follows: y i =argmax(p1,p2,…,p q ) In the formula, argmax(p1,p2,…,p q ) refers to the label of the entity category corresponding to the maximum output probability.

10. The method for rapidly constructing a railway bridge design standard knowledge graph according to claim 1, characterized in that: In S4-2, d min =5; in S4-3, the threshold is 0.2.

Images