A method for extracting entity relations in high-speed train bogie design standards

Abstract

This invention relates to a method for extracting entity relations from high-speed train bogie design standards. Based on the existing BERT-BiLSTM model, an improved RoBERTa-wwm-BISRU model optimized for bogie datasets and based on a dual attention concentration mechanism is proposed, forming the BRENet model. The dual attention concentration mechanism is combined with the RoBERTa-wwm-BISRU model, and the model loss function is optimized for the structure of the bogie dataset. The addition of two attention concentration layers improves the model's ability to understand contextual semantics. The improved RoBERTa-wwm model enhances word and sentence segmentation and encoding capabilities, thereby improving entity relation extraction performance on small datasets. Furthermore, the model is optimized for the bogie dataset, improving the model loss function and increasing the model's prediction accuracy.

Description

Technical Field

[0001] This invention relates to a method for extracting design standard entity relationships, specifically a method for extracting design standard entity relationships for high-speed train bogies. Background Technology

[0002] The design, manufacturing, and maintenance of high-speed train bogies involve a vast amount of data and knowledge, scattered across various business systems, including technical standards, design parameters, process parameters, and fault information. Complex relationships exist between these data, such as those between manufacturing parameters and fault information, and between design parameters and technical standards. By extracting relationships between entities, useful information can be extracted from this massive dataset, revealing the connections and providing crucial references for the design, manufacturing, and maintenance of high-speed train bogies. Therefore, relationship extraction is extremely important in the field of high-speed train bogies. Current research largely focuses on BERT-LSTM entity relationship extraction algorithms. The principle primarily involves using BERT or a language model improved from BERT to extract text features and obtain word-granular vector matrices. Then, a Long Short-Term Memory (LSTM) network is used to extract the word-to-word relationships between the input sentence and its context, thus outputting relationship labels. Current research focuses on reducing parameters using the BERT model, adding attention focus, and improving the model using bidirectional LSTM neural networks. However, these algorithm improvements all require training on massive datasets, and no research has yet proposed a specific method for extracting entity relationships from small datasets specifically for the high-speed train bogie domain.

[0003] Effective utilization and analysis of data from the design and manufacturing processes of high-speed train bogies can provide significant value for bogie performance optimization. However, the multi-source heterogeneity of data accumulated at various stages and the lack of proprietary corpora result in a lack of data completeness and consistency, leading to inaccurate entity relation extraction and hindering the effective realization of data value. Summary of the Invention

[0004] This invention provides the following technical solutions:

[0005] A method for extracting entity relationships from high-speed train bogie design standards includes steps S1-S2, wherein...

[0006] S1: Extract text data, perform data augmentation using synonym replacement and Chinese-English-Chinese translation, and create a dataset;

[0007] S2: Construct a network model and train the entity relationship extraction model.

[0008] Step S2 includes steps S21-S25:

[0009] S21: Construct an improved RoBERTa-wwm pre-trained model;

[0010] S22: Construct a self-attention concentration mechanism layer;

[0011] S23: Construct a bidirectional SRU network module;

[0012] S24: Construct a multi-head attention concentration mechanism layer;

[0013] S25: Improved loss function for the model.

[0014] In step S21, when constructing the improved RoBERTa-wwm pre-trained model, the backbone network of the RoBERTa model uses a Transformer encoder and a GELU non-linear activation function; the word embedding size is specified as E, the number of encoder layers as L, and the hidden layer size as H; the hidden layer size H is set to 1024, the number of attention heads is set to 16, the encoding layer size is set to 24, and the dataset size for each training session is 2K; RoBERTa uses a hybrid character and word-level encoding method to encode the input sentence to learn high-frequency byte combinations.

[0015] In step S22, the self-attention mechanism consists of three parts: the output word vectors of RoBERTa-wwm, the Tanh function, and the Softmax activation function; the calculation process of the self-attention concentration mechanism network structure is shown in equations (3) to (5):

[0016] M=tanh(H) (3)

[0017]

[0018] r = Hα T (5)

[0019] Where: In equation (3): H is the output of the first layer RoBERTa-wwm, and its dimension is n×Sw, where n is the dimension of the word vector matrix and Sw is the length of the word vector sequence;

[0020] In equation (4): α is the word vector matrix activated by the softmax activation function, and T represents the transpose of the matrix;

[0021] In equation (5): r is the output sentence weight vector;

[0022] The H matrix is ​​first processed by the Tanh activation function to obtain the word vector matrix M, and then processed by a softmax layer to obtain the weight matrix. At this point, the sum of all weights in α is 1. Multiplying the H word vector matrix by the normalized weight matrix yields the corresponding weighted global temporal feature vector.

[0023] In step S23, a bidirectional SRU network is used, with the forward SRU and backward SRU networks connected together, so that the model can obtain both forward semantic information and backward semantic information.

[0024] In step S24, the construction process of multi-head attention is as follows: Given an input sequence X = [x1, x2, ..., x...] n ], where n represents the sequence length, and each x i It is a vector representation; the multi-head attention mechanism maps the input sequence to h subspaces, each subspace having a different weight matrix W. Q W K W V These are used to calculate the query, key, and value vectors, respectively; for each subspace i, the query vector Q is calculated. i =XW i Q Key vector K i =XW i K Sum vector V i =XW i V Among them, W i Q W i K W i V It is d model ×d k The matrix, d model d is the dimension representing the vector in the model. k This is the dimension of the key vector; then, the attention score for the i-subspace is obtained by taking the dot product of the query vector and the key vector. And scale the attention score; next, scale the attention score S i The attention weights W are obtained by normalization using the Softmax function. i =softmax(S i ), representing the contribution of each position to subspace i; finally, the value vector V i And attention weight W i Perform a weighted average to obtain the output of multi-head attention. Finally, the outputs O1, O2, ..., O from multiple subspaces are... hConcatenating these elements forms the final output vector O = [O1, O2, ..., O2]. h ], where [O1,O2,...,O h ] indicates a vector concatenation operation;

[0025] S25: For the constructed high-speed train bogie relationship extraction dataset, the loss function of the overall BRENet model is improved, so that the pre-trained model can better adapt to the characteristics of the high-speed train bogie dataset. The improved loss function is shown in Equation (6):

[0026]

[0027] Where: y i Represents the actual value. 'c' represents the predicted value, and 'c' represents the length of the statement sequence. This improvement allows the model to pay more attention to the role of the original variables, that is, to let the original variables and the predicted variables jointly influence the prediction results.

[0028] Step S1 specifically includes steps S11-S14;

[0029] Step S11 is to extract data, which includes the following steps: collecting knowledge text about high-speed train bogies and obtaining the text as a data source through the network.

[0030] Step S12 is data augmentation; it includes the following steps:

[0031] (1) Data augmentation using synonym replacement. This method replaces a random number of words in a sentence with their synonyms. This method constructs new sentences by changing the synonyms in the sentences. (2) Translate the text into different languages ​​and then into Chinese to increase the detail of the corpus data. (3) Use the ALBERT-BILSTM+CNN-CRF model to extract text entities. This model uses BILSTM and CNN to extract features in parallel, and then labels the extracted entities.

[0032] Step S13: After entity extraction, manually label the data generated in step S12 with relationships, and label it in the format of a supervised learning dataset to form the dataset required by this invention.

[0033] Step S14: Control the sentence length. If the length of the input sentence is less than 20 characters, add zeros to both ends of the sentence to make the sentence length reach 20 characters.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] (1) The inventors found in experiments that the traditional BERT–BILSTM model relies on a large dataset for training and has a huge number of training parameters. In response, this invention discloses a method for extracting entity relationships from high-speed train bogie design standards. This method includes a RoBERTa-wwm-Self Attention-BiSRU-Multi-head Attention model based on a dual attention mechanism and an improved RoBERTa model to extract relationships between entities in the bogie design standard data. The model first uses a RoBERTa-wwm pre-trained model to convert sentence semantics into text vectors. Then, it dynamically adjusts the dynamic word vectors based on the features of the bogie design standard dataset. These word vectors are then fed into a self-attention mechanism layer to extract weighted semantic information. The weighted word vector data then flows through a BiSRU layer, which fully utilizes the advantages of GPU parallel computing and acquires contextual semantic information. Finally, the multi-head attention layer allows the model to simultaneously focus on information from different positions and aspects, thereby improving the model's performance. The model integrating the dual attention mechanism improves computational speed while fully extracting contextual semantics and information from different positions, greatly enhancing the model's ability to acquire entity relationships.

[0036] (2) The inventors discovered in experiments that the original BERT model used a character masking mechanism to cover up some text information. This masking method performed well in English datasets, but in Chinese datasets, character masking may segment Chinese words into individual characters, resulting in the loss of semantic information. Therefore, this invention improves upon character masking by using word masking technology. Specifically, during masking training, the language model from Harbin Institute of Technology is used to first segment the input sentence into words, and then masking training is performed on different words. This can expand the dataset while avoiding semantic loss and model underfitting.

[0037] (3) In experiments, the inventors found that existing technologies are weak in extracting complex relationships from bogie relationship datasets constructed using a pipeline model when performing entity relationship extraction tasks in the bogie domain. This invention first uses an entity extraction model to extract entities from the corpus, and then performs relationship annotation training on the extracted entities. However, since the entities and relationships in the high-speed train bogie dataset are proper nouns, the accuracy is low when using ordinary open-source pre-trained models for extraction. Therefore, a dual attention concentration mechanism is added to extract contextual semantics and information from different positions, and these are fused. In addition, an improved loss function is used to incorporate the original information during model evaluation, thereby improving the model's predictive performance.

[0038] (4) The inventors discovered in their experiments that pure convolutional neural networks such as TextCNN have insufficient encoding of syntactic information. Furthermore, using RNNs or LSTMs results in low computational efficiency due to the inability to perform parallel computations on GPUs. To address this, this invention proposes using BiSRU to resolve dependencies between words. The bidirectional network can extract semantic information from the context and fuse positional, lexical, syntactic, and semantic information. A multi-head attention mechanism is then used to focus on learning the feature information within the sequence. Simultaneously, the use of an SRU network solves the problem of the model's inability to perform parallel computations on GPUs, significantly improving the model's computational efficiency.

[0039] (5) The inventors control the length of the input statements in the model and limit the length of the labeled entity relation statements to 20 characters. According to our statistics, the average length of the entity relation statements in the constructed high-speed train bogie relation extraction dataset is about 16 characters. Using short words can effectively avoid the phenomenon of feature vanishing in the Bi-SRU layer. If the length of the statement is less than 16 characters, zero-padding is performed at both ends of the sentence to make the length of the sentence 20 characters.

[0040] (6) The inventors discovered in the experiment that there was a problem of insufficient design standards in the field of bogie design. In response, the present invention proposes to increase the existing corpus dataset by using synonyms and Chinese-English-Chinese translation, thereby preventing overfitting due to insufficient corpus and improving the accuracy of bogie model entity extraction. Attached image description:

[0041] Figure 1 This is a flowchart illustrating the overall structure of the model network.

[0042] Figure 2 This is a schematic diagram of the model calculation process;

[0043] Figure 3 This is a schematic diagram of the first layer of the self-attention concentration mechanism.

[0044] Figure 4 This is a schematic diagram of a bidirectional SRU network structure;

[0045] Figure 5 This is a schematic diagram of the second-layer multi-head attention mechanism structure. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0047] Therefore, the following detailed description of embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely illustrates some embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0048] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the embodiments of the present invention can be combined with each other.

[0049] This invention discloses a method for extracting entity relationships from the design standards of high-speed train bogies. Specifically, based on the existing BERT-BiLSTM model, an improved RoBERTa-wwm-BISRU model optimized for bogie datasets and based on a dual attention mechanism is proposed, forming a RoBERTa-wwm+Self-Attention+BiSRU+Multi-HeadAttention model (BRENet). The model innovatively combines the dual attention mechanism with the RoBERTa-wwm-BISRU model and optimizes the model loss function for the bogie dataset structure. The model is first pre-trained using the RoBERTa-wwm model. Sentence semantics are transformed into text vectors through full-word masking and an optimized BERT model. Then, dynamic word vectors are obtained by dynamically adjusting the features of a standard dataset designed for the bogie domain. Finally, the word vectors are fed into a self-attention layer, a bidirectional SRU layer, and a multi-head attention layer to extract feature information. The addition of two attention layers improves the model's understanding of contextual semantics. The improved RoBERTa-wwm model enhances word and sentence segmentation and encoding capabilities, thereby improving entity relation extraction performance in small datasets. Furthermore, the model is optimized for the bogie dataset, with an improved loss function, resulting in higher prediction accuracy.

[0050] This invention includes the following steps:

[0051] S1: Extract text data, perform data augmentation using synonym replacement and Chinese-English-Chinese translation, and create a dataset.

[0052] S2: Construct a network model and train the entity relationship extraction model.

[0053] Preferably, step S1 includes: extracting text data, data augmentation, and creating a dataset. Unlabeled corpus from a specified domain is collected to obtain the data required for training. After data augmentation using various methods, the relationships between entities in each sentence are then labeled. This invention first extracts named entities and then performs entity relationship dataset labeling to form the dataset used in this invention. Step S1 specifically includes steps S11-S14.

[0054] Preferably, step S11 involves data extraction. This includes the following steps: collecting knowledge texts about high-speed train bogies, obtaining a large amount of text from the internet as a data source. Due to the varying quality of the obtained texts, they are manually checked and cleaned to obtain the required text data related to high-speed train bogies.

[0055] Preferably, step S12 is data augmentation. It includes the following steps:

[0056] (1) Data augmentation using synonym replacement. This method replaces a random number of words in a sentence with their synonyms. This method constructs new sentences by changing the synonyms in the sentences, thereby increasing the amount of data. (2) Translate the text into different languages ​​and then into Chinese to increase the detail of the corpus data. (3) Use the ALBERT-BILSTM+CNN-CRF model to extract text entities. This model uses BILSTM and CNN to extract features in parallel. It can fuse the coarse-grained features and fine-grained features of the sentences, improving the accuracy of entity extraction. Then, the extracted entities are labeled.

[0057] Preferably, in step S13: the data generated in step S12 is labeled with relationships using manual annotation on the dataset after entity extraction, and labeled as a supervised learning dataset format, thus forming the dataset required by the present invention.

[0058] Preferably, step S14: control the sentence length. If the length of the input sentence is less than 20 characters, pad the sentence with zeros at both ends to make the sentence length reach 20 characters. Specifically, assuming the sentence length is n, if n is even, the number of zeros padded at both ends is as shown in equation (1), and if n is odd, the number of zeros padded at both ends is as shown in equation (2).

[0059]

[0060]

[0061] Preferably, in step S2, the entity relation extraction model is trained based on the data generated in S1 and the trained language model. During the training of the entity relation extraction model, the parameters of the language segmentation encoding model are also considered as parameters of the upper-level entity relation extraction model and trained together. Step S2 uses the BRENet entity relation extraction model network and trains the model on manually labeled entity relation extraction data. Then, by updating the parameters through gradient descent and repeating the training process, the loss function can be continuously reduced, resulting in increasingly better prediction results.

[0062] Step S2 includes steps S21-S25:

[0063] S21: Construct an improved RoBERTa-wwm pre-trained model;

[0064] S22: Construct a self-attention concentration mechanism layer;

[0065] S23: Construct a bidirectional SRU network module;

[0066] S24: Construct a multi-head attention concentration mechanism layer;

[0067] S25: Improved loss function for the model.

[0068] Preferably, S21: Construct an improved RoBERTa-wwm pre-trained model. The backbone network of the RoBERTa model uses a Transformer encoder and the GELU non-linear activation function. The word embedding size is specified as E, the number of encoder layers as L, and the hidden layer size as H. This invention sets the hidden layer size H to 1024, the number of attention heads to 16, and the encoding layer size to 24. During training, the dataset size for each training iteration is 2K. These parameters indicate that the RoBERTa model uses larger training parameters, resulting in better performance and improved word segmentation capabilities on the self-built bogie design standard relation extraction dataset. Simultaneously, RoBERTa uses a hybrid character and word-level encoding method to encode the input sentence, learning high-frequency byte combinations. Furthermore, to handle Chinese corpora, the RoBERTa model uses whole-word masking training technology (wwm) and the Language Technology Platform (LTP) from Harbin Institute of Technology for word segmentation of Chinese sentences. This avoids the semantic loss that occurs when segmenting Chinese words during masking training.

[0069] Preferably, S22: Construct a self-attention concentration mechanism layer. For example... Figure 3 As shown, the self-attention mechanism consists of three parts: the output word vectors of RoBERTa-wwm, the Tanh function, and the Softmax activation function. This invention uses a self-attention mechanism to process the output of RoBERTa-wwm, making the model's attention more focused on the entities to be extracted. The computation process of the self-attention focusing mechanism network structure used in this invention is shown in equations (3) to (5):

[0070] M=tanh(H) (3)

[0071]

[0072] r = Hα T(5)

[0073] Where: In equation (3): H is the output of the first layer RoBERTa-wwm, and its dimension is n×Sw, where n is the dimension of the word vector matrix and Sw is the length of the word vector sequence;

[0074] In equation (4): α is the word vector matrix activated by the softmax activation function, and T represents the transpose of the matrix;

[0075] In equation (5): r is the output sentence weight vector;

[0076] The H matrix is ​​first processed by the Tanh activation function to obtain the word vector matrix M, and then processed by a softmax layer to obtain the weight matrix. At this point, the sum of all weights in α is 1. Multiplying the H word vector matrix by the normalized weight matrix yields the corresponding weighted global temporal feature vector.

[0077] Preferably, S23: Construct a bidirectional SRU network module. Specifically, use a bidirectional SRU network, connecting the forward SRU and backward SRU networks, so that the model can obtain both forward and backward semantic information.

[0078] This invention uses an improved LSTM network, SRU, which can solve the problem of parallel computing, thereby improving computational efficiency. SRU is existing technology and will not be described in detail here. This invention uses a bidirectional SRU network, such as... Figure 4 As shown, the forward SRU and backward SRU networks are connected, allowing the model to acquire both forward and backward semantic information. This improves upon the problem often found in single-term SRU networks, where the inability to acquire forward semantic information leads to semantic loss. Furthermore, since the input sentence is limited to 20 characters, information loss during transmission due to excessively long sentences is avoided.

[0079] Preferably, S24: Construct a second-layer attention mechanism, namely a multi-head attention concentration mechanism layer.

[0080] After computation by the bidirectional SRU network, a series of word sequences are output. These word sequences lose semantic position information during the computation process. The multi-head attention mechanism, by mapping the input sequence to different representation spaces, captures different aspects of the input from different angles, enabling the model to better understand the input sequence. In the decoder of this model, the multi-head attention mechanism can help the model select the correct information from the input sequence to generate the output sequence.

[0081] like Figure 5 As shown, the construction process of multi-head attention is as follows: Given an input sequence X = [x1, x2, ..., x...], ...n ], where n represents the sequence length, and each x i It is a vector representation. The multi-head attention mechanism maps the input sequence to h subspaces, each subspace having a different weight matrix W. Q W K W V These are used to compute the query, key, and value vectors, respectively. For each subspace i, compute the query vector Q. i =XW i Q Key vector K i =XW i K Sum vector V i =XW i V Among them, W i Q W i K W i V It is d model ×d k The matrix, d model d is the dimension representing the vector in the model. k This is the dimension of the key vector. Then, by taking the dot product of the query vector and the key vector, the attention score for the i-subspace is obtained. And scale the attention score. Next, scale the attention score S. i The attention weights W are obtained by normalization using the Softmax function. i =softmax(S i Let V represent the contribution of each position to subspace i. Finally, the value vector V is... i And attention weight W i Perform a weighted average to obtain the output of multi-head attention. Finally, the outputs O1, O2, ..., O from multiple subspaces are... h Concatenating these elements forms the final output vector O = [O1, O2, ..., O2]. h ], where [O1,O2,...,O h ] indicates a vector concatenation operation.

[0082] S25: For the constructed high-speed train bogie relationship extraction dataset, the loss function of the overall BRENet model is improved, so that the pre-trained model can better adapt to the characteristics of the high-speed train bogie dataset. The improved loss function is shown in Equation (6):

[0083]

[0084] Where: y iRepresents the actual value. 'c' represents the predicted value, and 'c' represents the length of the statement sequence. This improvement allows the model to pay more attention to the role of the original variables, that is, to let the original variables and the predicted variables jointly influence the prediction results.

[0085] In text classification tasks, precision (P), recall (R), and F1 score are commonly used as evaluation metrics. The higher the values ​​of precision (P), recall (R), and F1 score, the better the performance.

[0086]

[0087]

[0088]

[0089] In equations (7)-(9): TP represents samples belonging to class C that are correctly classified into class C; FP represents samples not belonging to class C that are incorrectly classified into class C; FN represents samples belonging to class C that are incorrectly classified into other classes besides class C; FN represents samples not belonging to class C that are correctly classified into other classes besides class C. P is the percentage of correctly identified samples out of the total number of identified samples, R is the percentage of correctly identified samples out of the total number of samples, and F1 is the harmonic mean of P and R.

[0090] To test the advantages and disadvantages of the method proposed in this invention compared with the RoBERTa-wwm-Bi-SRU method, the RoBERTa-wwm-Self-Attention-Bi-SRU method, and the RoBERTa-wwm-Bi-SRU-Multi-Head Attention method, the following comparative experiment was conducted in this invention.

[0091] In this experiment, the initial learning rate was 0.005, the batch corpus consisted of 25,000 sentences, the number of SRU hidden units in both the forward and backward directions was 512 layers, and the number of training epochs was 110,000. The overall model used an improved loss function as the loss function calculation formula, an improved BiLSTM model with a dual attention focus mechanism, and a Softmax function as the classifier. The system optimization function used was the parameter-optimized Adam optimizer. To address potential overfitting and underfitting during model training, a dual attention focus mechanism was introduced, and an improved RoBERTa model based on full-word masking was used. The improved RoBERTa model was trained with large model parameters to increase the batch size, thereby improving prediction accuracy. The results were tested on a self-built bogie dataset, achieving a precision (P) of 86.31%, a recall (R) of 86.90%, and an F1 score of 86.60%.

[0092] The following table compares RoBERTa-wwm with three models using the same modules but different combinations under the same conditions. In the table, RoBERTa-wwm, Self-Attention, BiSRU, and Multi-Head Attention represent the different modules that make up the model, and P, R, and F1 are the model evaluation metrics. This table shows that all the modules mentioned in the patent are useful to the model.

[0093]

[0094] The following data compares the models with three different modules under the same conditions:

[0095]

[0096] The results of the comparative experiments show that the method proposed in this invention is superior to the methods in the prior art.

[0097] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the specific embodiments described above. Therefore, any modifications or equivalent substitutions to the present invention, as well as all technical solutions and improvements that do not depart from the spirit and scope of the invention, are covered within the scope of the claims of the present invention.

Claims

1. A high-speed train bogie design standard entity relationship extraction method, characterized by: Comprising steps S1-S2, Wherein, S1: extracting text data, using synonym replacement and Chinese-English-Chinese translation for data enhancement, and making a data set; S2: constructing a network model and training an entity relationship extraction model; Step S2 includes steps S21-S25: S21: constructing an improved RoBERTa-wwm pre-training model; S22: constructing a self-attention focusing mechanism layer; S23: constructing a bidirectional SRU network module; S24: constructing a multi-head attention focusing mechanism layer; S25: improving the loss function of the model; In step S22, the self-attention mechanism includes three parts: the output word vector of RoBERTa-wwm, the Tanh function and the Softmax activation function; the calculation process of the self-attention focusing mechanism network structure is shown in formula (3) to formula (5): M=tanh(H) (3) r＝Hα T (5) In formula (3): H is the output of the first layer RoBERTa-wwm, and the dimension is n x Sw, where n is the dimension of the word vector matrix, and Sw is the length of the word vector sequence; In formula (4): a is the word vector matrix activated by the softmax activation function, and T represents the transpose of the matrix; In formula (5): r is the output sentence weight vector; The H word vector matrix is first obtained by the Tanh activation function to obtain the word vector matrix M, and then a softmax layer is used to obtain the weight matrix, at this time the sum of all weights in a is 1, and the H word vector matrix is multiplied by the normalized weight matrix to obtain the corresponding weighted global timing feature vector; In step S23, a bidirectional SRU network is used, and the forward SRU and backward SRU networks are connected, so that the model can obtain forward semantic information and backward semantic information; In step S24, the construction process of multi-head attention is as follows: Given an input sequence X = [x1, x2, ..., x...] n ], where n represents the sequence length, and each x i It is a vector representation; the multi-head attention mechanism maps the input sequence to h subspaces, each subspace having a different weight matrix W. Q W K W V These are used to calculate the query, key, and value vectors, respectively; for each subspace i, the query vector Q is calculated. i =XW i Q Key vector K i =XW i K Sum vector V i =XW i V Among them, W i Q W i K W i V It is d model ×d k The matrix, d model d is the dimension representing the vector in the model. k This is the dimension of the key vector; then, the attention score for the i-subspace is obtained by taking the dot product of the query vector and the key vector. And scale the attention score; next, scale the attention score S i The attention weights W are obtained by normalization using the Softmax function. i =softmax(S i ), representing the contribution of each position to subspace i; finally, the value vector V i And attention weight W i Perform a weighted average to obtain the output of multi-head attention. Finally, the outputs O1, O2, ..., O from multiple subspaces are... h Concatenating these elements forms the final output vector O = [O1, O2, ..., O2]. h ], where [O1,O2,...,O h ] indicates a vector concatenation operation; S25: For the high-speed train bogie relationship extraction data set, the loss function of the overall BRENet model is improved, so that the pre-training model better adapts to the characteristics of the high-speed train bogie data set, and the improved loss function is shown in formula (6): where: y i represents the actual value, represents the predicted value, and c represents the length of the sentence sequence. After the improvement, the model can pay more attention to the role of the original variable, that is, the original variable and the predicted variable jointly affect the prediction result.

2. The method for extracting entity relationships in high-speed train bogie design standards as described in claim 1, characterized in that: Step S1 specifically includes steps S11-S14; step S11 is data extraction, including the following steps: collecting high-speed train bogie knowledge text, and obtaining text as a data source through a network.

3. The method of claim 2, wherein the design standard entity relationship extraction method is for a high-speed train bogie. Step S12 is data enhancement; including the following steps: ​ (1) using synonym replacement for data enhancement, which replaces a random number of words in a sentence and changes these words to their synonyms, which constructs a new sentence by changing the synonyms in the sentence; (2) translating the text into different languages and then translating it into Chinese to increase the details of the corpus data; (3) using the ALBERT-BILSTM+CNN-CRF model for text entity extraction, which uses BILSTM and CNN in parallel to extract features, and then labels the extracted entities.

4. The method of claim 3, wherein the design standard entity relationship extraction method of a high-speed train bogie is characterized in that: Step S13: After completing the entity extraction, the data generated in step S12 is annotated using manual annotation, which is annotated as a supervised learning data set format, and finally forms a target data set.

5. The method of claim 4, wherein the design standard entity relationship extraction method of a high-speed train bogie is characterized in that: Step S14: control the length of the sentence, if the length of the input sentence is less than 20 words, add 0 at both ends of the sentence to complete, so that the length of the sentence reaches 20 words.

6. The method of claim 5, wherein the design standard entity relationship extraction method of a high-speed train bogie is characterized in that: In the step S21 of constructing the improved RoBERTa-wwm pre-training model, the backbone network of the RoBERTa model uses a Transformer encoder and a GELU nonlinear activation function; the word embedding size E, the number of encoder layers L, and the hidden layer size H are specified; the hidden layer size H is set to 1024, the number of attention heads is set to 16, the encoding layer size is set to 24, and the data set size is divided by 2K during training; the RoBERTa uses a mixed encoding method of characters and word levels to mix encode the input sentence, which is used to learn high-frequency byte combinations.

Images