A method, device and equipment for extracting entity relations of Chinese equipment failure modes

Abstract

The present application relates to a method, apparatus, and device for extracting entity relationships from Chinese equipment failure modes. The method comprises: constructing a Chinese equipment failure mode entity relationship extraction model; wherein the Chinese equipment failure mode entity relationship extraction model is composed of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triple generation module; using a training sample set to train the Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model; inputting Chinese sentences with Chinese entity relationships to be extracted from the equipment failure mode into the trained Chinese equipment failure mode entity relationship extraction model to obtain entity relationship prediction triples of the Chinese sentences. This method can improve the accuracy of entity relationship extraction from Chinese equipment failure mode text data.

Description

Technical Field

[0001] The present application relates to the technical field of entity relationship extraction, and in particular to a method, device and equipment for extracting entity relationships of Chinese equipment failure modes. Background Art

[0002] Entity relationship extraction is a task in natural language processing that aims to identify semantic relationships between entities in text, forming entity relationship triplets, typically presented as subject, relationship, and object. Entity relationship extraction is a crucial step in fields such as information retrieval, knowledge graph construction, and text mining.

[0003] Extracting entity relationships from Chinese equipment failure pattern text data is a current research hotspot in natural language processing. However, existing entity relationship extraction methods fail to fully exploit the multi-feature information of Chinese language in Chinese equipment failure pattern text data. Therefore, accurate entity relationship extraction is difficult to achieve when Chinese equipment failure pattern text data sets are relatively insufficient. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device and equipment for extracting entity relationships of Chinese equipment failure modes to address the above technical problems.

[0005] A method for extracting entity relationships of Chinese equipment failure modes, the method comprising:

[0006] Construct a Chinese equipment failure mode entity relationship extraction model; the Chinese equipment failure mode entity relationship extraction model consists of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triple generation module;

[0007] The training sample set is used to train the Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model; wherein the training sample set includes Chinese sentence samples of equipment failure mode and corresponding entity relationship triple samples;

[0008] The Chinese sentences of the Chinese entity relations to be extracted in the equipment failure mode are input into the trained Chinese equipment failure mode entity relationship extraction model, and the Chinese character granularity embedding vector of the input Chinese sentence is generated according to the embedding vector generation module in the model. The subject-object local association is preliminarily extracted from the Chinese character granularity embedding vector according to the local feature preliminary extraction module, and the local and global feature iterative extraction module is used to iteratively extract the local and global associations of the preliminarily extracted subject and object features. After the iterative extraction is completed, the iteratively extracted subject and object features are tripled according to the table filling strategy adopted by the entity relationship triplet generation module to obtain the entity relationship prediction triplet of the Chinese sentence.

[0009] In one embodiment, the embedding vector generation module is the first module of the Chinese equipment failure mode entity relationship extraction model, the local feature preliminary extraction module is the second module of the Chinese equipment failure mode entity relationship extraction model, the local and global feature iterative extraction module includes the third module, the fourth module and the fifth module of the Chinese equipment failure mode entity relationship extraction model, and the entity relationship triplet generation module is the sixth module of the Chinese equipment failure mode entity relationship extraction model;

[0010] Among them, the output end of the first module is connected to the input end of the second module; the output end of the second module is connected to the input end of the third module; the output end of the third module is respectively connected to the input end of the fourth module and the input end of the sixth module; the output end of the fourth module is connected to the input end of the fifth module; the output end of the fifth module is connected to the input end of the third module; the input end of the first module is the input end of the Chinese equipment fault mode entity relationship extraction model; the output end of the sixth module is the output end of the Chinese equipment fault mode entity relationship extraction model.

[0011] In one embodiment, the first module is a pre-trained ChineseBERT model, which is used to generate a Chinese character granularity embedding vector for an input Chinese sentence and output the Chinese character granularity embedding vector to the input end of the second module; wherein the Chinese character granularity embedding vector is composed of pinyin and characters; wherein the i-th Chinese character c in the input Chinese sentence i The corresponding Chinese character granularity embedding vector is expressed as

[0012]

[0013] In one embodiment, the second module is a BiLSTM model, which is used to perform a preliminary extraction of subject-object local correlation on the Chinese character granularity embedding vector output by the first module using a forward-backward calculation method, obtain preliminary extracted subject features and object features, and output them to the input end of the third module;

[0014] Among them, the forward calculation and backward calculation of the BiLSTM model are symmetrical to each other. The forward calculation process is expressed as

[0015]

[0016] Among them, H s is the main feature extracted initially, H o is the initially extracted object feature, H is the Chinese sentence processed by vector encoding and BiLSTM model, the subscript 1 / 2 indicates two independent networks, and f t represents the BiLSTM forget gate at the tth iteration, Represents the BiLSTM forget gate weights corresponding to the subject and object; Indicates the BiLSTM forget gate bias corresponding to the subject and object; i t represents the first memory gate of BiLSTM at the tth iteration; σ() represents the sigmoid activation function, Represents the weight of the first memory gate of BiLSTM corresponding to the subject and object; Indicates the bias of the first memory gate of BiLSTM corresponding to the subject and object; represents the second memory gate of BiLSTM at the tth iteration, Represents the weight of the second memory gate of BiLSTM corresponding to the subject and object; Indicates the bias of the second memory gate of BiLSTM corresponding to the subject and the object; C t represents the first output gate of BiLSTM at the tth iteration, o t Represents the second output gate of BiLSTM at the tth iteration; Represents the weight of the second output gate of BiLSTM corresponding to the subject and object, Indicates the bias of the second output gate of BiLSTM corresponding to the subject and object, Indicates the second output gate of BiLSTM corresponding to the subject at the t-th iteration, Indicates that the subject at the tth iteration corresponds to the first output gate of BiLSTM, Represents the second output gate of BiLSTM corresponding to the object at the tth iteration, Represents the first output gate of BiLSTM corresponding to the object at the tth iteration.

[0017] In one embodiment, the third module is composed of a first fully connected layer, which is used to generate a table label feature for each relationship based on the subject features and object features initially extracted by the second module, and output the table label feature to the input of the fourth module before the iterative extraction of the subject and object features is completed; after the iterative extraction of the subject and object features is completed, the table label feature generated by the last iteration is output to the input of the sixth module;

[0018] Among them, the table label feature of relation r at the tth iteration is expressed as Each item in the table represents the label feature of the token pair, and the token pair is the vector representation of the entity after vector encoding. A pair of token pairs (w i ,w j )’s table label feature is represented as

[0019]

[0020] Among them, ReLU is the activation function, represents the token w extracted at the tth iteration i The corresponding subject features, represents the token w extracted at the tth iteration j The corresponding object characteristics, represents the Hadamard product, W r Represents the weight of the first fully connected layer corresponding to the relationship r, b r Represents the bias of the first fully connected layer corresponding to the relation r, and the subscripts i and j represent the index subscripts of the token pair.

[0021] In one embodiment, the fourth module is composed of a splicing module, a multi-head self-attention layer, a multi-head attention layer, and a second fully connected layer connected in sequence; the splicing module is used to horizontally splice the multi-head self-attention layer, the multi-head attention layer, and the second fully connected layer; the multi-head self-attention layer is used to obtain the table label features of each relationship output by the third module, and use the multi-head self-attention mechanism to mine the local association between each relationship; the multi-head attention layer is used to further mine the association between token pairs based on the local association between each relationship obtained by mining; the second fully connected layer is used to further mine the association between token pairs from the perspective of the entire Chinese sentence based on the association between the obtained token pairs, and output the mining results to the input end of the fifth module;

[0022] Among them, the calculation formula of the fourth module is expressed as

[0023]

[0024] Among them, MultiHeadSelfAtt represents the multi-head self-attention mechanism, MultiHeadAtt represents the multi-head attention mechanism, H represents the Chinese sentence processed by vector encoding and BiLSTM model, ReLU is the activation function, FM represents the table label features containing all relations after the t-th iteration of mining by the multi-head self-attention layer, (t) represents the table label features containing all relations at the t-th iteration, It represents the table label features containing all relations after the t-th iteration of the multi-head attention layer, H'( t ) represents the table label features containing all relations after the t-th iteration mining of the second fully connected layer, W represents the weight of the second fully connected layer, and b represents the bias of the second fully connected layer.

[0025] In one embodiment, the fifth module is composed of a maximum pooling layer, a third fully connected layer, and a residual neural network; the fifth module is used to obtain the result output by the fourth module, perform global correlation iterative extraction through the maximum pooling layer and the third fully connected layer, generate subject-related table features and object-related table features, and input the subject-related table features and object-related table features into the residual neural network, generate iteratively extracted subject features and object features through the residual neural network, and output them to the input end of the third module;

[0026] Among them, the subject-related table features generated at the t+1th iteration Object-related table features Respectively expressed as

[0027]

[0028] Among them, maxpool represents the maximum pooling, W s represents the weight of the third fully connected layer corresponding to the subject, b s Denotes the bias of the third fully connected layer corresponding to the main body, W o represents the weight of the third fully connected layer corresponding to the object, b o represents the bias of the third fully connected layer corresponding to the object;

[0029] The main features extracted in the t+1th iteration and object characteristics Respectively expressed as

[0030]

[0031] Among them, LayerNorm represents normalization processing, represents the main features extracted in the tth iteration, represents the object features extracted at the tth iteration.

[0032] In one embodiment, the sixth module is configured to obtain the table label features of each relationship output by the third module during the last iteration, fill the table label features of each relationship into a table according to a preset table filling strategy, extract an entity pair for each relationship by decoding the filled table, perform entity-relationship matching on each entity pair according to a preset relationship type, and generate an entity relationship prediction triplet; wherein the entity pair consists of a subject and an object, and the entity relationship prediction triplet consists of a subject, a relationship, and an object;

[0033] Among them, the form filling strategy is expressed as

[0034]

[0035] Among them, softmax is the normalized exponential function, tabler (i,j) is a token pair (w i ,w j )The relationship classification results in the table, Represents the table label features of relation r at the Nth iteration, Denotes the predicted token pair (w i ,w j ) All relationship classification results in the table, which contains multiple relationship results, represents the set of all relations L, Represents a set, L represents the number of relationships, and l represents the predicted relationship type; argmax represents finding all predicted relationships such that The relationship with the largest value is taken as the relationship classification result;

[0036] Among them, there are three parallel search routes in the table decoding process to extract entity pairs for each relationship, namely the forward search route, the reverse search route and the single search route; the forward search route generates entity pairs in the order from subject to object, the reverse search route generates entity pairs in the order from object to subject; the single search route generates a single entity pair.

[0037] In one embodiment, a training sample set is used to train a Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model, including:

[0038] Input the Chinese sentence samples of equipment failure modes in the training sample set into the Chinese equipment failure mode entity relationship extraction model to extract entity relationships and obtain entity relationship prediction triples;

[0039] The cross entropy loss function between the entity relationship prediction triples and the entity relationship triple samples in the training sample set is calculated, and the Chinese equipment failure mode entity relationship extraction model is trained according to the cross entropy loss function to obtain the trained Chinese equipment failure mode entity relationship extraction model.

[0040] A Chinese equipment failure mode entity relationship extraction device, the device comprising:

[0041] A model building unit is used to build a Chinese equipment failure mode entity relationship extraction model; wherein, the Chinese equipment failure mode entity relationship extraction model consists of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triple generation module;

[0042] A model training unit is used to train the Chinese equipment failure mode entity relationship extraction model using a training sample set to obtain a trained Chinese equipment failure mode entity relationship extraction model; wherein the training sample set includes Chinese sentence samples of equipment failure modes and corresponding entity relationship triple samples;

[0043] The entity relationship extraction unit is used to input the Chinese sentence of the Chinese entity relationship to be extracted in the equipment failure mode into the trained Chinese equipment failure mode entity relationship extraction model, generate the Chinese character granularity embedding vector of the input Chinese sentence according to the embedding vector generation module in the model, perform preliminary extraction of subject and object local association of the Chinese character granularity embedding vector according to the local feature preliminary extraction module, perform local and global association iterative extraction of the preliminary extracted subject and object features according to the local and global feature iterative extraction module, and after the iterative extraction is completed, generate triples for the iteratively extracted subject and object features according to the filling strategy adopted by the entity relationship triple generation module to obtain the entity relationship prediction triples of the Chinese sentence.

[0044] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0045] Construct a Chinese equipment failure mode entity relationship extraction model; the Chinese equipment failure mode entity relationship extraction model consists of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triple generation module;

[0046] The training sample set is used to train the Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model; wherein the training sample set includes Chinese sentence samples of equipment failure mode and corresponding entity relationship triple samples;

[0047] The Chinese sentences of the Chinese entity relations to be extracted in the equipment failure mode are input into the trained Chinese equipment failure mode entity relationship extraction model, and the Chinese character granularity embedding vector of the input Chinese sentence is generated according to the embedding vector generation module in the model. The subject-object local association is preliminarily extracted from the Chinese character granularity embedding vector according to the local feature preliminary extraction module, and the local and global feature iterative extraction module is used to iteratively extract the local and global associations of the preliminarily extracted subject and object features. After the iterative extraction is completed, the iteratively extracted subject and object features are tripled according to the table filling strategy adopted by the entity relationship triplet generation module to obtain the entity relationship prediction triplet of the Chinese sentence.

[0048] The above-mentioned Chinese equipment failure mode entity relationship extraction method, device and equipment extract entity relationships from equipment failure mode text data by constructing and training a Chinese equipment failure mode entity relationship extraction model. The constructed entity relationship extraction model consists of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module and an entity relationship triple generation module. Among them, the embedding vector generation module generates a Chinese character granularity embedding vector for the input Chinese sentence, which can help the model obtain high-quality embedding vectors even when the equipment failure mode text data set is relatively insufficient, and enriches the Chinese feature dimensions in the embedding vector. In addition, the local feature preliminary extraction module and the local and global feature iterative extraction module achieve full attention to the Chinese multi-feature information of the Chinese equipment failure mode text data by mining the local and global associations between the subject and object in the input sentence, thereby improving the accuracy of the entity relationship prediction triples generated by the subsequent entity relationship triple generation module. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of a flow chart of a method for extracting entity relationships of Chinese equipment failure modes in one embodiment;

[0050] Figure 2 A schematic diagram of the structure of a Chinese equipment failure mode entity relationship extraction model constructed in one embodiment;

[0051] Figure 3 A schematic diagram of a form filling strategy in one embodiment;

[0052] Figure 4 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0054] In one embodiment, Figure 1 As shown, a method for extracting entity relations of Chinese equipment failure modes is provided, comprising the following steps:

[0055] Step S1, constructing a Chinese equipment failure mode entity relationship extraction model; wherein, the Chinese equipment failure mode entity relationship extraction model consists of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triplet generation module.

[0056] Among them, the Chinese equipment failure mode entity relationship extraction model constructed by this application is also called the UREF model. The meanings of the letters in UREF are: (1) U stands for Unified, which means unified; (2) RE stands for Relation Extraction, that is, this model is used for relationship extraction; (3) F stands for Framework, which means framework. The overall meaning of UREF is: a unified entity relationship extraction framework for equipment failure modes constructed using BiLSTM and a large language model. The UREF model of this application can achieve good accuracy in both English general domains and Chinese equipment failure modes by filling in relationship tables based on local and global associations and performing entity relationship extraction.

[0057] Step S2: Use the training sample set to train the Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model; wherein the training sample set includes Chinese sentence samples of equipment failure modes and corresponding entity relationship triple samples.

[0058] In step S3, the Chinese sentence of the Chinese entity relationship to be extracted in the equipment failure mode is input into the trained Chinese equipment failure mode entity relationship extraction model, and the Chinese character granularity embedding vector of the input Chinese sentence is generated according to the embedding vector generation module in the model. The subject-object local association is preliminarily extracted for the Chinese character granularity embedding vector according to the local feature preliminary extraction module, and the local and global feature iterative extraction module is used to perform local and global association iterative extraction on the preliminarily extracted subject and object features. After the iterative extraction is completed, the iteratively extracted subject and object features are tripled according to the filling strategy adopted by the entity relationship triplet generation module to obtain the entity relationship prediction triplet of the Chinese sentence.

[0059] In one embodiment, Figure 2 As shown, the embedding vector generation module is the first module of the Chinese equipment failure mode entity relationship extraction model, the local feature preliminary extraction module is the second module of the Chinese equipment failure mode entity relationship extraction model, the local and global feature iterative extraction module includes the third module, the fourth module and the fifth module of the Chinese equipment failure mode entity relationship extraction model, and the entity relationship triplet generation module is the sixth module of the Chinese equipment failure mode entity relationship extraction model;

[0060] Among them, the output end of the first module is connected to the input end of the second module; the output end of the second module is connected to the input end of the third module; the output end of the third module is respectively connected to the input end of the fourth module and the input end of the sixth module; the output end of the fourth module is connected to the input end of the fifth module; the output end of the fifth module is connected to the input end of the third module; the input end of the first module is the input end of the Chinese equipment fault mode entity relationship extraction model; the output end of the sixth module is the output end of the Chinese equipment fault mode entity relationship extraction model.

[0061] In one embodiment, the first module is a pre-trained ChineseBERT model, which is used to generate a Chinese character granularity embedding vector for an input Chinese sentence and output the Chinese character granularity embedding vector to the input end of the second module; wherein the Chinese character granularity embedding vector is composed of pinyin and characters; wherein the i-th Chinese character c in the input Chinese sentence i The corresponding Chinese character granularity embedding vector is expressed as

[0062]

[0063] Among them, ChineseBERT starts from the two major characteristics of Chinese characters themselves and integrates the glyph and pinyin feature information of Chinese characters into the pre-training process of Chinese corpus. For glyph information, it vectorizes the text image and then uses the fully connected layer to obtain glyph embedding; for pinyin information, it uses CNN (convolutional neural network) to extract the information contained in the pinyin sequence corresponding to the text, and obtains the pinyin embedding after maximum pooling. The fusion vector obtained by fusing glyph embedding, pinyin embedding and word vector is the input of the pre-training model. Through Whole Word Masking (full word masking) and Character Masking (character masking) training, the model can learn the association information between Chinese characters, glyphs, pronunciations and contexts very well. Specifically, the open source ChineseBERT-base model on Huggingface is used in this embodiment. The Chinese character granularity embedding vector of the input Chinese sentence is generated by the ChineseBERT-base model, which can supplement the glyph and pinyin information in the Chinese character embedding vector, enrich the information of the embedding vector, and help to correctly complete the entity relationship extraction task.

[0064] In one embodiment, the second module is a BiLSTM model, which is used to perform a preliminary extraction of subject-object local correlation on the Chinese character granularity embedding vector output by the first module using a forward-backward calculation method, obtain preliminary extracted subject features and object features, and output them to the input end of the third module;

[0065] Among them, the forward calculation and backward calculation of the BiLSTM model are symmetrical to each other. The forward calculation process is expressed as

[0066]

[0067] Among them, H s is the main feature extracted initially, H o is the initially extracted object feature, H is the Chinese sentence processed by vector encoding and BiLSTM model, the subscript 1 / 2 indicates two independent networks, and f trepresents the BiLSTM forget gate at the tth iteration, Represents the BiLSTM forget gate weights corresponding to the subject and object; Indicates the BiLSTM forget gate bias corresponding to the subject and object; i t represents the first memory gate of BiLSTM at the tth iteration; σ() represents the sigmoid activation function, Represents the weight of the first memory gate of BiLSTM corresponding to the subject and object; Indicates the bias of the first memory gate of BiLSTM corresponding to the subject and object; represents the second memory gate of BiLSTM at the tth iteration, Represents the weight of the second memory gate of BiLSTM corresponding to the subject and object; Indicates the bias of the second memory gate of BiLSTM corresponding to the subject and the object; C t represents the first output gate of BiLSTM at the tth iteration, o t Represents the second output gate of BiLSTM at the tth iteration; Represents the weight of the second output gate of BiLSTM corresponding to the subject and object, Indicates the bias of the second output gate of BiLSTM corresponding to the subject and object, Indicates the second output gate of BiLSTM corresponding to the subject at the t-th iteration, Indicates that the subject at the tth iteration corresponds to the first output gate of BiLSTM, Represents the second output gate of BiLSTM corresponding to the object at the tth iteration, Represents the first output gate of BiLSTM corresponding to the object at the tth iteration.

[0068] In one embodiment, the third module is composed of a first fully connected layer, which is used to generate a table label feature for each relationship based on the subject features and object features initially extracted by the second module, and output the table label feature to the input of the fourth module before the iterative extraction of the subject and object features is completed; after the iterative extraction of the subject and object features is completed, the table label feature generated by the last iteration is output to the input of the sixth module;

[0069] Among them, the table label feature of relation r at the tth iteration is expressed as Each item in the table represents the label feature of the token pair, and the token pair is the vector representation of the entity after vector encoding. A pair of token pairs (w i ,w j )’s table label feature is represented as

[0070]

[0071] Among them, ReLU is the activation function, represents the token w extracted at the tth iteration i The corresponding subject features, represents the token w extracted at the tth iteration j The corresponding object characteristics, represents the Hadamard product, W r Represents the weight of the first fully connected layer corresponding to the relationship r, b r Represents the bias of the first fully connected layer corresponding to the relation r, and the subscripts i and j represent the index subscripts of the token pair.

[0072] In one embodiment, the fourth module is composed of a splicing module, a multi-head self-attention layer, a multi-head attention layer, and a second fully connected layer connected in sequence; the splicing module is used to horizontally splice the multi-head self-attention layer, the multi-head attention layer, and the second fully connected layer; the multi-head self-attention layer is used to obtain the table label features of each relationship output by the third module, and use the multi-head self-attention mechanism to mine the local association between each relationship; the multi-head attention layer is used to further mine the association between token pairs based on the local association between each relationship obtained by mining; the second fully connected layer is used to further mine the association between token pairs from the perspective of the entire Chinese sentence based on the association between the obtained token pairs, and output the mining results to the input end of the fifth module;

[0073] Among them, the calculation formula of the fourth module is expressed as

[0074]

[0075] Among them, MultiHeadSelfAtt represents the multi-head self-attention mechanism, MultiHeadAtt represents the multi-head attention mechanism, H represents the Chinese sentence processed by vector encoding and BiLSTM model, ReLU is the activation function, FM represents the table label features containing all relations after the t-th iteration of mining by the multi-head self-attention layer, (t) represents the table label features containing all relations at the tth iteration, where For FM (t) The parameterized representation of the tabular label features of the relation r, where for A pair of tokens (w i ,w j )'s table label feature parameterized representation; It represents the table label features containing all relations after the t-th iteration of the multi-head attention layer, H'( t) represents the table label features containing all relations after the t-th iteration mining of the second fully connected layer, W represents the weight of the second fully connected layer, and b represents the bias of the second fully connected layer.

[0076] In one embodiment, the fifth module is composed of a maximum pooling layer, a third fully connected layer, and a residual neural network; the fifth module is used to obtain the result output by the fourth module, perform global correlation iterative extraction through the maximum pooling layer and the third fully connected layer, generate subject-related table features and object-related table features, and input the subject-related table features and object-related table features into the residual neural network, generate iteratively extracted subject features and object features through the residual neural network, and output them to the input end of the third module;

[0077] Among them, the subject-related table features generated at the t+1th iteration Object-related table features Respectively expressed as

[0078]

[0079] Among them, maxpool represents the maximum pooling, W s represents the weight of the third fully connected layer corresponding to the subject, b s Denotes the bias of the third fully connected layer corresponding to the main body, W o represents the weight of the third fully connected layer corresponding to the object, b o represents the bias of the third fully connected layer corresponding to the object;

[0080] The main features extracted in the t+1th iteration and object characteristics Respectively expressed as

[0081]

[0082] Among them, LayerNorm represents normalization processing, represents the main features extracted in the tth iteration, represents the object features extracted at the tth iteration.

[0083] Among them, the self-attention mechanism can reveal the intrinsic relationships and dependencies of the input sequence, and the residual neural network helps to avoid the problem of gradient disappearance.

[0084] In one embodiment, the sixth module is configured to obtain the table label features of each relationship output by the third module during the last iteration, fill the table label features of each relationship into a table according to a preset table filling strategy, extract an entity pair for each relationship by decoding the filled table, perform entity-relationship matching on each entity pair according to a preset relationship type, and generate an entity relationship prediction triplet; wherein the entity pair consists of a subject and an object, and the entity relationship prediction triplet consists of a subject, a relationship, and an object;

[0085] Among them, Figure 3 As shown, the table filling strategy is expressed as

[0086]

[0087] Among them, softmax is the normalized exponential function, table r (i,j) is a token pair (w i ,w j )The relationship classification results in the table, Represents the table label features of relation r at the Nth iteration, Denotes the predicted token pair (w i ,w j ) All relationship classification results in the table, which contains multiple relationship results, represents the set of all relations L, Represents a set, L represents the number of relationships, and l represents the predicted relationship type; argmax represents finding all predicted relationships such that The relationship with the largest value is taken as the relationship classification result;

[0088] Among them, there are three parallel search routes in the table decoding process to extract entity pairs for each relationship, namely the forward search route, the reverse search route and the single search route; the forward search route generates entity pairs in the order from subject to object, the reverse search route generates entity pairs in the order from object to subject; the single search route generates a single entity pair.

[0089] Combine Figure 3 Further explain the search route of the table decoding of this application. Figure 3In the sentence, the token pair (i.e., token pair) (battery, abnormal) has the label "SOH". The algorithm searches forward, linking adjacent token pairs, until it finds the token pair (failure, noise) with the label "SOT". The forward search stops, and a complete (subject, object) pair (i.e., entity pair) is formed. However, the entity pair formed (battery failure, abnormal noise) is incorrect. The error is caused by nested entities in the sentence, such as "battery failure" and "battery failure lamp". Nested entities cause the forward search to stop early. In this case, the backward search first finds the token pair (lamp, noise) with the label "SOT", and then finds the token pair (battery, abnormal) with the label "SOH", resulting in the desired entity pair (battery failure lamp, abnormal noise).

[0090] In one embodiment, a training sample set is used to train a Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model, including:

[0091] Input the Chinese sentence samples of equipment failure modes in the training sample set into the Chinese equipment failure mode entity relationship extraction model to extract entity relationships and obtain entity relationship prediction triples;

[0092] The cross entropy loss function between the entity relationship prediction triples and the entity relationship triple samples in the training sample set is calculated, and the Chinese equipment failure mode entity relationship extraction model is trained according to the cross entropy loss function to obtain the trained Chinese equipment failure mode entity relationship extraction model.

[0093] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0094] In one embodiment, a device for extracting entity relationships of Chinese equipment failure modes is provided, comprising:

[0095] A model building unit is used to build a Chinese equipment failure mode entity relationship extraction model; wherein, the Chinese equipment failure mode entity relationship extraction model consists of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triple generation module;

[0096] A model training unit is used to train the Chinese equipment failure mode entity relationship extraction model using a training sample set to obtain a trained Chinese equipment failure mode entity relationship extraction model; wherein the training sample set includes Chinese sentence samples of equipment failure modes and corresponding entity relationship triple samples;

[0097] The entity relationship extraction unit is used to input the Chinese sentence of the Chinese entity relationship to be extracted in the equipment failure mode into the trained Chinese equipment failure mode entity relationship extraction model, generate the Chinese character granularity embedding vector of the input Chinese sentence according to the embedding vector generation module in the model, perform preliminary extraction of subject and object local association of the Chinese character granularity embedding vector according to the local feature preliminary extraction module, perform local and global association iterative extraction of the preliminary extracted subject and object features according to the local and global feature iterative extraction module, and after the iterative extraction is completed, generate triples for the iteratively extracted subject and object features according to the filling strategy adopted by the entity relationship triple generation module to obtain the entity relationship prediction triples of the Chinese sentence.

[0098] Regarding the specific limitations of the Chinese equipment failure mode entity relationship extraction device, please refer to the limitations of the Chinese equipment failure mode entity relationship extraction method above, which will not be repeated here. The various modules in the above-mentioned Chinese equipment failure mode entity relationship extraction device can be implemented in whole or in part through software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0099] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for extracting entity relationships of fault modes of Chinese equipment is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0100] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0101] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0102] Construct a Chinese equipment failure mode entity relationship extraction model; the Chinese equipment failure mode entity relationship extraction model consists of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triple generation module;

[0103] The training sample set is used to train the Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model; wherein the training sample set includes Chinese sentence samples of equipment failure mode and corresponding entity relationship triple samples;

[0104] The Chinese sentences of the Chinese entity relations to be extracted in the equipment failure mode are input into the trained Chinese equipment failure mode entity relationship extraction model, and the Chinese character granularity embedding vector of the input Chinese sentence is generated according to the embedding vector generation module in the model. The subject-object local association is preliminarily extracted from the Chinese character granularity embedding vector according to the local feature preliminary extraction module, and the local and global feature iterative extraction module is used to iteratively extract the local and global associations of the preliminarily extracted subject and object features. After the iterative extraction is completed, the iteratively extracted subject and object features are tripled according to the table filling strategy adopted by the entity relationship triplet generation module to obtain the entity relationship prediction triplet of the Chinese sentence.

[0105] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A Chinese equipment failure mode entity relationship extraction method, characterized by: The method comprises: Constructing a Chinese equipment failure mode entity relationship extraction model; wherein the Chinese equipment failure mode entity relationship extraction model consists of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triple generation module; Using a training sample set to train the Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model; wherein the training sample set includes Chinese sentence samples of equipment failure modes and corresponding entity relationship triple samples; The Chinese sentence of the Chinese entity relationship to be extracted in the equipment failure mode is input into the trained Chinese equipment failure mode entity relationship extraction model, and the Chinese character granularity embedding vector of the input Chinese sentence is generated according to the embedding vector generation module in the model. The subject-object local association is preliminarily extracted from the Chinese character granularity embedding vector according to the local feature preliminary extraction module, and the local and global feature iterative extraction module is used to perform local and global association iterative extraction on the preliminarily extracted subject and object features. After the iterative extraction is completed, the iteratively extracted subject and object features are tripled according to the filling-in strategy adopted by the entity relationship triplet generation module to obtain the entity relationship prediction triplet of the Chinese sentence.

2. The method according to claim 1, characterized in that The embedding vector generation module is the first module of the Chinese equipment failure mode entity relationship extraction model, the local feature preliminary extraction module is the second module of the Chinese equipment failure mode entity relationship extraction model, the local and global feature iterative extraction module includes the third module, the fourth module and the fifth module of the Chinese equipment failure mode entity relationship extraction model, and the entity relationship triplet generation module is the sixth module of the Chinese equipment failure mode entity relationship extraction model; Among them, the output end of the first module is connected to the input end of the second module; the output end of the second module is connected to the input end of the third module; the output end of the third module is respectively connected to the input end of the fourth module and the input end of the sixth module; the output end of the fourth module is connected to the input end of the fifth module; the output end of the fifth module is connected to the input end of the third module; the input end of the first module is the input end of the Chinese equipment fault mode entity relationship extraction model; the output end of the sixth module is the output end of the Chinese equipment fault mode entity relationship extraction model.

3. The method according to claim 2, characterized in that The first module is a pre-trained ChineseBERT model, which is used to generate a Chinese character granularity embedding vector of an input Chinese sentence and output the Chinese character granularity embedding vector to the input end of the second module; wherein the Chinese character granularity embedding vector is composed of pinyin and characters; wherein the i-th Chinese character c in the input Chinese sentence i The corresponding Chinese character granularity embedding vector is expressed as 4. The method according to claim 2, characterized in that The second module is a BiLSTM model, which uses a forward-backward calculation method to perform preliminary subject-object local correlation extraction on the Chinese character granularity embedding vector output by the first module to obtain preliminary extracted subject features and object features, and output them to the input end of the third module; The forward calculation and backward calculation of the BiLSTM model are symmetrical to each other. The forward calculation process is expressed as Among them, H s is the main feature extracted initially, H o is the initially extracted object feature, H is the Chinese sentence processed by vector encoding and BiLSTM model, the subscript 1 / 2 indicates two independent networks, and f t represents the BiLSTM forget gate at the tth iteration, Represents the BiLSTM forget gate weights corresponding to the subject and object; Indicates the BiLSTM forget gate bias corresponding to the subject and object; i t represents the first memory gate of BiLSTM at the tth iteration; σ() represents the sigmoid activation function, Represents the weight of the first memory gate of BiLSTM corresponding to the subject and object; Indicates the bias of the first memory gate of BiLSTM corresponding to the subject and object; represents the second memory gate of BiLSTM at the tth iteration, Represents the weight of the second memory gate of BiLSTM corresponding to the subject and object; Indicates the bias of the second memory gate of BiLSTM corresponding to the subject and the object; C t represents the first output gate of BiLSTM at the tth iteration, o t Represents the second output gate of BiLSTM at the tth iteration; Represents the weight of the second output gate of BiLSTM corresponding to the subject and object, Indicates the bias of the second output gate of BiLSTM corresponding to the subject and object, Indicates the second output gate of BiLSTM corresponding to the subject at the t-th iteration, Indicates that the subject at the tth iteration corresponds to the first output gate of BiLSTM, Represents the second output gate of BiLSTM corresponding to the object at the tth iteration, Represents the first output gate of BiLSTM corresponding to the object at the tth iteration.

5. The method according to claim 2, characterized in that The third module is composed of a first fully connected layer, which is used to generate a table label feature for each relationship based on the preliminary extracted subject features and object features output by the second module, and output the table label feature to the input end of the fourth module before the iterative extraction of subject and object features is completed; after the iterative extraction of subject and object features is completed, the table label feature generated by the last iteration is output to the input end of the sixth module; Among them, the table label feature of relation r at the tth iteration is expressed as Each item in the table represents the label feature of the token pair, and the token pair is the vector representation of the entity after vector encoding. A pair of token pairs (w i ,w j )’s table label feature is represented as Among them, ReLU is the activation function, represents the token w extracted at the tth iteration i The corresponding subject features, represents the token w extracted at the tth iteration j The corresponding object characteristics, represents the Hadamard product, W r Represents the weight of the first fully connected layer corresponding to the relationship r, b r Represents the bias of the first fully connected layer corresponding to the relation r, and the subscripts i and j represent the index subscripts of the token pair.

6. The method according to claim 2, characterized in that The fourth module is composed of a splicing module, a multi-head self-attention layer, a multi-head attention layer and a second fully connected layer connected in sequence; the splicing module is used to horizontally splice the multi-head self-attention layer, the multi-head attention layer and the second fully connected layer; the multi-head self-attention layer is used to obtain the table label features of each relationship output by the third module, and adopt the multi-head self-attention mechanism to mine the local association between each relationship; the multi-head attention layer is used to further mine the association between token pairs based on the local association between each relationship obtained by mining; the second fully connected layer is used to further mine the association between token pairs from the perspective of the entire Chinese sentence based on the association between the obtained token pairs, and output the mining result to the input end of the fifth module; The calculation formula of the fourth module is expressed as follows: Among them, MultiHeadSelfAtt represents the multi-head self-attention mechanism, MultiHeadAtt represents the multi-head attention mechanism, H represents the Chinese sentence processed by vector encoding and BiLSTM model, ReLU is the activation function, FM represents the table label features containing all relations after the t-th iteration of the multi-head self-attention layer mining. (t) represents the table label features containing all relations at the t-th iteration, H' represents the table label feature containing all relations after the t-th iteration of the multi-head attention layer mining. (t) It represents the table label features containing all relations after the t-th iteration mining of the second fully connected layer, W represents the weight of the second fully connected layer, and b represents the bias of the second fully connected layer.

7. The method according to claim 2, characterized in that The fifth module is composed of a maximum pooling layer, a third fully connected layer, and a residual neural network; the fifth module is used to obtain the result output by the fourth module, perform global correlation iterative extraction through the maximum pooling layer and the third fully connected layer, generate subject-related table features and object-related table features, and input the subject-related table features and object-related table features into the residual neural network. The residual neural network generates iteratively extracted subject features and object features, and outputs them to the input end of the third module; Among them, the subject-related table features generated at the t+1th iteration Object-related table features Respectively expressed as Among them, maxpool represents the maximum pooling, W s represents the weight of the third fully connected layer corresponding to the subject, b s Denotes the bias of the third fully connected layer corresponding to the main body, W o represents the weight of the third fully connected layer corresponding to the object, b o Represents the bias of the third fully connected layer corresponding to the object; The main feature H extracted in the t+1th iteration s ' (t+1) and object characteristics H o ' (t+1) Respectively expressed as Among them, LayerNorm represents normalization processing, represents the main features extracted in the tth iteration, represents the object features extracted at the tth iteration.

8. The method according to claim 2, characterized in that The sixth module is configured to obtain the table label features of each relationship output by the third module during the last iteration, fill the table label features of each relationship into a table according to a preset table filling strategy, extract an entity pair for each relationship by decoding the filled table, perform entity relationship matching on each entity pair according to a preset relationship type, and generate an entity relationship prediction triple; wherein the entity pair consists of a subject and an object, and the entity relationship prediction triple consists of a subject, a relationship, and an object; The form filling strategy is expressed as Among them, softmax is the normalized exponential function, table r (i,j) is a token pair (w i ,w j )The relationship classification results in the table, Represents the table label features of relation r at the Nth iteration, Denotes the predicted token pair (w i ,w j ) All relationship classification results in the table, which contains multiple relationship results, represents the set of all relations L, Represents a set, L represents the number of relationships, and l represents the predicted relationship type; argmax represents finding all predicted relationships such that The relationship with the largest value is taken as the relationship classification result; Among them, there are three parallel search routes in the table decoding process to extract entity pairs for each relationship, namely the forward search route, the reverse search route and the single search route; the forward search route generates entity pairs in the order from subject to object, the reverse search route generates entity pairs in the order from object to subject; the single search route generates a single entity pair.

9. The method according to claim 1, characterized in that The training sample set is used to train the Chinese equipment failure mode entity relationship extraction model to obtain a trained Chinese equipment failure mode entity relationship extraction model, including: Inputting Chinese sentence samples of equipment failure modes in the training sample set into the Chinese equipment failure mode entity relationship extraction model to extract entity relationships and obtain entity relationship prediction triples; Calculate the cross entropy loss function between the entity relationship prediction triples and the entity relationship triple samples in the training sample set, train the Chinese equipment failure mode entity relationship extraction model according to the cross entropy loss function, and obtain a trained Chinese equipment failure mode entity relationship extraction model.

10. A Chinese equipment failure mode entity relationship extraction device, characterized in that: The device comprises: A model construction unit is used to construct a Chinese equipment failure mode entity relationship extraction model; wherein the Chinese equipment failure mode entity relationship extraction model is composed of an embedding vector generation module, a local feature preliminary extraction module, a local and global feature iterative extraction module, and an entity relationship triple generation module; A model training unit is used to train the Chinese equipment failure mode entity relationship extraction model using a training sample set to obtain a trained Chinese equipment failure mode entity relationship extraction model; wherein the training sample set includes Chinese sentence samples of equipment failure modes and corresponding entity relationship triple samples; The entity relationship extraction unit is used to input the Chinese sentence of the Chinese entity relationship to be extracted in the equipment failure mode into the trained Chinese equipment failure mode entity relationship extraction model, generate the Chinese character granularity embedding vector of the input Chinese sentence according to the embedding vector generation module in the model, perform preliminary extraction of subject and object local association on the Chinese character granularity embedding vector according to the local feature preliminary extraction module, perform local and global association iterative extraction on the preliminary extracted subject and object features according to the local and global feature iterative extraction module, and after the iterative extraction is completed, generate triples for the iteratively extracted subject and object features according to the filling strategy adopted by the entity relationship triple generation module to obtain the entity relationship prediction triples of the Chinese sentence.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images