Triple extraction method and device for sparse sample in field of coal mine electromechanical equipment

Abstract

The invention provides a triple extraction method and device for sparse samples in the field of coal mine electromechanical equipment. The method comprises the following steps: performing text vectorization by using a BERT model as a training model; in the Span extraction process, the Bi-LSTM is used for fusing the Span context semantic relation; performing relation classification on the candidate Span entity pairs through a relation classifier; wherein in the Span extraction process, a sliding window mechanism is used, and each entity is expressed as a continuous subsequence. In the method, a sliding window mechanism is used in the Span method, each entity is expressed as a continuous subsequence instead of an independent mark, the problem of triple overlapping can be effectively solved, context semantics of the Span are fused by using Bi-LSTM in the entity extraction and relation prediction stage, the recognition effect on the overlapped triads is improved, and the recognition efficiency of the overlapped triads is improved. Compared with a common synonym replacement and back translation method, the method has a better effect on relation category enhancement in the field.

Description

Technical Field

[0001] The present application relates to the field of text processing technology, and more particularly to a method and apparatus for extracting triples from sparse samples in the field of coal mine electromechanical equipment. Background Art

[0002] With the ever-increasing demand for energy in modern society, coal mines, as one of the main sources of energy supply, play a vital role. In coal mine production operations, the management of electromechanical equipment is a key link in ensuring normal operation and safe production. Traditional coal mine electromechanical equipment management typically relies on human experience and manual operation. This approach suffers from low efficiency, difficulty in comprehensive monitoring, and high operator requirements. With the continuous development and application of information technology, a series of intelligent solutions have emerged in the field of coal mine electromechanical equipment management. Among them, triple extraction technology, as a key means of information extraction, provides new ideas for improving the efficiency and accuracy of coal mine electromechanical equipment management.

[0003] In the field of coal mining electromechanical equipment, due to its unique and specialized nature, labeled data in this area is often very limited and suffers from sparse relationships. This makes supervised learning methods in related technologies face data shortages and imbalanced data types when training models, making it difficult to achieve satisfactory results. Furthermore, datasets in this field contain complex entity names and a large number of overlapping triples. These issues greatly increase the difficulty of model extraction, making it difficult to overcome bottlenecks and further improve performance. The above background information does not necessarily constitute prior art. Summary of the Invention

[0004] The purpose of this application is to provide a method and device for extracting triples of sparse samples in the field of coal mine electromechanical equipment.

[0005] The technical solution of this application is achieved as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for extracting triples of sparse samples in the field of coal mine electromechanical equipment, comprising:

[0007] Use the BERT model as a training model for text vectorization;

[0008] Bi-LSTM is used to integrate the Span context semantic relationship during the Span extraction process;

[0009] Perform relationship classification on candidate Span entity pairs through relationship classifier;

[0010] Among them, the sliding window mechanism is used in the Span extraction process to represent each entity as a continuous subsequence.

[0011] In some embodiments, the BERT model consists of multiple Transformer layers, each of which contains multiple sublayers, including a multi-head self-attention mechanism and a feedforward neural network. Input text undergoes token embedding and position embedding processing, then passes through multiple Transformer layers to encode and extract information. Finally, a pooling layer aggregates the representations of all positions and outputs them to the output layer for task-related prediction or other downstream tasks.

[0012] In some embodiments, the use of Bi-LSTM to fuse Span contextual semantic relationships during the Span extraction process includes:

[0013] Span in the text is iteratively extracted using a sliding window. For sentence X, all possible entity span sets S = {S1, S2, S3, ... S n}, the number of spans generated for sentence X is shown in formula (1), where N is the length of the text of X,

[0014]

[0015] Use a length vector to represent the length of Span, the length is

[0016] SpanLen i =[S i-beg , S i-and ],(i∈S) (11)

[0017] Set the sliding window, starting at the beginning of the text vector encoded in the embedding layer, and take out the subsequence with the maximum step length of the window each time as the span

[0018] After the Span is traversed using the sliding window, the contextual semantic information in the span is integrated into the extraction model.

[0019] In some embodiments, the use of Bi-LSTM to fuse Span context semantic relationships during the Span extraction process further includes:

[0020] The context around the span traversed in the sliding window is used as input, further encoded using Bi-LSTM, and the encoded result is fused with the word vector inside the span;

[0021] Span after fusion i It will be used to determine whether the span is an entity and participate in the subsequent relationship classification; the vector representation of the span is

[0022] Span i =V i +Bi-LSTM(context)+Span Len i (12)

[0023] Finally, the softmax classifier is used to classify the types of Span. The categories of Span include the set ζ of entity labels, non-entity and the set of relations γ, where

[0024] Label s =soft max(Span x W x +b x ) (13).

[0025] In some embodiments, classifying the relationship of the candidate Span entity pairs using a relationship classifier includes:

[0026] The relation classification layer processes each candidate entity pair (S1, S2) from S, where the value range of the entity pair is the S Cartesian product, that is, S×S, and estimates whether any relationship in the relation set γ holds;

[0027] Span1 and Span2 and the contextual semantic relationship between them are classified by the softmax classifier, given a threshold θ; any relationship category with a score greater than or equal to θ is valid;

[0028] If the scores of all relationship classes are less than the threshold, it is assumed that there is no known relationship between the two entities, as shown in formula (5) and formula (6),

[0029]

[0030] The loss function is the sum of the entity classification loss function and the relationship loss function, as shown in formula (7),

[0031]

[0032] For the design of entity classification loss function, the cross entropy loss function is used, where v is the true label and C is the number of entity labels, as shown in formula (8),

[0033]

[0034] For the loss function of relation classification, a method combining sigmoid and binary classification cross entropy loss function is used, as shown in formula (9):

[0035]

[0036] In a second aspect, the present application provides a triplet extraction device for sparse samples in the field of coal mine electromechanical equipment, comprising:

[0037] The text vectorization module is used to perform text vectorization using the BERT model as a training model;

[0038] The fusion module is used to use Bi-LSTM to fuse the Span context semantic relationship during the Span extraction process;

[0039] The relationship classification module is used to classify the relationship between candidate Span entity pairs through the relationship classifier;

[0040] Among them, the sliding window mechanism is used in the Span extraction process to represent each entity as a continuous subsequence.

[0041] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory; the processor implements the method described in any embodiment of the present application when executing the running program stored in the memory.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present application.

[0043] This application proposes a triple extraction method for sparse samples in the field of coal mine electromechanical equipment. The method uses the BERT model as a training model for text vectorization. It also incorporates span contextual semantic relationships using a Bi-LSTM during span extraction. A relation classifier is then used to classify candidate span entity pairs. A sliding window mechanism is used during span extraction to represent each entity as a continuous subsequence, rather than as individual tokens. This method effectively addresses the issue of overlapping triples. Bi-LSTM is used during entity extraction and relation prediction to integrate span contextual semantics, improving the recognition of overlapping triples. This method also achieves better results in enhancing relation categories within the domain than conventional synonym replacement and back-translation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Shows a flowchart of a method for extracting triples of sparse samples in the field of coal mine electromechanical equipment provided by one embodiment of the present application.

[0045] Figure 2 Shown is a DA-SpBTE model structure diagram provided in one embodiment of the present application.

[0046] Figure 3 Shown is a BERT structure diagram provided by one embodiment of the present application.

[0047] Figure 4 Shows a pie chart of the relationship distribution of the training set provided by one embodiment of the present application.

[0048] Figure 5 Shows a block diagram of an electronic device structure provided by one embodiment of the present application.

[0049] Figure 6 Shown is a schematic diagram of a computer-readable storage medium provided by one embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to further illustrate the features and technical content of the embodiments of the present application, the following will be combined with the accompanying drawings and specific embodiments to further illustrate the technical solutions of the present application. The drawings are for reference only and are not intended to limit the scope of the embodiments of the present application.

[0051] Unless otherwise defined, all technical and scientific terms used in the examples of this application should have the same meaning as commonly understood by those skilled in the art. The use of these terms is for the purpose of describing the examples of this application only and is not intended to limit this application in any way.

[0052] In the following description, references to "some embodiments" represent a subset of all possible embodiments. However, it should be understood that "some embodiments" may be the same or different subsets of all possible embodiments, and may be combined without conflict. It should also be noted that if terms such as "first, second, and third" appear in the embodiments of this application, these terms are used solely to distinguish similar objects and do not represent a specific order or sequence. Therefore, the terms "first, second, and third" may be used interchangeably where permitted, and the embodiments of this application do not need to be executed in a strict order as described.

[0053] The triple extraction task involves extracting entities and relationships from unstructured text and predicting and classifying these relationships. For example, for the unstructured text "XGJL138 fully hydraulic crawler multifunctional pile frame is a new multifunctional pile-driving machine independently developed by Xinguo Heavy Machinery," the triple extraction task involves obtaining triplet knowledge data such as "<Xinguo Heavy Machinery, developed, XGJL138 fully hydraulic crawler multifunctional pile frame>."

[0054] Mainstream methods for deep learning relationship extraction currently fall into two categories: supervised and distantly supervised. Supervised entity relationship extraction can be categorized by extraction method: pipeline and joint. The pipeline method first extracts entities from unstructured text and then predicts relationships based on entity pairs. The joint method extracts both entities and relationships simultaneously in an end-to-end manner. A major drawback of the pipeline model is the error propagation problem. This is because the entity extraction and relationship classification models are independent and non-dependent. Errors such as recognition errors and omissions in the entity extraction stage cannot be corrected or altered, directly affecting the performance of the relationship classification stage.

[0055] Due to the confidentiality and privacy restrictions of data in the field of coal mine electromechanical equipment, relevant data is difficult to obtain. At the same time, the amount of text data written by professional experts in this field is limited, which makes the scarcity of the data set very obvious. Entities in the field of coal mine electromechanical equipment often involve multiple components, including equipment names, equipment parameters, equipment characteristics, etc. The way and order in which these entity components appear in different texts are different, and the components of the entities are complex, especially the types of electromechanical equipment and accessories, which increases the difficulty of entity recognition and extraction. For example, the entity "Komatsu PC500LC-10M0 excavator" contains both equipment models mixed with English and numbers, and Chinese equipment names, and even punctuation marks. Therefore, in the triple extraction task in the field of coal mine electromechanical equipment, it becomes extremely difficult to obtain the corresponding domain data set and the samples are extremely sparse, resulting in training. Training a model may face the problem of insufficient data, which can affect model performance and generalization. Furthermore, multiple relationship triplets may appear simultaneously in the data, leading to overlap. For example, a text may mention multiple parameters and characteristics of a device, making triple extraction more challenging. For example, "The Renault KERAX is equipped with a DXi 440 horsepower engine with a maximum output torque of 2000 Nm." The entity "Renault KERAX" has relationships with both the entities "DXi 440 horsepower engine" and "maximum output torque of 2000 Nm," resulting in overlapping triples. Related technologies for triple extraction in the field of coal mine electromechanical equipment face challenges such as difficulty in data acquisition, complex entity composition within the domain, and triple overlap. Targeted solutions are needed to improve model performance.

[0056] In response to the problems existing in the related art, the embodiment of the present application proposes a triplet extraction method for sparse samples in the field of coal mine electromechanical equipment, DA-SpBTE (Domain-Ausmented Span-BiLSTM Triplet Extraction Method for Sparse Samples in the Coal Mining Equipment Domain). The model uses the BERT model as a training model for text vectorization, and uses Bi-LSTM to fuse the Span context semantic relationship during the Span extraction process. Then, the candidate Span entity pairs are classified by the relationship classifier. The Span method uses a sliding window mechanism, which represents each entity as a continuous subsequence (Span) rather than a separate tag, which can effectively solve the triplet overlap problem. At the same time, a dataset MED (MineEquipDataset) in the field of coal mine electromechanical equipment and a dictionary MEL (MineEquip Lexicon is used to perform targeted data augmentation on sparse samples, enabling the model to better discern complex entities. Experimental results on a dataset augmented with domain-specific lexicon data show that the DA-SpBTE method achieves an F1 score of 67.41%, significantly improving the F1 score compared to six currently popular triplet extraction models, validating the effectiveness of the method. Ablation experiments also demonstrate that using specialized domain-specific lexicon data augmentation for sparse samples significantly outperforms other augmentation methods.

[0057] This embodiment of the present application provides a triple extraction method for sparse samples in the field of coal mine electromechanical equipment. It constructs an end-to-end Chinese triple joint extraction model, DA-SpBTE, using Chinese-BERT-WWM-EXT as a pre-trained model to enhance semantic understanding of complex entities. Bi-LSTM is used in the entity extraction and relationship prediction stages to integrate span contextual semantics, improving the recognition of overlapping triples. A coal mine electromechanical equipment dataset (MED) and a coal mine electromechanical equipment domain dictionary (MEL) are constructed to enhance corpus with sparse relationship categories in the dataset and address the sample sparsity issue. Experiments conducted on the sparse coal mine electromechanical equipment dataset and the dataset with enhanced relationship categories demonstrate that the proposed model significantly outperforms methods used in related technologies. Furthermore, the constructed coal mine electromechanical equipment domain dictionary is more effective in enhancing relationship categories within the domain than conventional synonym replacement and back translation methods.

[0058] refer to Figure 1 As shown, an embodiment of the present application provides a method for extracting triples of sparse samples in the field of coal mine electromechanical equipment, which may include:

[0059] S10. Use the BERT model as a training model for text vectorization;

[0060] S20. Use Bi-LSTM to integrate the Span context semantic relationship during the Span extraction process;

[0061] S30, performing relationship classification on the candidate Span entity pairs through a relationship classifier;

[0062] Among them, the sliding window mechanism is used in the Span extraction process to represent each entity as a continuous subsequence.

[0063] The DA-SpBTE triple extraction model mainly consists of three parts, namely the text embedding layer using the pre-trained model BERT, the span extraction layer based on Bi-LSTM span context semantic enhancement, and the relationship classification layer. The process structure of the model is as follows Figure 2 shown.

[0064] Use the text embedding layer of the pre-trained model BERT:

[0065] Since the experiments were conducted on a Chinese dataset, the Chinese pre-trained model, Chinese-BERT-WWM-EXT, was selected as the pre-training model in this module. Pre-trained on a large-scale Chinese corpus, Chinese-BERT-WWM-EXT is sensitive to and understands the characteristics of Chinese text, enabling it to better capture the complexity and semantic nuances of the Chinese language. Furthermore, it utilizes a character-level segmentation approach, which better handles the common splitting and merging of words in Chinese, thereby improving the model's understanding of Chinese context. It also employs a full-word masking strategy, which better learns the contextual relationships and semantic representations of words, helping to enhance the model's accuracy and generalization in extraction tasks.

[0066] BERT consists of multiple Transformer layers, each of which contains multiple sublayers, including a multi-head self-attention mechanism and a feed-forward neural network. The input text is processed by token embedding and position embedding, and then encoded and extracted by multiple Transformer layers. Finally, the representations of all positions are aggregated through the pooling layer and finally output to the output layer for task-related prediction or other downstream tasks. The main structure of BERT is as follows: Figure 3 shown.

[0067] Span Extraction Layer with Enhanced Span Context Semantics Based on Bi-LSTM:

[0068] Different from the traditional BIO annotation method that uses seq2seq to detect entities, the Span-based method is more intuitive and indirect. It directly annotates the entire scope of a named entity or relationship without the need to annotate each token. This makes it easier for the model to understand and learn the relationships between entities. Moreover, the BIO annotation method assigns only one annotation to each word, which means that an entity cannot be part of multiple entities at the same time, so it cannot solve the case of nested entities. For example, in "China XX Group", "China" has labels including "country" and "enterprise". If it is BIO annotation, it is impossible to assign two labels to it, and thus one entity will be lost.

[0069] In this module, a method similar to a sliding window is adopted to iteratively extract Spans in the text. For sentence X, first extract all possible entity Span sets s = {S1, S2, S3,... S n}, for example, for the sentence "China XX Group", its possible Spans are S = {(中), (中国)…(中国某某)…(中国某某集团)}. The number of Span generations for sentence X is shown in formula (1), where N is the text length of X.

[0070]

[0071] Since the boundaries and lengths of each Span in S are different, a length vector is needed to represent the length of the Span. The length formula is shown in formula (2). After counting the entity lengths in the dataset, the Span length is restricted to 20 because too long Spans are less likely to be entities and will greatly increase the computational complexity.

[0072] Span Len i =[S i-beg ,S i-and , (i ∈ S) (20)

[0073] Based on the experiment, a sliding window with a maximum stride of 20 is set, starting at the beginning of the text vector encoded in the embedding layer. Each time, a subsequence with a stride of 20 is extracted as a span. After traversing the span using the sliding window, it is necessary to consider how to integrate the contextual semantic information in the span into the extraction model to enhance its performance and generalization. Related methods often use max pooling to obtain contextual semantics. This method ignores the order of words in the context sequence, resulting in loss of sequence information. Furthermore, the window size of max pooling is fixed, typically specified by a hyperparameter. This results in the model only focusing on a fixed-length context and ignoring information in longer text sequences. When processing long texts, this can lead to the loss of important contextual semantic information. Max pooling only focuses on the maximum features of the partial words within the window, which may not fully capture the global contextual semantic relationships within the span. This can lead to poor recognition of complex entities or relationships within the span.

[0074] Therefore, a method of using Bi-LSTM to capture contextual semantics is proposed. Bi-LSTM is a bidirectional long short-term memory network that can process forward and backward context information simultaneously, thereby effectively capturing contextual semantic relationships in the sequence.

[0075] The context near the Span traversed in the sliding window is used as input, and Bi-LSTM is used to further encode it. The encoded result is fused with the word vector inside the Span to obtain a more comprehensive and rich Span representation. The fused Span i The representation will be used to determine whether the span is an entity and participate in subsequent relationship classification. The vector representation of span is shown in formula (3).

[0076] Span i =V i +Bi-LSTM(context)+Span Len i (twenty one)

[0077] Finally, the softmax classifier is used to classify the types of Span. The categories of Span include the set ζ of entity labels, non-entity And the relationship set 7 is shown in formula 4.

[0078] Label x =softmax(Span x W x +b x ) (twenty two).

[0079] Relationship classification layer:

[0080] The relationship classification layer processes each candidate entity pair (S1, S2) from S. Because overlapping entities are included, the value range of the entity pair is the Cartesian product of S, that is, S×S, and estimates whether any relationship in the relationship set γ holds. Span1 and Span2 and the contextual semantic relationship between them are classified by the softmx classifier. A threshold θ is given. After hyperparameter search, the threshold is set to 0.4 in the experiment. Any relationship category with a score greater than or equal to θ is considered valid. If the scores of all relationship categories are less than the threshold, it is assumed that there is no known relationship between the two entities. This is shown in Formulas (5) and (6).

[0081]

[0082] The loss function is the sum of the entity classification loss function and the relationship loss function, as shown in formula (7). For the design of the entity classification loss function, the cross entropy loss function is used, where v is the true label and c is the number of entity labels, as shown in formula (8). For the relationship classification loss function, a method combining sigmoid and binary classification cross entropy loss functions is used, as shown in formula (9).

[0083]

[0084]

[0085] experiment:

[0086] We used a crawler to crawl unstructured text about coal mine electromechanical equipment from websites such as the First Engineering Machinery Network, Sany Heavy Industry's official website, and Yigouji. After keyword extraction, segmentation, and noise cleaning, we annotated 500 pieces of raw data. The annotations of the relationship types, including overlapping triples, are shown in Table 1.

[0087] Table 1 Dataset relationship types

[0088]

[0089] Table 1 shows that the dataset distribution is highly uneven. This is partly due to the scarcity of online data, which prevents targeted data capture to supplement the scarce dataset. Furthermore, some relationships in overlapping triples appear relatively frequently, a natural problem inherent in the domain. For example, a company may develop multiple products but only have a few partners.

[0090] Using the entities extracted by the EMSS model from 500,000 raw data items scraped from the web, we constructed a dataset containing 18,504 entities in the field of coal mine electromechanical equipment. The entity types and numbers are shown in Table 2.

[0091] Table 2 Introduction to domain dictionaries

[0092]

[0093]

[0094] 100 samples were selected from the 500 original data as validation set to ensure class balance, and the remaining 400 samples were used as training set. The sparse samples in the training set were targeted for enhancement, while the non-sparse samples were randomly expanded to prevent overfitting, and the total number of enhancements was 8000. The data distribution is as follows Figure 4 The component statistics are shown in Table 3.

[0095] Table 3 Statistics of entity relationship components in the training set

[0096]

[0097] As can be seen from Table 3, in the training set, SEO and EPO account for 25% and 4.1% of the total relations, respectively. Overlapping triples account for 29.1% of the total, and complex entities account for 82.7% of the total number of entities.

[0098] Evaluation indicators

[0099] Since there are multiple classes of relationship categories, the multi-classification model evaluation indicators precision (Precision), recall (Recall), and Macro_F1 value are selected for evaluation. Precision refers to the actual correct proportion of samples predicted by the model as positive. Recall refers to the proportion of samples successfully predicted by the model as positive among all samples that actually have a relationship. The F1 score takes into account the precision and recall rates, and is the harmonic mean of the two, which is used to comprehensively evaluate the performance of the model. The formulas for precision and recall are shown in (10) and (11). For each category, the corresponding precision and recall rates are counted and then Macro_P and Macro_R are calculated. As shown in formulas (12-14).

[0100]

[0101] Among them, TP represents the number of correctly identified entity-relationship pairs, FP represents the number of incorrectly identified entity-relationship pairs, and FN represents the number of unrecognized entity-relationship pairs.

[0102] Experimental parameter settings:

[0103] The experiment was run on the Linux Ubuntu operating system, using Python 3.6.13 and PyTorch version 1.7.1. Based on the server's GPU size, the train_batch_size was set to 8, the eval_batch_size was set to 2, the number of training rounds was 400 epochs, and the dropout setting was 0.1. The pre-trained model used was Chinese-BERT-WWM-EXT. The remaining training parameters are shown in Table 4.

[0104] Table 4 Training parameters

[0105]

[0106] Comparative experiment setup and analysis:

[0107] To verify the effectiveness of the method, the proposed DA-SpBTE method was compared with other models. The comparison models selected in the experiment are mainstream models of entity relationship extraction in different fields and different methods in recent years:

[0108] The first model is NovelTagging, which provides richer information by annotating entities with their types, attributes, and relationships. It is well-suited for relation extraction tasks, especially when faced with complex relationships and multiple attributes.

[0109] The second model is CopyRE, which enhances the relation extraction model through a copy mechanism and can better handle unknown entities and scarce relations. It performs well in situations where there is a lack of training data or rare relations.

[0110] The third model, DPointer, uses a pointer network to solve the entity reference problem in relation extraction. It can effectively handle reference problems and entity resolution. It has good performance on complex tasks such as cross-sentence relation extraction.

[0111] The fourth model is Casrel, which uses a cascaded pointer network to model relationships between entities. This model effectively captures the interaction information between entity pairs. It performs well in relation extraction tasks that require considering contextual interactions.

[0112] The fifth model, ETL-Span, combines entity, type, and link span information into a multi-task learning framework. It can simultaneously consider entity type information in relation extraction tasks, providing a more comprehensive semantic representation.

[0113] The sixth model is RDA, which uses a head entity recognizer and a tail entity classifier to mine all possible relationships between the subject and the subject. It also uses multiple rounds of adversarial attacks to enhance the model's robustness and extract complex and implicit relationships.

[0114] Table 5 shows the comparative experimental results of six models on the MED dataset. Compared with existing mainstream triple extraction models, the proposed DA-SpBTE method achieves the highest recognition performance, verifying the effectiveness of the method.

[0115] Table 5 Comparative experimental results

[0116]

[0117] Specifically, the NovelTagging method achieved an F1 score of 30.80%. This method requires a lot of annotation work and manual labeling, so it may not be suitable when the dataset is small or the annotation resources are limited. In addition, its annotation method cannot solve the problem of overlapping triplets.

[0118] The CopyRE method achieved an F1 score of 43.06%, a 12.26% improvement over NovelTagging. Because CopyRE uses a generative approach, it can generate diverse relational representations. In the case of overlapping triples, the model can generate multiple different relational representations to accommodate the overlap. However, its reliance on surface entity information in the text may limit its accuracy for more complex relation extraction tasks, which may require more contextual semantic information.

[0119] DPointer uses a dual-pointer mechanism to extract certain overlapping entities, achieving an F1 score of 52.32%, a 9.26% improvement over CopyRE. However, it is limited by the length of the text; setting it too large will significantly increase the model training time.

[0120] Casrel proposed a new triple extraction paradigm that models relationships as functions from subject to object, rather than treating relationships and entities as discrete labels. This further improves the recognition of overlapping triples, achieving an F1 score of 57.68%, a 5.36% improvement over DPointer.

[0121] ETL-Span uses a span-based extraction method. In the first step of entity extraction, only entities that may participate in the target triple are extracted. Its F1 score is 57.57%, a 1.07% improvement over Casrel. As a multi-task learning framework, it may require more training data and labels, so it may not perform well when data is limited.

[0122] The RDA method also employs the cascaded pointer framework and uses BERT as a pre-trained model. Thanks to multiple rounds of adversarial attacks, the model's generalization performance is strong, with F1 scores improving by 4.93% and 3.86% over Casrel and ETL-Span, respectively. The proposed DA-SpBTE method achieves the best performance, with an F1 score improvement of 4.8% over the top-performing RDA method and 8.66% over the span-based ETL-Span method. This is due to the Bi-LSTM's ability to model the contextual semantics of token sequences within spans (continuous subsequences). When extracting spans, it captures sequential information between tokens, enabling a better understanding of the contextual semantic relationships between entities and relations. When processing text sequences with long-range dependencies, the Bi-LSTM can capture longer-term contextual information, enabling better modeling of complex semantic relationships between entities and relations.

[0123] When using MaxPool for contextual semantic modeling, the weight of each tag is fixed, while Bi-LSTM can dynamically adjust the importance of each tag according to the specific task and input text by learning dynamic weight distribution, thus more flexibly handling different sample situations.

[0124] In span-based triple extraction, the span length may be variable. Bi-LSTM can process spans of different lengths, while MaxPool can only output vectors of fixed length. This makes Bi-LSTM more flexible in processing spans of different lengths.

[0125] Therefore, the DA-SpBTE model has a better understanding of the semantics of complex entities within the domain and the relationships between entity pairs than other models, improving the accuracy of relationship recognition. Sample examples of the comparative experiment are shown in Table 6.

[0126] Table 6 Comparative experimental samples

[0127]

[0128]

[0129] Table 6 shows that the sample contains complex and overlapping entities, such as "TBM XX 237." The NovelTagging method only extracted two common triples, failed to identify overlapping triples, and inaccurately extracted complex entities. The CopyRE and DPointer methods recognized the complex overlapping entity but did not fully identify its other relationships. Casrel and ETL-Span recognized four triples, but their incomplete recognition of complex entities impacted subsequent knowledge graph construction. The RDA method, limited by pointer length, did not fully recognize complex entities. Increasing the pointer length exponentially increases training time, so setting the pointer length too large is not recommended. The proposed DA-SpBTE method performed best, not only identifying all overlapping triples but also accurately identifying complex entities. This method addresses the challenges encountered by other compared models and demonstrates its effectiveness.

[0130] Ablation experiment:

[0131] To verify the impact of data augmentation using the domain dictionary (MEL) proposed in this paper on experimental results, we conducted comparative experiments using different datasets. MED is the original dataset, while NLPCDA uses the nlpcda toolkit to augment the training set with data, including synonym replacement. BackTranslate uses the Baidu API to augment the training set with back-translation from 16 common languages. The MEL dataset uses the domain dictionary for data augmentation. Five experiments were conducted on each dataset, and the results were averaged. The ablation test results are shown in Table 7.

[0132] Table 7 Ablation experiment results

[0133]

[0134]

[0135] Table 7 shows that the training performance of the dataset augmented with domain dictionary data is 11.7% and 15.62% higher than that of the nlpcda synonym replacement and back-translation methods on the test set, respectively. The results show that after domain dictionary data augmentation, the F1 score of sparse categories (generate, collaborate, and have) improves significantly compared to the other two datasets. This is because using the domain dictionary for data augmentation ensures that the generated new samples are relevant to the domain, enhancing the model's generalization ability in a specific domain. Furthermore, words in the domain dictionary often have certain correlations with other words in the same domain. During data augmentation, replacing words based on the domain dictionary can better maintain the contextual relevance between words, making the generated new samples more natural and fluent.

[0136] Domain dictionary data augmentation is superior to common synonym replacement in maintaining semantic consistency, contextual relevance, and data diversity, which helps improve the generalization ability of the model and thus improve the performance of the model in specific domain tasks.

[0137] The results of the two models show that using Bi-LSTM to integrate contextual semantics achieves higher performance than the maxpooling method for each type of relationship. This is because using Bi-LSTM to capture contextual semantics more comprehensively captures the contextual semantic information of the span, thereby improving the model's performance in the triple extraction task. Furthermore, the bidirectional encoding feature of Bi-LSTM helps the model better process the contextual semantic information in the span, improving the model's extraction performance.

[0138] To address the sparse sample problem in the field of coal mine electromechanical equipment, this embodiment proposes a Chinese triple extraction method, DA-SpBTE, based on span annotation and using Bi-LSTM to integrate contextual semantic relationships during span extraction. A coal mine electromechanical equipment dataset (MED) and a coal mine electromechanical equipment domain dictionary (MEL) are constructed to expand the data volume. This method not only balances and enhances the relationship categories in the sparse data within the dataset, but also enables the model to better recognize complex entities, thereby improving model performance and robustness. Experiments demonstrate that the proposed DA-SpBTE method outperforms other traditional triple extraction models on the coal mine electromechanical equipment dataset.

[0139] In this application, a novel end-to-end Chinese triple extraction model, DA-SpBTE, is proposed. Using Chinese-BERT-WWM-EXT as a pre-trained model, it enhances semantic understanding of complex entities. Bi-LSTM is used in the entity extraction and relationship prediction stages to integrate span contextual semantics, improving the recognition of overlapping triples. A coal mine electromechanical equipment dataset (MED) and a coal mine electromechanical equipment domain dictionary (MEL) are constructed to enhance corpus with sparse relationship categories in the dataset and address the issue of sample sparsity. Experiments conducted on the sparse coal mine electromechanical equipment dataset and the dataset with enhanced relationship categories demonstrate that the proposed model significantly outperforms methods used in related technologies. Furthermore, the constructed coal mine electromechanical equipment domain dictionary demonstrates superior performance in enhancing relationship categories within the domain compared to conventional synonym replacement and back-translation methods.

[0140] To address issues in the field of coal mine electromechanical equipment, this application proposes a span-based Chinese triple extraction method, DA-SpBTE. This method incorporates contextual semantics into span vectors, enabling more accurate entity identification that better reflects domain characteristics. A dictionary, MEL, is constructed for the coal mine electromechanical equipment field. This can be used to augment NER datasets and triple extraction datasets, reducing data annotation time while improving model generalization under sparse conditions and enhancing model extraction performance. Comparative experimental results with six currently available mainstream models demonstrate the effectiveness of the DA-SpBTE method, and ablation experiments confirm that MEL is more effective than other data augmentation methods.

[0141] Another embodiment of the present application provides a triplet extraction device for sparse samples in the field of coal mine electromechanical equipment, comprising:

[0142] The text vectorization module is used to perform text vectorization using the BERT model as a training model;

[0143] The fusion module is used to use Bi-LSTM to fuse the Span context semantic relationship during the Span extraction process;

[0144] The relationship classification module is used to classify the relationship between candidate Span entity pairs through the relationship classifier;

[0145] Among them, the sliding window mechanism is used in the Span extraction process to represent each entity as a continuous subsequence.

[0146] For example, BERT consists of multiple Transformer layers, each of which contains multiple sublayers, including a multi-head self-attention mechanism and a feedforward neural network. Input text is processed through Token Embedding and Position Embedding, then passes through multiple Transformer layers for information encoding and extraction. Finally, a Pooling Layer aggregates the representations of all positions and outputs them to the output layer for task-related prediction or other downstream tasks.

[0147] For example, Bi-LSTM is used to fuse the Span context semantic relationship during the Span extraction process, including:

[0148] Span in the text is iteratively extracted using a sliding window. For sentence X, all possible entity span sets S = {S1, S2, S3, ... S n}, the number of spans generated for sentence X is shown in formula (1), where N is the length of the text of X,

[0149]

[0150] Use a length vector to represent the length of Span, the length is

[0151] Span Len i =[S i-beg , S i-and ], (i∈S);

[0152] Set the sliding window, starting at the beginning of the text vector encoded by the embedding layer, and take the subsequence with the maximum step length of the window as the span each time;

[0153] After the Span is traversed using the sliding window, the contextual semantic information in the span is integrated into the extraction model.

[0154] Exemplarily, the use of Bi-LSTM to fuse Span context semantic relationships during the Span extraction process further includes:

[0155] The context around the span traversed in the sliding window is used as input, further encoded using Bi-LSTM, and the encoded result is fused with the word vector inside the span;

[0156] The fused Spani representation will be used to determine whether the span is an entity and participate in subsequent relationship classification; the vector representation of the span is

[0157] Span i =V i +Bi-LSTM(context)+Span Len i ,

[0158] Finally, the softmax classifier is used to classify the types of Span. The categories of Span include the set ζ of entity labels, non-entity and the set of relations γ, where

[0159] Label x =softmax(Span x W x +b x ).

[0160] Exemplarily, the performing relationship classification on the candidate Span entity pairs by using a relationship classifier includes:

[0161] The relation classification layer processes each candidate entity pair (S1, S2) from S, where the value range of the entity pair is the S Cartesian product, that is, S×S, and estimates whether any relationship in the relation set γ holds;

[0162] Span1 and Span2 and the contextual semantic relationship between them are classified by the softmax classifier, given a threshold θ; any relationship category with a score greater than or equal to θ is valid;

[0163] If the scores of all relationship classes are less than the threshold, it is assumed that there is no known relationship between the two entities. The formulas are

[0164]

[0165] The loss function is the sum of the entity classification loss function and the relationship loss function, and the formula is

[0166]

[0167] For the design of entity classification loss function, the cross entropy loss function is used, where v is the true label and C is the number of entity labels. The formula is

[0168]

[0169] For the loss function of relation classification, a method combining sigmoid and binary classification cross entropy loss function is used, as shown in the following formula:

[0170]

[0171] Another embodiment of the present application provides an electronic device, comprising a processor and a memory; the processor implements the method described in any embodiment of the present application when executing the running program stored in the memory.

[0172] For example, Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device, provided as a specific example, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604. The processor 601, the communication interface 602, and the memory 603 communicate with each other via the communication bus 604. The memory 603 is used to store computer programs; the processor 601 is used to execute the programs stored in the memory 603, thereby implementing the methods provided in any embodiment of the present application.

[0173] In the above-mentioned electronic devices, the communication bus may be a peripheral component interconnect standard bus or an extended industry standard architecture bus. The communication bus may include an address bus, a data bus, and a control bus. The communication interface is used to implement data exchange between the electronic device and other devices.

[0174] Memory 603 may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk drive. Furthermore, memory 603 may include other storage devices located remotely from processor 601. Processor 601 may be a general-purpose processor, such as a central processing unit (CPU) or a network processor (NP). It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other types of programmable logic devices, discrete gate circuits or transistor logic devices, or discrete hardware components.

[0175] Another embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in any embodiment of the present application when the computer program is executed by a processor. Figure 6 As shown, Figure 6 The computer-readable storage medium shown is a disc 20, on which a computer program (ie, a program product) is stored. When the computer program is run by the processor, the method described in any embodiment of the present application can be implemented.

[0176] The computer-readable storage medium of this embodiment can be any type of medium that can be accessed by a computer, or a data storage device such as a server or data center that integrates multiple media. Common storage media types include magnetic media (such as floppy disks, hard disks, and magnetic tapes), optical media (such as DVDs), or semiconductor media (such as solid-state drives (SSDs)).

[0177] In the above embodiments, all or part of the functions can be implemented through software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the functions can be implemented through one or more computer instructions contained in a computer-readable storage medium. When a computer loads and executes these computer program instructions, it can partially or completely perform the processes or functions described in the embodiments of the present invention.

[0178] The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium and can also be transmitted from one computer-readable storage medium to another via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. For example, computer instructions can be transmitted from a website, computer, server, or data center to another computer, website, server, or data center.

[0179] It should be noted that:

[0180] In the embodiments of this application, the terms "comprise," "include," and any other variations thereof are intended to indicate a non-exclusive inclusion relationship, meaning that when a process, method, article, or apparatus is referred to as including certain elements, the process, method, article, or apparatus described includes not only those elements but also other elements not explicitly listed, or elements inherent to the process, method, article, or apparatus. Without further qualification, the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0181] Through the description of the above-mentioned embodiments and methods, those skilled in the art will clearly understand that these methods can be implemented through software plus the necessary general-purpose hardware platform. Of course, they can also be implemented through pure hardware, but in many cases, the former is usually a more optimal implementation method. Based on this understanding, the technical solutions of the embodiments of this application, or the technical contributions made by this application, can be embodied in the form of a software product. This software product is stored in a storage medium (such as ROM / RAM, disk, CD, etc.) and contains a number of instructions to enable an image display device (such as a mobile phone, computer, server, air conditioning equipment, or network equipment, etc.) to execute the various methods described in the embodiments of this application.

[0182] The above description is a specific implementation of the embodiment of the present application, but does not limit the scope of protection of the present application. Any changes or alternatives that can be thought of by any technician familiar with this technical field within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be determined according to the content of the claims.

Claims

1. A triplet extraction method for sparse samples in the field of coal mine electromechanical equipment, characterized in that: include: Use the BERT model as a training model for text vectorization; Bi-LSTM is used to integrate the Span context semantic relationship during the Span extraction process; Perform relationship classification on candidate Span entity pairs through relationship classifier; Among them, the sliding window mechanism is used in the Span extraction process to represent each entity as a continuous subsequence.

2. The method according to claim 1, characterized in that The BERT is composed of multiple Transformer layers, each of which contains multiple sublayers, including a multi-head self-attention mechanism and a feedforward neural network. The input text is processed by Token Embedding and Position Embedding, and then encoded and extracted by multiple Transformer layers. Finally, the representations of all positions are aggregated through the Pooling Layer and output to the output layer for task-related prediction or other downstream tasks.

3. The method according to claim 1, characterized in that The Bi-LSTM is used to fuse the Span context semantic relationship during the Span extraction process, including: Use sliding window to iteratively extract span in text, for sentence X , first extract all possible entity Span sets of X The number of spans generated for sentence X is shown in formula (1), where N is the length of the text of X. Use a length vector to represent the length of Span, the length is Set the sliding window, starting at the beginning of the text vector encoded by the embedding layer, and take the subsequence with the maximum step length of the window as the span each time; After traversing the span using the sliding window, the contextual semantic information in the span is integrated into the extraction model.

4. The method according to claim 3, characterized in that The use of Bi-LSTM to fuse Span context semantic relationships during the Span extraction process also includes: The context around the span traversed in the sliding window is used as input, further encoded using Bi-LSTM, and the encoded result is fused with the word vector inside the span; Span after fusion i It will be used to determine whether the span is an entity and participate in the subsequent relationship classification; the vector representation of the span is Span i =V i +Bi-LSTM(context)+Span Len i (3) Finally, the softmax classifier is used to classify the type of Span. The category of Span includes the set of entity labels. Non-entity and relation set 7, where 5. The method according to claim 1, wherein The process of performing relationship classification on the candidate Span entity pairs by using a relationship classifier includes: The relation classification layer processes each candidate entity pair (S1, S2) from S, where the value range of the entity pair is the Cartesian product of S, that is, S×S, and estimates whether any relationship in the relation set γ holds; Span1 and Span2 and the contextual semantic relationship between them are classified by the softmax classifier, given a threshold θ; any relationship category with a score greater than or equal to θ is valid; If the scores of all relationship classes are less than the threshold, it is assumed that there is no known relationship between the two entities, as shown in formula (5) and formula (6), The loss function is the sum of the entity classification loss function and the relationship loss function, as shown in formula (7), For the design of entity classification loss function, the cross entropy loss function is used, where y is the true label and C is the number of entity labels, as shown in formula (8), For the loss function of relation classification, a method combining sigmoid and binary classification cross entropy loss function is used, as shown in formula (9):

6. A triplet extraction device for sparse samples in the field of coal mine electromechanical equipment, characterized in that: include: The text vectorization module is used to perform text vectorization using the BERT model as a training model: The fusion module is used to use Bi-LSTM to fuse the Span context semantic relationship during the Span extraction process; The relationship classification module is used to classify the relationship between candidate Span entity pairs through the relationship classifier; Among them, the sliding window mechanism is used in the Span extraction process to represent each entity as a continuous subsequence.

7. An electronic device, characterized in that: The method comprises a processor and a memory; when the processor executes the operating program stored in the memory, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images