Event causal relationship mining method based on CRF mark learning

Abstract

The invention discloses an event causal relationship mining method based on CRF mark learning, which comprises the following steps: carrying out data preprocessing on a selected reference data set, and extracting tetrad data through relationship extraction; constructing a Re-net baseline model, combining a CRF algorithm in the Re-net baseline model, clustering to generate a pseudo mark for auxiliary discrimination, connecting forward and backward hidden states, observing context information of a tag sequence, performing tag sequence modeling by using CRF, and constructing a first mining model; inputting the preprocessed data into the first mining model for training and optimizing through a maximum probability criterion, and obtaining an optimized first mining model after the evaluation index reaches the standard; and performing event causal relationship mining on large-scale text data by utilizing the optimized first mining model.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and knowledge graph technology, and in particular to an event causal relationship mining method based on CRF labeling learning. Background Art

[0002] The large amount of text data makes natural language understanding tasks and natural language processing research face endless challenges and increasingly innovative requirements. The causal relationship between events can help people predict the events that are about to happen and make corresponding preventive or remedial measures based on the predictions. Therefore, event causal relationship identification has become an important research direction in natural language processing tasks.

[0003] The development of natural language processing (NLP) technology has made it possible to extract useful information from large amounts of text data. As an effective way to represent knowledge, knowledge graphs can intuitively express complex information by constructing a graph structure of entities and their relationships. However, building a high-quality knowledge graph requires solving the problem of how to efficiently and accurately extract entities and relationships from text.

[0004] As an important part of natural language processing, event causal relationship recognition plays a vital role in determining the causal relationship between events in text. At present, the methods for event causal relationship recognition mainly include rule-based methods, supervised learning-based methods, and deep learning-based methods. The rule-based method classifies the relationship between events through predefined linguistic rules. Although it is simple and effective, it has the problem of low recall rate. The supervised learning-based method uses labeled data to train the model, which has good flexibility and adaptability, but requires a large amount of labeled data. The deep learning-based method automatically learns features through a neural network model and can handle more complex causal relationships, but it relies on large-scale training data and computing resources.

[0005] Conditional Random Field (CRF), as a probabilistic graphical model for sequence labeling, can effectively capture contextual dependencies in sequence data and is widely used in tasks such as named entity recognition and part-of-speech tagging. In event causal relationship recognition, CRF can be used to label events in sequence data and learn the possible dependencies between these events. By introducing the CRF model, not only can the accuracy of causal relationship recognition be improved, but also the time series information can be effectively utilized to improve the robustness and efficiency of the model.

[0006] In recent years, multi-label learning technology has been widely used to learn semantically rich objects, but existing multi-label learning methods all assume that the importance of each label is equal, which is often difficult to hold in practical applications. For example, in event causal relationship recognition, the causal relationship of some events may be more important than that of other events. Therefore, a method that can distinguish the importance of different labels is needed to improve the accuracy of recognition. Summary of the invention

[0007] The present invention provides an event causal relationship mining method based on CRF labeling learning, so as to at least achieve the technical effect of improving the efficiency and accuracy of event causal relationship identification.

[0008] The present invention provides an event causal relationship mining method based on CRF labeling learning, comprising the following steps:

[0009] Selecting a reference data set, and performing data preprocessing on the reference data set, wherein the data preprocessing includes learning the reference data set using a Bert algorithm combined with a CRF-labeled relation extraction algorithm, integrating time objects with time labels, and extracting quadruple data through relation extraction; Build a Re-net baseline model and perform serial modeling on the temporal knowledge graph through global structural reasoning; In the Re-net baseline model, the CRF algorithm is combined to cluster and generate pseudo labels for auxiliary discrimination, connect the forward and backward hidden states, observe the context information of the label sequence, use the CRF algorithm to model the label sequence, and build the first mining model; Inputting the preprocessed data into the first mining model for training, optimizing the first mining model by using the maximum probability criterion, and obtaining the optimized first mining model after the evaluation index reaches the standard; The optimized first mining model is used to mine event causal relationships in large-scale text data.

[0010] Furthermore, the step of using the Bert algorithm combined with the CRF labeled relationship extraction algorithm to learn the reference data set includes: First, the input text is segmented and encoded, and then the BERT model is used to generate the context representation, which is then decoded through the CRF layer to obtain the tag sequence. Given an input sequence , BERT will generate the corresponding context representation , , by combining the CRF layer, given the output representation of BERT , the CRF layer will generate the optimal tag sequence , , Among them, A is the transfer matrix, which represents the transition from label Transfer to label where P is the label score for each position obtained from BERT.

[0011] Furthermore, the step of integrating the time objects with time stamps includes: Identify all time-tagged time objects from the text and extract time information through BERT's contextual representation and CRF algorithm's sequence annotation; Unify the format of identified time objects; Merge multiple time objects related to the same event to remove redundant information; By matching event and time objects, it ensures that all time information is associated with the corresponding event.

[0012] Furthermore, the step of extracting quadruple data by relation extraction includes: Make rules to save only relational data containing time objects; Sorting the relational data according to the time object, and sorting the events by timestamp; The CRF algorithm is combined with the Word2Vec method to extract features and extract the quadruple data of subject, relationship, object and time from the text.

[0013] Furthermore, the step of constructing the Re-net baseline model includes: establishing a local graph to model events occurring simultaneously in the same time window, and extrapolating the graph structure at future timestamps.

[0014] Furthermore, the steps of combining the CRF algorithm with the Re-net baseline model, clustering and generating pseudo labels for auxiliary discrimination, connecting forward and backward hidden states, observing context information of the label sequence, and using the CRF algorithm to model the label sequence to construct the first mining model include: Establishing implicit chain information in the Re-net baseline model; A feature function is defined, and node features in the quadruple data are clustered to generate a desired pseudo-label, the node features are obtained from the embedding updated by back-propagation in the baseline model, the pseudo-label is defined as the category to which the cluster belongs, and modeling is equivalent to the category to which the modeled embedding belongs to assist in discrimination, wherein the state space size of the pseudo-label is reduced to the number of clusters; The BiLSTM method is used to capture the contextual information in the label sequence; The CRF algorithm is used to model the conditional probability distribution of the label sequence. The modeling goal of the CRF algorithm is to calculate the probability of the label sequence under the condition of a given observation sequence. , the probability calculation formula of the CRF algorithm is as follows: , in, is a scoring function used to measure the consistency of a given observation sequence and label sequence. is a normalization factor that ensures that the sum of the probabilities of all possible label sequences is 1.

[0015] Furthermore, the reference data set uses the DuIE2.0 data set for training and verification.

[0016] This application has the following beneficial effects: On the basis of pre-training with large-scale text data, the traditional logical tags are enhanced by CRF tag learning technology to achieve efficient and accurate event causal relationship mining. Focus on solving the problems of low efficiency, low accuracy, poor robustness, etc. of the existing technology, and achieve efficient and accurate recognition requirements as much as possible. The method of the present invention can efficiently and accurately extract the causal relationship between events in large-scale text data. By introducing the CRF algorithm, not only the accuracy of causal relationship recognition can be improved, but also the time series information can be effectively utilized to improve the robustness and efficiency of the model, which has significant advantages for application scenarios that require rapid training and efficient deployment.

[0017] In particular, the method of the present invention has significant advantages in the following aspects: Improve recognition accuracy: By introducing the CRF labeling learning algorithm, the causal relationship between events can be more accurately identified, reducing misjudgments and missed judgments.

[0018] Enhance model robustness: By constructing a temporal network containing time series information, the connection between events is enhanced, and the robustness and adaptability of the model are improved.

[0019] Saving time and resources: The method of the present invention performs well with a low number of training times and can achieve a higher recognition accuracy in a shorter time, thus saving training time and computing resources.

[0020] Wide range of applications: The method of the present invention is not only applicable to the identification of event causal relationships, but can also be extended to other natural language processing tasks, such as named entity recognition, sentiment analysis, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of the overall process of an event causal relationship mining method based on CRF labeling learning provided by an embodiment of the present invention; Figure 2 A flow chart of a method for mining event causal relationships based on CRF labeling learning provided by an embodiment of the present invention; Figure 3 A graph structure of structured data provided by an embodiment of the present invention; Figure 4 The clustering category provided by an embodiment of the present invention is a function Probability diagram of formation; Figure 5 The specific structure of the recurrent neural network model provided by one embodiment of the present invention; Figure 6 A schematic diagram of a recurrent neural network model combined with a CRF model to model label correlation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] As a new learning paradigm, label distribution learning is a generalization of multi-label learning. It converts the logical labels of traditional multi-label learning into label distribution, reflects the different importance of different labels, and is more conducive to mining the intrinsic connection between labels to achieve more accurate classification results. By combining label distribution learning with CRF, the present invention can analyze different events in the text, extract key information and reduce the influence of confounding factors, so as to better judge the causal relationship between events.

[0025] like Figure 1 As shown, it shows a flow chart of an event causal relationship mining method based on CRF labeling learning of the present application, which specifically includes the following steps: Step S101, selecting a reference data set, and performing data preprocessing on the reference data set, wherein the data preprocessing includes using the Bert algorithm combined with the CRF-labeled relation extraction algorithm to learn the reference data set, integrating the time objects with time labels, and extracting quadruple data through relation extraction; In this step, this embodiment selects the DuIE2.0 dataset as the basis for training and verification. The dataset contains more than 430,000 triples, 210,000 Chinese sentences and 48 predefined relationship types. The Bert algorithm is combined with the CRF-labeled relationship extraction algorithm to learn the existing dataset. <t>That is, the objects of time are integrated, relevant rules are formulated and saved in combination with the extracted relationships.

[0026] Step S102, construct a Re-net baseline model, and perform serialization modeling on the temporal knowledge graph through global structural reasoning.

[0027] In this step, the Re-net model is used as a baseline, which serializes and models the temporal knowledge graph through global structural reasoning to predict the occurrence of future events.

[0028] Step S103, combining the CRF algorithm in the Re-net baseline model, clustering to generate pseudo labels for auxiliary discrimination, connecting forward and backward hidden states, observing the context information of the label sequence, using CRF to model the label sequence, and constructing a first mining model; In this step, the CRF algorithm is introduced into the baseline model to enhance the connection between event timing and causality, and improve the ability to mine event causal relationships from network text end-to-end. The model uses solutions based on link prediction and automatic reasoning, and the information from the two perspectives complements each other to achieve performance improvement.

[0029] Step S104, inputting the preprocessed data into the first mining model for training, optimizing the first mining model by using the maximum probability criterion, and obtaining the optimized first mining model after the evaluation index reaches the standard.

[0030] Step S105: Use the optimized first mining model to mine event causal relationships in large-scale text data.

[0031] In one embodiment, the step of learning the reference data set using the Bert algorithm combined with the CRF labeled relationship extraction algorithm includes: First, the input text is segmented and encoded, and then the BERT model is used to generate the context representation, which is then decoded through the CRF layer to obtain the token sequence, where Given an input sequence , BERT will generate the corresponding context representation , , by combining the CRF layer, given the output representation of BERT , the CRF layer will generate the optimal tag sequence , , Among them, A is the transfer matrix, which represents the transition from label Transfer to label where P is the label score for each position obtained from BERT.

[0032] In one embodiment, the step of integrating the time objects with time stamps includes: Identify all time-tagged time objects from the text and extract time information through BERT’s contextual representation and CRF’s sequence annotation; Unify the format of identified time objects; Merge multiple time objects related to the same event to remove redundant information; By matching event and time objects, it ensures that all time information is associated with the corresponding event.

[0033] It should be noted that the BERT+CRF model is used to identify all tags from the text <t>Through the context representation of BERT and the sequence labeling capability of CRF, the time information can be accurately extracted.

[0034] The identified time objects need to be normalized and unified into a standard format. In the paper, the time format is unified as For example, "July 29, 2023" will be normalized to "2023-07-29".

[0035] Merge multiple time objects related to the same event to remove redundant information. Match events and time objects to ensure that all time information is associated with the corresponding event.

[0036] In one embodiment, the step of extracting quadruple data by relation extraction includes: Make rules to save only relational data containing time objects; Sorting the relational data according to time objects and sorting the events by timestamps; The CRF tagging algorithm is combined with the Word2Vec method to extract features and extract the quadruple data of subject, relationship, object and time from the text.

[0037] It should be noted that the rules are formulated to save only relational data containing time objects to ensure the temporal order and causality of the data, thereby ensuring the accuracy of event reasoning. In addition, relational data needs to be sorted according to time objects, and events are sorted by timestamps to ensure that the temporal relationship of events is clear, which is convenient for subsequent causal reasoning and analysis. To this end, the CRF tagging algorithm is combined with the Word2Vec method to extract features and extract (s, r, o, t) quadruple data from the text.

[0038] The steps of relation extraction are as follows: Use Word2Vec to vectorize text.

[0039] First, the input text is segmented, and then the Word2Vec model is used to convert the segmented text into a vector representation. Each word is mapped to a high-dimensional space and represented as a vector : .

[0040] The CRF tagging algorithm is used to extract four-tuple data, namely subject (s), relation (r), object (o) and time (t).

[0041] The word vector representation of the text is input into the CRF model, and the four-tuple data is extracted through the sequence labeling method, namely subject (s), relation (r), object (o) and time (t). Given the feature vector of the input sequence , the CRF model maximizes the conditional probability To predict the labeled sequence Y: , in, is the transfer matrix, is the feature vector score for each position obtained from Word2Vec.

[0042] Based on historical interaction information, multiple relationships between nodes are mined and embedded into a low-dimensional feature space.

[0043] For each node (entity) and its related historical interaction information, the features extracted by the Word2Vec model and the CRF tagging algorithm are embedded into a low-dimensional feature space. The specific steps include: Feature representation: Represent nodes and interaction information as feature vectors.

[0044] Low-dimensional embedding: embedding high-dimensional feature vectors into low-dimensional feature space to reduce dimensionality and improve computational efficiency.

[0045] Construct a temporal network based on the historical interaction relationship between nodes to enhance the connection between event timing and causality.

[0046] Combined with historical interaction data, a temporal network is constructed in chronological order. In a temporal network, nodes represent entities and edges represent interaction behaviors with timestamps: , Among them, the node set V represents all entities, and the edge set represents all interactions before time t. Each edge From the starting point , terminal node and timestamp constitute.

[0047] In one embodiment, the step of constructing the Re-net baseline model includes: establishing a local graph to model events occurring simultaneously in the same time window, and extrapolating the graph structure at future timestamps.

[0048] It should be noted that the Re-net model serializes and models the time series knowledge graph through global structural reasoning to predict the occurrence of future events. It can effectively extract the temporal and structural information in the time series knowledge graph, thereby demonstrating a strong event prediction capability. It models events that occur simultaneously in the same time window by building a local graph, which can be used to extrapolate the graph structure at future timestamps.

[0049] Predicting future events at multiple time points can be abstracted as a sequence and multi-step inference problem.

[0050] Temporally adjacent events may have related semantics and information patterns, which can help further predict future events (temporal information).

[0051] Multiple events may occur simultaneously in the same time window and show structural dependencies between entities. To this end, an autoregressive architecture is used to define the joint probability distribution of all events in the time series knowledge graph. At the current time point, the conditional probability distribution of simultaneous events is constrained by the previous events.

[0052] like Figure 3 As shown, a structured data provided by an embodiment of the present invention is used to model the events occurring simultaneously in the same time window by establishing a local graph, so that the graph structure at the future timestamp can be extrapolated; Figure 3 As a detailed explanation, Re-net is a component of a graph neural network model that encodes entities and relationships. Unlike other neural networks, it aggregates events that occur in the same time window. Specifically, the model uses a recurrent event encoder (REE) to summarize information from past event sequences and aggregates information about events that occur simultaneously in the same time window through aggregation operations. With this information, the decoder can define the joint probability of the current event.

[0053] Among them, assuming and Depends only on events related to entity s, where represents the set of neighbor nodes that have event interactions with entity s at time t. The part is still retained because the s entity is unknown, so the event set must be taken into consideration.

[0054]

[0055] in is a learnable embedding associated with entity s and relation r. Represents the history vector, which encodes the past neighbor information and graph structure of s Global information in . It is a code for all past information, which is input into a linear softmax classifier to calculate different The probability of occurrence, are the parameters of the classifier. Models all local information of past time s, The model is The global information in the graph structure.

[0056] For each time , ,and It retains information about past events and updates them using a recurrent network: , in is an aggregate function, It means the event related to entity s at the current time t. For each subject entity s, it may act on multiple entities under multiple relationships at the same time t, generating events, that is, contains multiple events, so it is necessary to define an efficient aggregation function g to The aggregation function g in the experiment uses Mean Aggregator.

[0057] In one embodiment, the steps of combining the CRF algorithm with the Re-net baseline model, clustering to generate pseudo labels for auxiliary discrimination, connecting forward and backward hidden states, observing context information of the label sequence, using CRF to model the label sequence, and constructing a first mining model include: Establishing implicit chain information in the Re-net baseline model; A feature function is defined, and node features in the quadruple data are clustered to generate a desired pseudo-label, the node features are obtained from the embedding updated by back-propagation in the baseline model, the pseudo-label is defined as the category to which the cluster belongs, and modeling is equivalent to the category to which the modeled embedding belongs to assist in discrimination, wherein the state space size of the pseudo-label is reduced to the number of clusters; The BiLSTM method is used to capture the contextual information in the label sequence; Use CRF to model the conditional probability distribution of the label sequence. The goal of CRF modeling is to calculate the probability of the label sequence given the observation sequence. , the probability calculation formula of CRF is as follows: , in, is a scoring function used to measure the consistency of a given observation sequence and label sequence. is a normalization factor that ensures that the sum of the probabilities of all possible label sequences is 1.

[0058] It should be noted that the CRF algorithm is introduced into the Re-net model, and the global optimization characteristics of CRF are used to enhance the connection between event timing and causality. The following are the specific optimization steps: Modeling of implicit chain information: Assumptions Definition: In the baseline model, it is assumed that entity s is The probability of interacting with entity o It also depends on the definition of the full graph G (the set of all time graphs), that is, any entity s in Objects that interact The ordered set This ordered set implies a one-to-one correspondence , in the expectation that by modeling To estimate a single target information .

[0059] To solve this problem, the conditional random field model is used, which has the following advantages: The model is able to exploit the contextual relationships between events in a sequence, thereby improving model performance. It can model the interdependencies between events, taking into account the order of events and the interactions between them. This is critical for processing events in time series graphs, as events are often affected by previous and subsequent events.

[0060] The model has the ability to handle incomplete time series data. In the time series graph, there may be missing or noisy data for events. The model can fill in missing events or correct erroneous data by modeling the correlation between events, thereby improving the completeness and accuracy of the data.

[0061] The model allows flexible definition and use of various features to represent time series data. These features can be based on event attributes, contextual information, semantic associations, etc. By selecting and designing appropriate features, the model can effectively capture important patterns and regularities in time series data.

[0062] Simplified problem: In the optimization of this application, the assumption under the naive case is first considered, that is, In this case, the problem can be simplified to Based on this, by enhancing and The relationship between to model the CRF model , in this case, ,in is the probability normalization factor.

[0063] The definition process of state characteristic function and transfer characteristic function is as follows: Characteristic function definition: According to the state characteristic function in crf Definition of the transfer characteristic function , considering the transfer characteristic function requires at least However, in the naive case, the magnitude of the state space of y is too large for the naive form to hold.

[0064] Pseudo-markup generation: In a given dataset, only Therefore, pseudo-labeling needs to be generated. To assist in discrimination. Since s and o come from a common node set, they also come from the same feature space. The features of the nodes in the set can be clustered to generate the required pseudo-labels. These node features can be obtained from the embedding updated by back propagation in the baseline model. In this case, z can be defined as the category to which the cluster belongs, and the modeling Equivalent to modeling The embedding category of z is used to assist in discrimination. At this point, the state space size of z is reduced to the number of clusters.

[0065] In summary, the modeling object is ,in express The clustering category to which the feature belongs is the function Form a probability map, such as Figure 4 shown.

[0066] Introducing the BiLSTM model: In order to better describe the relationship between X and the state characteristic function , the BiLSTM model is used as the extractor of the relationship between X, forming When mapping to the label space, we use softmax to normalize to form a probability distribution In this way, the state characteristic function, that is, the emission matrix, is constructed. The transfer characteristic function Then the gradient descent solution is performed through the network layer.

[0067] BiLSTM is a variant of recurrent neural network (RNN) with hidden states in both forward and backward directions, which can capture contextual information in sequence data. Figure 5 shown.

[0068] Suppose there is a sequence data ,in represents the i-th observation. The goal is to label each observation, that is, to find the corresponding label sequence .

[0069] The calculation of the forward and backward hidden states of BiLSTM can be described by the following formula:

[0070] in, represents the update operation of LSTM, and denote the hidden states of the previous and next time steps respectively.

[0071] Then, the forward and backward hidden states are concatenated to obtain the combined hidden state at each time step These combined hidden states can capture the contextual information of the observation sequence.

[0072] Label sequence modeling using CRF Next, we use CRF to model the conditional probability distribution of the label sequence. Assume represents a tag sequence, Represents the corresponding comprehensive hidden state sequence. The goal of CRF modeling is to calculate the probability of the label sequence given the observation sequence The probability calculation formula of CRF is as follows:

[0073] in, It is a scoring function used to measure the consistency of a given observation sequence and label sequence. is a normalization factor that ensures that the sum of the probabilities of all possible label sequences is 1.

[0074] Comprehensively build an overall model In the BiLSTM-CRF model, a linear layer is used to map the comprehensive hidden state to the scoring function of the label sequence, that is, ,in and are learnable parameters.

[0075] Then there is a further overall model structure such as Figure 6 shown.

[0076] In one embodiment, the preprocessed data is input into the first mining model for training, the first mining model is optimized by a maximum probability criterion, and the optimized first mining model is obtained after the evaluation index reaches the standard.

[0077] It should be noted that Model training: The DuIE2.0 dataset is used for model training, and the parameters of the CRF model are optimized by the maximum probability criterion.

[0078] Model Evaluation: In order to evaluate the quality and performance of relation completion, the Top-K accuracy is used, which measures the accuracy of the top K predicted relations in the completion result. It refers to the probability that, given an entity, the predicted relation contains the true relation of the entity and the relation is ranked in the top K positions.

[0079] To address this problem, a certain variant of this indicator is used, Hit-1 and Hit-1+Top-1, which are marked with the original Top-1 and Top-2, where Hit-1 represents the Top-1 guided by the CRF algorithm.

[0080] The trained model is used for event causal relationship mining.

[0081] The following is an example to illustrate: In this embodiment, the DuIE2.0 dataset is used for model training, and the parameters of the CRF model are optimized by the maximum probability criterion. In order to evaluate the quality and performance of relationship completion, the Top-K accuracy is used, which measures the accuracy of the top K predicted relationships in the completion result. It refers to the probability that, given an entity, the predicted relationship contains the true relationship of the entity and the relationship is ranked in the top K positions. For this problem, a certain variant of this indicator is used, Hit-1 and Hit-1+Top-1, and the original Top-1 and Top-2 are marked, where Hit-1 represents the Top-1 guided by the CRF algorithm.

[0082] In the experiment, we introduced pseudo-labels formed by clustering on the basis of the RE-NET baseline, and obtained the hidden temporal relationship between pseudo-labels through conditional random fields (CRF). This design aims to enable the model to achieve significant performance improvements with low training times. Specifically, when the model predicts an event, the accuracy of the prediction result is positively correlated with the number of model training times.

[0083] The experimental results show that when the number of training times is low, the model has a higher accuracy after the introduction of the CRF algorithm with fewer training times than when the CRF tagging algorithm is not introduced. This shows that CRF can effectively utilize the temporal relationship between pseudo-labels formed by clustering, thereby improving the learning effect of the model in the initial stage. Due to the significant improvement in performance at low training times, the method of this application has obvious advantages in saving time and resource costs. This is especially important for application scenarios that require rapid deployment and iteration.

[0084] However, the experiment also found that when the number of training times increased to a certain extent, the performance of the model actually decreased. Analysis shows that this may be due to the large number of parameters introduced by the CRF emission matrix and transfer matrix, which caused the model to overfit when the number of training times was high. In order to further analyze the impact of CRF on the prediction effect of the model, two evaluation indicators were used: single item accuracy and double item accuracy.

[0085] The single-item accuracy is a traditional evaluation indicator, which indicates the probability that the model's prediction result is completely correct. The results show that when the number of training times is low, the single-item accuracy of the model with CRF is significantly higher than that of the model without CRF. The double-item accuracy is a new evaluation indicator, which indicates the probability that at least one of the two most likely results in the model's prediction results is correct. The experimental results show that the model with the CRF algorithm performs well in double-item accuracy, and maintains a high double-item accuracy even when the number of training times is high. This shows that the CRF algorithm has enhanced the model's ability to capture multiple possibilities to a certain extent and reduced the negative impact of model overfitting.

[0086] In summary, the model that introduces the CRF algorithm performs well at low training times, significantly improving the accuracy and efficiency of the model and saving time and resource costs. However, at high training times, the model may overfit due to the large number of parameters introduced by CRF, resulting in a decrease in the accuracy of a single item. However, through the evaluation of the accuracy of two items, it is further found that the model that introduces the CRF algorithm has better performance in terms of prediction robustness and the ability to capture multiple possibilities. The advantages and disadvantages of this method need to be weighed according to the specific application scenario. For scenarios that require fast training and efficient deployment, the introduction of the CRF algorithm is undoubtedly an effective enhancement method.

[0087] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.< / t> < / t>

Claims

1. A method for event causal relationship mining based on CRF labeling learning, characterized in that: The following steps are involved: Selecting a reference data set, and performing data preprocessing on the reference data set, wherein the data preprocessing includes learning the reference data set using a Bert algorithm combined with a CRF-labeled relation extraction algorithm, integrating time objects with time labels, and extracting quadruple data through relation extraction; Build a Re-net baseline model and perform serial modeling on the temporal knowledge graph through global structural reasoning; In the Re-net baseline model, the CRF algorithm is combined to cluster and generate pseudo labels for auxiliary discrimination, connect the forward and backward hidden states, observe the context information of the label sequence, use the CRF algorithm to model the label sequence, and build the first mining model; Inputting the preprocessed data into the first mining model for training, optimizing the first mining model by using the maximum probability criterion, and obtaining the optimized first mining model after the evaluation index reaches the standard; The optimized first mining model is used to mine event causal relationships in large-scale text data.

2. The event causal relationship mining method based on CRF labeling learning according to claim 1 is characterized in that: The step of using the Bert algorithm combined with the CRF-labeled relation extraction algorithm to learn the reference data set includes: First, the input text is segmented and encoded, and then the BERT model is used to generate the context representation, which is then decoded through the CRF layer to obtain the tag sequence. Given an input sequence , BERT will generate the corresponding context representation , , by combining the CRF layer, given the output representation of BERT , the CRF layer will generate the optimal tag sequence , ， Among them, A is the transfer matrix, which represents the transition from label Transfer to label where P is the label score for each position obtained from BERT.

3. The event causal relationship mining method based on CRF labeling learning according to claim 2 is characterized in that: The step of integrating the time objects with time stamps includes: Identify all time-tagged time objects from the text and extract time information through BERT's contextual representation and CRF algorithm's sequence annotation; Unify the format of identified time objects; Merge multiple time objects related to the same event to remove redundant information; By matching event and time objects, it ensures that all time information is associated with the corresponding event.

4. The event causal relationship mining method based on CRF labeling learning according to claim 3 is characterized in that: The step of extracting quadruple data by relation extraction includes: Make rules to save only relational data containing time objects; Sorting the relational data according to the time object, and sorting the events by timestamp; The CRF algorithm is combined with the Word2Vec method to extract features and extract the quadruple data of subject, relationship, object and time from the text.

5. The event causal relationship mining method based on CRF labeling learning according to claim 4 is characterized in that: The step of constructing the Re-net baseline model includes: establishing a local graph to model events occurring simultaneously in the same time window, and extrapolating the graph structure at future timestamps.

6. The event causal relationship mining method based on CRF labeling learning according to claim 5 is characterized in that: The steps of combining the CRF algorithm in the Re-net baseline model, clustering and generating pseudo labels for auxiliary discrimination, connecting forward and backward hidden states, observing context information of the label sequence, using the CRF algorithm to model the label sequence, and constructing the first mining model include: Establishing implicit chain information in the Re-net baseline model; A feature function is defined, and node features in the quadruple data are clustered to generate a desired pseudo-label, the node features are obtained from the embedding updated by back-propagation in the baseline model, the pseudo-label is defined as the category to which the cluster belongs, and modeling is equivalent to the category to which the modeled embedding belongs to assist in discrimination, wherein the state space size of the pseudo-label is reduced to the number of clusters; The BiLSTM method is used to capture the contextual information in the label sequence; The CRF algorithm is used to model the conditional probability distribution of the label sequence. The modeling goal of the CRF algorithm is to calculate the probability of the label sequence under the condition of a given observation sequence. , the probability calculation formula of the CRF algorithm is as follows: , in, is a scoring function used to measure the consistency of a given observation sequence and label sequence. is a normalization factor that ensures that the sum of the probabilities of all possible label sequences is 1.

7. The event causal relationship mining method based on CRF labeling learning according to claim 1 is characterized in that: The reference data set uses the DuIE2.0 data set for training and verification.

Images