Method and apparatus for estimating causal strength between events
By introducing an external causal knowledge base and ALBERT pre-trained model, combining semantic information and adversarial training methods, the adaptability and common sense knowledge integration problems of causal intensity estimation in the existing technology are solved, and a more accurate and efficient causal intensity analysis is achieved.
Patent Information
- Application Number
- CN202410558036.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-07
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-05-07
AI Technical Summary
When estimating the causal intensity between events, the statistical method only considers the co-occurrence frequency. Deep learning methods independently train causal intensity and causal intensity. The twin network method creates negative samples at the mechanism, resulting in the model's dependence on specific domain data sets. Not very adaptable and fail to fully integrate common sense causal prior knowledge.
By introducing an external causal knowledge base, combining semantic information, ALBERT pre-training model and adversarial training method are used to establish parameter sharing between the basic model and the causal information enhancement model, integrate common sense causal prior knowledge, and form more compatible A model of adaptive causal intensity calculation.
The recognition ability of causal semantics is improved, so that the causal intensity estimation process is not limited to the statistics of causal sharing frequency, the understanding and analysis accuracy of causal semantics is improved, and the adaptability and recognition ability of the model are enhanced.
Smart Images

Figure CN118377846B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of natural language processing, and in particular to a method and device for estimating the causal strength between events. Background Art
[0002] A causal pair refers to a binary group of events formed by causal extraction technology, generally written as (cause event, result event). The causal relationship between events has not only the analytical dimension of existence or not, but also the analytical dimension of the strength of the association, that is, the causal strength. This is because causal relationships, as higher-level semantic relationships, have a certain degree of complexity. The causal relationship between events is often not deterministic, but is established with a certain probability. For example, "hackers stealing data" may cause "contract leakage", but not every data theft will lead to contract leakage.
[0003] In related technologies, statistical methods only consider the co-occurrence frequency of causal pairs; in deep learning-based methods, the training processes of event pairs with stronger causal strength and event pairs with weaker causal strength are relatively independent, so that the improvement of causal strength calculation results only depends on the prior knowledge of the second round of pre-training, and the two processes are relatively separated; the way negative samples are constructed in the twin network-based method is random, which makes the model too dependent on data sets in specific fields and has low adaptability. Summary of the invention
[0004] The method and device for estimating the causal strength between events provided by the exemplary embodiments of the present disclosure can at least solve the above technical problems and other technical problems not mentioned above.
[0005] According to one aspect of the present disclosure, a method for estimating the causal strength between events is provided, the method comprising: obtaining first input data, wherein the first input data comprises two event texts; searching from an external causal knowledge base to obtain information enhancement words that respectively match the core trigger words in the two event texts and have consistent directionality, or information enhancement words that have the highest semantic similarity to the core trigger words in the two event texts and have consistent directionality; concatenating the two event texts with the corresponding information enhancement words, respectively, to obtain second input data; and estimating the causal strength between the two event texts based on the semantic information of the first input data and the second input data.
[0006] Optionally, the estimating the causal strength between the two event texts based on the semantic information of the first input data and the second input data includes: based on the ALBERT pre-trained model, respectively learning the semantic information of the first input data and the second input data; and estimating the causal strength between the two event texts based on the similarity of the semantic information between the first input data and the second input data.
[0007] Optionally, the causal strength between the two event texts is estimated based on the semantic information of the first input data and the second input data, including: obtaining a first feature vector corresponding to the first input data based on a basic model, wherein the first feature vector contains the semantic information of the first input data; obtaining a second feature vector corresponding to the second input data based on a causal information enhancement model, wherein the second feature vector contains the semantic information of the second input data; and obtaining a causal strength score between the two event texts by calculating through a fully connected network a final feature vector obtained by adding the first feature vector and the second feature vector; wherein the basic model and the causal information enhancement model respectively include an ALBERT pre-trained model and a pooling layer, and the ALBERT pre-trained model is used to obtain a corresponding word embedding vector based on the input data, and the word embedding vector is used to represent the semantic information of the corresponding input data.
[0008] Optionally, the final feature vector obtained by adding the first feature vector and the second feature vector is calculated through a fully connected network to obtain a causal strength score between the two event texts, including:
[0009]
[0010] C=σ(W(F)+b)
[0011] Among them, F represents the final eigenvector, F B represents the first eigenvector, F R represents the second feature vector, W is the fully connected network parameter, b is the bias, σ is the sigmoid function, and C is the causal strength score between the two event texts.
[0012] Optionally, the method further includes: training the basic model and the causal information enhancement model with the goal of making the basic model as close as possible to the causal information enhancement model.
[0013] Optionally, the training of the basic model and the causal information enhancement model with the goal of making the basic model as close as possible to the causal information enhancement model includes: adjusting parameters of the basic model and the causal information enhancement model with the goal of minimizing a preset loss function and maximizing the similarity between the first feature vector and the second feature vector through parameter sharing between the basic model and the causal information enhancement model; wherein the preset loss function is used to make a first causal strength score as close as possible to a second causal strength score during the minimization process, the first causal strength score being the causal strength score between the two event texts estimated based on the first input data, and the second causal strength score being the causal strength score between the two event texts estimated based on the second input data.
[0014] Optionally, the similarity between the first feature vector and the second feature vector is calculated by the following formula:
[0015]
[0016] Among them, F B represents the first eigenvector, F R represents the second eigenvector, |·| represents the absolute value, b i Indicates F B The i-th element in r i Indicates F R The i-th element in , p is a prime number.
[0017] Optionally, the first causal strength score is calculated by the following formula:
[0018] C B =Score(E1,E2)
[0019] Among them, E1 and E2 represent the anchor samples B i the two event texts in the first input data;
[0020] The calculation method of the second causal strength score is expressed as:
[0021] C R =Score(R1,R2)
[0022] Among them, R1 and R2 represent the positive samples R i In the second input data, a text obtained by concatenating the two event texts with the corresponding information enhancement words respectively;
[0023] Positive sample R i and anchor sample B i The following conditions are met:
[0024] Score(R i )>Score(B i )
[0025] in, is the set containing all anchor samples, is a set containing all positive samples;
[0026] The preset loss function is expressed by the following formula:
[0027]
[0028] Among them, ε is a parameter representing the error range.
[0029] According to another aspect of the present disclosure, a device for estimating the causal strength between events is also provided, the device comprising: an acquisition unit, configured to: acquire first input data, wherein the first input data includes two event texts; a search unit, configured to: search from an external causal knowledge base to obtain information enhancement words that match the core trigger words in the two event texts and have consistent directionality, or information enhancement words that have the highest semantic similarity to the core trigger words in the two event texts and have consistent directionality; a splicing unit, configured to: splice the two event texts with the corresponding information enhancement words, respectively, to obtain second input data; an estimation unit, configured to: estimate the causal strength between the two event texts based on the semantic information of the first input data and the second input data.
[0030] Optionally, the estimation unit is configured to: based on the ALBERT pre-trained model, respectively learn the semantic information of the first input data and the second input data; based on the similarity of the semantic information between the first input data and the second input data, estimate the causal strength between the two event texts.
[0031] Optionally, the estimation unit is configured to: obtain a first feature vector corresponding to the first input data based on a basic model, wherein the first feature vector contains semantic information of the first input data; obtain a second feature vector corresponding to the second input data based on a causal information enhancement model, wherein the second feature vector contains semantic information of the second input data; and calculate the causal strength score between the two event texts through a fully connected network based on a final feature vector obtained by adding the first feature vector and the second feature vector; wherein the basic model and the causal information enhancement model respectively include an ALBERT pre-trained model and a pooling layer, and the ALBERT pre-trained model is used to obtain a corresponding word embedding vector based on the input data, and the word embedding vector is used to represent the semantic information of the corresponding input data.
[0032] Optionally, the estimation unit is configured to: obtain a final feature vector based on the addition of the first feature vector and the second feature vector by the following formula, and obtain a causal strength score between the two event texts by calculating through a fully connected network:
[0033]
[0034] C=σ(W(F)+b)
[0035] Among them, F represents the final eigenvector, F B represents the first eigenvector, F R represents the second feature vector, W is the fully connected network parameter, b is the bias, σ is the sigmoid function, and C is the causal strength score between the two event texts.
[0036] Optionally, the device also includes: a training unit configured to: train the basic model and the causal information enhancement model with the goal of making the basic model as close as possible to the causal information enhancement model.
[0037] Optionally, the training unit is configured to adjust parameters of the basic model and the causal information enhancement model by sharing parameters between the basic model and the causal information enhancement model, with the goal of minimizing a preset loss function and maximizing the similarity between the first feature vector and the second feature vector; wherein the preset loss function is used to make the first causal strength score as close as possible to the second causal strength score during the minimization process, the first causal strength score being the causal strength score between the two event texts estimated based on the first input data, and the second causal strength score being the causal strength score between the two event texts estimated based on the second input data.
[0038] Optionally, the training unit is configured to: calculate the similarity between the first feature vector and the second feature vector by using the following formula:
[0039]
[0040] Among them, F B represents the first eigenvector, F R represents the second eigenvector, |·| represents the absolute value, b i Indicates F B The i-th element in r i Indicates F R The i-th element in , p is a prime number.
[0041] Optionally, the training unit is configured to: calculate the first causal strength score by the following formula:
[0042] C B =Score(E1,E2)
[0043] Among them, E1 and E2 represent the anchor samples B i the two event texts in the first input data;
[0044] The second causal strength score is calculated by the following formula:
[0045] C R =Score(R1,R2)
[0046] Among them, R1 and R2 represent the positive samples R i In the second input data, a text obtained by concatenating the two event texts with the corresponding information enhancement words respectively;
[0047] Positive sample R i and anchor sample B i The following conditions are met:
[0048] Score(R i )>Score(B i )
[0049] in, is the set containing all anchor samples, is a set containing all positive samples;
[0050] The preset loss function is expressed by the following formula:
[0051]
[0052] Among them, ε is a parameter representing the error range.
[0053] According to another aspect of an embodiment of the present disclosure, an electronic device is also provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute any of the methods for estimating the causal strength between events as described above.
[0054] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium storing instructions is also provided. When the instructions are executed by at least one processor, the at least one processor is prompted to execute any of the methods for estimating the causal strength between events as described above.
[0055] According to another aspect of an embodiment of the present disclosure, a system is also provided, comprising at least one computing device and at least one storage device storing instructions, wherein when the instructions are executed by the at least one computing device, the at least one computing device is prompted to execute any of the methods described above for estimating the causal strength between events.
[0056] According to another aspect of an embodiment of the present disclosure, a computer program product is also provided, including a computer program / instruction, which, when executed by a processor, implements the method for estimating the causal strength between events as described in any one of the above.
[0057] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:
[0058] According to the method, device, electronic device, storage medium, system and computer program product for estimating the causal strength between events disclosed in the present invention, causal information can be enhanced by introducing an external causal knowledge base, so that common-sense causal prior knowledge can be integrated into the process of estimating the causal strength between event texts; in the process of estimating the causal strength between event texts, semantic information is combined, so that the process of estimating the causal strength is not limited to the statistics of the shared frequency of causal pairs, thereby improving the recognition ability of causal semantics.
[0059] In addition, a deep learning framework can be used, through the currently popular ALBERT pre-training model, to realize that the calculation process of causal strength is not limited to the statistics of causal pair sharing frequency, and to integrate semantic information into the causal strength calculation.
[0060] In addition, the adversarial training method can be integrated to apply the pre-trained model ALBERT to the training of causal information enhancement, establish parameter sharing between the basic model and the causal information enhancement model, and realize the joint calculation of causal strength through two sets of causal pairs.
[0061] In addition, by introducing an external causal knowledge base to form the input of the causal information enhancement model, the basic model can be better trained with the help of common-sense causal semantics, thereby integrating common-sense causal prior knowledge into the overall model composed of the basic model and the causal information enhancement model.
[0062] In addition, by introducing an external causal knowledge base to enhance the causal information of cause-effect event pairs (hereinafter referred to as causal pairs), and adopting an adversarial training framework to calculate the causal strength of causal pairs, the causal strength analysis results can be made more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.
[0064] Figure 1 A flowchart showing a method for estimating causal strength between events in an exemplary embodiment of the present disclosure;
[0065] Figure 2 A simplified flow chart showing a method for estimating causal strength between events in an exemplary embodiment of the present disclosure;
[0066] Figure 3 A flow chart showing a method for estimating causal strength between events in an exemplary embodiment of the present disclosure;
[0067] Figure 4 A flowchart showing a method for estimating causal strength between events in an exemplary embodiment of the present disclosure;
[0068] Figure 5 A block diagram showing an apparatus for estimating causal strength between events in an exemplary embodiment of the present disclosure;
[0069] Figure 6 A block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0070] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0071] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.
[0072] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.
[0073] A cause-effect event pair (hereinafter referred to as a cause-effect pair) refers to a binary event formed by causal relationship extraction technology, generally written as (cause event, effect event). For example, in the sentence "the economic crisis was generated due to the sub-prime mortgage mess, brought about through government intervention in the market to promote progressive ideals." the cause event is "mortgage mess", and its core noun is "mess"; the effect event is "economic crisis", and its core noun is "crisis".
[0074] The causal relationship between events not only has the analytical dimension of existence or not, but also has the analytical dimension of the strength of the association, that is, the causal strength. This is because the causal relationship, as a higher-level semantic relationship, has a certain complexity. The causal relationship between events is often not deterministic, but is established with a certain probability. For example, "hackers stealing data" may cause "contract leakage", but not every data theft will lead to contract leakage.
[0075] In the related art, the solutions for causal strength calculation are mainly divided into three categories:
[0076] (1) Statistical-based method: This method uses the co-occurrence of statistical causal events to replace the original word co-occurrence method to calculate the causal strength between events, so as to alleviate the sparsity of event texts, and model from the aspects of necessity and sufficiency to reflect the causal strength between causal event pairs.
[0077] (2) Method based on deep learning: This method uses a pre-trained language model to model the calculation of causal strength as a text matching task. The model is further pre-trained with causal data through a two-stage pre-training method, so that the model learns the prior knowledge of text distribution and causal relationship. Then, the cause and effect event texts are input, and the model outputs the causal relationship strength score of the two events.
[0078] (3) Twin network-based method: This method adopts the idea of contrastive learning and models the causal strength calculation task as a short text matching task based on a pre-trained model. Negative samples are generated by randomly replacing cause / effect events and input into the dual-tower twin network structure in parallel with positive samples. The causal strength scores of causal event pairs can be calculated relatively quickly.
[0079] However, the statistical method only considers the co-occurrence frequency of causal pairs and ignores the semantic influence. When semantically similar words play the same role in the same event in the causal pair, the causal strength of the two groups of causal pairs should also be similar. For example, the possibility of "stealing" and "looting" as causes leading to the result of "leakage" should be similar.
[0080] In the deep learning-based method, the training processes of event pairs with strong causal strength and event pairs with weak causal strength are relatively independent, so that the improvement of causal strength calculation results only depends on the prior knowledge of the second round of pre-training, and the two processes are relatively separated. The method based on the twin network constructs negative samples in a random way and fails to fully integrate common sense causal prior knowledge, making the model too dependent on data sets in specific fields and not very adaptable.
[0081] In order to solve the above problems, the present disclosure provides a method, device, electronic device, storage medium, system and computer program product for estimating the causal strength between events. Causal information can be enhanced by introducing an external causal knowledge base, so that common-sense causal prior knowledge can be integrated into the estimation process of the causal strength between event texts; semantic information is combined in the estimation process of the causal strength between event texts, so that the estimation process of the causal strength is not limited to the statistics of the shared frequency of causal pairs, thereby improving the recognition ability of causal semantics.
[0082] In addition, with the goal of integrating semantic information into the calculation of causal strength, the deep learning framework of the present disclosure can be used. Through the currently popular pre-training model, the calculation process of causal strength is not limited to the statistics of the sharing frequency of causal pairs; with the goal of jointly calculating the causal strength of two groups of causal pairs, the present disclosure can integrate the adversarial training method to establish parameter sharing between the basic model and the causal information enhancement model; with the goal of integrating common-sense causal prior knowledge into the model, the present disclosure can enhance causal information by introducing an external causal knowledge base, thereby improving the model's recognition ability of causal semantics.
[0083] Next, we will refer to Figures 1 to 6 The present invention specifically describes a method, apparatus, electronic device, storage medium, system, and computer program product for estimating causal strength between events.
[0084] Figure 1 A flowchart showing a method for estimating causal strength between events in an exemplary embodiment of the present disclosure is shown.
[0085] Reference Figure 1 In step 101, first input data may be obtained, wherein the first input data may include two event texts.
[0086] According to an exemplary embodiment of the present disclosure, an event extraction technology can be used to identify and extract two related event texts from the first input data, which may include the type of event, core trigger words, and arguments (such as time, location, participants, etc.). The present disclosure takes the core trigger words as an example, and the first input data contains only two event texts as an example for explanation.
[0087] For example, the first input data may be a cause-effect pair sample ("cardiovascular collapse", "caffeine overdose").
[0088] In step 102, a search may be performed from an external causal knowledge base to obtain information enhancement words that match the core trigger words in the two event texts and have consistent directionality, or information enhancement words that have the highest semantic similarity to the core trigger words in the two event texts and have consistent directionality.
[0089] According to an exemplary embodiment of the present disclosure, the causal knowledge base is technical data disclosed in the art. The causal pairs in the causal knowledge base are generally recognized to have a strong causal relationship, and the data is cleaned and restored to parts of speech.
[0090] According to an exemplary embodiment of the present disclosure, directional consistency means that two information enhancement words have the same attributes (cause / effect) as the corresponding two core trigger words, or in other words, the two situations of "rain causes floods" and "floods cause rain" do not belong to situations of directional consistency.
[0091] According to an exemplary embodiment of the present disclosure, after obtaining the first input data, the core trigger words of the two event texts in the first input data can be extracted. For example, if the two event texts are "'cardiovascular collapse', 'caffeine overdose'", then the core trigger words are "'collapse', 'overdose'". Before searching from the external causal knowledge base, the core trigger words can be restored to their roots. When searching from the external causal knowledge base, and when it is possible to obtain information enhancement words that match the core trigger words in the above two event texts and have consistent directions, the obtained information enhancement words are "'collapse', 'overdose'". If it is not possible to search for information enhancement words that match the core trigger words in the two event texts and have consistent directions, it is possible to search for information enhancement words that have the highest semantic similarity (for example, synonyms) and consistent directions with the core trigger words in the two event texts.
[0092] In step 103, the two event texts may be concatenated with the corresponding information enhancement words respectively to obtain second input data.
[0093] According to an exemplary embodiment of the present disclosure, for example, the two event texts “cardiovascular collapse” and “caffeine overdose” may be concatenated with the corresponding information enhancement words “collapse” and “overdose”, respectively, to obtain the second input data “cardiovascular collapse, collapse, caffeine overdose, overdose”.
[0094] In step 104, the causal strength between the two event texts may be estimated based on the semantic information of the first input data and the second input data.
[0095] According to an exemplary embodiment of the present disclosure, the first input data and the second input data can be converted into vector representations, for example, word embeddings (Word2Vec, GloVe, FastText, etc.) or sentence embeddings (BERT, RoBERTa, GPT, etc.) can be used, which can capture the semantic information of the text. Then, the semantic relationship between the two events in the first input data and the second input data can be analyzed, for example, the order, dependency relationship between the events, and whether there are words or phrases representing causal relationships, etc. can be determined. For words and phrases that explicitly or implicitly represent causal relationships in the text, they can be direct (such as "lead to", "because") or indirect (through logical reasoning and semantic understanding, etc.). Further, based on the recognition results of event embedding, text embedding, and causal words and phrases, a series of features can be constructed to represent the semantic relationship and potential causal strength between the two events; these features may include the following: similarity or distance between event embeddings, similarity or distance between text embeddings, frequency or weight of causal words and phrases, time order between events, syntactic dependency, other related words or phrases in the context, etc. Optionally, a dataset labeled with causal strength can be used to train a machine learning model so that the model can predict the causal strength between two events based on the above features; the model selection can include logistic regression, decision tree, random forest, gradient boosting machine, neural network, etc. In one example, a validation set can be used to evaluate the performance of the model and adjust and optimize it as needed. This may include feature selection, model parameter adjustment, ensemble learning, etc.
[0096] Since the second input data is obtained by concatenating the two event texts contained in the first input data with the corresponding information enhancement words, when estimating the causal strength between the two event texts based on the semantic information of the first input data and the second input data, the estimation result of the causal strength between the two event texts can be more accurate and not limited to a specific field because it includes common-sense causal priors and balances the semantics of the first input data itself.
[0097] According to an exemplary embodiment of the present disclosure, semantic information of the first input data and the second input data can be learned respectively based on the ALBERT pre-training model; the causal strength between the two event texts can be estimated based on the similarity of the semantic information between the first input data and the second input data.
[0098] According to an exemplary embodiment of the present disclosure, the first input data and the second input data can be respectively input into the ALBERT pre-trained model, so that the embedding vector output by the ALBERT pre-trained model is used as the semantic information representation of the first input data and the second input data. Then, the similarity of the semantic information between the first input data and the second input data can be obtained by calculating the distance between the embedding vectors, and the estimated values of the causal strength between the two event texts respectively calculated based on the first input data and the second input data can be weighted averaged according to the similarity value to obtain the final estimated value of the causal strength between the two event texts.
[0099] Figure 2 A simplified flow chart of a method for estimating causal strength between events in an exemplary embodiment of the present disclosure is shown.
[0100] Reference Figure 2 According to an exemplary embodiment of the present disclosure, the first input data and the second input data can be respectively input into a basic model (Base Model) and a causal information enhancement model (Reinforced Model) constructed based on the ALBERT pre-training model, and the input end of the causal information enhancement model has an additional information enhancement module based on an external causal knowledge base.
[0101] The first input data may be two causal event texts E1 and E2, for example, "cardiovascular collapse" and "caffeine overdose". The core trigger words in the two events may be extracted and restored to their roots. For example, the core trigger word (Trigger1) of "cardiovascular collapse" is "collapse", which is "collapse" after root restoration, and the core trigger word (Trigger2) of "caffeine overdose" is "overdose", which is "overdose" after root restoration. Then, the causal knowledge base may be searched for the core trigger words. If the match is successful, the words obtained by the match may be used as information enhancement words (Trigger1' and Trigger2'). If the match is unsuccessful, the synonyms of the core trigger words may be searched in the causal knowledge base by finding synonyms, which may be used as information enhancement words (Trigger1' and Trigger2'). Then, the two event texts E1 and E2 may be concatenated with the corresponding information enhancement words Trigger1' and Trigger2' to obtain the second input data.
[0102] The first input data can be input into the basic model, and the second input data can be input into the causal information enhancement model. Therefore, two embedding vectors can be obtained by the ALBERT pre-trained model as the semantic information representation of the first input data and the second input data respectively. Then, the similarity between the two embedding vectors can be calculated by the Minkowski distance (Euclidean distance, cosine similarity, etc. can also be used), and finally the causal strength score is output as the final estimated value of the causal strength between the two event texts.
[0103] According to an exemplary embodiment of the present disclosure, a first feature vector corresponding to the first input data can be obtained based on the basic model, wherein the first feature vector contains semantic information of the first input data; a second feature vector corresponding to the second input data can be obtained based on the causal information enhancement model, wherein the second feature vector contains semantic information of the second input data; a causal strength score between two event texts can be calculated through a fully connected network based on a final feature vector obtained by adding the first feature vector and the second feature vector; wherein the basic model and the causal information enhancement model respectively include an ALBERT pre-trained model and a pooling layer, and the ALBERT pre-trained model is used to obtain a corresponding word embedding vector based on the input data, and the word embedding vector can be used to represent the semantic information of the corresponding input data.
[0104] Figure 3 A flowchart of a method for estimating causal strength between events in an exemplary embodiment of the present disclosure is shown.
[0105] Reference Figure 3 According to an exemplary embodiment of the present disclosure, the network structure of the causal information enhancement model and the basic model can be basically the same, and the only difference is that an information enhancement module based on an external causal knowledge base is added to the input end of the causal information enhancement model, so that the feature vector obtained by the causal information enhancement model contains more information that is helpful for judging the strength of causality.
[0106] The first input data of the basic model can be two event texts and E1 and E2 can be concatenated and input into the ALBERT pre-trained model (Base Model). The base model can calculate a context-dependent word embedding vector T1,…,T for each word in the text. m ,…,T m+n , this process can be expressed as:
[0107] T1,…,T m ,…,T m+n =ALBERT(E1,E2)
[0108] The word embedding vector can be obtained as a fixed-length vector F after passing through a pooling layer. B , F B The first eigenvector output as the basic model.
[0109] F B =Pooler(T1,…,T m ,…,T m+n )
[0110] The input of the causal enhancement model is E1, E2 and the output of the information enhancement module. The event text contains the core trigger word x of the event. S If the core trigger word of E1 and E2's core trigger words After restoring the root word, they can be successfully matched in the external causal knowledge base and the direction is consistent. Then E1 can be matched with Splice to R1, connect E2 with The concatenation is R2, and then R1 and R2 are concatenated and input into the ALBERT pre-trained model (Reinforced Model, causal information enhancement model), and the model calculates a context-related word embedding vector.
[0111] T1,…,T m ,…,T m+p ,…,T m+p+n ,…,T m+p+n+q =ALBERT(R1,R2)
[0112] If no match is found, select and The most semantically similar And ensure the directionality is consistent, then E1 and Splice to R1′, connect E2 with Concatenate them into R2′, then concatenate R1′ and R2′ and input them into the ALBERT pre-trained model, and the model calculates a context-related word embedding vector.
[0113] T1,…,T m ,…,T m+p′ ,…,T m+p′+n ,…,T m+p+n+q′ =ALBERT(R1′,R2′)
[0114] The word embedding vector can be obtained through a pooling layer to obtain a fixed-length vector F R , F R The second feature vector as the output of the causal enhancement model.
[0115] FR =Pooler(T1,…,T m ,…,T m+p / p ,…,T m+p / p′+n ,…,T m+p / p′+n+q / q′ )
[0116] For example, the first input data may be a causal pair sample (“cardiovascular collapse”, “caffeine overdose”), whose input to the basic model is “cardiovascular collapse, caffeineoverdose”, and whose input to the causal information enhancement model is “cardiovascular collapse, collapse, caffeine overdose, overdose”.
[0117] The specific pseudo code operation flow of the above information enhancement module is shown in the following table:
[0118]
[0119]
[0120] According to an exemplary embodiment of the present disclosure, with the goal of making the basic model as close as possible to the causal information enhancement model, the basic model and the causal information enhancement model are trained so that the overall model consisting of the trained basic model and the causal information enhancement model can be used to obtain a causal strength score that incorporates external causal knowledge.
[0121] According to an exemplary embodiment of the present disclosure, the parameters of the basic model and the causal information enhancement model can be adjusted by sharing parameters between the basic model and the causal information enhancement model, with the goal of minimizing a preset loss function and maximizing the similarity between the first eigenvector and the second eigenvector; wherein the preset loss function can be used to make the first causal strength score as close as possible to the second causal strength score during the minimization process, the first causal strength score being the causal strength score between the two event texts estimated based on the first input data, and the second causal strength score being the causal strength score between the two event texts estimated based on the second input data.
[0122] Return to reference Figure 3According to an exemplary embodiment of the present disclosure, the basic model and the causal information enhancement model can share parameters during the training process through adversarial training. The process of synchronously updating the training parameters of the basic model (i.e., the weights learned by the neural network) along with the training parameters of the causal information enhancement model can be regarded as realizing parameter sharing between the two models, so as to form a dynamic process in which the causal information enhancement model drives the basic model to continuously learn and capture the causal relationship semantics in the external knowledge base.
[0123] F B 、F R They are the feature vectors learned by passing neural network parameters for the basic model and the causal information enhanced model.
[0124] According to an exemplary embodiment of the present disclosure, the similarity between two feature vectors can be calculated using Euclidean distance, cosine similarity, Minkowski distance, etc. The purpose of similarity calculation is to push the outputs of the two models closer and closer in combination with the loss function. In the present disclosure, the first feature vector F B and the second eigenvector F R The similarity between Sim(F B ; F R ), Minkowski distance can be used for calculation, and the formula can be as follows:
[0125]
[0126] Among them, |·| represents the absolute value, b i Indicates F B The i-th element in r i Indicates F R The i-th element in , p is a prime number.
[0127] According to an exemplary embodiment of the present disclosure, a ranking cross entropy loss can be used to train the model, and each causal pair will become an anchor sample B. i , the corresponding positive sample R i is a strong causal pair after information enhancement. The calculation method of the causal strength score can be expressed by Score(). Therefore, the first causal strength score can be calculated by the following formula:
[0128] C B =Score(E1,E2)
[0129] Among them, E1 and E2 represent the anchor samples B i The two event texts in the first input data.
[0130] The calculation method of the second causal strength score is expressed as:
[0131] C R =Score(R1,R2)
[0132] Among them, R1 and R2 represent the positive samples R i In the second input data, the text is obtained by concatenating two event texts with the corresponding information enhancement words.
[0133] The goal of training is to make C B and C R As close as possible, and for the positive sample R i and anchor sample B i , the following conditions must be met:
[0134] Score(R i )>Score(B i )
[0135] in, is the set containing all anchor samples, is the set containing all positive samples.
[0136] The following loss function can be used as the preset loss function, and the model is trained by minimizing the following preset loss function, that is, the preset loss function can be expressed by the following formula:
[0137]
[0138] Among them, ε is a parameter representing the error range. ε is similar to the bias and m in the ranking loss function, which is used to control a boundary range to prevent the model from overfitting; for example, in ε(a), the value of a can be close to 0 within a certain range, but it cannot be directly equal to 0, otherwise, it is easy to make incorrect classification results for positive samples that are close to 0.
[0139] According to an exemplary embodiment of the present disclosure, based on the first feature vector F B and the second eigenvector F R The final eigenvector F obtained by addition is:
[0140]
[0141] The final feature vector F fine-tuned by the above loss function can be input into the fully connected network, and the causal strength score C between the two event texts can be obtained after mapping:
[0142] C=σ(W(F)+b)
[0143] Among them, W is the fully connected network parameter, b is the bias, and σ is the sigmoid function.
[0144] The function of the fully connected network is to normalize the causal strength score to ensure that its value is between 0 and 1. Therefore, the fully connected network can be implemented using softmax. The causal strength score between two event texts can be obtained through the fully connected network calculation.
[0145] Figure 4 A flowchart showing a method for estimating causal strength between events in an exemplary embodiment of the present disclosure is shown.
[0146] Reference Figure 4 According to an exemplary embodiment of the present disclosure, an event text containing a causal pair can be used as the input of a basic model, and an information enhancement module constructed based on an external causal knowledge base can be introduced to form the input of a causal information enhancement model, so that the overall model composed of the basic model and the causal information enhancement model can be better trained with the help of common sense causal semantics; then, the pre-trained model ALBERT can be applied to the training of causal information enhancement, a suitable loss function (Loss Function) can be established, and adversarial training can be used to establish parameter sharing between the basic model and the causal information enhancement model; finally, the final causal strength score can be obtained after a fully connected network based on the similarity between the obtained feature vectors.
[0147] According to an exemplary embodiment of the present disclosure, a deep learning framework combined with adversarial training can be used to divide the entire model into a basic model, a causal information enhancement model, a similarity calculation layer, and an output causal strength score.
[0148] Compared with statistical-based methods, it mainly uses the co-occurrence of statistical causal events to calculate the causal strength between events in order to alleviate the sparsity of event text. It only considers the frequency of causal event pairs in the dataset used, and does not consider the semantic correlation between causal event pairs, between cause events, and between result events. As a result, the causal strength calculation results are subject to specific datasets and are not cross-domain adaptable.
[0149] The exemplary embodiments of the present disclosure can utilize the ALBERT pre-trained model to fully consider the deep semantic relationship between cause events and result events, and introduce a common-sense external causal knowledge base, which will not limit the causal information enhancement model to prior knowledge in a specific field.
[0150] Compared with deep learning-based methods, it requires a two-stage pre-training method to further pre-train the model with causal data. The training processes of strong causal pairs and weak causal pairs are relatively independent and fragmented, and the semantic correlation between events with the same causal role cannot be learned well.
[0151] The exemplary embodiments of the present disclosure may adopt an adversarial training method to establish parameter sharing between the basic model and the causal information enhancement model, and by designing a loss function, the learning ability of the basic model is made as close as possible to the causal information enhancement model, which can better coordinate the joint training of the two models. The meaning of the basic model being close to the causal information enhancement model may include: the output of the basic model is close to the output of the causal information enhancement model, or the parameters of the basic model are close to the parameters of the causal information enhancement model, etc.
[0152] Compared with the twin network-based method, it mainly generates negative samples by randomly replacing cause / result events, and inputs them into the dual-tower twin network structure in parallel with positive samples. This method cannot integrate causal prior knowledge into the comparative learning of positive and negative samples, making the model not well adaptable to cross-domain causal data.
[0153] The exemplary embodiments of the present disclosure may use causal pairs as anchor samples, generate positive samples by introducing an information enhancement module of an external causal knowledge base, and integrate common-sense causal prior knowledge into the model training process, so that the model can better identify causal semantic relationships.
[0154] Figure 5 A block diagram showing an apparatus for estimating causal strength between events in an exemplary embodiment of the present disclosure.
[0155] Reference Figure 5 The exemplary embodiment of the present disclosure further provides a device 500 for estimating the causal strength between events, which may include but is not limited to an acquisition unit 501 , a search unit 502 , a splicing unit 503 , and an estimation unit 504 .
[0156] The acquisition unit 501 may acquire first input data, wherein the first input data may include two event texts.
[0157] The search unit 502 can search from an external causal knowledge base to obtain information enhancement words that match the core trigger words in the two event texts and have the same direction, or information enhancement words that have the highest semantic similarity with the core trigger words in the two event texts and have the same direction.
[0158] The concatenation unit 503 may concatenate the two event texts with the corresponding information enhancement words respectively to obtain the second input data.
[0159] The estimation unit 504 may estimate the causal strength between the two event texts based on the semantic information of the first input data and the second input data.
[0160] According to an exemplary embodiment of the present disclosure, the estimation unit 504 can learn the semantic information of the first input data and the second input data respectively based on the ALBERT pre-training model; and can estimate the causal strength between the two event texts based on the similarity of the semantic information between the first input data and the second input data.
[0161] According to an exemplary embodiment of the present disclosure, the estimation unit 504 can obtain a first feature vector corresponding to the first input data based on the basic model, wherein the first feature vector can include semantic information of the first input data; can obtain a second feature vector corresponding to the second input data based on the causal information enhancement model, wherein the second feature vector can include semantic information of the second input data; can calculate the causal strength score between the two event texts through a fully connected network based on a final feature vector obtained by adding the first feature vector and the second feature vector; wherein the basic model and the causal information enhancement model can respectively include an ALBERT pre-trained model and a pooling layer, and the ALBERT pre-trained model can be used to obtain a corresponding word embedding vector based on the input data, and the word embedding vector can be used to represent the semantic information of the corresponding input data.
[0162] According to an exemplary embodiment of the present disclosure, the estimation unit 504 may obtain the causal strength score between the two event texts by using the following formula based on the final feature vector obtained by adding the first feature vector and the second feature vector, and calculating the causal strength score between the two event texts through a fully connected network:
[0163]
[0164] C=σ(W(F)+b)
[0165] Among them, F represents the final eigenvector, F B represents the first eigenvector, F R represents the second eigenvector, W is the fully connected network parameter, b is the bias, σ is the sigmoid function, and C is the causal strength score between the two event texts.
[0166] According to an exemplary embodiment of the present disclosure, the above-mentioned device 500 for estimating the causal strength between events may also include a training unit 505 (not shown in the figure), which can train the basic model and the causal information enhancement model with the goal of making the basic model as close as possible to the causal information enhancement model.
[0167] According to an exemplary embodiment of the present disclosure, the training unit 505 can adjust the parameters of the basic model and the causal information enhancement model by sharing parameters between the basic model and the causal information enhancement model, with the goal of minimizing a preset loss function and maximizing the similarity between the first eigenvector and the second eigenvector; wherein the preset loss function is used to make the first causal strength score as close as possible to the second causal strength score during the minimization process, the first causal strength score being the causal strength score between the two event texts estimated based on the first input data, and the second causal strength score being the causal strength score between the two event texts estimated based on the second input data.
[0168] According to an exemplary embodiment of the present disclosure, the training unit 505 may calculate the similarity between the first feature vector and the second feature vector by the following formula:
[0169]
[0170] Among them, F B represents the first eigenvector, F R represents the second eigenvector, |·| represents the absolute value, b i Indicates F B The i-th element in r i Indicates F R The i-th element in , p is a prime number.
[0171] According to an exemplary embodiment of the present disclosure, the training unit 505 may calculate the first causal strength score by the following formula:
[0172] C B =Score(E1,E2)
[0173] Among them, E1 and E2 represent the anchor samples B i The two event texts in the first input data;
[0174] The second causal strength score is calculated by the following formula:
[0175] C R =Score(R1,R2)
[0176] Among them, R1 and R2 represent the positive samples R i In the second input data, the text is obtained by concatenating the two event texts with the corresponding information enhancement words respectively;
[0177] Positive sample R i and anchor sample B i The following conditions are met:
[0178] Score(R i)>Score(B i )
[0179] in, is the set containing all anchor samples, is a set containing all positive samples;
[0180] The default loss function is expressed by the following formula:
[0181]
[0182] Among them, ε is a parameter representing the error range.
[0183] It can be understood that in the exemplary embodiment of the above-mentioned device 500 for estimating the causal strength between events, the specific implementation process is roughly the same as the exemplary embodiment of the method for estimating the causal strength between events, and will not be repeated here. The device 500 for estimating the causal strength between events can be configured as software, hardware, firmware or any combination of the above items to perform specific functions. For example, these devices may correspond to dedicated integrated circuits, pure software codes, or units and modules that combine software and hardware. In addition, one or more functions implemented by these devices may also be uniformly executed by components in physical entity devices (for example, processors, clients or servers, etc.).
[0184] Figure 6 A block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure.
[0185] Reference Figure 6 The electronic device 600 includes at least one memory 601 and at least one processor 602, wherein the at least one memory 601 stores a set of computer executable instructions, and when the computer executable instruction set is executed by the at least one processor 602, a method for estimating causal strength between events according to an exemplary embodiment of the present disclosure is performed.
[0186] As an example, the electronic device 600 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instruction set. Here, the electronic device 600 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instruction (or instruction set) individually or in combination. The electronic device 600 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.
[0187] In the electronic device 600, the processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.
[0188] The processor 602 may execute instructions or codes stored in the memory 601, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.
[0189] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 601 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 601 and the processor 602 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 602 can read files stored in the memory.
[0190] In addition, the electronic device 600 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 600 may be connected to each other via a bus and / or a network.
[0191] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the above-mentioned method of estimating the causal strength between events.
[0192] Examples of computer-readable storage media include read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R, BD-RW, etc. , BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers. It should be noted that the instructions can also be used to execute additional steps in addition to the above steps or to perform more specific processing when executing the above steps. These additional steps and further processing contents have been mentioned in the description of the relevant methods, so they will not be repeated here to avoid repetition.
[0193] Another embodiment of the present disclosure relates to a system including at least one computing device and at least one storage device storing instructions, wherein when the instructions are executed by the at least one computing device, the at least one computing device is prompted to perform the above-mentioned method for estimating the causal strength between events.
[0194] It should be noted that the system according to the exemplary embodiments of the present disclosure can completely rely on the execution of computer programs or instructions to realize corresponding functions, that is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (e.g., lib library) to realize the corresponding functions.
[0195] On the other hand, when the above-mentioned system is implemented in software, firmware, middleware or microcode, the program code or code segment for performing the corresponding operation can be stored in a computer-readable medium such as a storage medium, so that at least one processor or at least one computing device can perform the corresponding operation by reading and running the corresponding program code or code segment.
[0196] According to an exemplary embodiment of the present disclosure, the storage device may be integrated with the computing device, for example, RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the storage device may include an independent device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The storage device and the computing device may be operationally coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the computing device can read instructions stored in the storage device.
[0197] Another embodiment of the present disclosure relates to a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for estimating the causal strength between events as described in any one of the above.
[0198] According to the method, device, electronic device, storage medium, system and computer program product for estimating the causal strength between events provided by the present disclosure, causal information can be enhanced by introducing an external causal knowledge base, so that common-sense causal prior knowledge can be integrated into the process of estimating the causal strength between event texts; in the process of estimating the causal strength between event texts, semantic information is combined, so that the process of estimating the causal strength is not limited to the statistics of the shared frequency of causal pairs, thereby improving the recognition ability of causal semantics.
[0199] In addition, a deep learning framework can be used, through the currently popular ALBERT pre-training model, to realize that the calculation process of causal strength is not limited to the statistics of causal pair sharing frequency, and to integrate semantic information into the causal strength calculation.
[0200] In addition, the adversarial training method can be integrated to apply the pre-trained model ALBERT to the training of causal information enhancement, establish parameter sharing between the basic model and the causal information enhancement model, and realize the joint calculation of causal strength through two sets of causal pairs.
[0201] In addition, by introducing an external causal knowledge base to form the input of the causal information enhancement model, the overall model composed of the basic model and the causal information enhancement model can be better trained with the help of common-sense causal semantics, thereby integrating common-sense causal prior knowledge into the overall model composed of the basic model and the causal information enhancement model.
[0202] In addition, by introducing an external causal knowledge base to enhance the causal information of cause-effect event pairs (hereinafter referred to as causal pairs), and adopting an adversarial training framework to calculate the causal strength of causal pairs, the causal strength analysis results can be made more accurate.
[0203] The above describes various exemplary embodiments of the present disclosure, and it should be understood that the above description is only exemplary and not exhaustive, and the present disclosure is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present disclosure, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present disclosure should be based on the scope of the claims.
Claims
1. A method for estimating the causal strength between events, characterized in that The method comprises: Acquire first input data, wherein the first input data includes two event texts; Searching from an external causal knowledge base to obtain information enhancement words that match the core trigger words in the two event texts and have consistent directions, or information enhancement words that have the highest semantic similarity to the core trigger words in the two event texts and have consistent directions; The two event texts are respectively concatenated with corresponding information enhancement words to obtain second input data; Based on the semantic information of the first input data and the second input data, estimating the causal strength between the two event texts; The estimating the causal strength between the two event texts based on the semantic information of the first input data and the second input data includes: Based on the ALBERT pre-training model, respectively learn and obtain semantic information of the first input data and the second input data; Based on the similarity of semantic information between the first input data and the second input data, the causal strength between the two event texts is estimated.
2. The method for estimating causal strength between events according to claim 1, characterized in that: The estimating the causal strength between the two event texts based on the semantic information of the first input data and the second input data further includes: Obtaining a first feature vector corresponding to the first input data based on a basic model, wherein the first feature vector contains semantic information of the first input data; Obtaining a second feature vector corresponding to the second input data based on a causal information enhancement model, wherein the second feature vector includes semantic information of the second input data; Based on a final feature vector obtained by adding the first feature vector and the second feature vector, a causal strength score between the two event texts is calculated through a fully connected network; Among them, the basic model and the causal information enhancement model respectively include an ALBERT pre-trained model and a pooling layer. The ALBERT pre-trained model is used to obtain a corresponding word embedding vector based on input data, and the word embedding vector is used to represent the semantic information of the corresponding input data.
3. The method for estimating causal strength between events as claimed in claim 2, characterized in that: The final feature vector obtained by adding the first feature vector and the second feature vector is calculated by a fully connected network to obtain a causal strength score between the two event texts, including: in, represents the final eigenvector, represents the first eigenvector, represents the second eigenvector, is the fully connected network parameter, is the bias, is the sigmoid function, is the causal strength score between the two event texts.
4. The method for estimating causal strength between events according to claim 2, characterized in that: The method further comprises: The basic model and the causal information enhancement model are trained with the goal of making the basic model as close as possible to the causal information enhancement model.
5. The method for estimating causal strength between events according to claim 4, characterized in that: The training of the basic model and the causal information enhancement model with the goal of making the basic model as close as possible to the causal information enhancement model includes: By sharing parameters between the basic model and the causal information enhancement model, the parameters of the basic model and the causal information enhancement model are adjusted with the goal of minimizing a preset loss function and maximizing the similarity between the first feature vector and the second feature vector; In which, the preset loss function is used to make the first causal strength score as close as possible to the second causal strength score in the minimization process, the first causal strength score is the causal strength score between the two event texts estimated based on the first input data, and the second causal strength score is the causal strength score between the two event texts estimated based on the second input data.
6. The method for estimating causal strength between events according to claim 5, characterized in that: The similarity between the first feature vector and the second feature vector is calculated by the following formula: in, represents the first eigenvector, represents the second eigenvector, represents the absolute value, express The elements, express The elements, is a prime number.
7. The method for estimating causal strength between events according to claim 5, characterized in that: The first causal strength score is calculated by the following formula: in, and They are represented as anchor samples the two event texts in the first input data; The calculation method of the second causal strength score is expressed as: in, and Represents a positive sample In the second input data, a text obtained by concatenating the two event texts with the corresponding information enhancement words respectively; Positive Sample and anchor samples The following conditions are met: in, , , is the set containing all anchor samples, is a set containing all positive samples; The preset loss function is expressed by the following formula: in, is a parameter representing the error range.
8. A device for estimating the causal strength between events, characterized in that The device comprises: An acquisition unit is configured to: acquire first input data, wherein the first input data includes two event texts; A search unit is configured to: search from an external causal knowledge base to obtain information enhancement words that match the core trigger words in the two event texts and have consistent directions, or information enhancement words that have the highest semantic similarity with the core trigger words in the two event texts and have consistent directions; A concatenation unit is configured to: concatenate the two event texts with the corresponding information enhancement words respectively to obtain second input data; an estimating unit, configured to: estimate the causal strength between the two event texts based on the semantic information of the first input data and the second input data; Wherein, the estimation unit is configured as follows: Based on the ALBERT pre-training model, respectively learn and obtain semantic information of the first input data and the second input data; Based on the similarity of semantic information between the first input data and the second input data, the causal strength between the two event texts is estimated.
9. The device for estimating causal strength between events according to claim 8, characterized in that: The estimation unit is further configured to: Obtaining a first feature vector corresponding to the first input data based on a basic model, wherein the first feature vector contains semantic information of the first input data; Obtaining a second feature vector corresponding to the second input data based on a causal information enhancement model, wherein the second feature vector includes semantic information of the second input data; Based on a final feature vector obtained by adding the first feature vector and the second feature vector, a causal strength score between the two event texts is calculated through a fully connected network; Among them, the basic model and the causal information enhancement model respectively include an ALBERT pre-trained model and a pooling layer. The ALBERT pre-trained model is used to obtain a corresponding word embedding vector based on input data, and the word embedding vector is used to represent the semantic information of the corresponding input data.
10. The device for estimating causal strength between events according to claim 9, characterized in that: The estimation unit is configured to: The final feature vector is obtained by adding the first feature vector and the second feature vector through the following formula, and the causal strength score between the two event texts is calculated through a fully connected network: in, represents the final eigenvector, represents the first eigenvector, represents the second eigenvector, is the fully connected network parameter, is the bias, is the sigmoid function, is the causal strength score between the two event texts.
11. The device for estimating causal strength between events according to claim 9, characterized in that: The device also includes: The training unit is configured to train the basic model and the causal information enhancement model with the goal of making the basic model as close as possible to the causal information enhancement model.
12. The device for estimating causal strength between events according to claim 11, characterized in that: The training unit is configured as follows: By sharing parameters between the basic model and the causal information enhancement model, the parameters of the basic model and the causal information enhancement model are adjusted with the goal of minimizing a preset loss function and maximizing the similarity between the first feature vector and the second feature vector; In which, the preset loss function is used to make the first causal strength score as close as possible to the second causal strength score in the minimization process, the first causal strength score is the causal strength score between the two event texts estimated based on the first input data, and the second causal strength score is the causal strength score between the two event texts estimated based on the second input data.
13. The device for estimating causal strength between events according to claim 12, characterized in that: The training unit is configured as follows: The similarity between the first feature vector and the second feature vector is calculated by the following formula: in, represents the first eigenvector, represents the second eigenvector, represents the absolute value, express The elements, express The elements, is a prime number.
14. The device for estimating causal strength between events according to claim 12, characterized in that: The training unit is configured as follows: The first causal strength score is calculated by the following formula: in, and They are represented as anchor samples the two event texts in the first input data; The second causal strength score is calculated by the following formula: in, and Represents a positive sample In the second input data, a text obtained by concatenating the two event texts with the corresponding information enhancement words respectively; Positive Sample and anchor samples The following conditions are met: in, , , is the set containing all anchor samples, is a set containing all positive samples; The preset loss function is expressed by the following formula: in, is a parameter representing the error range.
15. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to perform the method for estimating the causal strength between events according to any one of claims 1 to 7.
16. A computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by at least one processor, the at least one processor is prompted to perform the method for estimating causal strength between events according to any one of claims 1 to 7.
17. A system comprising at least one computing device and at least one storage device storing instructions, characterized in that: When the instructions are executed by the at least one computing device, the at least one computing device is prompted to perform the method for estimating causal strength between events as described in any one of claims 1 to 7.
18. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method for estimating the causal strength between events as claimed in any one of claims 1 to 7 is implemented.
