Entity relationship extraction method for text data in network security field based on multi-task learning

Through a multi-task learning-based method, using technologies such as ERNIE and Bi-GRU to identify entities and relationships in text data in the field of network security, the problem of loose structure and heterogeneous diversity is solved, and the accuracy and efficiency of entity relationship extraction is improved. .

CN113988075BActive Publication Date: 2024-11-19GUANGZHOU JIANHUA SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111264565.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-28
Publication Date
2024-11-19
Estimated Expiration
2041-10-28

AI Technical Summary

Technical Problem

The existing technology has the problem of loose structure and heterogeneous diversity in text data in the field of network security, resulting in low efficiency and low accuracy of entity relationship extraction.

Method used

Using a multi-task learning method, the pre- and post-text information is captured through ERNIE pre-training language model, Bi-GRU, and the pointer network identification entity and relationship pairing are used to decode entity relationship triple-tuple decoding with the global matrix, and the model effect is optimized through multiple trainings. .

Benefits of technology

It significantly improves the accuracy and efficiency of entity relationship extraction in text data in the field of network security, solves the problems of loose structure and heterogeneous diversity, and is suitable for large-scale Internet data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113988075B_ABST
    Figure CN113988075B_ABST
Patent Text Reader

Abstract

The present invention provides a method for extracting entity relationships from text data in the field of network security based on multi-task learning. The method solves the problem of classifying relationships between target entities in the original text, and to a considerable extent solves the defects of loose structure organization and heterogeneous diversity in large-scale Internet data. Entity relationships are an important step in building complex knowledge base systems, such as text summarization, automatic question answering, machine translation, search engines, knowledge graphs, etc. This technology has become a key factor in the development of technologies such as natural language processing, machine intelligence learning, and big data mining. It also means that this technology is related to the future development of industry and informatization in my country, and has a very broad application prospect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology for artificial intelligence security, and more specifically, to a method for extracting entity relationships from text data in the field of network security based on multi-task learning. Background Art

[0002] The main goal of entity relationship extraction is to identify and determine the specific relationship between entity pairs from natural language texts. As a core task in the fields of information retrieval, information extraction, and natural language understanding, this has always been a hot topic in the field of natural language processing. After years of exploration and research by Chinese and foreign scholars, relatively rich research results have been achieved. From the initial traditional rule-based and dictionary-driven methods to traditional machine learning methods. In recent years, with the rise of deep learning, scholars have applied deep learning methods such as Bi-LSTM and GCN to entity relationship extraction tasks, which greatly improved the accuracy of the task. In recent years, the introduction of attention mechanisms and pre-training models has not only improved the accuracy of the model, but also greatly improved the training efficiency of the model.

[0003] Currently available solution technologies:

[0004] Medical entity relationship extraction method, device, computer equipment and readable storage medium (2019103304707), the method uses a model to perform medical named entity recognition on data; and extracts entity relationships based on the entity recognition results, obtains entity pairs with entity relationships, calculates the confidence of the entity pairs based on the entity relationships, and generates target data based on each entity pair, entity relationship and corresponding confidence, which solves the problem of time-consuming and labor-intensive and low efficiency of manual extraction of medical entity relationships in the prior art. However, this method cannot make good use of the contextual relationship of the data set, nor does it solve the problem of missing a small number of entities.

[0005] A text entity relationship extraction method for the chemical industry (2020109787338) discloses a text entity relationship extraction method for the chemical industry. The corresponding data set is collected through web crawlers and crowdsourcing technology, and a more reasonable, comprehensive and highly accurate text entity relationship extraction method for the chemical industry is obtained through the corresponding relationship. However, the data set for model training is small, and the number of occurrences of individual features is too small, which is insufficient for learning such features.

[0006] Drug entity relationship extraction method and system based on attention mechanism neural network (2018115105663), the method relates to a drug entity relationship extraction method and system based on attention mechanism neural network. The invention adds a classification method that focuses on the weight of entity category information in the attention mechanism, which can reduce the impact of incorrect dependency analysis results in long sentences and improve the accuracy of drug entity relationship extraction. However, the biggest problem is that it requires a lot of space resources, and each round of training takes a long time.

[0007] An open Chinese entity relationship extraction method using dependency analysis (2015103213396), the invention discloses an open Chinese entity relationship extraction method using dependency analysis, the method first performs dependency analysis on the sentence, then extracts the relationship words in combination with the Chinese grammar heuristic rules and the result of dependency analysis, then determines the position of the named entity according to the distance, and finally outputs the triple. Experiments were conducted on the SogouCA and SogouCS corpora, and the results show that the invention is suitable for large-scale corpora and has good portability. The invention fundamentally overcomes the limitations of the inherent properties of Chinese such as complex grammar, diverse expressions, and rich semantics. However, the model training process is relatively cumbersome and complicated, and the accuracy is not high enough.

[0008] A remote supervised entity relationship extraction method based on human-computer interaction (2018113966422), the invention discloses a remote supervised entity relationship extraction method based on human-computer interaction, which is characterized by adding crowdsourcing technology to the model training of the relationship extraction task, adjusting the crowdsourcing strategy in time according to the feedback of the model to obtain new data and input it into the model until all data are cleaned or the model performance is no longer improved. Compared with the prior art, the invention has the advantages of high quality of crowdsourcing results and low crowdsourcing cost. It does not need to rely on experts for labeling and can be processed in parallel by multiple workers. It better solves the problem of noise data reducing model performance in remote supervision of entity relationship extraction. However, they need to consume more computing resources and the cost is very high. Summary of the invention

[0009] The present invention provides a method for extracting entity relationships from text data in the field of network security based on multi-task learning, which solves the defects of loose structure organization and heterogeneous diversity in large-scale Internet data.

[0010] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0011] A method for extracting entity relations from text data in the field of network security based on multi-task learning, comprising the following steps:

[0012] S1: Clean and annotate the collected cybersecurity text data;

[0013] S2: Use large-scale unlabeled corpus related to the cybersecurity text field to pre-train the language model ERNIE;

[0014] S3: Use ERNIE to analyze the sentence S = {s1, s2...s n} is encoded and outputs a fixed-dimensional word vector W = {w1, w2…w n};

[0015] S4: The word vector W output by ERNIE is W = {w1, w2…w n} as input, predict the set of relations R that may exist in the sentence;

[0016] S5: Concatenate the word vector W output from step S2 with the relation set R from step S3, and then use Bi-GRU to capture the hidden information of the context that can effectively improve the accuracy of the model, and output a sequence H containing the hidden information = {h1, h2…h n};

[0017] S6: Identify the entity set E in the sentence through the pointer network, use the global matrix to pair the entities with the relations, and complete the entity-relation triple decoding;

[0018] S7: Repeat steps S2-S6 for multiple trainings until the optimal effect is achieved.

[0019] Furthermore, the process of collecting network security text data in step S1 is:

[0020] S11: First, we use crawlers to crawl websites such as the Cyberspace Administration of China. The crawled modules include first-level websites such as governance and supervision, early warning notifications, and network security review. Then, we crawl second-level websites based on the first-level websites until all data texts under the second-level websites are crawled. Then, we collect reported data related to network security, data on the entire network security industry, and industry development reports.

[0021] S12: Segment the collected cybersecurity texts, segment the sentences using periods, question marks, exclamation marks, and ellipsis marks as markers, and obtain sentence-level text data, which is stored in a way that one sentence occupies one line. Then, the sentence-level text data is deduplicated and filtered, and data not related to cybersecurity is deleted, thereby achieving data cleaning.

[0022] S13: After cleaning the data, the "BIO" scheme is used to manually annotate the data: "B" indicates the starting position of the entity, "I" indicates the middle position of the entity, and "O" indicates a non-entity word; after the annotation is completed, 80% of them are randomly selected as training sets, 10% as development sets, and 10% as test sets. In order to ensure the balance of the data, the ratio of the relationship volume to the data volume of the three data sets is basically the same. At this point, the collection of network security text data is completed.

[0023] Furthermore, the specific process of step S3 is:

[0024] The text sequence S in the network security dataset is n} Input in the format required by ERNIE, use the twice pre-trained language model ERNIE to encode the sequence, and output a fixed-dimensional word vector W = {w1, w2…w n}, and at the same time generate a global matrix, where n is the length of the sequence.

[0025] Furthermore, the specific process of step S4 is:

[0026] For the word vector W = {w1, w2…w n}, predict the possible relationship set in the sentence, and output the relationship set R; model the relationship prediction as a multi-label binary classification task, and perform relationship prediction through formulas (1) and (2). If the probability exceeds a certain threshold λ1, the corresponding vector will be assigned a label "1", otherwise, the corresponding relationship will be assigned a label "0"; the word vector with the label "1" is recorded as r t Add to the relationship set R, t is initialized to 1, and increases according to the number of relationships. The prediction principle is as follows:

[0027] H avg =Avgpool(X) (1)

[0028] P rel =σ(W r H avg +b v ) (2)

[0029] Among them, Avgpool is the average pooling operation, W r is the relationship weight matrix, b v is the relation bias vector.

[0030] Furthermore, in step S5, the word vector sequence W={w1, w2…w n}, and the relation set R = {r1, r2…r m} are concatenated into X = {x1, x2…xn}, and then use Bi-GRU to encode to capture the hidden information of the previous and following texts, and finally output the sequence H = {h1, h2…h n}, the GRU principle formula is expressed as follows:

[0031] GRU's update gate calculation:

[0032] z t =σ(W z ·[h t-1 , x t ]+b z ) (3)

[0033] Among them, σ represents the sigmoid activation function, W z is the weight matrix of the update gate, t-1 is the output of the GRU unit at the previous moment, x t is the word vector input at the current moment, b z is the bias vector of the update gate;

[0034] GRU reset gate calculation:

[0035] r t =σ(W r ·[ t-1 , x t ]+b r ) (4)

[0036] Among them, σ represents the sigmoid activation function, W r is the weight matrix of the reset gate, h t-1 is the output of the GRU unit at the previous moment, x t is the word vector input at the current moment, b r is the bias vector of the reset gate, which generates new memory information according to the update gate

[0037]

[0038] Among them, tanh represents the tanh activation function, W is the hidden state weight matrix, r t is the output of the reset gate, h t-1 is the output of the hidden state at the previous moment, x t is the word vector input at the current moment, is the hidden state bias vector;

[0039] GRU output calculation at the current moment:

[0040]

[0041] Among them, zt is the output of the update gate, h t-1 is the output of GRU at the previous moment, t It is in a hidden state;

[0042] Using the above five formulas, we can get the forward GRU output vector as → t , and the backward GRU output vector is ← t , integrate the previous and next vectors to get the output of Bi-GRU t :

[0043]

[0044] Word vector sequence X = {x1, x2, x3…x n}, after encoding, the final output contains the hidden information sequence H = {h1, h2…h n};

[0045] Two identical binary classifiers are used to mark whether the current vector corresponds to the start or end position of the object, thereby identifying the entity set E in the sentence;

[0046] represents the probability that the i-th vector is identified as the starting position of the object, and represents the probability that the i-th vector is recognized as the end position of the object. If the probability exceeds a certain threshold λ2, the corresponding vector will be assigned a label of "1", otherwise it will be assigned a label of "0", and the word vector with the label "1" is recorded as e j Add to the entity set E, j is initialized to 1, and increases according to the number of entities. The prediction principle is as follows:

[0047]

[0048] Among them, σ represents the sigmoid activation function, W start is the weight matrix for predicting the starting position of the entity, h i is the current predicted word vector, b start is the bias vector for predicting the entity's start position

[0049]

[0050] Among them, σ represents the sigmoid activation function, W end is the weight matrix for predicting the end position of the entity, b end is the bias vector for predicting the end position of the entity.

[0051] Furthermore, the specific process of step S6 is:

[0052] The global matrix generated in step S3, the sequence H containing hidden information output in step S5, and the output entity set E are concatenated, and then the entity relationship triples are extracted using the following formula:

[0053] P(s,r,o)=σ(W g [E; H]+b g ) (11)

[0054] Among them, P(s, r, o) is the extracted entity-relation triple, σ represents the sigmoid activation function, E is the entity vector, R is the relationship vector, and W g is the weight matrix of the predicted triples, h i is the current predicted word vector, b g is the bias vector for the prediction triplet.

[0055] Furthermore, the specific process of step S7 is:

[0056] The triples extracted in step S6 are compared with the triples in the actual labeled data. After calculating the current loss value loss, the loss value loss is optimized and trained using the stochastic gradient descent method. During the training process, the matrix parameters of each weight in the model are adjusted using back propagation. The above training process is repeated many times until the effect is optimal.

[0057] Furthermore, the role of the global matrix is ​​to predict the pairing scores between the relationship set and the entity set in order to realize the extraction of entity relationship triples; the left side is the subject and the right side is the object. The one marked as "1" is the beginning of the two subjects, and the one marked as "2" is the end of the two subjects. In this way, the paired subjects and objects in the triples can be determined, and then the relationship probabilities corresponding to the subjects and objects are predicted based on the relationship set. At this point, the extraction of entity relationship triples is completed.

[0058] Furthermore, the relationship prediction is modeled as a multi-label binary classification task. The relationship set in the sentence is first obtained, and then entity recognition is performed to effectively alleviate entity redundancy.

[0059] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0060] The present invention solves the problem of classifying the relationships between target entities in the original text, and to a considerable extent solves the defects of loose structure organization and heterogeneous diversity in large-scale Internet data. Entity relationships are an important step in building complex knowledge base systems, such as text summarization, automatic question answering, machine translation, search engines, knowledge graphs, etc. This technology has become a key factor in the development of technologies such as natural language processing, machine intelligence learning, and big data mining. It also means that this technology is related to the future development of industry and informatization in my country, and has a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a flow chart of the method of the present invention;

[0062] Figure 2 is the set of relations obtained in the instance;

[0063] Figure 3 is the entity set obtained in the instance;

[0064] Figure 4 is the global matrix obtained in the example. DETAILED DESCRIPTION

[0065] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0066] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0067] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0068] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0069] like Figure 1 As shown, a method for extracting entity relations from text data in the field of network security based on multi-task learning includes the following steps:

[0070] S1: Clean and annotate the collected cybersecurity text data;

[0071] S2: Use large-scale unlabeled corpus related to the cybersecurity text field to pre-train the language model ERNIE;

[0072] S3: Use ERNIE to analyze the sentence S = {s1, s2...s n} is encoded and outputs a fixed-dimensional word vector W = {w1, w2…w n};

[0073] S4: The word vector W output by ERNIE is W = {w1, w2…w n} as input, predict the set of relations R that may exist in the sentence;

[0074] S5: Concatenate the word vector W output from step S2 with the relation set R from step S3, and then use Bi-GRU to capture the hidden information of the context that can effectively improve the accuracy of the model, and output a sequence H containing the hidden information = {h1, h2…h n};

[0075] S6: Identify the entity set E in the sentence through the pointer network, use the global matrix to pair the entities with the relations, and complete the entity-relation triple decoding;

[0076] S7: Repeat steps S2-S6 for multiple trainings until the optimal effect is achieved.

[0077] The process of collecting network security text data in step S1 is:

[0078] S11: First, we use crawlers to crawl websites such as the Cyberspace Administration of China. The crawled modules include first-level websites such as governance and supervision, early warning notifications, and network security review. Then, we crawl second-level websites based on the first-level websites until all data texts under the second-level websites are crawled. Then, we collect reported data related to network security, data on the entire network security industry, and industry development reports.

[0079] S12: Segment the collected cybersecurity texts, segment the sentences using periods, question marks, exclamation marks, and ellipsis marks as markers, and obtain sentence-level text data, which is stored in a way that one sentence occupies one line. Then, the sentence-level text data is deduplicated and filtered, and data not related to cybersecurity is deleted, thereby achieving data cleaning.

[0080] S13: After cleaning the data, the "BIO" scheme is used to manually annotate the data: "B" indicates the starting position of the entity, "I" indicates the middle position of the entity, and "O" indicates a non-entity word; after the annotation is completed, 80% of them are randomly selected as training sets, 10% as development sets, and 10% as test sets. In order to ensure the balance of the data, the ratio of the relationship volume to the data volume of the three data sets is basically the same. At this point, the collection of network security text data is completed.

[0081] The specific process of step S3 is:

[0082] The text sequence S in the network security dataset is n} Input in the format required by ERNIE, use the twice pre-trained language model ERNIE to encode the sequence, and output a fixed-dimensional word vector W = {w1, w2…w n}, and at the same time generate a global matrix, where n is the length of the sequence.

[0083] The specific process of step S4 is:

[0084] For the word vector W = {w1, w2…w n}, predict the possible relationship set in the sentence, and output the relationship set R; model the relationship prediction as a multi-label binary classification task, and perform relationship prediction through formulas (1) and (2). If the probability exceeds a certain threshold λ1, the corresponding vector will be assigned a label "1", otherwise, the corresponding relationship will be assigned a label "0"; the word vector with the label "1" is recorded as r t Add to the relationship set R, t is initialized to 1, and increases according to the number of relationships. The prediction principle is as follows:

[0085] H avg =Avgpool(X) (1)

[0086] P rel =σ(W r H avg +b v ) (2)

[0087] Among them, Avgpool is the average pooling operation, W r is the relationship weight matrix, b v is the relation bias vector.

[0088] Furthermore, in step S5, the word vector sequence W={w1, w2…w n}, and the relation set R = {r1, r2…r m} are concatenated into X = {x1, x2…x n}, and then use Bi-GRU to encode to capture the hidden information of the previous and following texts, and finally output the sequence H = {h1, h2…h n}, the GRU principle formula is expressed as follows:

[0089] GRU's update gate calculation:

[0090] z t =σ(W z ·[h t-1 , x t ]+b z ) (3)

[0091] Among them, σ represents the sigmoid activation function, W z is the weight matrix of the update gate, h t-1 is the output of the GRU unit at the previous moment, x t is the word vector input at the current moment, b z is the bias vector of the update gate;

[0092] GRU reset gate calculation:

[0093] r t =σ(Wr ·[h t-1 , x t ]+b r ) (4)

[0094] Among them, σ represents the sigmoid activation function, W r is the weight matrix of the reset gate, h t-1 is the output of the GRU unit at the previous moment, x t is the word vector input at the current moment, b r is the bias vector of the reset gate, which generates new memory information according to the update gate

[0095]

[0096] Among them, tanh represents the tanh activation function, W h is the hidden state weight matrix, r t is the output of the reset gate, h t-1 is the output of the hidden state at the previous moment, x t is the word vector input at the current moment, is the hidden state bias vector;

[0097] GRU output calculation at the current moment:

[0098]

[0099] Among them, z t is the output of the update gate, h t-1 is the output of GRU at the previous moment, It is in a hidden state;

[0100] Using the above five formulas, we can get the forward GRU output vector as And the backward GRU output vector is Integrate the previous and next vectors to get the output h of Bi-GRU t :

[0101]

[0102] Word vector sequence X = {x1, x2, x3…x n}, after encoding, the final output contains the hidden information sequence H = {h1, h2…h n};

[0103] Two identical binary classifiers are used to mark whether the current vector corresponds to the start or end position of the object, thereby identifying the entity set E in the sentence;

[0104] represents the probability that the i-th vector is identified as the starting position of the object, and represents the probability that the i-th vector is recognized as the end position of the object. If the probability exceeds a certain threshold λ2, the corresponding vector will be assigned a label of "1", otherwise it will be assigned a label of "0", and the word vector with the label "1" is recorded as e j Add to the entity set E, j is initialized to 1, and increases according to the number of entities. The prediction principle is as follows:

[0105]

[0106] Among them, σ represents the sigmoid activation function, W start is the weight matrix for predicting the starting position of the entity, h i is the current predicted word vector, b start is the bias vector for predicting the entity's start position

[0107]

[0108] Among them, σ represents the sigmoid activation function, end is the weight matrix for predicting the end position of the entity, end is the bias vector for predicting the end position of the entity.

[0109] The specific process of step S6 is:

[0110] The global matrix generated in step S3, the sequence H containing hidden information output in step S5, and the output entity set E are concatenated, and then the entity relationship triples are extracted using the following formula:

[0111] P(s,r,o)=σ(W g [E; H]+b g ) (11)

[0112] Among them, P(s, r, o) is the extracted entity-relation triple, σ represents the sigmoid activation function, E is the entity vector, R is the relationship vector, and W g is the weight matrix of the predicted triples, h i is the current predicted word vector, b g is the bias vector for the prediction triplet.

[0113] The specific process of step S7 is:

[0114] The triples extracted in step S6 are compared with the triples in the actual labeled data. After calculating the current loss value loss, the loss value loss is optimized and trained using the stochastic gradient descent method. During the training process, the matrix parameters of each weight in the model are adjusted using back propagation. The above training process is repeated many times until the effect is optimal.

[0115] The role of the global matrix is ​​to predict the pairing scores between the relationship set and the entity set in order to realize the extraction of entity relationship triples; the left side is the subject and the right side is the object. The one marked as "1" is the beginning of the two subjects, and the one marked as "2" is the end of the two subjects. This can determine the paired subjects and objects in the triples, and then predict the corresponding relationship probabilities of the subjects and objects based on the relationship set. At this point, the extraction of entity relationship triples is completed; the relationship prediction is modeled as a multi-label binary classification task. The relationship set in the sentence is first obtained, and then entity recognition is performed to effectively alleviate entity redundancy.

[0116] Examples:

[0117] The sentence in the dataset: "Firewalls can effectively resist hacker attacks" is processed using this method:

[0118] S1: First, the text sequence S = {s1, s2...s n} is input into the language model ERNIE that has been pre-trained twice, encoded, and the word vector sequence W = {w1, w2…w n};

[0119] S2: Then, the relationship is predicted based on the word vector W output by ERNIE to obtain the relationship set R, such as Figure 2 As shown;

[0120] S3: Combine the two parts R and W into X = {x1, x2…x n} and input into Bi-GRU, using forward and backward GRU to obtain the hidden information of the previous and next texts respectively, and output sequence H = {h1, h2…h n};

[0121] S4: Input sequence H = {h1, h2...h n}, use two identical binary classifiers to extract the entity set E in the text, such as Figure 3 As shown;

[0122] S5: Concatenate the sequence H containing hidden information and the entity set E, and then pair them according to the global matrix, such as Figure 4 shown.

[0123] S6: Finally, the entity relationship triples are decoded: (firewall, resist, hacker), (firewall, resist, attack).

[0124] The same or similar reference numerals correspond to the same or similar components;

[0125] The positional relationships described in the drawings are only for illustrative purposes and should not be construed as limiting the present patent.

[0126] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for extracting entity relations from text data in the field of network security based on multi-task learning, characterized in that: The following steps are involved: S1: Clean and annotate the collected cybersecurity text data; S2: Use large-scale unlabeled corpus related to the cybersecurity text field to pre-train the language model ERNIE; S3: Use ERNIE to analyze the sentence S = {s1, s2...s n } is encoded and outputs a fixed-dimensional word vector W = {w1, w2…w n }; S4: The word vector W output by ERNIE is W = {w1, w2…w n } as input, predict the set of relations R that may exist in the sentence; for the word vector W = {w1, w2…w n }, predict the possible relationship set in the sentence, and output the relationship set R; model the relationship prediction as a multi-label binary classification task, and perform relationship prediction through formulas (1) and (2). If the probability exceeds a certain threshold λ1, the corresponding vector will be assigned a label "1", otherwise, the corresponding relationship will be assigned a label "0"; the word vector with the label "1" is recorded as r t Add to the relationship set R, t is initialized to 1, and increases according to the number of relationships. The prediction principle is as follows: H avg =Avgpool(W) (1) r t =σ(W r H avg +b v ) (2) Among them, Avgpool is the average pooling operation, W r is the relationship weight matrix, b v is the relation bias vector; S5: Concatenate the word vector W output from step S3 with the relation set R from step S4, and then use Bi-GRU to capture the hidden information of the context that can effectively improve the accuracy of the model, and output a sequence H containing the hidden information = {h1, h2…h n }; S6: Identify the entity set E in the sentence through the pointer network, use the global matrix to pair the entity and the relationship, and complete the entity relationship triple decoding; concatenate the global matrix generated in step S3, the sequence H containing hidden information in step S5, and the entity set E output in step S6, and then use the following formula to extract the entity relationship triples: P(s,r,o)=σ(W g [E;H]+b g ) (11) Among them, P(s, r, o) is the extracted entity relationship triple, σ represents the sigmoid activation function, E is the entity set, W g is the weight matrix of the predicted triples; b g It is the bias vector of the predicted triple. The function of the global matrix is ​​to predict the score of the pairing between the relationship set and the entity set to realize the extraction of entity relationship triples. The left side is the subject and the right side is the object. The "1" is the beginning of the two subjects and the "2" is the end of the two subjects. The paired subject and object in the triple can be determined, and then the relationship probability corresponding to the subject and object is predicted according to the relationship set. At this point, the entity relationship triple extraction is completed; S7: Repeat steps S2-S6 for multiple trainings until the optimal effect is achieved.

2. According to the multi-task learning-based network security field text data entity relationship extraction method according to claim 1, it is characterized in that: The process of collecting network security text data in step S1 is: S11: First, we use crawlers to crawl websites such as the Cyberspace Administration of China. The crawled modules include first-level websites such as governance and supervision, early warning notifications, and network security review. Then, we crawl second-level websites based on the first-level websites until all data texts under the second-level websites are crawled. Then, we collect reported data related to network security, data on the entire network security industry, and industry development reports. S12: Segment the collected cybersecurity texts, segment the sentences using periods, question marks, exclamation marks, and ellipsis marks as markers, and obtain sentence-level text data, which is stored in a way that one sentence occupies one line. Then, the sentence-level text data is deduplicated and filtered, and data not related to cybersecurity is deleted, thereby achieving data cleaning. S13: After cleaning the data, the "BIO" scheme is used to manually annotate the data: "B" indicates the starting position of the entity, "I" indicates the middle position of the entity, and "O" indicates a non-entity word; after the annotation is completed, 80% of them are randomly selected as training sets, 10% as development sets, and 10% as test sets. In order to ensure the balance of the data, the ratio of the relationship volume to the data volume of the three data sets is basically the same. At this point, the collection of network security text data is completed.

3. The network security field text data entity relationship extraction method based on multi-task learning according to claim 2 is characterized in that: The specific process of step S3 is: The text sequence S in the network security dataset is n } Input in the format required by ERNIE, use the twice pre-trained language model ERNIE to encode the sequence, and output a fixed-dimensional word vector W = {w1, w2…w n }, and at the same time generate a global matrix, where n is the length of the sequence.

4. The network security field text data entity relationship extraction method based on multi-task learning according to claim 3 is characterized in that: In step S5, the word vector sequence W={w1, w2…w n }, and the relation set R = {r1, r2…r m } are concatenated into X = {x1, x2…x n }, and then use Bi-GRU to encode to capture the hidden information of the previous and following texts, and finally output the sequence H = {h1, h2…h n }, the GRU principle formula is expressed as follows: GRU's update gate calculation: z t =σ(W z ·[h t-1 ,x t ]+b z ) (3) Among them, σ represents the sigmoid activation function, W z is the weight matrix of the update gate, h t-1 is the output of the GRU unit at the previous moment, x t is the word vector input at the current moment, b z is the bias vector of the update gate; GRU reset gate calculation: r t =σ(W r ·[h t-1 ,x t ]+b r ) (4) Among them, σ represents the sigmoid activation function, W r is the weight matrix of the reset gate, h t-1 is the output of the GRU unit at the previous moment, x t is the word vector input at the current moment, b r is the bias vector of the reset gate, which generates new memory information according to the update gate Among them, tanh represents the tanh activation function, W h is the hidden state weight matrix, r t is the output of the reset gate, h t-1 is the output of the hidden state at the previous moment, x t is the word vector input at the current moment, is the hidden state bias vector; GRU output calculation at the current moment: Among them, z t is the output of the update gate, h t-1 is the output of GRU at the previous moment, It is in a hidden state; Executing formulas (3)(4)(5)(6) from front to back on the input text, we can get the forward GRU output vector as follows: And execute formula (3)(4)(5)(6) from back to front on the input text to get the backward GRU output vector: Integrate the previous and next vectors to get the output h of Bi-GRU t : Word vector sequence X = {x1, x2, x3…x n }, after encoding, the final output contains the hidden information sequence H = {h1, h2…h n }.

5. The network security field text data entity relationship extraction method based on multi-task learning according to claim 4 is characterized in that: In step S6, two identical binary classifiers are used to mark whether the current vector corresponds to the start position or the end position of the object, thereby identifying the entity set E in the sentence; represents the probability that the i-th vector is identified as the starting position of the object, and represents the probability that the i-th vector is recognized as the end position of the object. If the probability exceeds a certain threshold λ2, the corresponding vector will be assigned a label "1", otherwise it will be assigned a label "0", and the word vector with the label "1" is recorded as e j Add to the entity set E, j is initialized to 1, and increases according to the number of entities. The prediction principle is as follows: Among them, σ represents the sigmoid activation function, W start is the weight matrix for predicting the starting position of the entity, h i is the current predicted word vector, b start is the bias vector for predicting the entity's start position Among them, σ represents the sigmoid activation function, W end is the weight matrix for predicting the end position of the entity, b end is the bias vector for predicting the end position of the entity.

6. The network security field text data entity relationship extraction method based on multi-task learning according to claim 5 is characterized in that: The specific process of step S7 is: The triples extracted in step S6 are compared with the triples in the actual labeled data. After calculating the current loss value loss, the loss value loss is optimized and trained using the stochastic gradient descent method. During the training process, the matrix parameters of each weight in the model are adjusted using back propagation. The above training process is repeated many times until the effect is optimal.