Entity relation joint extraction method and device, equipment, medium and program product

Abstract

The present invention relates to the technical field of data processing, and discloses an entity relationship joint extraction method, apparatus, device, medium and program product, the method comprising: acquiring Internet of Vehicles security data for constructing a security database, the Internet of Vehicles security data comprising structured data and unstructured data; constructing a security ontology model based on the Internet of Vehicles security data; fusing the structured data into the unstructured data to obtain a fused knowledge embedding vector; and on the basis of the security ontology model, encoding and decoding the knowledge embedding vector to obtain a mark sequence of the Internet of Vehicles security data. Therefore, through knowledge fusion of the Internet of Vehicles security data, semantic understanding in the field of Internet of Vehicles security is significantly enhanced, and the accuracy of entity relationship extraction is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment, medium and program product for joint extraction of entity relationships. Background Art

[0002] In view of the complex and ever-changing threats faced by Internet of Vehicles security, combining Internet of Vehicles security threat intelligence, building an Internet of Vehicles security knowledge graph, and completing Internet of Vehicles security threat modeling are of great significance to improving Internet of Vehicles security protection capabilities and enhancing threat response capabilities. The key step in building an Internet of Vehicles security knowledge graph is network security knowledge extraction, which specifically involves subtasks such as entity recognition and relationship extraction.

[0003] In the related technologies, network security knowledge extraction mainly includes rule-based and learning-based methods, and mainly targets network security knowledge extraction in general fields. Rule-based methods extract relevant knowledge in a conventional form by applying predefined rules, which is limited to extracting basic security knowledge such as names and place names, and cannot identify complex entities such as attack methods. With the advancement of deep learning and natural language processing technology, deep learning models have begun to be used to extract more complex security intelligence entities, but the knowledge extraction of pre-trained models lacks understanding of the semantics of specific fields and lacks extraction of relationships between security entities. Therefore, how to quickly and effectively extract entities and relationships has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the present invention provides a method, apparatus, device, medium and program product for joint extraction of entity relations to solve the problem that error propagation is easily caused by performing entity recognition before relation extraction.

[0005] In a first aspect, the present invention provides a method for joint extraction of entity relationships, the method comprising:

[0006] Acquire Internet of Vehicles security data for building a security database, where the Internet of Vehicles security data includes structured data and unstructured data;

[0007] Build a security ontology model based on Internet of Vehicles security data;

[0008] Fuse structured data into unstructured data to obtain the fused knowledge embedding vector;

[0009] Based on the security ontology model, the knowledge embedding vector is encoded and decoded to obtain the tag sequence of the Internet of Vehicles security data. The tag sequence is used to represent the joint extraction results of the entities and relations of the Internet of Vehicles security data.

[0010] The entity relationship joint extraction method of the present invention obtains the Internet of Vehicles security data used to build a security database, then builds a security ontology model based on the Internet of Vehicles security data, fuses the structured data into the unstructured data, obtains the fused knowledge embedding vector, and encodes and decodes the knowledge embedding vector based on the security ontology model to obtain a label sequence of the Internet of Vehicles security data. Therefore, through the knowledge fusion of the Internet of Vehicles security data, the semantic understanding of the Internet of Vehicles security field is significantly enhanced, and the accuracy of entity relationship extraction is effectively improved.

[0011] In an optional implementation, obtaining Internet of Vehicles security data for building a security database includes:

[0012] Store structured data in the form of knowledge triples in a secure database;

[0013] De-noise the unstructured data to obtain text sentences;

[0014] Perform sequence labeling on text sentences to obtain labeled text;

[0015] Stores annotation text in a secure database.

[0016] The entity relationship joint extraction method of the present invention performs denoising on unstructured data to obtain text sentences, and performs sequence annotation on the text sentences to obtain annotated text, and further stores the annotated text in a secure database. Thus, comprehensive and accurate basic data is obtained, providing accurate data support for entity relationship extraction, thereby effectively improving the accuracy of entity relationship extraction.

[0017] In an optional implementation, a security ontology model is constructed based on Internet of Vehicles security data, including:

[0018] Acquire expert knowledge for building a security ontology model. Expert knowledge is used to characterize the basic annotation rules of data categories of IoV security data.

[0019] Based on expert knowledge and Internet of Vehicles security data, the data categories, the hierarchical structure of the data categories, the data attributes of the data categories, and the relationship attributes between the categories included in the security ontology model for characterizing the Internet of Vehicles security data are determined.

[0020] The entity relationship joint extraction method of the present invention obtains expert knowledge for building a security ontology model, and based on the expert knowledge and the Internet of Vehicles security data, determines the data categories, hierarchical structure of data categories, data attributes of data categories, and relationship attributes between categories included in the security ontology model for characterizing the Internet of Vehicles security data. As a result, the security ontology model for entity relationship extraction of Internet of Vehicles security data is reasonably and accurately divided into categories and hierarchies, thereby effectively improving the accuracy and precision of entity relationship extraction of Internet of Vehicles security data.

[0021] In an optional implementation, merging structured data into unstructured data includes:

[0022] Querying multiple target entities from the Internet of Vehicles security data, querying the target entities contained in the structured data and the corresponding insertion positions of the target entities in the unstructured data;

[0023] Based on the target entity and the insertion position, a structured core triple is obtained;

[0024] Insert the structured core triples into the unstructured data to obtain a sentence tree;

[0025] Convert sentence trees into linear structured text data;

[0026] According to the sentence tree, the knowledge embedding vector is obtained.

[0027] The entity relationship joint extraction method of the present invention obtains multiple target entities from the Internet of Vehicles security data, queries the target entities contained in the structured data and the corresponding insertion positions of the target entities in the unstructured data, and obtains the structured core triples based on the target entities and the insertion positions, thereby inserting the structured core triples into the unstructured data to obtain a sentence tree, converting the sentence tree into linear structured text data, and then obtaining a knowledge embedding vector based on the sentence tree. As a result, the error propagation problem caused by the pipeline-style execution of entity recognition first and then relationship extraction in the related technology is effectively avoided, and the efficiency and accuracy of entity relationship extraction are significantly improved.

[0028] In an optional implementation, obtaining a knowledge embedding vector according to a sentence tree includes:

[0029] Construct a visible matrix that is adapted to the positional relationship of the sentence tree as a knowledge embedding vector.

[0030] In an optional implementation, encoding and decoding the knowledge embedding vector based on the security ontology model includes:

[0031] Add the knowledge embedding vector to the self-attention function to perform attention calculation;

[0032] Based on the result of attention calculation, the precoding result is obtained through feedforward neural network, residual calculation, and normalization processing;

[0033] Perform feature extraction on the pre-coding result to obtain the coding output result;

[0034] Entity extraction is performed based on the encoding output results to obtain a tag sequence.

[0035] The entity relationship joint extraction method of the present invention adds the knowledge embedding vector to the self-attention function, performs attention calculation, obtains the precoding result based on the result of the attention calculation through a feedforward neural network, residual calculation, and normalization processing, performs feature extraction on the precoding result, obtains the coding output result, and performs entity extraction based on the coding output result to obtain a tag sequence. Thus, the joint extraction of entities and relationships is achieved, which effectively solves the error propagation problem caused by the pipeline-style method of first performing entity recognition and then performing relationship extraction in the related art, and effectively improves the efficiency of the entity relationship extraction model.

[0036] In a second aspect, the present invention provides an entity relationship joint extraction device, the device comprising:

[0037] A data collection module, used to obtain Internet of Vehicles security data for building a security database, where the Internet of Vehicles security data includes structured data and unstructured data;

[0038] Ontology model building module, used to build a security ontology model based on Internet of Vehicles security data;

[0039] The knowledge fusion module is used to fuse structured data into unstructured data to obtain the fused knowledge embedding vector;

[0040] The joint extraction module is used to encode and decode the knowledge embedding vector based on the security ontology model to obtain a tag sequence of the Internet of Vehicles security data. The tag sequence is used to represent the joint extraction results of the entities and relationships of the Internet of Vehicles security data.

[0041] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the entity relationship joint extraction method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0042] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the entity relationship joint extraction method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0043] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the entity relationship joint extraction method of the first aspect or any corresponding embodiment thereof.

[0044] The entity relationship joint extraction method of the present invention obtains the Internet of Vehicles security data used to build a security database, then builds a security ontology model based on the Internet of Vehicles security data, fuses the structured data into the unstructured data, obtains the fused knowledge embedding vector, and encodes and decodes the knowledge embedding vector based on the security ontology model to obtain a label sequence of the Internet of Vehicles security data. Therefore, through the knowledge fusion of the Internet of Vehicles security data, the semantic understanding of the Internet of Vehicles security field is significantly enhanced, and the accuracy of entity relationship extraction is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a flow chart of a method for joint extraction of entity relationships according to an embodiment of the present invention;

[0047] Figure 2 is a flowchart of another entity relationship joint extraction method according to an embodiment of the present invention;

[0048] Figure 3 is a flowchart of a specific application example of the entity relationship joint extraction method according to an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of an entity relationship joint extraction model structure according to a specific application example of the entity relationship joint extraction method according to an embodiment of the present invention;

[0050] Figure 5 is a structural block diagram of an entity relationship joint extraction device according to an embodiment of the present invention;

[0051] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0053] In the related art, the extraction of entities and relations mostly adopts a pipeline model, first performing entity recognition and then performing relation extraction, ignoring the correlation between entity recognition and relation extraction, which easily leads to error propagation. The embodiments of the present invention provide a method, device, equipment, medium and program product for joint extraction of entity relations to solve the problem that first performing entity recognition and then performing relation extraction easily leads to error propagation.

[0054] According to an embodiment of the present invention, an embodiment of a method for joint extraction of entity relationships is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0055] In this embodiment, a method for joint extraction of entity relationships is provided, which can be used in a server for data processing, etc. Figure 1 is a flowchart of a method for joint extraction of entity relationships according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0056] Step S101, obtaining Internet of Vehicles security data for building a security database, where the Internet of Vehicles security data includes structured data and unstructured data.

[0057] In some optional embodiments of the present invention, the structured data of the Internet of Vehicles security data may include information such as vulnerabilities in a certain version of the open source busybox, and the unstructured data may include threat intelligence data related to the field of automotive network security. For example, a large amount of structured data and unstructured data can be collected from open source intelligence data such as CVE (Common Vulnerabilities & Exposures) vulnerability databases, vulnerability reports, security forums, social media, etc. that are publicly available on the Internet. Furthermore, the structured data can be stored in the database in the form of knowledge triples of <entity, relationship, entity>. The unstructured data is first filtered and denoised, and punctuation marks and special characters in the text sentences of the unstructured data are removed. The text sentences of the unstructured data are sequenced and annotated using a text annotation tool, and stored in the security database as training samples for the entity-relationship joint extraction model.

[0058] Step S102: construct a security ontology model based on the Internet of Vehicles security data.

[0059] In some optional embodiments of the present invention, the security ontology model is used to classify and describe entities and relationships. Here, the ontology model can be constructed in combination with expert knowledge. For example, it is possible to combine expert knowledge and Internet of Vehicles security data, refer to the actual architecture of the Internet of Vehicles and the existing security ontology, and construct a security ontology model suitable for entity relationship extraction of Internet of Vehicles security data. Specifically, the classes, class hierarchies, data attributes of classes, and relationship attributes between classes contained in the ontology model can be determined. Among them, the classes of the security ontology model can include five types of security entities: assets, vulnerabilities, weaknesses, attacks, and attackers. The class hierarchy represents the hierarchical relationship between classes. For example: the asset class can include equipment and sensitive data at all levels of the Internet of Vehicles system, the vulnerability class includes vulnerabilities that attackers can exploit to launch attacks through technical defects, the weakness class includes defects or other errors in the software and hardware of the target assets, the attack class includes related concepts such as the mode or method of attack taken by the attacker in the Internet of Vehicles environment, and the attacker class represents the identity-related information of the person or organization that launches the attack. The data type of the class in the security ontology model represents the attribute information of a single class. The relationship attributes between classes represent the relationships between different classes. The relationships in the security ontology model include "existence", "possession" or "use", for example: <asset, existence, vulnerability>, <asset, possession, weakness>, <weakness, composition, vulnerability>, <attack, exploit, weakness>, <attacker, use, vulnerability>, <attacker, launch, attack>. Thus, the security ontology model of IoV security data is obtained.

[0060] Step S103, merging the structured data into the unstructured data to obtain a fused knowledge embedding vector.

[0061] In some optional embodiments of the present invention, structured data can be fused to unstructured data by introducing a sentence tree and constructing a visible matrix to obtain a fused knowledge embedding vector. The unstructured data is in the form of input text, and the sentence tree is introduced to express the unstructured data in the form of a sentence tree, so that after the structured data is fused to the unstructured data, the obtained text can be displayed in the form of a sentence tree. The visible matrix is ​​constructed to ensure the visibility of each token (the smallest unit in text or sequence data) when jointly extracting entities and relationships from the Internet of Vehicles safety data through an entity-relationship joint extraction model, so as to avoid changing the meaning of the original input sentence of the Internet of Vehicles safety data due to excessive knowledge injection.

[0062] Step S104, based on the security ontology model, the knowledge embedding vector is encoded and decoded to obtain a tag sequence of the Internet of Vehicles security data, and the tag sequence is used to represent the joint extraction result of the entities and relationships of the Internet of Vehicles security data.

[0063] In some optional embodiments of the present invention, structured data can be fused into unstructured data to obtain a fused knowledge embedding vector which is input into an entity relationship joint extraction model for encoding and decoding to obtain a final tag sequence, thereby completing entity relationship extraction.

[0064] Specifically, the entity-relationship joint extraction model is the model used by the embodiment of the present invention for the entity-relationship joint extraction. The entity-relationship joint extraction model can be trained in advance based on the above steps S103 to S104 to obtain the model parameters of the entity-relationship joint extraction model. When obtaining the Internet of Vehicles safety data to be extracted, based on the above steps S103 to S104, a tag sequence of the Internet of Vehicles safety data to be extracted is obtained, which is recorded as the joint extraction result of the entities and relationships of the Internet of Vehicles safety data.

[0065] It should be noted that the entity relationship joint extraction method provided in the embodiment of the present invention can perform training, updating, etc. of the entity relationship joint extraction model based on step S103 to step S104. During the training and updating process of the entity relationship joint extraction model, the tag sequence of the Internet of Vehicles safety data obtained in step S104 can be compared with the actual annotations in the training sample of the Internet of Vehicles safety data obtained in step S101. When the difference between the two meets the set threshold range, it is determined that the training, updating, etc. of the entity relationship joint extraction model is completed.

[0066] Furthermore, based on the trained entity relationship joint extraction model, entity relationship joint extraction is performed through the above steps S103 to S104.

[0067] The entity relationship joint extraction method provided in this embodiment obtains the Internet of Vehicles security data used to build a security database, then builds a security ontology model based on the Internet of Vehicles security data, fuses the structured data into the unstructured data, obtains the fused knowledge embedding vector, and encodes and decodes the knowledge embedding vector based on the security ontology model to obtain a label sequence of the Internet of Vehicles security data. Therefore, through the knowledge fusion of the Internet of Vehicles security data, the semantic understanding of the Internet of Vehicles security field is significantly enhanced, and the accuracy of entity relationship extraction is effectively improved.

[0068] In this embodiment, a method for joint extraction of entity relationships is provided, which can be used in a server for data processing, etc. Figure 2 is a flowchart of a method for joint extraction of entity relationships according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0069] Step S201, obtaining Internet of Vehicles security data for building a security database, where the Internet of Vehicles security data includes structured data and unstructured data.

[0070] In an optional implementation, step S201 may include:

[0071] Step S2011, storing the structured data in the form of knowledge triples in a security database.

[0072] In some optional embodiments of the present invention, a large amount of Internet of Vehicles security data can be collected from open source intelligence data such as CVE vulnerability databases, vulnerability reports, security forums, social media, etc., which are publicly available on the Internet. Among them, structured security data can be directly stored in the security database in the form of knowledge triples of <entity, relationship, entity>.

[0073] Step S2012, denoising the unstructured data to obtain text sentences.

[0074] In some optional implementations of the present invention, the unstructured data may be first filtered and denoised to remove punctuation marks, special characters, etc. in the text sentences of the unstructured data to obtain text sentences.

[0075] Step S2013, sequence labeling is performed on the text sentences to obtain labeled text.

[0076] In some optional embodiments of the present invention, any applicable text annotation tool can be used to perform sequence annotation on the text sentence obtained in step S2012 to obtain annotated text. Sequence Tagging is the most basic task in the natural language recognition process, and the sequence tagging here belongs to the category of named entity recognition.

[0077] Step S2014, storing the annotated text in a secure database.

[0078] After completing operations such as denoising and sequence labeling on unstructured data, the annotated text can be stored in a secure database.

[0079] The entity relationship joint extraction method of the present invention performs denoising on unstructured data to obtain text sentences, and performs sequence annotation on the text sentences to obtain annotated text, and further stores the annotated text in a secure database. Thus, comprehensive and accurate basic data is obtained, providing accurate data support for entity relationship extraction, thereby effectively improving the accuracy of entity relationship extraction.

[0080] Step S202: construct a security ontology model based on the Internet of Vehicles security data.

[0081] In an optional implementation, step S202 may include:

[0082] Step S2021, obtaining expert knowledge for building a security ontology model, where the expert knowledge is used to characterize basic labeling rules for data categories of Internet of Vehicles security data.

[0083] In some optional embodiments of the present invention, expert knowledge may be industry-wide rules for entity classification of Internet of Vehicles security data, which may be obtained from the Internet through legal means using technologies such as real-time crawlers, etc. It may also be obtained in advance through other means and stored in a server for training an entity relationship joint extraction model.

[0084] Step S2022, based on expert knowledge and Internet of Vehicles security data, determine the data categories, hierarchical structure of data categories, data attributes of data categories, and relationship attributes between categories included in the security ontology model for characterizing Internet of Vehicles security data.

[0085] In some optional embodiments of the present invention, a security ontology model suitable for entity relationship extraction of Internet of Vehicles security data can be constructed by combining expert knowledge and Internet of Vehicles security data, referring to the actual architecture of the Internet of Vehicles and the existing security ontology. Specifically, the classes, class hierarchies, data attributes of classes, and relationship attributes between classes contained in the ontology model can be determined. Among them, the classes of the security ontology model can include five types of security entities: assets, vulnerabilities, weaknesses, attacks, and attackers. The class hierarchy represents the hierarchical relationship between classes. For example: the asset class can include equipment and sensitive data at all levels of the Internet of Vehicles system, the vulnerability class includes vulnerabilities that attackers can exploit to launch attacks through technical defects, the weakness class includes defects or other errors in the software and hardware of the target assets, the attack class includes related concepts such as the mode or method of attack adopted by the attacker in the Internet of Vehicles environment, and the attacker class represents the identity-related information of the person or organization that launches the attack. The data type of the class in the security ontology model represents the attribute information of a single class. The relationship attributes between classes represent the relationships between different classes. The relationships in the security ontology model include "existence", "possession" or "use", for example: <asset, existence, vulnerability>, <asset, possession, weakness>, <weakness, composition, vulnerability>, <attack, exploit, weakness>, <attacker, use, vulnerability>, <attacker, launch, attack>. Thus, the security ontology model of IoV security data is obtained.

[0086] The entity relationship joint extraction method of the present invention obtains expert knowledge for building a security ontology model, and based on the expert knowledge and the Internet of Vehicles security data, determines the data categories, hierarchical structure of data categories, data attributes of data categories, and relationship attributes between categories included in the security ontology model for characterizing the Internet of Vehicles security data. As a result, the security ontology model for entity relationship extraction of Internet of Vehicles security data is reasonably and accurately divided into categories and hierarchies, thereby effectively improving the accuracy and precision of entity relationship extraction of Internet of Vehicles security data.

[0087] Step S203, merging the structured data into the unstructured data to obtain a fused knowledge embedding vector.

[0088] In an optional implementation, step S203 includes:

[0089] Step S2031, querying and obtaining multiple target entities from the Internet of Vehicles security data, querying the target entities contained in the structured data and the corresponding insertion positions of the target entities in the unstructured data.

[0090] Specifically, the unstructured data of the Internet of Vehicles safety data is used as an input text sentence and input into the entity-relationship joint extraction model. Multiple target entities related to expert knowledge are queried in the input text sentence, and multiple target entities related to the entities contained in the input text sentence are screened out from the structured data of the Internet of Vehicles safety data. The target entities contained in the structured data and the corresponding insertion positions of the target entities in the unstructured data are queried using the string fuzzy matching method.

[0091] Step S2032, obtaining a structured core triple based on the target entity and the insertion position.

[0092] Specifically, structured data based on the safety data of the Internet of Vehicles can obtain triples of structured data. After obtaining all structured data triplets related to the structured data, the data needs to be screened and filtered. Here, the semantic relevance of structured data triplets and triplets extracted from unstructured data can be sorted, and the knowledge triples can be screened according to the threshold of the semantic relevance score, and only the knowledge triples with semantic relevance higher than the specified threshold are retained. In this way, common triples are filtered out and important triples are retained as structured core triples. Here, semantic relevance sorting can be performed by TF-IDF (term frequency-inverse text frequency) processing.

[0093] Step S2033, inserting the structured core triples into the unstructured data to obtain a sentence tree.

[0094] The Internet of Vehicles safety data includes structured data and unstructured data. The unstructured data of the Internet of Vehicles safety data is in the form of input text sentences, also known as unstructured sentences. Here, the structured core triples are inserted into the unstructured data to obtain a sentence tree.

[0095] Step S2034, converting the sentence tree into text data with a linear structure.

[0096] Specifically, the sentence tree can be transformed into a linear structure. The structured core triples are inserted into the unstructured data to obtain a sentence tree, and then the tree structure is transformed into a linear structure and input into the entity relationship joint extraction model.

[0097] Step S2035, obtaining a knowledge embedding vector according to the sentence tree.

[0098] Specifically, the input of the sentence tree incorporates threat intelligence text data (I1, I2, I3, ..., I n ). Threat intelligence text data (I1, I2, I3, ..., I n ) can be represented as a token embedding (T1,T2,T3,...,T n), fragment embedding (E A ,E A ,E A ,...,E A ), soft position embedding (SP1, SP2, SP3, ..., SP n ), where token embedding is the encoding of each token, fragment embedding represents the sentence to which the token belongs, and soft position embedding represents the position of the token in the sentence. The relative encoding of the token at different depths in the sentence tree stores the structural information of the sentence tree.

[0099] In an optional implementation, a visible matrix adapted to the positional relationship of the sentence tree may be constructed as a knowledge embedding vector, thereby implementing the above step S2035.

[0100] Specifically, in order to reduce the impact of inserted knowledge on the original input vehicle network safety data, an improved visibility matrix construction method is adopted to construct a visibility matrix M of size N×N as the input of the self-attention mechanism according to the positional relationship of the sentence tree. The constructed improved visibility matrix enables the inserted knowledge branch to receive semantic information from the main branch, but does not allow the main branch to directly obtain semantics from the knowledge branch. The construction rule of the visibility matrix is: all tokens in the original sentence are visible to the inserted knowledge, the inserted knowledge is invisible relative to the original input, and the inserted knowledge is invisible to each other.

[0101] Right now

[0102] Where wi represents the word with hard position "i", indicates that wi and wj are on the same branch, W represents the set of input tokens, and w j ∈W means that the word with hard position “j” comes from the original input, K represents the fused knowledge set, and w i ∈K indicates that the word with hard position “i” comes from the fused knowledge.

[0103] The entity relationship joint extraction method of the present invention obtains multiple target entities from the Internet of Vehicles security data, queries the target entities contained in the structured data and the corresponding insertion positions of the target entities in the unstructured data, and obtains the structured core triples based on the target entities and the insertion positions, thereby inserting the structured core triples into the unstructured data to obtain a sentence tree, converting the sentence tree into linear structured text data, and then obtaining a knowledge embedding vector based on the sentence tree. As a result, the error propagation problem caused by the pipeline-style execution of entity recognition first and then relationship extraction in the related technology is effectively avoided, and the efficiency and accuracy of entity relationship extraction are significantly improved.

[0104] Step S204: Based on the security ontology model, the knowledge embedding vector is encoded and decoded to obtain a tag sequence of the Internet of Vehicles security data, where the tag sequence is used to represent the joint extraction result of entities and relationships of the Internet of Vehicles security data.

[0105] In an optional implementation, step S204 may include:

[0106] Step S2041, adding the knowledge embedding vector to the self-attention function to perform attention calculation.

[0107] Specifically, after obtaining the visible matrix and sentence tree embedding through the knowledge fusion module, the modified Transformer bidirectional encoder can be used to encode the string, and attention calculation can be performed on each word of the sentence, all words in the sentence, and the obtained visible matrix.

[0108] Step S2042, based on the result of attention calculation, the precoding result is obtained through feedforward neural network, residual calculation, and normalization processing.

[0109] Specifically, the visible matrix M can be added to the softmax function of self-attention.

[0110] h i+1 =S i+1 V i+1 ;

[0111] Among them, Q i+1 ,S i+1 ,V i+1 =h i W q ,h i W k ,h i W v .

[0112] Thus, the relationship between words and the internal structure of the sentence tree of the original input vehicle network safety data are obtained, and an encoded output is obtained by passing through a feedforward neural network, residual summation and layer normalization. The model parameters in the pre-trained model BERT can be used here.

[0113] Step S2043, extract features from the pre-coding result to obtain a coding output result.

[0114] Specifically, after obtaining the output after pre-training encoding in step S2042, further encoding is performed to extract features H = {h1, h2..., h n}, here we can use the BiLSTM layer for encoding, connecting the forward and backward LSTM hidden states as the context word representation as Among them, LSTM (Long Short-Term Memory Network) discards useless information through the forget gate and input gate, and then obtains the result through the multiplication of the output gate and the memory cell. Its formula is expressed as:

[0115] i t =σ(W xi x t +W hi h t-1 +W ci c t-1 +b i );

[0116] z t =tanh(W xc x t +W hc h t-1 +b c );

[0117] f t =σ(W xf x t +W hf h t-1 +W cf c t-1 +b f );

[0118] c t =f t c t-1 +i t z t ;

[0119] o t =tanh(W xo x t +W ho h t-1 +W co c t +b o );

[0120] h t =o t tanh(c t );

[0121] Among them, σ is the activation function, W is the weight matrix, b is the bias vector, and z t is the content to be added, c t is the updated state at time t, i t ,f t ,o t are the output results of the input gate, forget gate and output gate respectively, h t It is the output of the entire LSTM unit at time t.

[0122] It should be noted here that further weights need to be assigned to the relationships. The attention mechanism can be used to assign different weights to the words in the sentences of the Internet of Vehicles safety data under each relationship. Specifically:

[0123] h g =avg{h1,h2…,h n};

[0124] e tk =v T tanh(W r r k +W g h g +W h h t );

[0125]

[0126] Among them, h g Represented as the overall representation of the sentence, r k represents the kth relation embedded, W r , W g , W h are all trainable parameters, and the generated attention scores reflect the importance of the words in the context of the sentence and the importance of the relation expression in the current relation. k Below the relationship, the sentence representation is generated by the weighted sum of the words in the sentence, and the attention layer combines to generate l k .

[0127] Here, the sentence representation l output by the encoding layer can be k As the input of the decoding layer in step S2044.

[0128] Step S2044: perform entity extraction based on the encoding output result to obtain a tag sequence.

[0129] Specifically, a decoding layer can be used to generate k The label sequence of the sentence is obtained, and the relation triples are returned through the BIO tagging scheme, and another BiLSTM is used to generate the sentence representation And generate the sequence score Z = {z1,z2,...,z n}.

[0130] Further, using the features from the encoding layer and the attention layer, we get the representation Finally, use CRF (Conditional Random Fields) to decode the sequence. For any sequence X = (x1, x2, ... x n ), where P is assumed to be the output score matrix of BiLSTM, and the size of P is n×k. Where n is the number of words, k is the number of tags, and P i,j represents the score of the jth label of the ith word. For the prediction sequence Y=(y1,y2,…,y n ), its score function is:

[0131]

[0132] Among them, A represents the transfer score matrix, A ij Represents the fraction of label i being transferred to label j, and the size of A is k+2.

[0133] The probability of predicting sequence Y is Taking the logarithm of both ends, we get the likelihood function of the predicted sequence. in, represents the real label sequence, YX represents all possible label sequences, and the output sequence with the maximum score after decoding is

[0134] Thus, we get the labeled sequence Y * ,The label sequence can represent the extraction results of entities and relations of ,Internet of Vehicles security data. ,Furthermore, the entities and relations are stored in triples to complete the ,joint extraction of entities and relations.

[0135] The entity relationship joint extraction method of the present invention adds the knowledge embedding vector to the self-attention function, performs attention calculation, obtains the precoding result based on the result of the attention calculation through a feedforward neural network, residual calculation, and normalization processing, performs feature extraction on the precoding result, obtains the coding output result, and performs entity extraction based on the coding output result to obtain a tag sequence. Thus, the joint extraction of entities and relationships is achieved, which effectively solves the error propagation problem caused by the pipeline-style method of first performing entity recognition and then performing relationship extraction in the related art, and effectively improves the efficiency of the entity relationship extraction model.

[0136] In this embodiment, a specific application example of an entity relationship joint extraction method is provided, which can be used in a server for data processing, etc. Figure 3 is a flowchart of a specific application example of the entity relationship joint extraction method according to an embodiment of the present invention, Figure 4 2 is a schematic diagram of the structure of an entity relationship joint extraction model according to a specific application example of the entity relationship joint extraction method according to an embodiment of the present invention. Figure 3 and Figure 4 A specific application example of the entity relationship joint extraction method according to an embodiment of the present invention is described.

[0137] like Figure 3 As shown, the process includes the following steps:

[0138] Step S301, obtaining Internet of Vehicles security data for building a security database, where the Internet of Vehicles security data includes structured data and unstructured data.

[0139] Step S302: construct a security ontology model based on the Internet of Vehicles security data.

[0140] refer to Figure 4 As shown, based on the structured data and unstructured data included in the Internet of Vehicles security data, an Internet of Vehicles security ontology model can be constructed. The security ontology model can include entity categories and relationship categories.

[0141] It should be noted that Figure 4 The Internet of Vehicles security ontology model and ontology model are consistent with the security ontology model in the embodiment of the present invention.

[0142] Step S303: fuse the structured data into the unstructured data to obtain a fused knowledge embedding vector.

[0143] refer to Figure 4 As shown in the figure, the structured data in the Internet of Vehicles safety data is filtered and used as the basis for sentence tree construction. After clarifying the target entity and the insertion position, the structured core triples are inserted into the unstructured data to construct a sentence tree. The sentence tree is converted into linear structured text data to obtain the knowledge embedding vector.

[0144] Step S304, based on the security ontology model, the knowledge embedding vector is encoded and decoded to obtain a tag sequence of the Internet of Vehicles security data, and thereby obtain the parameters of the entity relationship joint extraction model.

[0145] Specifically, the tag sequence can be compared with the entity categories and relationship categories of the security ontology model constructed in step S302. When the difference between the entity categories and relationship categories of the tag sequence and the security ontology model constructed in step S302 is less than a set threshold, the parameters of the entity-relationship joint extraction model are determined.

[0146] Step S305 , based on the parameters of the entity-relationship joint extraction model determined in step S304 , joint extraction of entities and relationships is performed on the Internet of Vehicles safety data to be extracted.

[0147] Specifically, the above steps S301 and S303 to S304 may be executed to obtain a tag sequence of the Internet of Vehicles security data to be extracted, and the tag sequence is used to represent the joint extraction result of the entities and relations of the Internet of Vehicles security data.

[0148] Step S306: based on the extraction result of the joint extraction of entities and relationships of the Internet of Vehicles safety data to be extracted, the entity-relationship joint extraction model is updated.

[0149] For more details on the specific application examples of the entity relationship joint extraction method according to the embodiment of the present invention, please refer to Figure 1 and Figure 2 The implementation details of the illustrated embodiment will not be described in detail here.

[0150] In the present embodiment, a device for joint extraction of entity relations is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0151] This embodiment provides an entity relationship joint extraction device, such as Figure 5 As shown, including:

[0152] A data collection module 501 is used to obtain Internet of Vehicles security data for building a security database, where the Internet of Vehicles security data includes structured data and unstructured data;

[0153] An ontology model building module 502 is used to build a security ontology model based on Internet of Vehicles security data;

[0154] The knowledge fusion module 503 is used to fuse the structured data into the unstructured data to obtain the fused knowledge embedding vector;

[0155] The joint extraction module 504 is used to encode and decode the knowledge embedding vector based on the security ontology model to obtain a tag sequence of the Internet of Vehicles security data, and the tag sequence is used to represent the joint extraction result of the entities and relationships of the Internet of Vehicles security data.

[0156] In an optional implementation, the data collection module 501 includes:

[0157] A first collection unit, used for storing the structured data in the form of knowledge triples into a secure database;

[0158] A denoising unit is used to denoise the unstructured data to obtain text sentences;

[0159] The sequence unit is used to perform sequence annotation on text sentences to obtain annotated text;

[0160] The second collection unit is used to store the annotated text in a security database.

[0161] In an optional implementation, the ontology model building module 502 includes:

[0162] An acquisition unit, used to acquire expert knowledge for building a security ontology model, where the expert knowledge is used to characterize basic annotation rules for data categories of Internet of Vehicles security data;

[0163] The construction unit is used to determine the data categories, hierarchical structure of data categories, data attributes of data categories, and relationship attributes between categories included in the security ontology model for characterizing the security data of the Internet of Vehicles based on expert knowledge and the security data of the Internet of Vehicles.

[0164] In an optional implementation, the knowledge fusion module 503 includes:

[0165] A query unit, used to query multiple target entities from the Internet of Vehicles security data, query the target entities contained in the structured data and the corresponding insertion positions of the target entities in the unstructured data;

[0166] A triple unit, used to obtain structured core triples based on the target entity and the insertion position;

[0167] An insertion unit, used to insert structured core triples into unstructured data to obtain a sentence tree;

[0168] A conversion unit, used to convert the sentence tree into text data with a linear structure;

[0169] The embedding unit is used to obtain the knowledge embedding vector according to the sentence tree.

[0170] In an optional embodiment, the embedding unit includes:

[0171] The matrix subunit is used to construct a visible matrix that is adapted to the positional relationship of the sentence tree as a knowledge embedding vector.

[0172] In an optional implementation, the joint extraction module 504 includes:

[0173] Attention unit, used to add the knowledge embedding vector to the self-attention function and perform attention calculation;

[0174] A precoding unit is used to obtain a precoding result based on the result of attention calculation through a feedforward neural network, residual calculation, and normalization processing;

[0175] An extraction unit, used for performing feature extraction on the pre-coding result to obtain a coding output result;

[0176] The extraction unit is used to extract entities based on the encoding output result to obtain a tag sequence.

[0177] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0178] The entity relationship joint extraction device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0179] The embodiment of the present invention also provides a computer device having the above Figure 5 The entity relationship joint extraction device shown.

[0180] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0181] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0182] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0183] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0184] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0185] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0186] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0187] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0188] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for joint extraction of entity relations, characterized in that: The method comprises: Acquiring Internet of Vehicles security data for building a security database, wherein the Internet of Vehicles security data includes structured data and unstructured data; Based on the Internet of Vehicles security data, a security ontology model is constructed; Fusion the structured data with the unstructured data to obtain a fused knowledge embedding vector; Based on the security ontology model, the knowledge embedding vector is encoded and decoded to obtain a tag sequence of the Internet of Vehicles security data, and the tag sequence is used to represent the joint extraction result of the entities and relationships of the Internet of Vehicles security data.

2. The method according to claim 1, characterized in that The obtaining of Internet of Vehicles security data for building a security database includes: Storing the structured data in the secure database in the form of knowledge triples; Performing denoising processing on the unstructured data to obtain text sentences; Performing sequence labeling on the text sentences to obtain labeled text; The annotated text is stored in the secure database.

3. The method according to claim 1, characterized in that The constructing of a security ontology model based on the Internet of Vehicles security data includes: Acquire expert knowledge for constructing the security ontology model, wherein the expert knowledge is used to characterize basic annotation rules for data categories of the Internet of Vehicles security data; Based on the expert knowledge and the Internet of Vehicles security data, determine the data categories included in the security ontology model for characterizing the Internet of Vehicles security data, the hierarchical structure of the data categories, the data attributes of the data categories, and the relationship attributes between categories.

4. The method according to claim 1, characterized in that: The step of fusing the structured data into the unstructured data comprises: Querying and obtaining a plurality of target entities from the Internet of Vehicles security data, and querying the target entities contained in the structured data and the corresponding insertion positions of the target entities in the unstructured data; Based on the target entity and the insertion position, a structured core triple is obtained; Inserting the structured core triples into the unstructured data to obtain a sentence tree; Converting the sentence tree into text data with a linear structure; According to the sentence tree, the knowledge embedding vector is obtained.

5. The method according to claim 4, characterized in that The step of obtaining the knowledge embedding vector according to the sentence tree includes: A visible matrix adapted to the positional relationship of the sentence tree is constructed as the knowledge embedding vector.

6. The method according to claim 4, characterized in that The encoding and decoding of the knowledge embedding vector based on the security ontology model includes: Adding the knowledge embedding vector to the self-attention function to perform attention calculation; Based on the result of the attention calculation, a precoding result is obtained through a feedforward neural network, residual calculation, and normalization processing; Performing feature extraction on the pre-coding result to obtain a coding output result; Entity extraction is performed based on the encoding output result to obtain the tag sequence.

7. An entity relationship joint extraction device, characterized in that: The device comprises: A data collection module, used to obtain Internet of Vehicles security data for building a security database, wherein the Internet of Vehicles security data includes structured data and unstructured data; An ontology model building module, used to build a security ontology model based on the Internet of Vehicles security data; A knowledge fusion module, used to fuse the structured data into the unstructured data to obtain a fused knowledge embedding vector; A joint extraction module is used to encode and decode the knowledge embedding vector based on the security ontology model to obtain a tag sequence of the Internet of Vehicles security data, and the tag sequence is used to characterize the joint extraction result of the entities and relationships of the Internet of Vehicles security data.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the entity relationship joint extraction method described in any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the entity relationship joint extraction method described in any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises computer instructions, and the computer instructions are used to enable a computer to execute the entity relationship joint extraction method described in any one of claims 1 to 6.

Images