A method for extracting key elements of network access resources
Through the combination of recurrent neural networks and knowledge graphs, key entity information in Internet data is extracted, unstructured data processing problems are solved, and automated processing capabilities and the accuracy of text semantic extraction are improved.
Patent Information
- Application Number
- CN202311546201.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-20
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-11-20
AI Technical Summary
The prior art is difficult to effectively process and extract unstructured data from Internet data, especially in relational rule mining and text semantic extraction. There are problems of inconsistent noise and format, resulting in insufficient automation processing capabilities.
Using the key element extraction method of network lead resource based on recurrent neural network, the key entity information in structured and unstructured text is identified through preprocessing, feature extraction, classification model training and entity information fusion, and the knowledge graph and attention mechanism are used. Improve the accuracy of text semantic feature extraction.
It realizes the automation processing and standardized production capacity of Internet data, reduces model complexity and resource consumption, improves the ability to capture important text information and entity correlation, solves the problems of ambiguity and one-sidedness, and improves text semantics. Comprehensiveness and accuracy of the characterization.
Smart Images

Figure CN117573869B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of element extraction, and in particular to a method for extracting key elements of network access resources. Background Art
[0002] The raw data collected is usually a large amount of unstructured data. On the one hand, unstructured data itself has many forms of expression and non-uniform formats, which makes it difficult to adapt to the requirements of integrated data mining tools for input data formats, and often relies on manual intervention for format conversion; on the other hand, the Internet data detected, crawled, and collected contains noise. How to mine association rules for a large amount of disordered Internet data, obtain useful information, and extract the semantics of web page text is one of the problems currently faced by the preprocessing of massive multi-source data. Summary of the invention
[0003] To solve the above problems, the present invention provides a method for extracting key elements of network access resources, which is based on the key element extraction technology of access resources based on recurrent neural networks, accesses and aggregates Internet data resources, uses recurrent neural networks to extract contextual semantic features of texts, integrates entity information, realizes feature extraction, and finally identifies key entity information from structured and unstructured text data.
[0004] To achieve the above object, the present invention discloses a method for extracting key elements of network access resources, the method comprising:
[0005] S1, preprocessing the network access resource data to obtain text representation and entity representation; the text representation represents the network access resource data represented by word embedding; the entity representation represents the word embedding representation corresponding to the entity object in the network access resource data;
[0006] S2. Using the initial classification model, the text representation and the entity representation are processed to obtain the initial category of the network access resource data; the initial category is a key element or a non-key element;
[0007] The initial classification model includes a first feature extraction module, a first attention calculation module, a feature fusion module, and a Highway network module;
[0008] S3. Based on the initial category, the initial classification model is trained using a loss function to obtain a classification model;
[0009] S4. Using the classification model, the data to be tested is processed to obtain the category of the data to be tested.
[0010] As an optional implementation, the network-linked resource data is preprocessed to obtain text representation and entity representation, including:
[0011] S11, performing word segmentation and stop word removal processing on the network access resource data to obtain second network access resource data;
[0012] S12, performing word embedding processing on the second network access resource data to obtain text representation;
[0013] S13. Based on the knowledge graph, the second network access resource data is processed to obtain entity representation.
[0014] As an optional implementation manner, the processing of the second network-linked resource data based on the knowledge graph to obtain entity representation includes:
[0015] S131, using a preset training model, extracting and encoding the information of the knowledge graph from the second network-linked resource data to obtain word embedding of the entity object;
[0016] S132. Fusion of two different expression vector spaces, word embedding and language model, of entity objects to obtain entity representation.
[0017] As an optional implementation, the initial classification model is used to process the text representation and the entity representation to obtain the initial category of the network access resource data, including:
[0018] S21, using the first feature extraction module to extract text context semantic features from the text representation to obtain text context semantic features;
[0019] S22, using the first attention calculation module to perform attention calculation on the text context semantic feature to obtain a first text feature;
[0020] S23, using the feature fusion module to perform fusion feature processing on the text representation and the preset entity representation to obtain a second text feature;
[0021] S24: Using a Highway network module, process the first text feature and the second text feature to obtain an initial category.
[0022] As an optional implementation, the first feature extraction module uses a bidirectional long short-term memory recurrent neural network;
[0023] The step of extracting text context semantic features from the text representation using the first feature extraction module to obtain text context semantic features includes:
[0024] S211, inputting the text representation into the forward gated recurrent unit of the first feature extraction module to obtain a first word embedding feature matrix;
[0025] The first word embedding feature matrix H1 is:
[0026]
[0027] In the formula, is the i-th word vector W in the text representation i The corresponding word embedding feature vector after being processed by the forward gated recurrent unit of the first feature extraction module, i∈{1,…,q}; q represents the number of word vectors contained in the text representation;
[0028] S212, inputting the text representation into the backward gated recurrent unit of the first feature extraction module to obtain a second word embedding feature matrix;
[0029] The second word embedding feature matrix H2 is:
[0030]
[0031] In the formula, is the i-th word vector W in the text representation i The corresponding word embedding feature vector after being processed by the forward gated recurrent unit of the first feature extraction module, i∈{1,…,q}; q represents the number of word vectors contained in the text representation;
[0032] S213, connecting the first word embedding feature matrix, the text representation and each vector in the second word embedding feature matrix to obtain text context semantic features;
[0033] The text context semantic feature X=[X1,X2,X3,…X q ];
[0034] In the formula, X i Depend on W i , After concatenation, we get i∈{1,…,q}.
[0035] As an optional implementation manner, the using the first attention calculation module to perform attention calculation on the text context semantic feature to obtain the first text feature includes:
[0036] S221, calculating the text context semantic feature using a tanh function to obtain a second text context semantic feature;
[0037] The second text context semantic feature M=tanh(X);
[0038] In the formula, X represents the semantic features of text context;
[0039] S222, calculating the second text context semantic features using a softmax function to obtain a semantic weight matrix;
[0040] The semantic weight matrix
[0041] In the formula, α is the semantic weight matrix, ω represents the trainable parameter vector, and ω T represents the transposed ω, and M represents the semantic feature of the second text context;
[0042] S223, calculating the text context semantic features based on the semantic weight matrix to obtain text context semantic attention features;
[0043] The text context semantic attention feature v = Xα T ;
[0044] S224, using the tanh function to process the text context semantic attention feature to obtain a first text feature; the first text feature v * =tanh(v);
[0045] As an optional implementation, the feature fusion module is used to perform fusion feature processing on the text representation and the preset entity representation to obtain a second text feature, including:
[0046] S231, using a first mean model, summing and averaging all word vectors in the text representation to obtain a text word vector mean;
[0047] The first mean model is:
[0048]
[0049] In the formula, v D represents the mean value of the text word vector; x wi represents the i-th word vector in the text representation; q represents the number of word vectors contained in the text representation;
[0050] S232, using the second mean model, summing and averaging all word vectors in the preset entity representation to obtain the entity word vector mean;
[0051] The second mean model is:
[0052]
[0053] In the formula, v e represents the mean value of the text word vector; x ei represents the i-th word vector in the preset entity representation; p represents the number of word vectors contained in the entity representation; βei Represents word vector x ei The weighting coefficient of
[0054] S233, using the tanh activation function to calculate the mean of the entity word vector to obtain a second text feature; the second text feature v e * =tanh(ν e );
[0055] As an optional implementation, the entity word vector x ei The weighting coefficient β ei The calculation model is:
[0056]
[0057]
[0058]
[0059] In the formula, C ei Represents x ei With ν D The cosine similarity of x ei Represents the i-th word vector in the preset entity representation, v D represents the mean of the text word vector, ei represents the entity word vector x ei In the entity representation, the serial number, P ei Represents the entity word vector x ei The probability of the entity represented by is extracted, represents the splicing operation, ω ei represents the weight matrix, b ei Represents the bias term.
[0060] As an optional implementation, the using of the Highway network module to process the first text feature and the second text feature to obtain an initial category includes:
[0061] S241, concatenating the first text feature and the second text feature to obtain a second text vector representation;
[0062] S242, performing column processing on the second text vector representation to obtain a text feature column vector set; the text feature column vector set includes n text feature column vectors;
[0063] S243, performing character embedding processing on the text representation to obtain a text character representation matrix; the text character representation matrix includes character embeddings of m characters;
[0064] S244, fusing the text feature column vector set and the text character representation matrix to obtain a third text feature;
[0065] S245, using the Highway network to process the third text feature to obtain a fourth text feature;
[0066] The fourth text feature z=τ⊙σ2(w h γ+b h )+(1-τ)⊙γ;
[0067] Where z is the fourth text feature, τ is the conversion gate, which is used to convert part of the input, τ = σ1(W τ γ+b t ), σ1 and σ2 are nonlinear activation functions, γ represents the third text feature, W τ and W h represents the weight matrix, b t and b h represents the bias term;
[0068] S246, using Softmax to classify the fourth text feature to obtain an initial category;
[0069]
[0070]
[0071] In the formula, represents the probability that the text D is of category y, where D is the network-linked resource data, The initial category of network access resource data, W (D) is the weight matrix, b (D) is the bias term.
[0072] As an optional implementation manner, the fusing process of the text feature column vector set and the text character representation matrix to obtain the third text feature includes:
[0073] S2441, processing any text feature vector in the text feature column vector set to obtain a word embedding representation corresponding to the any text feature vector; the word embedding representation corresponding to the any text feature vector is:
[0074]
[0075] In the formula, ξ i represents any text feature vector, h, k are the starting and ending positions of the i-th word after the text representation is embedded in the character embedding, s j represents the j-th character embedding, the γ i Represents ξi The corresponding word embedding representation;
[0076] S2442: Perform granular fusion processing on all word embedding representations obtained in step S2441 to obtain a third text feature, where the third text feature is γ = [γ1, γ2, ..., γ n ].
[0077] As an optional implementation, based on the initial category, the initial classification model is trained using a loss function to obtain a classification model, including:
[0078] S31, construct a loss function; the loss function is the initial category label The negative log-likelihood of is expressed as
[0079]
[0080] In the formula, J(θ) represents the category y i The negative log-likelihood of y i Indicates category; t i is the one-hot representation of the true label, m is the number of categories, and λ is the L2 regularization parameter;
[0081] S32, training the initial classification model by minimizing the negative log-likelihood loss function;
[0082] S33. When the negative log-likelihood loss function no longer decreases, the training is stopped to obtain a classification model.
[0083] As an optional implementation, the method further includes: evaluating the classification model to obtain an evaluation result of the classification model, including:
[0084] S51, randomly dividing the network access resource data into k packets;
[0085] S52, taking one of the k packages as a test set and the remaining k-1 packages as a training set for training, and obtaining the precision and recall of the classification model;
[0086] S53: Based on the precision and recall, the F1 metric is used as an evaluation indicator to obtain an evaluation result of the classification model.
[0087] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0088] 1) Use recurrent neural networks to extract contextual semantic features of text, integrate entity information, extract important context features, and ultimately identify entity key element information from structured and unstructured text data, thereby improving the automated processing and standardized production capabilities of linked resources;
[0089] 2) In terms of the model, a variant LSTM+Attention+RNN learning model is adopted, which reduces the overall model complexity and resource consumption, improves the model operation efficiency, and avoids the problems of long sequence dependency, inability to process in parallel, and limited processing capacity during the model processing;
[0090] 3) Combining natural language processing and integrating entity information from knowledge graphs can quickly mine important syntactic and semantic features at multiple granular levels of text resources, and improve the actual relevance of captured and extracted important text information to entities; in addition, combining text expressions at different levels such as character embedding and word embedding can improve the ambiguity and one-sidedness of text semantic representation, and improve the comprehensiveness, accuracy and uniqueness of its contextual representation. BRIEF DESCRIPTION OF THE DRAWINGS
[0091] Figure 1 A flow chart of a method for extracting key elements of network access resources disclosed in an embodiment of the present invention;
[0092] Figure 2 A schematic diagram of an initial classification model structure disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0093] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0094] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.
[0095] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0096] The present invention discloses a method for extracting key elements of network access resources, which is beneficial to improving the accuracy and efficiency of classification analysis of network security events, thereby improving the adaptability of network security event analysis capabilities. Detailed descriptions are given below.
[0097] Embodiment 1
[0098] See also Figure 1 , Figure 1 This is a flow chart of a method for extracting key elements of network access resources disclosed in an embodiment of the present invention. Figure 1 As shown, the method for extracting key elements of network access resources may include the following operations:
[0099] S1. Preprocess the network-linked resource data to obtain text representation and entity representation;
[0100] In the embodiment of the present invention, the text representation represents the network access resource data represented by word embedding; the entity representation represents the word embedding representation corresponding to the entity object in the network access resource data;
[0101] S2. Using the initial classification model, the text representation and the entity representation are processed to obtain the initial category of the network access resource data; the initial category is a key element or a non-key element;
[0102] In the embodiment of the present invention, Figure 2 As shown, the initial classification model includes a first feature extraction module, a first attention calculation module, a feature fusion module, and a Highway network module;
[0103] S3. Based on the initial category, the initial classification model is trained using a loss function to obtain a classification model;
[0104] S4. Using the classification model, the data to be tested is processed to obtain the category of the data to be tested.
[0105] In another optional embodiment, the network-based resource data is preprocessed to obtain text representation and entity representation, including:
[0106] S11, performing word segmentation and stop word removal processing on the network access resource data to obtain second network access resource data;
[0107] S12, performing word embedding processing on the second network access resource data to obtain text representation;
[0108] S13. Based on the knowledge graph, the second network access resource data is processed to obtain entity representation.
[0109] In this optional embodiment, the processing of the second network-linked resource data based on the knowledge graph to obtain entity representation includes:
[0110] S131, using a preset training model, extracting and encoding the information of the knowledge graph from the second network-linked resource data to obtain word embedding of the entity object;
[0111] Specific:
[0112] a1) Extract entities and entity relations in text.
[0113] For unstructured data: First, combined with neural networks, the idea of sliding window classification is adopted to map the text content into a sequence of byte fragments of length N. By learning the representation of each sequence, it is predicted whether the byte sequence is the target entity; then, a segmented convolutional neural network is used to realize entity relationship extraction.
[0114] For structured data: Mainly based on the knowledge graph meta-knowledge representation model, the direct mapping method is used to realize entity-machine relationship extraction. Specifically, the database table is mapped to a class, the database table column is mapped to an attribute; each row in the database table is mapped to a resource or entity; the value of each cell in the database table is mapped to a text value, and the original entity-relationship data is represented based on the target vocabulary.
[0115] For semi-structured data: mainly through the generation, update and maintenance of wrappers, semi-structured data is converted into structured form, and then entities and relationships are extracted based on the structured data.
[0116] a2) Link these entities to the corresponding entity objects in the knowledge graph database to form an entity link relationship network.
[0117] a3) Find the word embeddings corresponding to these entity objects.
[0118] S132, fusing two different expression vector spaces of the word embedding and language model of the entity object to obtain the entity representation;
[0119] Specifically: Based on the BERT model, an improvement was made and a pre-training goal related to the knowledge graph was added: filtering out the alignment relationship between the index and the entity, and requiring the model to select appropriate entity text from the entities in the graph for alignment; then combining the entity link network as input, jointly training BERT, and merging entity representations in an end-to-end manner.
[0120] In yet another optional embodiment, the initial classification model is used to process the text representation and the entity representation to obtain the initial category of the network access resource data, including:
[0121] S21, using the first feature extraction module to extract text context semantic features from the text representation to obtain text context semantic features;
[0122] S22, using the first attention calculation module to perform attention calculation on the text context semantic feature to obtain a first text feature;
[0123] S23, using the feature fusion module to perform fusion feature processing on the text representation and the preset entity representation to obtain a second text feature;
[0124] S24: Using a Highway network module, process the first text feature and the second text feature to obtain an initial category.
[0125] In this optional embodiment, the first feature extraction module uses a bidirectional long short-term memory recurrent neural network;
[0126] The step of extracting text context semantic features from the text representation using the first feature extraction module to obtain text context semantic features includes:
[0127] S211, inputting the text representation into the forward gated recurrent unit of the first feature extraction module to obtain a first word embedding feature matrix;
[0128] The first word embedding feature matrix H1 is:
[0129]
[0130] In the formula, is the i-th word vector W in the text representation i The corresponding word embedding feature vector after being processed by the forward gated recurrent unit of the first feature extraction module, i∈{1,…,q}; the above q represents the number of word vectors contained in the text representation;
[0131] S212, inputting the text representation into the backward gated recurrent unit of the first feature extraction module to obtain a second word embedding feature matrix;
[0132] The second word embedding feature matrix H2 is:
[0133]
[0134] In the formula, is the i-th word vector W in the text representation i The corresponding word embedding feature vector after being processed by the forward gated recurrent unit of the first feature extraction module, i∈{1,…,q}; q represents the number of word vectors contained in the text representation;
[0135] S213, connecting the first word embedding feature matrix, the text representation and each vector in the second word embedding feature matrix to obtain text context semantic features;
[0136] The text context semantic feature X=[X1,X2,X3,…X q ];
[0137] In the formula, X i Depend on W i , After concatenation, we get i∈{1,…,q}.
[0138] Optionally, in order to avoid the problem that the obtained text context semantic features are too large, resulting in large amount of calculation and long training time, it is necessary to reduce the dimension of the feature matrix.
[0139] In this implementation, the unsupervised principal component analysis (PCA) method is used to map high-dimensional data to a low-dimensional space through a certain linear projection, and it is expected that the variance of the data in the projected dimension is the largest, so that the high-dimensional feature vectors are merged into low-dimensional feature attributes, while retaining more characteristics of the original data points and ensuring that the feature attributes are independent of each other. Specifically, it includes:
[0140] b1) Data centering: Zero-mean each row (i.e., a feature attribute) in the sample set X, that is, subtract the mean of this row;
[0141] b2) Find the covariance matrix of the matrix X after data centering; b3) Find the eigenvalues and eigenvectors of the covariance matrix
[0142] b4) Arranging the eigenvectors in columns from large to small according to their eigenvalues is called a matrix, and obtaining the first k columns of data to form the matrix W;
[0143] b5) Perform matrix multiplication on the matrix W and the sample set X to obtain the final data matrix reduced to k dimensions.
[0144] In this optional embodiment, the using the first attention calculation module to perform attention calculation on the text context semantic feature to obtain the first text feature includes:
[0145] S221, calculating the text context semantic feature using a tanh function to obtain a second text context semantic feature;
[0146] The second text context semantic feature M=tanh(X);
[0147] In the formula, X represents the semantic features of text context;
[0148] S222, calculating the second text context semantic features using a softmax function to obtain a semantic weight matrix;
[0149] The semantic weight matrix
[0150] In the formula, α is the semantic weight matrix, ω represents the trainable parameter vector, and ω T represents the transposed ω, and M represents the semantic feature of the second text context;
[0151] S223, calculating the text context semantic features based on the semantic weight matrix to obtain text context semantic attention features;
[0152] The text context semantic attention feature v = Xα T ;
[0153] S224, using the tanh function to process the text context semantic attention feature to obtain a first text feature; the first text feature v * =tanh(v);
[0154] In this optional embodiment, the feature fusion module is used to perform fusion feature processing on the text representation and the preset entity representation to obtain the second text feature, including:
[0155] S231, using a first mean model, summing and averaging all word vectors in the text representation to obtain a text word vector mean;
[0156] The first mean model is:
[0157]
[0158] In the formula, v D represents the mean value of the text word vector; x wi represents the i-th word vector in the text representation; q represents the number of word vectors contained in the text representation;
[0159] S232, using the second mean model, summing and averaging all word vectors in the preset entity representation to obtain the entity word vector mean;
[0160] The second mean model is:
[0161]
[0162] In the formula, v e represents the mean value of the text word vector; x ei represents the i-th word vector in the preset entity representation; p represents the number of word vectors contained in the entity representation; β ei Represents word vector x ei The weighting coefficient of
[0163] S233, using the tanh activation function to calculate the mean of the entity word vector to obtain a second text feature; the second text feature v e * =tanh(ν e );
[0164] Optionally, the entity word vector x ei The weighting coefficient β ei The calculation model is:
[0165]
[0166]
[0167]
[0168] In the formula, C ei Represents x ei With ν D The cosine similarity of x ei Represents the i-th word vector in the preset entity representation, v D represents the mean of the text word vector, ei represents the entity word vector x ei In the entity representation, the serial number, P ei Represents the entity word vector x ei The probability of the entity represented by is extracted, represents the splicing operation, ω ei represents the weight matrix, b ei Represents the bias term; using C ei and P ei Two features are used to obtain the entities that are most relevant to the text and have clear semantics.
[0169] In this optional embodiment, the use of the Highway network module to process the first text feature and the second text feature to obtain an initial category includes:
[0170] S241, concatenating the first text feature and the second text feature to obtain a second text vector representation;
[0171] S242, performing column processing on the second text vector representation to obtain a text feature column vector set; the text feature column vector set includes n text feature column vectors;
[0172] It should be noted that the second text vector represents the principle of column segmentation processing: that is, the principle of text segmentation, which is mainly based on statistical methods, and the model is trained through a large amount of annotated data corpus to automatically learn the word segmentation rules of the text; then, on the basis of part-of-speech tagging of the target text through natural language processing, the target text segmentation is achieved, and the number n of characteristic words in the target text is obtained.
[0173] S243, performing character embedding processing on the text representation to obtain a text character representation matrix; the text character representation matrix includes character embeddings of m characters;
[0174] It should be noted that the principle of character embedding processing for the text representation is the character-based text classification principle, which is similar to the text segmentation model, except that the segmentation level is applied to the character level;
[0175] At the same time, in the process of vector representation of characters, the character division principle is mainly based on the English dictionary and Chinese frequency: for English text, 26 letters + 10 numbers + 34 other characters = 70 characters are mainly used to construct the English character dictionary. In terms of encoding method, the one-hot code method can be used, and then the original text data set is encoded one by one to generate training data that can be input into CNN; for Chinese text, since the number of characters contained is obviously more than English, a maximum threshold can be set based on the above processing, and the most commonly used characters can be filtered according to the frequency of occurrence of the characters to form a dictionary.
[0176] In terms of model features, the features of a certain original word can be expanded into multiple character features at the character level based on the character splitting results;
[0177] Then, based on deep learning training, the number m of characteristic characters in the target text is obtained.
[0178] S244, fusing the text feature column vector set and the text character representation matrix to obtain a third text feature;
[0179] S245, using the Highway network to process the third text feature to obtain a fourth text feature;
[0180] The fourth text feature z=τ⊙σ2(w h γ+b h )+(1-τ)⊙γ;
[0181] Where z is the fourth text feature, τ is the conversion gate, which is used to convert part of the input, τ = σ1(W τ γ+b t ), σ1 and σ2 are nonlinear activation functions, γ represents the third text feature, W τ and W h represents the weight matrix, b t and b h represents the bias term;
[0182] S246, using Softmax to classify the fourth text feature to obtain an initial category;
[0183]
[0184]
[0185] In the formula, represents the probability that the text D is of category y; the D is the network access resource data, The initial category of network access resource data, W (D) is the weight matrix, b (D) is the bias term.
[0186] In another embodiment of the present invention, the step of fusing the text feature column vector set and the text character representation matrix to obtain the third text feature comprises:
[0187] S2441, processing any text feature vector in the text feature column vector set to obtain a word embedding representation corresponding to the any text feature vector; the word embedding representation corresponding to the any text feature vector is:
[0188]
[0189] In the formula, ξ i represents any text feature vector, h, k are the starting and ending positions of the i-th word after the text representation is embedded in characters, and s j represents the jth character embedding, the γ i Represents ξ i The corresponding word embedding representation;
[0190] S2442: Perform granular fusion processing on all word embedding representations obtained in step S2441 to obtain a third text feature, where the third text feature is γ = [γ1, γ2, ..., γ n ].
[0191] In yet another optional embodiment, the initial classification model is trained using a loss function based on the initial category to obtain a classification model, including:
[0192] S31, construct a loss function; the loss function is the initial category label The negative log-likelihood of is expressed as
[0193]
[0194] In the formula, J(θ) represents the category y i The negative log-likelihood of y i Indicates category; t i is the one-hot representation of the true label, m is the number of categories, and λ is the L2 regularization parameter;
[0195] S32, training the initial classification model by minimizing the negative log-likelihood loss function;
[0196] S33. When the negative log-likelihood loss function no longer decreases, the training is stopped to obtain a classification model.
[0197] In yet another optional embodiment, the method further includes: evaluating the classification model to obtain an evaluation result of the classification model, including:
[0198] S51, randomly dividing the network access resource data into k packets; k is a positive integer not less than 10;
[0199] S52, taking one of the k packages as a test set and the remaining k-1 packages as a training set for training, and obtaining the precision and recall of the classification model;
[0200] S53, based on the precision and recall, using the F1 metric as an evaluation indicator to obtain an evaluation result of the classification model;
[0201] Specific:
[0202] c1) Construct confusion matrix
[0203]
[0204]
[0205] c2) Calculate the precision and recall
[0206] The precision rate reflects the proportion of true positive samples in the positive examples determined by the classifier; the precision rate calculation formula is:
[0207]
[0208] The recall rate reflects the proportion of correctly judged positive examples to the total positive examples; the recall rate calculation formula is:
[0209]
[0210] c3) Calculate the F1 metric
[0211] The F1 metric is the harmonic mean of precision and recall, and is calculated as follows:
[0212]
[0213] The higher the F1 value, the more effective the classification model is.
[0214] Finally, it should be noted that the method for extracting key elements of network access resources disclosed in the embodiment of the present invention only discloses a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for extracting key elements of network access resources, characterized in that: The method comprises: S1. Preprocessing the network access resource data to obtain text representation and entity representation; the network access resource data represents access to aggregated Internet data resources; the text representation represents the network access resource data represented by word embedding; the entity representation represents the word embedding representation corresponding to the entity object in the network access resource data; S2, using the initial classification model, processing the text representation and entity representation to obtain the initial category of the network access resource data; the initial category is a key element, or a non-key element; the initial classification model includes a first feature extraction module, a first attention calculation module, a feature fusion module and a Highway network module; S3. Based on the initial category, the initial classification model is trained using a loss function to obtain a classification model; S4. Using the classification model, the test data is processed to obtain the category of the test data; In step S2, the text representation and entity representation are processed using the initial classification model to obtain the initial category of the network access resource data, including: S21, using the first feature extraction module to extract text context semantic features from the text representation to obtain text context semantic features; S22, using the first attention calculation module to perform attention calculation on the text context semantic feature to obtain a first text feature; S23, using the feature fusion module to perform fusion feature processing on the text representation and the preset entity representation to obtain a second text feature; S24, using the Highway network module, processing the first text feature and the second text feature to obtain an initial category, specifically: S241, concatenating the first text feature and the second text feature to obtain a second text vector representation; S242, performing column processing on the second text vector representation to obtain a text feature column vector set; the text feature column vector set includes n text feature column vectors; S243, performing character embedding processing on the text representation to obtain a text character representation matrix; the text character representation matrix includes character embeddings of m characters; S244, fusing the text feature column vector set and the text character representation matrix to obtain a third text feature, specifically: S2441, processing any text feature vector in the text feature column vector set to obtain a word embedding representation corresponding to the any text feature vector; the word embedding representation corresponding to the any text feature vector is: In the formula, ξ i represents any text feature vector, h, k are the starting and ending positions of the i-th word after the text representation is embedded in the character, s j represents the j-th character embedding, γ i Represents ξ i The corresponding word embedding representation; S2442: Perform granular fusion processing on all word embedding representations obtained in step S2441 to obtain a third text feature, where the third text feature is γ = [γ1, γ2, ..., γ n ]; S245, using the Highway network to process the third text feature to obtain a fourth text feature; The fourth text feature z=τ⊙σ2(W h γ+b h )+(1-τ)⊙γ; Where z is the fourth text feature, τ is the conversion gate, which is used to convert part of the input, τ = σ1(W τ γ+b t ), σ1 and σ2 are nonlinear activation functions, γ represents the third text feature, W τ and W h represents the weight matrix, b t and b h represents the bias term; S246, using Softmax to classify the fourth text feature to obtain an initial category; In the formula, represents the probability that the text D is of category y, where D is the network access resource data, The initial category of network access resource data, W (D) is the weight matrix, b (D) is the bias term.
2. The method for extracting key elements of network access resources according to claim 1, characterized in that: The first feature extraction module uses a bidirectional long short-term memory recurrent neural network; The step of extracting text context semantic features from the text representation using the first feature extraction module to obtain text context semantic features includes: S211, inputting the text representation into the forward gated recurrent unit of the first feature extraction module to obtain a first word embedding feature matrix; The first word embedding feature matrix H1 is: In the formula, is the i-th word vector W in the text representation i The corresponding word embedding feature vector after being processed by the forward gated recurrent unit of the first feature extraction module, i∈{1,…,q}; q represents the number of word vectors contained in the text representation; S212, inputting the text representation into the backward gated recurrent unit of the first feature extraction module to obtain a second word embedding feature matrix; The second word embedding feature matrix H2 is: In the formula, is the i-th word vector W in the text representation i The corresponding word embedding feature vector after being processed by the forward gated recurrent unit of the first feature extraction module, i∈{1,…,q}; q represents the number of word vectors contained in the text representation; S213, connecting the second word embedding feature matrix, the text representation, and each vector in the second word embedding feature matrix to obtain text context semantic features; The text context semantic feature X=[X1,X2,X3,…X q ]; Where, X i Depend on W i , After concatenation, we get i∈{1,…,q}.
3. The method for extracting key elements of network access resources according to claim 1, characterized in that: The using the first attention calculation module to perform attention calculation on the text context semantic feature to obtain a first text feature includes: S221, calculating the text context semantic feature using a tanh function to obtain a second text context semantic feature; The second text context semantic feature M=tanh(X); In the formula, X represents the semantic features of text context; S222, calculating the second text context semantic features using a softmax function to obtain a semantic weight matrix; The semantic weight matrix In the formula, α is the semantic weight matrix, ω represents the trainable parameter vector, and ω T represents the transposed ω, and M represents the semantic feature of the second text context; S223, calculating the text context semantic features based on the semantic weight matrix to obtain text context semantic attention features; The text context semantic attention feature v = Xα T ; S224, using the tanh function to process the text context semantic attention feature to obtain a first text feature; the first text feature v * =tanh(v).
4. The method for extracting key elements of network access resources according to claim 1, characterized in that: The step of using the feature fusion module to perform fusion feature processing on the text representation and the preset entity representation to obtain a second text feature includes: S231, using a first mean model, summing and averaging all word vectors in the text representation to obtain a text word vector mean; The first mean model is: In the formula, v D represents the mean value of the text word vector; x wi represents the i-th word vector in the text representation; q represents the number of word vectors contained in the text representation; S232, using the second mean model, summing and averaging all word vectors in the preset entity representation to obtain the entity word vector mean; The second mean model is: In the formula, v e represents the mean value of the text word vector; x ei represents the i-th word vector in the preset entity representation; p represents the number of word vectors contained in the entity representation; β ei Represents word vector x ei The weighting coefficient of S233, using the tanh activation function to calculate the mean of the entity word vector to obtain a second text feature; the second text feature v e * =tanh(ν e ).
5. The method for extracting key elements of network access resources according to claim 1, characterized in that: The initial classification model is trained based on the initial category using a loss function to obtain a classification model, including: S31, construct a loss function; the loss function is the initial category label The negative log-likelihood of is expressed as: In the formula, J(θ) represents the category y i The negative log-likelihood of y i Indicates category; t i is the one-hot representation of the true label, m is the number of categories, and λ is the L2 regularization parameter; S32, training the initial classification model by minimizing the negative log-likelihood loss function; S33. When the negative log-likelihood loss function no longer decreases, the training is stopped to obtain a classification model.
6. The method for extracting key elements of network access resources according to claim 1, characterized in that: The method further comprises the following steps of evaluating the classification model to obtain an evaluation result of the classification model; specifically: S51, randomly dividing the network access resource data into k packets; k is a positive integer not less than 10; S52, taking one of the k packages as a test set and the remaining k-1 packages as a training set for training, and obtaining the precision and recall of the classification model; S53: Based on the precision and recall, the F1 metric is used as an evaluation indicator to obtain an evaluation result of the classification model.
