A text extraction method based on a standard knowledge graph

Abstract

The application discloses a text extraction method based on a standard knowledge graph, comprising the following steps: obtaining a text to be extracted, preprocessing the text to be extracted, extracting keywords of the preprocessed text to be extracted, inputting the keywords into a recognition model to recognize named entities, calculating the relevance of the named entities and existing knowledge graph entities, matching the named entities and the existing knowledge graph entities according to the relevance, obtaining linked entities according to the matching, inputting the linked entities into a first algorithm to extract initial attributes, inputting the initial attributes into a second algorithm to extract attributes, extracting the relationship between entities according to the attributes, clustering the entities according to the relationship, and extracting the text according to the clustering. The method can not only improve the text extraction accuracy, but also has good interpretability and can be directly applied to a standard knowledge graph text extraction system.

Description

Technical Field

[0001] This invention relates to the field of text extraction, and more particularly to a text extraction method based on a standard knowledge graph. Background Technology

[0002] Text extraction technology is increasingly widely used in the field of standard knowledge graphs, helping administrators to extract text efficiently and timely, and enabling adjustments to text extraction methods. Currently, standard knowledge graphs are characterized by massive amounts of text information, diverse data types, and high information density. This leads to numerous uncertainties in text extraction methods for standard knowledge graph management, resulting in significant uncertainty. Although some text extraction methods for standard knowledge graph management have been developed, they still cannot effectively solve the uncertainty problem. Summary of the Invention

[0003] The purpose of this invention is to provide a text extraction method based on standard knowledge graphs.

[0004] To achieve the above objectives, the present invention is implemented according to the following technical solution:

[0005] This invention includes the following steps:

[0006] A acquires the text to be extracted, preprocesses the text to be extracted, extracts the keywords from the preprocessed text to be extracted, and inputs the keywords into the recognition model to recognize named entities. The preprocessing includes word segmentation, part-of-speech tagging, and vectorization.

[0007] B calculates the correlation between the named entity and existing knowledge graph entities, matches the named entity with existing knowledge graph entities based on the correlation, and obtains linked entities based on the matching.

[0008] C inputs the linked entity into the first algorithm to extract initial attributes, and inputs the initial attributes into the second algorithm to extract attributes;

[0009] D extracts the relationships between entities based on the attributes, clusters the entities based on the relationships, and extracts text based on the clusters.

[0010] Furthermore, the method for extracting keywords from the preprocessed text to be extracted includes:

[0011] Candidate keywords are selected based on the following rules: phrase length not exceeding three words, not being proper nouns, and not starting or ending with a stop word. The formula for extracting the term frequency-inverse document frequency feature of the keywords is as follows:

[0012]

[0013] The frequency of phrase p in document d is f(p,d), the number of files containing phrase p in the global corpus is N, the size of the global corpus is C(d), the number of words in document d is C(d), and the number of documents containing phrase p in the global corpus is df(p).

[0014] The formula for calculating the candidate keyword score using the Naive Bayes model is as follows:

[0015]

[0016] The number of candidate keywords that are keywords is Y, the number of candidate keywords that are not keywords is N, and the feature value is t. Candidate keywords with a score greater than 0.8 are used as keywords.

[0017] Furthermore, the method for inputting the keywords into the recognition model to recognize named entities includes:

[0018] The preprocessed text to be extracted is input into the recognition model. The input text to be extracted is pre-trained. The character-level feature information of the text to be extracted is learned by dynamic fine-tuning. The calculation formula of the character sequence vector is obtained.

[0019]

[0020]

[0021] The calculation result of the single-head attention mechanism is R(a,b,c), with dimension d. b The query is 'a', the key is 'b', the value is 'c', the softmax function is 'S', the time step is 'T', and the character sequence vector of feature i is 'f'. i The weight matrix for the linear mapping of query, key, and value is: Where b and c correspond one-to-one;

[0022] In the encoding layer, contextual features are extracted from the output vector of the embedding layer; different weights are assigned to the features based on multi-head self-attention; in the decoding layer, the features output by the encoding layer are decoded using CRF to obtain the predicted named entities.

[0023] Furthermore, the method for matching the named entity with existing knowledge graph entities includes:

[0024] The formula for calculating the relevance between existing knowledge graph entities and named entities is as follows: (This formula is used to parse existing knowledge graph entities into morphemes and generate morphemes.)

[0025]

[0026] The regulating factors are k1, k2, and b, and the morpheme is C. iNamed entity U i The semantic relevance between morphemes and named entities is f. i The length of knowledge graph j, and the average length of named entities (avgU) 1 Morpheme parsing decomposes the entities in an existing knowledge graph into their smallest semantic units.

[0027] The formula for calculating the similarity score by weighted summation of morphemes and named entities is as follows:

[0028]

[0029] The number of named entities to be retrieved is N, including C. i The number of knowledge graphs is n(C i Let i be the named entity and j be the entity in the knowledge graph. Sort the similarity scores from largest to smallest, and pair the named entities corresponding to the top three scores with the existing entities in the knowledge graph.

[0030] Furthermore, the method for inputting the linked entity into the first algorithm to extract initial attributes includes:

[0031] The linked entities are input into the first algorithm, using existing knowledge graph entities as the initial rule seed set and attribute value set; the initial rule seed set is used to traverse and match the attribute values ​​in the linked entities to obtain a candidate attribute set; the formula for calculating the credibility of each attribute in the candidate attribute value set is as follows:

[0032]

[0033] The attribute of the link entity 'o' is f. v (o), the attribute of the rule seed h is f v (h);

[0034] Sort the confidence levels in descending order and add the attribute values ​​corresponding to the top three confidence levels to the seed attribute value set. If convergence is achieved, iterate through the text using the attribute value set and generate a candidate template set by matching the words in the context of the attribute values. Calculate the confidence level of the candidate template set and sort it from largest to smallest. Output the initial attributes of the candidate template sets corresponding to the top three confidence levels.

[0035] Furthermore, the method for inputting the initial attributes into the second algorithm to extract attributes includes:

[0036] The initial attributes are input into the second algorithm, and the initial attributes are used to generate a meta-pattern candidate set through sequence pattern mining. The quality evaluation function of the candidate patterns is learned, and features with rich domain-independent contexts are selected to mine quality meta-patterns. The evaluation guides context-aware segmentation, and the entity type distribution in the meta-pattern candidates is checked. Initial attributes with fewer than 3 initial attributes in the meta-pattern candidate set are deleted. The score of the initial attributes in the meta-pattern candidate set is calculated using the attribute score function, and the initial attributes with a score higher than 0.85 are taken as attributes.

[0037] Furthermore, the method for extracting relationships between entities based on the aforementioned attributes includes:

[0038] The attributes are encoded using a convolutional neural network, and the relationships between entity pairs are classified based on the encoding results and the activation function combined with a fully connected layer.

[0039] The entity vector is obtained by concatenating the word vector and the position vector. i A standard convolutional neural network is used to perform window cropping on entities whose attributes have been extracted. The formula for calculating the i-th window is:

[0040] s i =w i , i+1∈R z×e

[0041] The convolution kernel is z×e; the k-th convolution kernel is W. k The formula for calculating the effect on the i-th window is:

[0042] t k,i =f(W k s i +L)

[0043] Where L is the maximum value obtained by max pooling, and the entity representation is obtained by performing a nonlinear transformation on the maximum value. The formula for calculating the relationship probability of entity x is as follows:

[0044] O=Mx+d

[0045]

[0046] The weight matrix M to be learned, the bias term d to be learned, and the k-th element of O are all represented by O. k The number of relation categories n r The feature vector for characterizing the relation extraction task is r, and the parameter is θ. The relationship between entities is determined based on the relation probability.

[0047] The beneficial effects of this invention are:

[0048] This invention is a text extraction method based on standard knowledge graphs. Compared with existing technologies, this invention has the following technical advantages:

[0049] This invention improves the accuracy of text extraction from standard knowledge graphs through preprocessing, keyword extraction, and attribute extraction steps, thereby enhancing the precision of text extraction. It intelligently extracts text from knowledge graphs, significantly saving resources and manpower, improving work efficiency, and enabling automatic text extraction. It allows for real-time automatic extraction and improvement of the text being processed, which is of great significance for text extraction based on standard knowledge graphs. It can adapt to text extraction needs of different standard knowledge graphs and different users, demonstrating a certain degree of universality. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the steps of a text extraction method based on a standard knowledge graph according to the present invention. Detailed Implementation

[0051] The present invention will be further described below through specific embodiments. The illustrative embodiments and descriptions herein are used to explain the present invention, but are not intended to limit the present invention.

[0052] The text extraction method based on standard knowledge graphs in this invention includes the following steps:

[0053] like Figure 1 As shown, this embodiment includes the following steps:

[0054] A acquires the text to be extracted, preprocesses the text to be extracted, extracts the keywords from the preprocessed text to be extracted, and inputs the keywords into the recognition model to recognize named entities. The preprocessing includes word segmentation, part-of-speech tagging, and vectorization.

[0055] In actual evaluation, a product analysis report is used as the research object, and a passage from it is presented as an example: "This reporter learned from the State Administration for Market Regulation that recently, the State Administration for Market Regulation organized a national supervision and spot check on the quality of XX types of online-sold products. This spot check included children's and infant clothing, school stationery, school bags, children's toothbrushes, projectors, electric hand warmers, automatic rice cookers, power adapters, reading and writing lamps, power banks, electronic door locks, and balance bikes—XX types of online-sold products. A total of XX batches of products from XX companies were sampled, and XXX batches of products were found to be substandard, with a failure rate of XX.X%." The pre-processed text reads: "This magazine / reporter / State Administration for Market Regulation / learned / State Administration for Market Regulation / organized / force / " The keywords extracted from the search query are "State Administration for Market Regulation / online sales products / quality / national supervision / spot checks / children / infant clothing / student stationery / student backpacks / children's toothbrushes / electric hand warmers / electric clothes racks / automatic rice cookers / power adapters / balance bikes / electronic door locks / online sales products / spot checks / enterprises / production / batch products / detection / batch products / non-compliance / spot checks / non-compliance rate". The named entities are: infant clothing, student stationery, student backpacks, children's toothbrushes, electric hand warmers, automatic rice cookers, power adapters, and balance bikes.

[0056] B calculates the correlation between the named entity and existing knowledge graph entities, matches the named entity with existing knowledge graph entities based on the correlation, and obtains linked entities based on the matching.

[0057] In the actual evaluation, the scores for named entities infant clothing, school stationery, school bags, children's toothbrushes, and children's products were 0.82, 0.91, 0.85, and 0.81, respectively. The scores for named entities electric hand warmers, automatic rice cookers, power adapters, balance bikes, and electronic products were 0.73, 0.91, 0.85, 0.83, and 0.87, respectively. Based on the correlation, the named entities selected were infant clothing, school stationery, school bags, automatic rice cookers, power adapters, and balance bikes. The named entities infant clothing, school stationery, and school bags were matched with children's products, and the named entities automatic rice cookers, power adapters, and balance bikes were matched with electronic products.

[0058] C inputs the linked entity into the first algorithm to extract initial attributes, and inputs the initial attributes into the second algorithm to extract attributes;

[0059] In actual evaluation, the initial attributes of infant and toddler clothing, school stationery, and school bags are appearance, shape, color, and material; the initial attributes of automatic rice cookers, power adapters, and balance bikes are function, appearance design, charging and discharging, and performance parameters.

[0060] D extracts the relationships between entities based on the attributes, clusters the entities based on the relationships, and extracts text based on the clusters;

[0061] In actual assessments, the relationship between infant and toddler clothing, school stationery, school bags, and electronic products is included, while the relationship between automatic rice cookers, power adapters, balance bikes, and electronic products is included.

[0062] In this embodiment, the method for extracting keywords from the preprocessed text to be extracted includes:

[0063] Candidate keywords are selected based on the following rules: phrase length not exceeding three words, not being proper nouns, and not starting or ending with a stop word. The formula for extracting the term frequency-inverse document frequency feature of the keywords is as follows:

[0064]

[0065] The frequency of phrase p in document d is f(p,d), the number of files containing phrase p in the global corpus is N, the size of the global corpus is C(d), the number of words in document d is C(d), and the number of documents containing phrase p in the global corpus is df(p).

[0066] The formula for calculating the candidate keyword score using the Naive Bayes model is as follows:

[0067]

[0068] The number of candidate keywords that are keywords is Y, the number of candidate keywords that are not keywords is N, and the feature value is t. Candidate keywords with a score greater than 0.8 are used as keywords.

[0069] In this embodiment, the method for inputting the keyword into the recognition model to recognize named entities includes:

[0070] The preprocessed text to be extracted is input into the recognition model. The input text to be extracted is pre-trained. The character-level feature information of the text to be extracted is learned by dynamic fine-tuning. The calculation formula of the character sequence vector is obtained.

[0071]

[0072]

[0073] The calculation result of the single-head attention mechanism is R(a,b,c), with dimension d. b The query is 'a', the key is 'b', the value is 'c', the softmax function is 'S', the time step is 'T', and the character sequence vector of feature i is 'f'. i The weight matrix for the linear mapping of query, key, and value is: Where b and c correspond one-to-one;

[0074] In the encoding layer, contextual features are extracted from the output vector of the embedding layer; different weights are assigned to the features based on multi-head self-attention; in the decoding layer, the features output by the encoding layer are decoded using CRF to obtain the predicted named entities.

[0075] In this embodiment, the method for matching the named entity with existing knowledge graph entities includes:

[0076] The formula for calculating the relevance between existing knowledge graph entities and named entities is as follows: (This formula is used to parse existing knowledge graph entities into morphemes and generate morphemes.)

[0077]

[0078] The regulating factors are k1, k2, and b, and the morpheme is C. i Named entity U i The semantic relevance between morphemes and named entities is f. i The length of knowledge graph j, and the average length of named entities (avgU) 1 ;

[0079] The formula for calculating the similarity score by weighted summation of morphemes and named entities is as follows:

[0080]

[0081] The number of named entities to be retrieved is N, including C. i The number of knowledge graphs is n(C i Let i be the named entity and j be the entity in the knowledge graph. Sort the similarity scores from largest to smallest, and pair the named entities corresponding to the top three scores with the existing entities in the knowledge graph.

[0082] In this embodiment, the method for inputting the linked entity into the first algorithm to extract initial attributes includes:

[0083] The linked entities are input into the first algorithm, using existing knowledge graph entities as the initial rule seed set and attribute value set; the initial rule seed set is used to traverse and match the attribute values ​​in the linked entities to obtain a candidate attribute set; the formula for calculating the credibility of each attribute in the candidate attribute value set is as follows:

[0084]

[0085] The attribute of the link entity 'o' is f. v (o), the attribute of the rule seed h is f v (h);

[0086] Sort the confidence levels in descending order and add the attribute values ​​corresponding to the top three confidence levels to the seed attribute value set. If convergence is achieved, iterate through the text using the attribute value set and generate a candidate template set by matching the words in the context of the attribute values. Calculate the confidence level of the candidate template set and sort it from largest to smallest. Output the initial attributes of the candidate template sets corresponding to the top three confidence levels.

[0087] In this embodiment, the method of inputting the initial attribute into the second algorithm to extract the attribute includes:

[0088] The initial attributes are input into the second algorithm, and the initial attributes are used to generate a meta-pattern candidate set through sequence pattern mining. The quality evaluation function of the candidate patterns is learned, and features with rich domain-independent contexts are selected to mine quality meta-patterns. The evaluation guides context-aware segmentation, and the entity type distribution in the meta-pattern candidates is checked. Initial attributes with fewer than 3 initial attributes in the meta-pattern candidate set are deleted. The score of the initial attributes in the meta-pattern candidate set is calculated using the attribute score function, and the initial attributes with a score higher than 0.85 are taken as attributes.

[0089] In this embodiment, the method for extracting relationships between entities based on the attributes includes:

[0090] The attributes are encoded using a convolutional neural network, and the relationships between entity pairs are classified based on the encoding results and the activation function combined with a fully connected layer.

[0091] The entity vector is obtained by concatenating the word vector and the position vector. i A standard convolutional neural network is used to perform window cropping on entities whose attributes have been extracted. The formula for calculating the i-th window is:

[0092] s i =w i , i+1∈R z×e

[0093] The convolution kernel is z×e; the k-th convolution kernel is W. kThe formula for calculating the effect on the i-th window is:

[0094] t k,i =f(W k s i +L)

[0095] Where L is the maximum value obtained by max pooling, and the entity representation is obtained by performing a nonlinear transformation on the maximum value. The formula for calculating the relationship probability of entity x is as follows:

[0096] O=Mx+d

[0097]

[0098] The weight matrix M to be learned, the bias term d to be learned, and the k-th element of O are all represented by O. k The number of relation categories n r The feature vector for characterizing the relation extraction task is r, and the parameter is θ. The relationship between entities is determined based on the relation probability.

[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A text extraction method based on standard knowledge graphs, characterized in that, Includes the following steps: A acquires the text to be extracted, preprocesses the text to be extracted, extracts the keywords of the preprocessed text to be extracted, and inputs the keywords into the recognition model to recognize named entities. The preprocessing includes word segmentation, part-of-speech tagging, and vectorization. B calculates the correlation between the named entity and existing knowledge graph entities, matches the named entity with existing knowledge graph entities based on the correlation, and obtains linked entities based on the matching. C inputs the linked entity into a first algorithm to extract initial attributes, and inputs the initial attributes into a second algorithm to extract attributes; including: The linked entities are input into the first algorithm, using existing knowledge graph entities as the initial rule seed set and attribute value set; the initial rule seed set is used to traverse and match the attribute values ​​in the linked entities to obtain a candidate attribute set; the formula for calculating the credibility of each attribute in the candidate attribute value set is as follows: ； The attributes of the link entity 'o' are: The attribute of the rule seed h is ; Sort the confidence levels in descending order and add the attribute values ​​corresponding to the top three confidence levels to the seed attribute value set. If convergence is achieved, use the attribute value set to traverse the text and generate a candidate template set by using the same words in the context of the matching attribute values. Calculate the confidence level of the candidate template set and sort it from largest to smallest. Output the attributes in the candidate template set corresponding to the top three confidence levels as the initial attributes. The initial attributes are input into the second algorithm, and the initial attributes are used to generate a meta-pattern candidate set by sequence pattern mining. The quality evaluation function of the candidate patterns is learned, and features with rich domain-independent context are selected to mine quality meta-patterns. The evaluation guides context-aware segmentation, and the entity type distribution in the meta-pattern candidates is checked. Initial attributes with fewer than 3 initial attributes in the meta-pattern candidate set are deleted. The score of the initial attributes in the meta-pattern candidate set is calculated using the attribute score function, and the initial attributes with a score higher than 0.85 are taken as attributes. D extracts the relationships between entities based on the attributes, clusters the entities based on the relationships, and extracts text based on the clusters.

2. The text extraction method based on a standard knowledge graph according to claim 1, characterized in that, A method for extracting keywords from the preprocessed text to be extracted includes: Candidate keywords are selected based on the following rules: phrase length not exceeding three words, not being proper nouns, and not starting or ending with a stop word. The formula for extracting the term frequency-inverse document frequency feature of the keywords is as follows: ； The frequency of phrase p in document d is The global corpus contains the number of files with phrase p, the size of the global corpus is N, and the number of words in document d is... The number of documents containing the phrase p in the global corpus is ; The formula for calculating the candidate keyword score using the Naive Bayes model is as follows: ； The number of candidate keywords that are keywords is Y, the number of candidate keywords that are not keywords is N, and the feature value is t. Candidate keywords with a score greater than 0.8 are used as keywords.

3. The text extraction method based on a standard knowledge graph according to claim 1, characterized in that, A method for inputting the keywords into a recognition model to identify named entities includes: The preprocessed text to be extracted is input into the recognition model. The model is then pre-trained on the input text, and dynamic fine-tuning is used to learn the character-level feature information of the text to be extracted. The formula for calculating the character sequence vector is as follows: ； The calculation results for the single-head attention mechanism are as follows: , dimension The query is 'a', the key is 'b', the value is 'c', the softmax function is 'S', the time step is 'T', and the character sequence vector of feature i is 'i'. The weight matrix for the linear mapping of query, key, and value is: , where b and c correspond one-to-one; In the encoding layer, contextual features are extracted from the output vector of the embedding layer; different weights are assigned to the features based on multi-head self-attention; in the decoding layer, the features output by the encoding layer are decoded using CRF to obtain the predicted named entities.

4. The text extraction method based on a standard knowledge graph according to claim 1, characterized in that, The method for matching named entities with existing knowledge graph entities includes: The formula for calculating the relevance between existing knowledge graph entities and named entities is as follows: (This formula is used to parse existing knowledge graph entities into morphemes and generate morphemes.) ； Where the adjustment factor is , b, morpheme Named entity is The semantic relevance between morphemes and named entities is The length of knowledge graph j, and the average length of named entities. ; The formula for calculating the similarity score by weighted summation of morphemes and named entities is as follows: ； The number of named entities to be retrieved is N, including The number of knowledge graphs is Let i be the named entity and j be the entity in the knowledge graph. Sort the similarity scores from largest to smallest, and pair the named entities corresponding to the top three scores with the existing entities in the knowledge graph.

5. The text extraction method based on a standard knowledge graph according to claim 1, characterized in that, The method for extracting relationships between entities based on the attributes includes: The attributes are encoded using a convolutional neural network, and the relationships between entity pairs are classified based on the encoding results and the activation function combined with a fully connected layer. Concatenating the word vector and the position vector yields the entity vector. A standard convolutional neural network is used to perform window cropping on entities whose attributes have been extracted. The formula for calculating the i-th window is: ； The convolution kernel is z The k-th convolutional kernel The formula for calculating the effect on the i-th window is: ； Where L is the maximum value obtained by max pooling, and the entity representation is obtained by performing a nonlinear transformation on the maximum value. The formula for calculating the relationship probability of entity x is as follows: ； The weight matrix M to be learned, the bias term d to be learned, and the k-th element in O are... Number of relation categories The feature vector for the relation extraction task is r, and the parameters are... The relationships between entities are determined based on the probability of those relationships.

Images