A government affairs field knowledge automatic discovery method and system

Abstract

The application discloses a kind of government affair field knowledge automatic discovery method and system, it is related to artificial intelligence technical field.The application includes: obtaining domain corpus by domain corpus acquisition module, and the domain corpus is preprocessed;The preprocessed domain corpus is input to open information extraction module, and the fact triple in corpus is obtained;The fact triple is input to relationship type discovery module, and the relationship type of fact triple is obtained;The relationship type is input to integration naming module, and the domain relationship type naming result is obtained.The application can be in government affair field text, design specific model structure extraction fact triple and discover the relationship type implied therein, complete the automatic discovery of type-containing domain knowledge.The application can obtain a relationship type implied in a domain without the intervention of domain expert, and it is important for the automatic construction of domain knowledge graph.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for automatic knowledge discovery in the field of government affairs. Background Technology

[0002] Knowledge in the field of government affairs often exists in documents within that field, including books, papers, announcements, and online texts. However, the forms in which this knowledge exists are not uniform, making it difficult to quickly locate the required knowledge units and establish connections between them. The emergence of knowledge graphs effectively solves these problems. They organize knowledge in the form of triples (head entity, relation type, tail entity) and store it in a graph database, facilitating knowledge retrieval and reasoning.

[0003] Generally, constructing a domain knowledge graph requires the participation of domain experts in designing the graph's schema layer, especially the types of relationships existing within the domain. However, because domain experts are unfamiliar with the operational mechanisms of knowledge graphs, the design of the schema layer often requires multiple rounds of communication between domain experts and knowledge graph engineers. This process is time-consuming, labor-intensive, and inefficient. Furthermore, it is more common to find domain experts to participate in the design of the graph's schema layer. Therefore, researching a method for automatically discovering relationship types from domain text to assist in the construction of knowledge graphs is of great significance. Summary of the Invention

[0004] This invention addresses the problem of high time consumption and low efficiency in traditional methods that rely on domain experts to manually design relationship types.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] On the one hand, the present invention provides an automatic discovery method for knowledge in the government domain. The method is implemented by an automatic discovery system for knowledge in the government domain, which includes a domain corpus acquisition module, an open information extraction module, a relation type discovery module, and a type integration and naming module.

[0007] The methods include:

[0008] S1. Obtain domain corpus through the domain corpus acquisition module and preprocess the domain corpus.

[0009] S2. Input the preprocessed domain corpus into the open information extraction module to obtain fact triples in the corpus.

[0010] S3. Input the fact triples into the relation type discovery module to obtain the relation type of the fact triples.

[0011] S4. Input the relation type into the integration naming module to obtain the domain relation type naming result.

[0012] Optionally, the domain corpus acquisition module in S1 acquires domain corpus, including:

[0013] Web crawling is used to obtain domain-specific corpora, and optical character recognition (OCR) is used to obtain domain-specific corpora.

[0014] Among them, web crawling to obtain domain-specific corpora includes: obtaining web page text data from domain-related websites using web crawling technology.

[0015] OCR acquisition of domain-specific corpora includes converting scanned image data into text data from domain-related scanned image data using OCR technology.

[0016] Optionally, preprocessing of the domain corpus in S1 includes:

[0017] After cleaning, deduplication, and merging, the domain corpus is divided into multiple records according to chapters and paragraphs, and the multiple records are saved in JSON format.

[0018] Optionally, the open information extraction module in S2 includes an input construction module, a feature extraction and fusion module, a feature interaction module, and a decoding module.

[0019] The preprocessed domain corpus is input into the open information extraction module to obtain fact triples from the corpus, including:

[0020] S21. Input the preprocessed domain corpus into the input construction module to obtain Query and Context pairs.

[0021] S22. Input the Query and Context pairs into the feature extraction and fusion module to obtain text features and part-of-speech features and fuse them.

[0022] S23. Input the fused features into the feature interaction module to obtain deeper semantic information.

[0023] S24. Input the deep semantic information into the decoding module to obtain the fact triples in the corpus.

[0024] Optionally, the decoding module employs multi-answer selection decoding.

[0025] Optionally, the relation type discovery module in S3 includes an input construction module, a feature encoding module, and a contrastive learning module.

[0026] The training process of the relation type discovery module includes:

[0027] S31. Obtain an open-source relation extraction dataset.

[0028] S32. Input the relation extraction dataset and fact triples into the input construction module to obtain the natural language template.

[0029] S33. Input the natural language template into the feature encoding module to obtain a vector representation of the relation type.

[0030] S34. By constructing a set of positive and negative samples from the relation extraction dataset and fact triples through the contrastive learning module, the parameters of the relation type discovery model with the minimum loss are obtained.

[0031] Optionally, the relation type is words after synonym replacement.

[0032] Optionally, in S4, the relation type is input into the integration naming module to obtain the domain relation type naming results, including:

[0033] S41. Convert the format of the fact triples and input the converted fact triples into the trained relation type discovery module to obtain the relation type embedding representation of each record.

[0034] S42. Cluster the relation type Embedding representation of each record to obtain the relation type.

[0035] S43. Name the relation types to obtain the domain relation type naming results.

[0036] Optionally, naming relation types in S43 includes: manual summary naming and frequency-based automatic naming.

[0037] Among them, frequency-based automatic naming includes:

[0038] Calculate the set of relation terms for each relation type and the frequency of each relation term.

[0039] The top two most frequent relation terms are selected as candidate relation descriptions.

[0040] Calculate the sum of the frequencies of the first two most frequent relation terms. If the sum of the frequencies is greater than a preset threshold, set the relation description to be the concatenation of the first two most frequent relation terms; otherwise, remove this relation type.

[0041] On the other hand, the present invention provides an automatic knowledge discovery system for the government domain. This system is applied to the implementation of an automatic knowledge discovery method for the government domain. The system includes a domain corpus acquisition module, an open information extraction module, a relation type discovery module, and a type integration and naming module.

[0042] in:

[0043] The domain corpus acquisition module is used to acquire domain corpus and preprocess the domain corpus.

[0044] An open information extraction module is used to obtain fact triples from the preprocessed domain corpus.

[0045] The relation type discovery module is used to determine the relation type of a fact triple based on the fact triple.

[0046] The integrated naming module is used to obtain domain relationship type naming results based on the relationship type.

[0047] Optionally, the domain corpus acquisition module is further used for:

[0048] Web crawling is used to obtain domain-specific corpora, and optical character recognition (OCR) is used to obtain domain-specific corpora.

[0049] Among them, web crawling to obtain domain-specific corpora includes: obtaining web page text data from domain-related websites using web crawling technology.

[0050] OCR acquisition of domain-specific corpora includes converting scanned image data into text data from domain-related scanned image data using OCR technology.

[0051] Optionally, the domain corpus acquisition module is further used for:

[0052] After cleaning, deduplication, and merging, the domain corpus is divided into multiple records according to chapters and paragraphs, and the multiple records are saved in JSON format.

[0053] Optionally, the open information extraction module includes an input construction module, a feature extraction and fusion module, a feature interaction module, and a decoding module.

[0054] The open information extraction module can be further used for:

[0055] S21. Input the preprocessed domain corpus into the input construction module to obtain Query and Context pairs.

[0056] S22. Input the Query and Context pairs into the feature extraction and fusion module to obtain text features and part-of-speech features and fuse them.

[0057] S23. Input the fused features into the feature interaction module to obtain deeper semantic information.

[0058] S24. Input the deep semantic information into the decoding module to obtain the fact triples in the corpus.

[0059] Optionally, the decoding module employs multi-answer selection decoding.

[0060] Optionally, the relation type discovery module includes an input construction module, a feature encoding module, and a contrastive learning module.

[0061] The training process of the relation type discovery module includes:

[0062] S31. Obtain an open-source relation extraction dataset.

[0063] S32. Input the relation extraction dataset and fact triples into the input construction module to obtain the natural language template.

[0064] S33. Input the natural language template into the feature encoding module to obtain a vector representation of the relation type.

[0065] S34. By constructing a set of positive and negative samples from the relation extraction dataset and fact triples through the contrastive learning module, the parameters of the relation type discovery model with the minimum loss are obtained.

[0066] Optionally, the relation type is words after synonym replacement.

[0067] Optionally, the naming module can be integrated for further use in:

[0068] S41. Convert the format of the fact triples and input the converted fact triples into the trained relation type discovery module to obtain the relation type embedding representation of each record.

[0069] S42. Cluster the relation type Embedding representation of each record to obtain the relation type.

[0070] S43. Name the relation types to obtain the domain relation type naming results.

[0071] Optionally, naming relation types can include: manual summarization naming and frequency-based automatic naming.

[0072] Among them, frequency-based automatic naming includes:

[0073] Calculate the set of relation terms for each relation type and the frequency of each relation term.

[0074] The top two most frequent relation terms are selected as candidate relation descriptions.

[0075] Calculate the sum of the frequencies of the first two most frequent relation terms. If the sum of the frequencies is greater than a preset threshold, set the relation description to be the concatenation of the first two most frequent relation terms; otherwise, remove this relation type.

[0076] The above technical solution has at least the following advantages compared with the existing technology:

[0077] The above approach establishes a method for extracting factual knowledge and relation types within a new domain by leveraging open-source relational data, without the intervention of domain experts, thereby aiding in the construction of knowledge graphs. Furthermore, this method can be used to perform knowledge discovery based on textual corpora from other domains, thus possessing portability. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a schematic diagram of the automatic knowledge discovery method in the government domain provided in an embodiment of the present invention;

[0080] Figure 2 An overall flowchart for domain relationship type discovery provided in this embodiment of the invention;

[0081] Figure 3 A flowchart for acquiring domain corpus provided in an embodiment of the present invention;

[0082] Figure 4 Design diagram of the fact triple extraction model provided in the embodiments of the present invention;

[0083] Figure 5 Design diagram of the relationship type discovery model provided in the embodiments of the present invention;

[0084] Figure 6 This is a block diagram of the automatic knowledge discovery system in the government domain provided in an embodiment of the present invention. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0086] like Figure 1 , 2 As shown, this embodiment of the invention provides a method for automatic discovery of knowledge in the government domain, which can be implemented by a government domain knowledge automatic discovery system. Figure 1 , 2The flowchart shown illustrates an automatic knowledge discovery method in the government affairs domain. The processing flow of this method may include the following steps:

[0087] S1. Obtain domain corpus through the domain corpus acquisition module and preprocess the domain corpus.

[0088] Optionally, the domain corpus acquisition module in S1 acquires domain corpus, including:

[0089] Web crawling is used to obtain domain-specific corpora, and optical character recognition (OCR) is used to obtain domain-specific corpora.

[0090] S1.1. Crawling to obtain domain-specific corpora includes: obtaining web page text data from domain-related websites using web crawling technology.

[0091] One feasible implementation method is, for example Figure 3 As shown, in order to ensure the accuracy of knowledge and the reliability of data, the target websites for crawling are generally government official websites or large government affairs Q&A websites, such as the State Taxation Administration (http: / / www.chinatax.gov.cn / ).

[0092] Furthermore, the corpus data crawled from the internet is publicly accessible, not restrictive or containing privacy concerns, and the crawling process follows website crawler protocols rather than maliciously scraping to cause target server anomalies. Using Python's third-party library requests, network requests are initiated to retrieve webpage data. Then, the policy list within the webpage is parsed, and for each specific policy text page, information such as its title, body, issuing organization, and date of issuance is crawled. Finally, depending on whether there is a next page, a new network request is initiated.

[0093] S1.2 OCR acquisition of domain-specific corpora includes: using OCR technology to convert scanned image data into text data from domain-related scanned image data.

[0094] In one feasible implementation, scanned image materials such as books or papers related to the field are identified and converted into text using OCR (Optical Character Recognition) technology. The resulting raw TXT text corpus is then obtained. The text is then segmented by sentence (filtering out sentences shorter than 5 or longer than 256 characters), with each sentence treated as a record, and each record occupying one line, stored in a TXT file.

[0095] Optionally, preprocessing of the domain corpus in S1 includes:

[0096] After cleaning, deduplication, and merging, the domain corpus is divided into multiple records according to chapters and paragraphs, and the multiple records are saved in JSON format.

[0097] In one feasible implementation, methods S1.1 and S1.2 are used to obtain a large amount of raw text data in the domain. Preprocessing operations such as cleaning illegal characters and removing duplicate files are performed on these texts to obtain high-quality domain corpus. To facilitate subsequent analysis and model training, the files are segmented according to their chapter entries, and the results are stored in JSON format.

[0098] S2. Input the preprocessed domain corpus into the open information extraction module to obtain fact triples in the corpus.

[0099] One feasible implementation method is, for example Figure 4 As shown, a lot of important knowledge in vertical fields exists in field-related texts. For example, knowledge in the field of government affairs is mostly found in laws, regulations, or notices issued by relevant national departments.

[0100] Furthermore, a large amount of factual knowledge exists in texts. This factual knowledge can be narrowly understood as (subject, predicate, object), and broadly described as (head entity, relational word, tail entity). Such triadic results are termed fact triples. This invention designs an open information extraction model that integrates part-of-speech features and textual features to mine fact triples existing in texts.

[0101] Optionally, the open information extraction module in S2 includes an input construction module, a feature extraction and fusion module, a feature interaction module, and a decoding module.

[0102] Specifically, step S2 above may include the following steps S21-S24:

[0103] S21. Input the preprocessed domain corpus into the input construction module to obtain Query and Context pairs.

[0104] In one feasible implementation, the goal of this model is to extract fact triples from text, where each fact triple contains a head entity, a relational term, and a tail entity. The relational term connects the head and tail entities; therefore, the following query is constructed first: "What are the 'relational terms' in the following sentence? Taxpayers who enjoy the above tax policies should follow the regulations of the tax authorities." Then, to extract the head entity for each relational term, the relational term is included when constructing the head entity query: "What are the subjects corresponding to the relational term 'enjoy'? Taxpayers who enjoy the above tax policies should follow the regulations of the tax authorities." Similarly, when constructing the tail entity query, information from both the head entity and the relational term is included: "What are the subjects corresponding to the subject 'taxpayer' and the relational term 'enjoy'? Taxpayers who enjoy the above tax policies should follow the regulations of the tax authorities."

[0105] Another part of the model's input is part-of-speech tagging (POS). This part of the query is processed by labeling all queries as "Other (o)" because it doesn't actually affect the extraction of fact triples. For the main text, the Harbin Institute of Technology's LTP tool is used for POS tagging, resulting in POS inputs such as "ooooooooooooooonnnv vn nnnnn wp vpnnnnnnvv wp". The relevant POS tags are shown in Table 1.

[0106] Table 1

[0107]

[0108] S22. Input the Query and Context pairs into the feature extraction and fusion module to obtain text features and part-of-speech features and fuse them.

[0109] In one feasible implementation, text feature extraction utilizes the Embedding Layer of the BERT (Bidirectional Encoder Representation from Transformers) series of pre-trained language models, denoted as...

[0110] t1,t2,…,t d =ertEmbedding(s_1,s_2,…,s_L) (1)

[0111] t=(t1,t2,…,t d (2)

[0112] Where L represents the sentence length, s i This represents the i-th character in the sentence. Part-of-speech features are encoded using one-hot encoding, denoted as...

[0113] c1,c2,…,c v =onehot(s′1,s′2,…,s′ L (3)

[0114] c = (c1, c2, ..., c v (4)

[0115] Where s′ i This represents the part-of-speech tag of the i-th character in the sentence. The fusion of part-of-speech features and text features is achieved by introducing a learnable transformation matrix with one parameter. The part-of-speech features and text features are mapped to the same dimension and then added together to obtain the fused features.

[0116] h = t + cW f (5)

[0117] S23. Input the fused features into the feature interaction module to obtain deeper semantic information.

[0118] In one feasible implementation, after obtaining the fused features, this invention uses a self-attention mechanism to complete the internal interaction of the features and learn deep semantic information. The self-attention mechanism is designed with reference to the Encoder part in the Transformer model. The fused features are then fed into an interaction layer containing N Encoders to obtain the final sentence embedding.

[0119] h α =Encoder(h α-1 ),α∈[1,N] (6)

[0120] To extract the target answer, the sentence embedding needs to be mapped to two binary vectors through a non-linear transformation. One vector represents the start position (start), and the other represents the end position (end). Furthermore, each element in the vector is classified as either a start or end position (0-1), where 1 indicates that the position is either start or end, and 0 indicates that it is not. This structure allows for multiple start and end points, whereas traditional methods using softmax to calculate probabilities can only yield at most one set of start and end points.

[0121]

[0122]

[0123] in, and These represent the probabilities that the i-th character is the start and end position of the result, respectively. If this probability is greater than a threshold (e.g., 0.5), it is marked as 1; otherwise, it is marked as 0. i is the embedding representation of the i-th character, W is the trainable parameter matrix, b represents the bias, and σ represents the sigmoid activation function.

[0124] S24. Input the deep semantic information into the decoding module to obtain the fact triples in the corpus.

[0125] Optionally, the decoding module employs multi-answer selection decoding, which allows multiple possible answers to be decoded to resolve multiple fact triples present in the text.

[0126] One feasible implementation involves introducing a multi-answer selection mechanism during feature interaction and decoding to address the problem of extracting multiple factual knowledge from text.

[0127] Specifically, the decoding module is responsible for converting the 0-1 marker matrix of S23 into the final result text. To decode multiple answers, this invention uses two nested loops to decode each possible answer sequentially. First, a position j marked as 1 is found in the end matrix. Then, starting from the j-th position in the start matrix, the first position i marked as 1 is found backwards (if not found, it is skipped). This decodes a result (x). i ,x j Then continue to traverse the end matrix and repeat the above operation until all possible results are found.

[0128] Furthermore, this model can be used to extract the fact triples contained in each record in step one and organize them into the form {"text":"...","spos":[{"head":"...","relation":"...","tail":"..."}]}. All extracted records are stored in a JSON file.

[0129] S3. Input the fact triples into the relation type discovery module to obtain the relation type of the fact triples.

[0130] One feasible implementation method is, for example Figure 5 As shown, relation type discovery refers to finding the same type of relation among a large number of existing fact triples. For example, (wages, payable, personal income tax) and (labor remuneration, legally paid, personal income tax) actually both express the relation of "payable".

[0131] Optionally, the relation type discovery module in S3 includes an input construction module, a feature encoding module, and a contrastive learning module.

[0132] The training process of the relation type discovery module includes S31-S35:

[0133] S31. Obtain an open-source relation extraction dataset.

[0134] In one feasible implementation, part of the training dataset is an open-source relation extraction dataset, which can be obtained directly from the network; the other part comes from the set of fact triples obtained in step 2.

[0135] S32. Input the relation extraction dataset and fact triples into the input construction module to obtain the natural language template.

[0136] In one feasible implementation, to stimulate the text understanding ability of a pre-trained language model, a specific natural language template (Prompt) needs to be designed to guide the model in completing a word-filling task. The word-filling task is a native training task of the pre-trained language model; this approach reduces the difference between training and fine-tuning, making the model's use more natural. The Prompt designed in this invention is "[Text] where the relationship between (head, relation, tail) can be represented by [MASK]", for example, "Taxpayers independently develop tax declaration software that complies with relevant regulations. Where the relationship between (tax declaration software, complies, relevant regulations) can be represented by [MASK]". Here, (head, relation, tail) comes from the results of open information extraction or supervised relation extraction triples. It is worth noting that the relation here is the relation word in the fact triple, not the final relation type; the relation type is unknown in the input data. The input sentence is denoted as l. i .

[0137] l i =[w1,w2,…,w mask ,…,w T (9)

[0138] Where T represents the number of tokens (word symbols, generally referring to indivisible semantic units; Chinese characters are commonly used as tokens, while English words or sub-words are commonly used) in the sentence, and w mask This indicates the token corresponding to [MASK] in the Prompt. i This represents the i-th sentence in a batch of input data.

[0139] S33. Input the natural language template into the feature encoding module to obtain a vector representation of the relation type.

[0140] In one feasible implementation, the pre-trained model used by the feature encoding module is DomainBert, which is obtained by further training on vertical domain corpora. DomainBert consists of 12 encoders, each containing 12 different head attention modules, forming a multi-head attention mechanism. This structure allows the model to perceive semantic information at different levels. For a given sentence l... i DomainBERT is used to encode it as an embedding vector:

[0141]

[0142]

[0143] Where d represents the dimension of the embedding, hi This represents the relation type embedding after encoding the i-th sentence. Take the vector b corresponding to [MASK]. mask This is represented as a relation embedding. Furthermore, the process of extracting relational features from a sentence is defined as a function f:h i =f(l i [MASK]).

[0144] S34. By constructing a set of positive and negative samples from the relation extraction dataset and fact triples through the contrastive learning module, the parameters of the relation type discovery model with the minimum loss are obtained.

[0145] Optionally, the relation type is words after synonym replacement.

[0146] In one feasible implementation, the relation type discovery model constructs a prompt template as input, then uses a language model trained on a domain corpus as an encoder, and introduces a positive and negative sample construction strategy based on synonym expansion to achieve contrastive learning. Ultimately, the model learns the ability to represent relation types.

[0147] Specifically, the relation type discovery model uses cue learning to obtain relation type representations between head and tail entities from a pre-trained model, and then uses contrastive learning to enable the model to learn to distinguish different relation types. The cue learning template (Prompt) used in the model's input construction module is " <text>The relationship (head, relation, tail) can be represented by [MASK]. The model's feature extraction layer uses BERT as the encoder and takes the embedding at the [MASK] position as the vector representation of the relation type. The labeling of the model's contrastive learning layer involves pairwise combinations of the n input samples to form an n*n matrix M. If the i-th sample and the j-th sample belong to the same type, then M(i,j) = 1; otherwise, M(i,j) = 0. The contrastive learning layer calculates the cosine similarity between each pair of the feature layer and the extracted sample embeddings, and calculates the cross-entropy loss with the labels. After multiple rounds of training, the model parameters with the minimum loss are saved as the final model.

[0148] Typically, contrastive learning requires establishing sets of positive and negative samples, narrowing the distance between positive samples (similar samples), and widening the distance between negative samples (dissimilar samples) to achieve discriminative power. This invention employs a hybrid supervised and self-supervised sample construction strategy and designs a sample generation method with minimal human intervention. Specifically, we use two types of datasets as the final datasets: the first type is an open-source multi-domain supervised relation extraction dataset. This dataset is used because there is currently no dataset specifically designed for Chinese relation type discovery, and supervised relation extraction datasets can acquire a large amount of data with a lower barrier to entry. The second type is fact triple data from vertical domains obtained through S2. Although their relation types are not yet certain, some typical relation types usually appear in relation words. The method for constructing positive samples is as follows: for supervised relation extraction datasets, since relation types naturally exist, positive samples can be distinguished; that is, samples of the same relation type are positive samples. For fact triple datasets, the frequency of relation words is counted, and then several typical relation words with high frequency are selected as a relation type to construct positive samples. For example, the frequency statistics of factual triple relation words in the field of government affairs are shown in Table 2 (a portion is selected from high to low frequency):

[0149] Table 2

[0150]

[0151]

[0152] To prevent the model from overfitting the input relation words and thus only focusing on the relation words themselves, the input to the model is not the original relation type, but a word after synonym replacement. For each relation type, a list of synonyms of a certain length is given, and each time an input relation word is input, one is randomly selected from this list as the replacement.

[0153] S35. Following the model design described above, the cross-entropy loss function is used to train the model. During training, the model with the minimum loss is saved as the final model.

[0154] S4. Input the relation type into the integration naming module to obtain the domain relation type naming result.

[0155] Optionally, step S4 above may include the following steps S41-S43:

[0156] S41. Convert the format of the fact triples and input the converted fact triples into the trained relation type discovery module to obtain the relation type embedding representation of each record.

[0157] S42. Cluster the relation type Embedding representation of each record to obtain the relation type.

[0158] S43. Name the relation types to obtain the domain relation type naming results.

[0159] Optionally, naming relation types in S43 includes: manual summary naming and frequency-based automatic naming.

[0160] Among them, frequency-based automatic naming includes:

[0161] Calculate the set of relation terms for each relation type and the frequency of each relation term.

[0162] The top two most frequent relation terms are selected as candidate relation descriptions.

[0163] Calculate the sum of the frequencies of the first two most frequent relation terms. If the sum of the frequencies is greater than a preset threshold, then set the relation description to be the concatenation of the first two most frequent relation terms.

[0164] In one feasible implementation, the data obtained in step 2 is organized into the data format of the input layer in step 3. Using the relation type discovery model stored in S3, predictions are made on the data obtained in S2 in the government affairs domain to obtain the relation type embedding representation for each record. This invention uses the K-Means method to cluster these embeddings, obtaining several categories, which are the final relation types. It is worth noting that the names of these categories are simply "1,2,3…k". To provide category names that are easy for humans to read and understand, manual summarization or an automatic naming method proposed in this invention can be used: First, the set of relation terms and the frequency of each relation term in each category are counted; then, the two most frequent relation terms are used as candidate relation descriptions; if the sum of the frequencies of these two relation terms is less than or equal to a certain threshold, it indicates that the relation in this category is unclear and should be eliminated; if the sum of the frequencies of these two relation terms is greater than a certain threshold, the relation description is set as a concatenation of the two relation terms.

[0165] The implementation of this invention is based on using an open-source multi-domain relation extraction dataset as a training foundation. First, domain-specific text corpora are obtained from books and websites related to the government affairs field, and fact triples are extracted from the corpora using the Open IE (Open Information Extraction) model designed in this invention. Then, the relation discovery model based on cue learning and contrastive learning, designed in this invention, is trained using the open-source relation extraction dataset, enabling the model to learn the ability to distinguish different relation types. Finally, the model is applied to the government affairs field to obtain relation type embeddings for fact triples, and the final relation type is determined through clustering.

[0166] In this embodiment of the invention, a method is developed that, without the intervention of domain experts, leverages open-source relation extraction data to obtain factual knowledge and relation types contained in a new domain, thereby assisting in the construction of a knowledge graph. Furthermore, this method can be used to perform knowledge discovery based on textual corpora obtained from other domains, thus possessing portability.

[0167] like Figure 6 As shown, this embodiment of the invention provides an automatic knowledge discovery system for the government domain. This system is applied to implement an automatic knowledge discovery method for the government domain. The system includes a domain corpus acquisition module, an open information extraction module, a relationship type discovery module, and a type integration and naming module.

[0168] in:

[0169] The domain corpus acquisition module is used to acquire domain corpus and preprocess the domain corpus.

[0170] An open information extraction module is used to obtain fact triples from the preprocessed domain corpus.

[0171] The relation type discovery module is used to determine the relation type of a fact triple based on the fact triple.

[0172] The integrated naming module is used to obtain domain relationship type naming results based on the relationship type.

[0173] Optionally, the domain corpus acquisition module is further used for:

[0174] Web crawling is used to obtain domain-specific corpora, and optical character recognition (OCR) is used to obtain domain-specific corpora.

[0175] Among them, web crawling to obtain domain-specific corpora includes: obtaining web page text data from domain-related websites using web crawling technology.

[0176] OCR acquisition of domain-specific corpora includes converting scanned image data into text data from domain-related scanned image data using OCR technology.

[0177] Optionally, the domain corpus acquisition module is further used for:

[0178] After cleaning, deduplication, and merging, the domain corpus is divided into multiple records according to chapters and paragraphs, and the multiple records are saved in JSON format.

[0179] Optionally, the open information extraction module includes an input construction module, a feature extraction and fusion module, a feature interaction module, and a decoding module.

[0180] The open information extraction module can be further used for:

[0181] S21. Input the preprocessed domain corpus into the input construction module to obtain Query and Context pairs.

[0182] S22. Input the Query and Context pairs into the feature extraction and fusion module to obtain text features and part-of-speech features and fuse them.

[0183] S23. Input the fused features into the feature interaction module to obtain deeper semantic information.

[0184] S24. Input the deep semantic information into the decoding module to obtain the fact triples in the corpus.

[0185] Optionally, the decoding module employs multi-answer selection decoding.

[0186] Optionally, the relation type discovery module includes an input construction module, a feature encoding module, and a contrastive learning module.

[0187] The training process of the relation type discovery module includes:

[0188] S31. Obtain an open-source relation extraction dataset.

[0189] S32. Input the relation extraction dataset and fact triples into the input construction module to obtain the natural language template.

[0190] S33. Input the natural language template into the feature encoding module to obtain a vector representation of the relation type.

[0191] S34. By constructing a set of positive and negative samples from the relation extraction dataset and fact triples through the contrastive learning module, the parameters of the relation type discovery model with the minimum loss are obtained.

[0192] Optionally, the relation type is words after synonym replacement.

[0193] Optionally, the naming module can be integrated for further use in:

[0194] S41. Convert the format of the fact triples and input the converted fact triples into the trained relation type discovery module to obtain the relation type embedding representation of each record.

[0195] S42. Cluster the relation type Embedding representation of each record to obtain the relation type.

[0196] S43. Name the relation types to obtain the domain relation type naming results.

[0197] Optionally, naming relation types can include: manual summarization naming and frequency-based automatic naming.

[0198] Among them, frequency-based automatic naming includes:

[0199] Calculate the set of relation terms for each relation type and the frequency of each relation term.

[0200] The top two most frequent relation terms are selected as candidate relation descriptions.

[0201] Calculate the sum of the frequencies of the first two most frequent relation terms. If the sum of the frequencies is greater than a preset threshold, set the relation description to be the concatenation of the first two most frequent relation terms; otherwise, remove this relation type.

[0202] In this embodiment of the invention, a method is developed that, without the intervention of domain experts, leverages open-source relation extraction data to obtain factual knowledge and relation types contained in a new domain, thereby assisting in the construction of a knowledge graph. Furthermore, this method can be used to perform knowledge discovery based on textual corpora obtained from other domains, thus possessing portability.

[0203] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0204] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.< / text>

Claims

1. A method for automatic knowledge discovery in the field of government affairs, characterized in that, The method is implemented by an automatic knowledge discovery system for the government domain, which includes a domain corpus acquisition module, an open information extraction module, a relation type discovery module, and a type integration and naming module. The method includes: S1. Obtain domain corpus through the domain corpus acquisition module and preprocess the domain corpus; S2. Input the preprocessed domain corpus into the open information extraction module to obtain fact triples in the corpus; S3. Input the fact triples into the relation type discovery module to obtain the relation type of the fact triples; S4. Input the relationship type into the integration naming module to obtain the domain relationship type naming result; The open information extraction module in S2 includes an input construction module, a feature extraction and fusion module, a feature interaction module, and a decoding module; The step of inputting the preprocessed domain corpus into the open information extraction module to obtain fact triples in the corpus includes: S21. Input the preprocessed domain corpus into the input construction module, sequentially construct queries for relation words, head entities and tail entities in fact triples, and perform part-of-speech tagging on the preprocessed domain corpus to obtain Query and Context pairs; S22. Input the Query and Context pair into the feature extraction and fusion module to obtain text features and part-of-speech features and fuse them; S23. Input the fused features into the feature interaction module, obtain the sentence embedding through the interaction layer containing N layers of encoders, and map the sentence embedding to two binary classification vectors through nonlinear transformation, one vector representing the start position and the other vector representing the end position, allowing multiple start and end points to obtain deep semantic information. S24. Input the deep semantic information into the decoding module, and use two loops to decode the possible answers one by one to obtain the fact triplet contained in each record; The relation type discovery module in S3 includes an input construction module, a feature encoding module, and a contrastive learning module. The training process of the relation type discovery module includes: S31. Obtain an open-source relation extraction dataset; S32. Input the relation extraction dataset and fact triples into the input construction module to obtain a natural language template and obtain a sentence using the natural language template; S33. Input the natural language template into the feature encoding module, and use the DomainBERT model to encode the sentence using the natural language template into an embedding vector to obtain a vector representation of the relation type; S34. Construct a set of positive and negative samples from the relation extraction dataset and fact triples using the contrastive learning module, and then obtain the parameters of the relation type discovery model with the minimum loss. Among them, positive samples are constructed by extracting relation words of the same relation type and fact triples in the relation extraction dataset with a frequency higher than a preset threshold as a relation type.

2. The method according to claim 1, characterized in that, The acquisition of domain corpus through the domain corpus acquisition module in S1 includes: Web crawling to obtain domain-specific corpora and optical character recognition (OCR) to obtain domain-specific corpora; The process of obtaining domain-specific corpora by crawling includes: obtaining webpage text data from domain-related websites using web crawling technology; The OCR acquisition of domain-specific corpus includes: converting scanned image data into text data from domain-related scanned image data using OCR technology.

3. The method according to claim 1, characterized in that, The preprocessing of the domain corpus in S1 includes: After cleaning, deduplication, and merging, the domain corpus is divided into multiple records according to chapters and paragraphs, and the multiple records are saved in JSON format.

4. The method according to claim 1, characterized in that, The decoding module employs a multi-answer selection decoding method.

5. The method according to claim 1, characterized in that, The relationship type is words that have been replaced with synonyms.

6. The method according to claim 1, characterized in that, In step S4, the relationship type is input into the integration naming module to obtain the domain relationship type naming result, including: S41. The fact triples are format-converted, and the format-converted fact triples are input into the trained relation type discovery module to obtain the relation type Embedding representation of each record; S42. Cluster the relation type Embedding representation of each record to obtain the relation type; S43. Name the relationship type to obtain the domain relationship type naming result.

7. The method according to claim 6, characterized in that, The naming of the relationship type in S43 includes: manual summarization naming and frequency-based automatic naming; The frequency-based automatic naming includes: Analyze the set of relation terms for each relation type and the frequency of each relation term. The top two most frequent relation terms are selected as candidate relation descriptions; Calculate the sum of the frequencies of the first two frequency relation words. If the sum of the frequencies is greater than a preset threshold, then set the relation description as a concatenation of the first two frequency relation words; otherwise, discard the relation type.

8. An automatic knowledge discovery system for the government domain, the system being used to implement the automatic knowledge discovery method for the government domain as described in any one of claims 1-7, characterized in that, The system includes a domain corpus acquisition module, an open information extraction module, a relation type discovery module, and a type integration and naming module. in: The domain corpus acquisition module is used to acquire domain corpus and preprocess the domain corpus; The open information extraction module is used to obtain fact triples from the preprocessed domain corpus. The relation type discovery module is used to obtain the relation type of the fact triples based on the fact triples. The integrated naming module is used to obtain the domain relationship type naming result based on the relationship type; The open information extraction module includes an input construction module, a feature extraction and fusion module, a feature interaction module, and a decoding module; The step of inputting the preprocessed domain corpus into the open information extraction module to obtain fact triples in the corpus includes: S21. Input the preprocessed domain corpus into the input construction module, sequentially construct queries for relation words, head entities and tail entities in fact triples, and perform part-of-speech tagging on the preprocessed domain corpus to obtain Query and Context pairs; S22. Input the Query and Context pair into the feature extraction and fusion module to obtain text features and part-of-speech features and fuse them; S23. Input the fused features into the feature interaction module, obtain the sentence embedding through the interaction layer containing N layers of encoders, and map the sentence embedding to two binary classification vectors through nonlinear transformation, one vector representing the start position and the other vector representing the end position, allowing multiple start and end points to obtain deep semantic information. S24. Input the deep semantic information into the decoding module, and use two loops to decode the possible answers one by one to obtain the fact triplet contained in each record; The relation type discovery module includes an input construction module, a feature encoding module, and a contrastive learning module; The training process of the relation type discovery module includes: S31. Obtain an open-source relation extraction dataset; S32. Input the relation extraction dataset and fact triples into the input construction module to obtain a natural language template and obtain a sentence using the natural language template; S33. Input the natural language template into the feature encoding module, and use the DomainBERT model to encode the sentence using the natural language template into an embedding vector to obtain a vector representation of the relation type; S34. Construct a set of positive and negative samples from the relation extraction dataset and fact triples using the contrastive learning module, and then obtain the parameters of the relation type discovery model with the minimum loss. Among them, positive samples are constructed by extracting relation words of the same relation type and fact triples in the relation extraction dataset with a frequency higher than a preset threshold as a relation type.

Images