A method and device for extracting basic rules of audit business

Abstract

The present invention discloses a method and device for extracting basic rules of auditing business, constructs a set of professional basic dictionaries and word vector generation models in the field of power auditing, fully considers the characteristics of business rules in the field of auditing, and the importance differences between words, and directly extracts the relationship from the text as an entity by converting the relationship between entities into an entity category, not limited to the known auditing business relationship. It effectively solves the problem of extracting business rules in the field of auditing, and fully considers the semantic information of the text and the importance differences between words. Through a triple sequential extraction model of power auditing business rules that labels the relationship as an entity and is based on pattern matching, it effectively avoids the limitation that the classification model in the traditional relationship extraction method must pre-plan the relationship type, and improves the accuracy of entity relationship extraction.

Description

Technical Field

[0001] The present invention relates to the technical field of power business auditing, and in particular to a method and device for extracting basic rules for auditing business. Background Art

[0002] Business rules refer to the description of business definitions and constraints, which are used to maintain business structure or control and influence business behavior. Business rules can essentially be understood as a set of conditions and operations under these conditions. They are a set of accurate and concise statements used to describe, constrain and control the structure, operation and strategy of an enterprise. They are a piece of business logic in an application. Its theoretical basis is: set a set of conditions, and when this set of conditions is met, one or more actions are triggered.

[0003] In the field of natural language processing, information extraction has always attracted people's attention. Information extraction mainly includes three subtasks: entity extraction, relationship extraction and event extraction, and relationship extraction is the core task and important link in the field of information extraction. The tasks involved in business rule extraction are entity and relationship extraction. The main goal of entity relationship extraction is to identify and determine the specific relationship between entity pairs from natural language text, which provides basic support for intelligent retrieval, semantic analysis, etc., helps to improve search efficiency and promote the automatic construction of knowledge base.

[0004] Unlike entity relationship extraction tasks in general open fields, business rule extraction involves specific domain knowledge. The relevant text corpus describing business rules in the power audit field does not have polysemy, and usually has a specific rule of "entity-relationship-entity" order. The entity relationship is relatively obvious in the text, but the entity description is relatively complex, with many professional terms, and many types of entity relationships that are difficult to summarize and organize. It requires the labelers to have domain expertise, manual labeling is difficult and costly, and has strict requirements on the accuracy of the extraction results. At present, there is no relevant research on business rule extraction in the power audit field.

[0005] In the existing technology, it is usually based on open fields or data in specific fields such as medical care, lacks attention to the knowledge of the power audit business field, cannot be directly transplanted, lacks professional basic dictionaries in the power audit field, and lacks relevant research on entity relationship extraction in the power audit field. Only considering the semantic information of words, it is impossible to distinguish the importance differences between words. In reality, the relationships between entities in the business rules of the power audit field are numerous and complex, and the existing technology cannot exhaust the types of relationships. Summary of the invention

[0006] The present invention provides a method and device for extracting basic audit business rules, which can directly extract the relationship from the text as an entity by converting the relationship between entities into an entity category, not limited to known audit business relationships.

[0007] According to one aspect of the present invention, a method for extracting basic audit business rules is provided, comprising the following steps:

[0008] Performing text preprocessing on the input data to obtain the input corpus; the input data includes: ordinary text corpus in the audit field, annotated training text corpus and text corpus for extracting rules;

[0009] Construct a word vector generation model in the field of power auditing; obtain the word vector of each word in the input corpus text based on Word2vec and the improved word frequency-inverse document frequency TF-IDF model, and obtain the word vector matrix of each text;

[0010] Construct a rule extraction model based on a multi-head self-attention mechanism, use the word vector matrix of the annotated training text corpus as input data, and continuously iterate the model to complete the model establishment and parameter tuning;

[0011] According to the rule extraction model after training, the word vector matrix of a text in the text corpus to be extracted rules is used as input data, and the entity relationship label of each word in the text is output;

[0012] According to the entity relationship labels, a set of triples of basic audit business rules corresponding to the text of each rule to be extracted is obtained.

[0013] The text preprocessing includes:

[0014] Remove the null characters in the input data text;

[0015] Replace common periods in sentences and at the end of sentences with spaces;

[0016] Keep the punctuation marks commonly used in official documents, keep numbers, letters, and uppercase and lowercase letters unchanged, and remove other punctuation marks and special characters;

[0017] Based on general dictionaries and independently constructed professional basic dictionaries in the audit field, Chinese word segmentation tools are used to segment the input text to obtain input idioms.

[0018] The word vector of each word in the input corpus text is obtained by weighting based on Word2vec and the improved word frequency-inverse document frequency method, and the word vector matrix of each text is obtained, including:

[0019] Copy a copy of the input familiar corpus D and save it as corpus D′;

[0020] The corpus D′ is used to train the Word2Vec word embedding model based on CBOW, and the word embedding representation E={e1, e2, …, e N};

[0021] Use the constructed domain stop word dictionary to filter the stop words in corpus D and remove the stop words in the corpus;

[0022] Use the corpus D that has completed stop word filtering to train the TF-IDF model and get the tfidf values ​​of all words in corpus D:

[0023] tfidf i，j =tf i，j ×idf i

[0024]

[0025]

[0026] Among them, tfidf i，j Representation word t i For text d in corpus D j The importance of tf i，j Representation word t i In text d j The frequency of occurrence in i Representation word t i A measure of the general importance of i，j Representation word t i In text d j The number of occurrences in ;∑ k n k，j Indicates that in the text j The sum of the occurrence times of all words in the corpus; |D| represents the total number of texts in the corpus, |{j:t i ∈d j}| means containing word t i The total number of texts;

[0027] For a text d in corpus D′ j The stop words in the text are initialized to 0 with tfidf values; the tfidf values ​​of all words in the text are added by 1 to obtain the current text d j The weight of all words where w i，j =tfidf i，j +1, indicating the weight of the i-th word;

[0028] The text d j Input the trained Word2Vec word embedding model and map the word embedding representations of all words in the text. i , i = 1, 2, 3...n; embed the obtained words to represent the weights w corresponding to the words i，j Multiply them together to get the final word vector matrix of the text:

[0029] V j = {v 1，j , v 2，j , …, v n，j}

[0030] v i，j =W i，j ×e i

[0031] Among them, v i，j Represents text d j The final word vector of the i-th word in ; w i，j Represents text d j The weight of the i-th word in e i Represents text d j The Word2Vec word embedding representation of the i-th word in;

[0032] Repeat the above steps for all texts in corpus D′ to obtain the word vector matrix corresponding to each text.

[0033] The construction method of the rule extraction model based on the multi-head self-attention mechanism is as follows:

[0034] The word vector matrix of the annotated training text corpus is used as input data, and the word vector plus the position vector code is used to represent the order of the sequence; the position code adopts the calculation method of sin and cos, and the final addition result is sent to the simplified Transformer model to extract text features;

[0035] The feature extraction results of the text are sent to the conditional random field CRF classifier to output the probability score of the label sequence:

[0036]

[0037] Where W = (w1, w2, ..., w n ) is the given text; y=(y1,y2,…,y n ) is the predicted label sequence; M is the state transfer matrix; Indicates that from label y i Move to label y i+1 probability; Indicates that the i-th word is labeled with label y i The probability of ; S(W, y) is the probability score of the input text W being predicted as the label sequence y;

[0038] In the back propagation process, the Adam optimization algorithm is used to train the model, and the parameters are continuously updated to finally obtain the label prediction model; preferably, the negative log-likelihood function is used as the model loss function, as follows:

[0039]

[0040] in, The input text W is predicted as a label sequence The probability score of ; S(W, y) is the probability score of the input text W being predicted as the true label sequence y.

[0041] The word vector matrix of the labeled training text corpus is used as input data, and the model is continuously iterated to complete the establishment and parameter tuning of the model, including:

[0042] According to the constructed business rule extraction model, the text corpus of the rules to be extracted is used as input data, and the entity relationship label of each word in each text in the text corpus is output;

[0043] Obtain the word vector matrix corresponding to all texts in the text corpus to be extracted rules one by one;

[0044] The word vector matrix is ​​used as input data and sent to the trained rule extraction model based on the multi-head self-attention mechanism for prediction, and the entity relationship label of each word in the text is output to obtain the label sequence result of the current text.

[0045] The method of obtaining a set of three-tuples of basic audit business rules corresponding to the text of each rule to be extracted according to the entity relationship label includes:

[0046] An audit business rule triple extraction model based on rule expressions is established, and the relationship triples in the text are extracted in the order of "entity-relationship-entity" in the label sequence. The set of audit business basic rule triples corresponding to the text of each rule to be extracted is output to generate the basic audit business rules.

[0047] According to another aspect of the present invention, there is provided a device for extracting basic rules of audit business, comprising:

[0048] The preprocessing unit is used to perform text preprocessing on the input data to obtain the input corpus; the input data includes: ordinary text corpus in the audit field, annotated training text corpus and text corpus for extracting rules;

[0049] A word vector generation unit is used to construct a word vector generation model in the field of power auditing; based on Word2vec and the improved word frequency-inverse document frequency TF-IDF model, the word vector of each word in the input corpus text is obtained to obtain a word vector matrix for each text;

[0050] A business rule learning unit is used to build a rule extraction model based on a multi-head self-attention mechanism, using the word vector matrix of the annotated training text corpus as input data, and continuously iterating the model to complete the model establishment and parameter tuning;

[0051] An entity relationship extraction unit is used to extract the entity relationship label of each word in the text according to the rule extraction model after training, taking the word vector matrix of a text in the text corpus of the rule to be extracted as input data;

[0052] The audit business output unit is used to obtain the audit business basic rule triple set corresponding to the text of each rule to be extracted according to the entity relationship label.

[0053] The pre-processing unit specifically comprises:

[0054] The character processing subunit is used to remove the null characters in the input data text;

[0055] The period processing subunit is used to replace the common period in the middle of a sentence and the period at the end of a sentence with a space;

[0056] The special character processing subunit is used to retain the punctuation marks commonly used in official documents, keep numbers, letters, and uppercase and lowercase letter formats unchanged, and remove other punctuation marks and special characters;

[0057] The word segmentation processing subunit is used to segment the input text using Chinese word segmentation tools based on the general dictionary and the independently constructed professional basic dictionary in the audit field to obtain the input idiom.

[0058] The word vector generating unit specifically includes:

[0059] The stop word processing subunit is used to copy a copy of the input familiar corpus D and save it as corpus D′, and use the corpus D′ to train the Word2Vec word embedding model based on CBOW to obtain a word embedding representation E with semantic information = {e1, e2, ..., e N}; Use the constructed domain stop word dictionary to filter the stop words in corpus D and remove the stop words in the corpus;

[0060] The model training subunit is used to train the TF-IDF model using the corpus D that has been filtered for stop words, and obtain the tfidf values ​​of all words in the corpus D:

[0061] tfidf i，j =tf i，j ×idf i

[0062]

[0063]

[0064] Among them, tfidf i，j Representation word t i For text d in corpus D j The importance of tf i，j Representation word t i In text d j The frequency of occurrence in i Representation word t i A measure of the general importance of i，j Representation word t i In text d j The number of occurrences in ;∑ k n k，j Indicates that in the text j The sum of the occurrence times of all words in the corpus; |D| represents the total number of texts in the corpus, |{j:t i ∈d j}| means containing word t i The total number of texts;

[0065] For a text d in corpus D′ j The stop words in the text are initialized to 0 with tfidf values; the tfidf values ​​of all words in the text are added by 1 to obtain the current text d j The weight of all words where w i，j =tfidf i，j +1, indicating the weight of the i-th word;

[0066] The text d j Input the trained Word2Vec word embedding model and map the word embedding representations of all words in the text. i , i = 1, 2, 3...n; embed the obtained words to represent the weights w corresponding to the words i，j Multiply them together to get the final word vector matrix of the text:

[0067] V j = {v 1，j , v 2，j , …, v n，j}

[0068] v i，j =w i，j ×e i

[0069] Among them, v i，j Represents text d j The final word vector of the i-th word in ; w i，j Represents text d jThe weight of the i-th word in e i Represents text d j The Word2Vec word embedding representation of the i-th word in;

[0070] The word vector matrix processing subunit is used to repeatedly perform the above steps on all the texts in the corpus D′ to obtain the word vector matrix corresponding to each text.

[0071] The business rule learning unit is specifically used for:

[0072] The text feature extraction subunit is used to use the word vector matrix of the labeled training text corpus as input data, and use the word vector plus the position vector encoding to represent the order of the sequence; the position encoding adopts the calculation method of sin and cos, and the final addition result is sent to the simplified Transformer model to extract text features;

[0073] The probability prediction subunit is used to send the text feature extraction results to the conditional random field CRF classifier and output the probability score of the label sequence:

[0074]

[0075] Where W = (w1, w2, ..., w n ) is the given text; y=(y1,y2,…,y n ) is the predicted label sequence; M is the state transfer matrix; Indicates that from label y i Move to label y i+1 probability; Indicates that the i-th word is labeled with label y i The probability of ; S(W, y) is the probability score of the input text W being predicted as the label sequence y;

[0076] The probability acquisition subunit is used to train the model using the Adam optimization algorithm during the back propagation process, continuously update the parameters, and finally obtain the prediction model of the label; preferably, the negative log-likelihood function is used as the model loss function, as follows:

[0077]

[0078] in, The input text W is predicted as a label sequence The probability score of ; S(W, y) is the probability score of the input text W being predicted as the true label sequence y.

[0079] By adopting the technical solution of the present invention, a scheme for extracting basic audit business rules is proposed, and a set of professional basic dictionaries and word vector generation models in the field of power audit are constructed. The characteristics of business rules in the audit field and the importance differences between words are fully considered. By converting the relationship between entities into an entity category, it is not limited to known audit business relationships, and the relationship is directly extracted from the text as an entity.

[0080] The embodiment of the present invention effectively solves the problem of extracting business rules in the audit field, and fully considers the semantic information of the text and the difference in importance between words. It effectively avoids the limitation that the classification model in the traditional relationship extraction method must pre-plan the relationship type through a triple sequential extraction model of power audit business rules that entity labels the relationship and is based on pattern matching, thereby improving the accuracy of entity relationship extraction.

[0081] The scheme of the present invention constructs a set of professional basic dictionaries in the field of power audit, and in the text cleaning process, according to the characteristics and rules of the text corpus in the specific field of power audit, the cleaning rules of retaining the commonly used punctuation marks (including "" [】()〔〕():-) in official documents and retaining the unchanged format of numbers, letters and uppercase letters are adopted. When calculating the weight of a word in a text, the tfidf values ​​of all non-stop words are first calculated, and the tfidf value of the stop word is initialized to 0, and then the tfidf value of all words in the text is added by 1 to obtain the weight of the word, and the weight of all words in the text is multiplied by its Word2Vec word embedding representation, so as to integrate the semantic information and importance of the word. Compared with directly calculating the tfidf values ​​of all words in the text and then weighting them, this method takes into account the influence of stop words, reduces its tfidf value, that is, the importance, to the minimum, and adds 1 to the tfidf values ​​of all words to avoid the situation where the word vectors of all stop words become zero vectors after weighted processing.

[0082] The present invention has built a set of word vector generation models in the field of power audit that integrates semantic information and importance. By inputting a text, its word vector matrix can be automatically mapped, which is the first in the field of power audit. According to the characteristics of the power audit business rule description text being relatively standardized, having strong regularity and unfixed relationships, the present invention proposes a method for entity labeling of relationships, which effectively avoids the limitation that the classification model in the traditional relationship extraction method must pre-determine the relationship type, and constructs a model for sequential extraction of power audit business rule triples based on pattern matching, so as to achieve effective extraction of business rule triples.

[0083] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0085] Figure 1 This is a flow chart of the principle of extracting basic rules for audit business in an embodiment of the present invention;

[0086] Figure 2 It is a flowchart of a method for extracting basic rules of audit business based on an attention mechanism in an embodiment of the present invention;

[0087] Figure 3 A model structure diagram of a method for extracting basic rules of audit business based on an attention mechanism in an embodiment of the present invention;

[0088] Figure 4 Schematic diagram of the structure of the basic rules extraction device for audit business in an embodiment of the present invention. DETAILED DESCRIPTION

[0089] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0090] The input data used by the method for extracting basic audit business rules based on the attention mechanism in an embodiment of the present invention includes: ordinary text corpus in the audit field, annotated training text corpus, and text corpus of rules to be extracted, and the output result is a set of triples of basic audit business rules corresponding to the text of each rule to be extracted.

[0091] In the embodiment of the present invention, information extraction is mainly a technology for automatically extracting specific messages from a large amount of text data for database access. An entity is something that is distinguishable and exists independently within itself. But it does not need to exist physically. In particular, abstractions and legal fictions are also often regarded as entities. Relationship is an explicit or implicit semantic connection between entities. Word2Vec is a group of related models used to generate word vectors. After training, each word can be mapped to a vector to represent the relationship between words. Word embedding is a general term for language models and representation learning technologies in natural language processing (NLP). Conceptually, it refers to embedding a high-dimensional space with a dimension equal to the number of all words into a continuous vector space with a much lower dimension, and each word or phrase is mapped to a vector on the real number field. Transformer is a classic NLP model proposed by the Google team in 2017. The Transformer model uses the Self-Attention mechanism and does not adopt the sequential structure of RNN, so that the model can be trained in parallel and have global information.

[0092] Figure 1 FIG. 1 is a flowchart of extracting basic rules for auditing business in an embodiment of the present invention. Figure 1 As shown, the basic rules extraction process of the audit business includes the following steps:

[0093] Step 101: Perform text preprocessing on input data to obtain input idioms.

[0094] In the embodiment of the present invention, the input data includes: ordinary text corpus in the audit field, annotated training text corpus and text corpus for extracting rules.

[0095] In the embodiment of the present invention, a set of professional basic dictionaries in the field of power audit and text cleaning rules targeting the characteristics of business text corpus in the field of power audit are constructed. All input data are cleaned and segmented. The detailed steps are as follows:

[0096] Remove the blank characters from the text;

[0097] Replace common periods in sentences and at the end of sentences (including,,;.) with spaces;

[0098] Keep the punctuation marks commonly used in official documents (including 《》【】()〔〕():-), keep numbers, letters, and uppercase and lowercase letters unchanged, and remove other punctuation marks and special characters;

[0099] Based on the general dictionary and the self-built professional basic dictionary in the audit field, the input text is segmented using the jieba Chinese word segmentation tool to obtain the familiar corpus D. The comparison of the word segmentation results before and after citing the professional basic dictionary in the audit field is shown in Table 1 below:

[0100] Table 1 Comparison of word segmentation results before and after citing the audit field dictionary

[0101]

[0102] Step 102: construct a word vector generation model in the field of power auditing; obtain the word vector of each word in the input corpus text based on Word2vec and the improved word frequency-inverse document frequency TF-IDF model, and obtain the word vector matrix of each text.

[0103] In the embodiment of the present invention, a word vector generation model in the field of power auditing is built, which integrates semantic information and importance. By inputting a text, its word vector matrix can be automatically mapped. The vectorized representation of each text is obtained through the word vector generation module.

[0104] In the embodiment of the present invention, a word vector generation model in the field of power audit is constructed, and the word vector of each word in the text is obtained by weighting based on Word2vec and improved TF-IDF (term frequency-inverse document frequency), thereby obtaining the word vector matrix of each text. The detailed steps are as follows:

[0105] Copy the word-segmented corpus D and save it as corpus D′.

[0106] The corpus D′ is used to train the Word2Vec word embedding model based on CBOW, and the word embedding representation E={e1, e2, …, e N Considering that the corpus D′ is not large in size, the word embedding dimension is set to the commonly used 128 dimensions.

[0107] Use the constructed domain stop word dictionary to filter the stop words in corpus D and remove the stop words in the corpus.

[0108] Use the corpus D that has completed stop word filtering to train the TF-IDF model and obtain the tfidf values ​​of all words in corpus D. The calculation formula is as follows:

[0109] tfidf i，j =tf i，j ×idf i

[0110]

[0111]

[0112] Among them, tfidf i，j Representation word t i For text d in corpus D j The importance of tf i，j Representation word t i In text d j The frequency of occurrence, idf i Representation word t i A measure of the general importance of i，j Representation word t i In text d j The number of occurrences in ,∑ k n k，j Indicates that in the text j The sum of the occurrence times of all words in the corpus, |D| represents the total number of texts in the corpus, |{j:t i ∈d j}| means containing word t i The total number of texts.

[0113] For a text d in corpus D′ j For example, it consists of non-stop words with tfidf values ​​and stop words without tfidf values. The tfidf value of the stop words in the text is initialized to 0, so that the impact of the stop words on the text is minimized, and then the tfidf values ​​of all words in the text are added by 1 to avoid the situation where the word vectors of all stop words become zero vectors after weighted processing. Thus, the current text d is finally obtained. j The weight of all words where w i，j =tfidf i，j +1 indicates the weight of the i-th word, that is, its importance. This step proposes an improved TFIDF value calculation method, which effectively reduces the impact of stop words on the text and avoids the occurrence of zero vectors.

[0114] D j Input the trained Word2Vec word embedding model to map the word embedding representations of all words in the text. i , i = 1, 2, 3...n, and the obtained word embedding represents the weight w corresponding to each word obtained in step s205 i，j Multiply them together to get the final word vector matrix of the text. The calculation formula is as follows:

[0115] V j = {v 1，j , v 2，j , …, v n，j}

[0116] v i，j =w i，j ×e i

[0117] where v i，j Represents text d j The final word vector of the i-th word in , w i，j Represents text d j The weight of the i-th word in i Represents text d j The Word2Vec word embedding representation of the i-th word in .

[0118] Repeat the above operation for all texts in corpus D′ to obtain the word vector matrix corresponding to each text.

[0119] Step 103, construct a rule extraction model based on a multi-head self-attention mechanism, use the word vector matrix of the labeled training text corpus as input data, and continuously iterate the model to complete model establishment and parameter tuning.

[0120] In the embodiment of the present invention, according to the constructed business rule extraction model, the text corpus of the rules to be extracted is used as input data, and the entity relationship label of each word in each text in the text corpus is output.

[0121] In an embodiment of the present invention, the entity relationship label of each word in the text is obtained through the business rule extraction module. In this step, a rule extraction model based on a multi-head self-attention mechanism is constructed. In the process of building the model, the word vector matrix of the training text corpus that has been annotated is used as input data, and the model is continuously iterated to complete the establishment of the model and parameter tuning. Then the trained model is embedded in the business rule extraction module, and the word vector matrix of a text in the text corpus to be extracted is used as input data, and the entity relationship label of each word in the text is output after calculation.

[0122] In the embodiment of the present invention, the detailed construction steps of the rule extraction model based on the multi-head self-attention mechanism are as follows:

[0123] The word vector matrix of the annotated training text corpus is used as input data, and the word vector plus the position vector code is used to represent the order of the sequence. Here, the position code adopts the calculation method of sin and cos. The final addition result is sent to the simplified Transformer model to extract text features. Considering the limitation of the number of annotated samples and to reduce the risk of overfitting, the present invention builds a simplified Transformer model consisting of 2 encoders and 2 decoders to extract features.

[0124] The output result is sent to the conditional random field (CRF) classifier to output the probability score of the label sequence. The calculation formula is as follows:

[0125]

[0126] Where W = (w1, w2, ..., w n ) is the given text, y=(y1,y2,…,y n ) is the predicted label sequence, M is the state transfer matrix, Indicates that from label y i Move to label y i+1 The probability of Indicates that the i-th word is labeled with label y i The probability of S(W, y) is the probability score that the input text W is predicted to be the label sequence y.

[0127] In the back propagation process, the Adam optimization algorithm is used to train the model, and the parameters are continuously updated to finally obtain the label prediction model. The present invention adopts the negative log-likelihood function as the model loss function, which is as follows:

[0128]

[0129] in The input text W is predicted as a label sequence The probability score of S(W, y) is the probability score of the input text W being predicted as the true label sequence y.

[0130] Step 104, according to the rule extraction model after training, the word vector matrix of a text in the text corpus for extracting rules is used as input data, and the entity relationship label of each word in the text is output.

[0131] In the embodiment of the present invention, after the rule extraction model based on the multi-head self-attention mechanism is constructed, a complete business rule extraction module is formed. The input data can be passed through the business rule extraction module to obtain the label sequence result of the text. The detailed steps are as follows:

[0132] Obtain the corresponding word vector matrix for all texts in the text corpus to be extracted rules one by one;

[0133] The word vector matrix is ​​used as input data and sent to the trained rule extraction model based on the multi-head self-attention mechanism for prediction, and the entity relationship label of each word in the text is output, so as to obtain the label sequence result of the current text.

[0134] Step 105, based on the entity relationship label, obtain a set of triples of basic audit business rules corresponding to the text of each rule to be extracted.

[0135] In the embodiment of the present invention, the basic rules of audit business are output. Considering that the description text of audit business rules is relatively uniform, and the corresponding label sequence shows a certain regularity, a triple extraction model of audit business rules based on regular expressions is established, and the relationship triples in the text are extracted in the order of "entity-relationship-entity" in the label sequence output in step s312, and finally the set of triples of basic rules of audit business corresponding to the text of each rule to be extracted is output to generate the basic rules of audit business.

[0136] See also Figure 2 , which is a flowchart of the basic rules extraction method for audit business based on the attention mechanism. Figure 3 This is a model structure diagram of the basic rules extraction method for audit business based on the attention mechanism.

[0137] The embodiment of the present invention constructs a set of professional basic dictionaries in the field of power auditing, and in the text cleaning process, according to the characteristics and rules of the text corpus in the specific field of power auditing, adopts the cleaning rules of retaining the commonly used punctuation marks in official documents (including "》【】()〔〕():-), retaining the numbers, letters and the uppercase and lowercase letter formats unchanged.

[0138] When calculating the weight of a word in a text, first calculate the tfidf values ​​of all non-stop words, and initialize the tfidf value of the stop words to 0. Then add 1 to the tfidf values ​​of all words in the text to get the weight of the word, and use the weight of all words in the text multiplied by their Word2Vec word embedding representation to integrate the semantic information and importance of the word. Compared with directly calculating the tfidf values ​​of all words in the text and then weighting them, this method takes into account the impact of stop words, reduces their tfidf values, i.e., importance, to the minimum, and adds 1 to the tfidf values ​​of all words to avoid the situation where the word vectors of all stop words become zero vectors after weighting.

[0139] The embodiment of the present invention has built a word vector generation model in the field of power auditing that integrates semantic information and importance. By inputting a text, its word vector matrix can be automatically mapped out, which is the first of its kind in the field of power auditing.

[0140] According to the characteristics of the description text of power audit business rules, which is relatively standardized, has strong regularity and has non-fixed relationships, an embodiment of the present invention proposes a method for entity labeling of relationships, which effectively avoids the limitation that the classification model in the traditional relationship extraction method must pre-plan the relationship type, and constructs a triple sequential extraction model of power audit business rules based on pattern matching to achieve effective extraction of business rule triples.

[0141] In order to realize the above process, the technical solution of the present invention also provides an audit business basic rule extraction device, such as Figure 4 As shown, the audit business basic rule extraction device includes:

[0142] The preprocessing unit 21 is used to perform text preprocessing on the input data to obtain the input corpus; the input data includes: ordinary text corpus in the audit field, annotated training text corpus and text corpus for extracting rules;

[0143] The word vector generation unit 22 is used to construct a word vector generation model in the field of power auditing; the word vector of each word in the input corpus text is obtained by weighting based on Word2vec and the improved word frequency-inverse document frequency TF-IDF model, and the word vector matrix of each text is obtained;

[0144] The business rule learning unit 23 is used to construct a rule extraction model based on a multi-head self-attention mechanism, use the word vector matrix of the labeled training text corpus as input data, and continuously iterate the model to complete the model establishment and parameter tuning;

[0145] An entity relationship extraction unit 24 is used to extract the rule model after training, take the word vector matrix of a text in the text corpus to be extracted as input data, and output the entity relationship label of each word in the text;

[0146] The audit business output unit 25 is used to obtain a set of audit business basic rule triples corresponding to the text of each rule to be extracted according to the entity relationship label.

[0147] The pre-processing unit 21 specifically includes:

[0148] The character processing subunit is used to remove the null characters in the input data text;

[0149] The period processing subunit is used to replace the common period in the middle of a sentence and the period at the end of a sentence with a space;

[0150] The special character processing subunit is used to retain the punctuation marks commonly used in official documents, keep numbers, letters, and uppercase and lowercase letter formats unchanged, and remove other punctuation marks and special characters;

[0151] The word segmentation processing subunit is used to segment the input text using Chinese word segmentation tools based on the general dictionary and the independently constructed professional basic dictionary in the audit field to obtain the input idiom.

[0152] The word vector generating unit 22 specifically includes:

[0153] The stop word processing subunit is used to copy a copy of the input familiar corpus D and save it as corpus D′, and use the corpus D′ to train the Word2Vec word embedding model based on CBOW to obtain a word embedding representation E with semantic information = {e1, e2, ..., e N}; Use the constructed domain stop word dictionary to filter the stop words in corpus D and remove the stop words in the corpus;

[0154] The model training subunit is used to train the TF-IDF model using the corpus D that has been filtered for stop words, and obtain the tfidf values ​​of all words in the corpus D:

[0155] tfidf i，j =tf i，j ×idf i

[0156]

[0157]

[0158] Among them, tfidf i，j Representation word t i For text d in corpus D j The importance of tf i，j Representation word t i In text d j The frequency of occurrence in i Representation word t i A measure of the general importance of i，j Representation word t i In text d j The number of occurrences in ;∑ k n k，j Indicates that in the text j The sum of the occurrence times of all words in the corpus; |D| represents the total number of texts in the corpus, |{j:t i ∈d j}| means containing word t i The total number of texts;

[0159] For a text d in corpus D′ j The stop words in the text are initialized to 0 with tfidf values; the tfidf values ​​of all words in the text are added by 1 to obtain the current text d j The weight of all words where w i，j =tfidf i，j +1, indicating the weight of the i-th word;

[0160] The text d jInput the trained Word2Vec word embedding model and map the word embedding representations of all words in the text. i , i = 1, 2, 3...n; embed the obtained words to represent the weights w corresponding to the words i，j Multiply them together to get the final word vector matrix of the text:

[0161] V j = {v 1，j , v 2，j , …, v n，j}

[0162] v i，j =w i，j ×e i

[0163] Among them, v i，j Represents text d j The final word vector of the i-th word in ; w i，j Represents text d j The weight of the i-th word in e i Represents text d j The Word2Vec word embedding representation of the i-th word in;

[0164] The word vector matrix processing subunit is used to repeatedly perform the above steps on all the texts in the corpus D′ to obtain the word vector matrix corresponding to each text.

[0165] The business rule learning unit 23 is specifically used for:

[0166] The text feature extraction subunit is used to use the word vector matrix of the labeled training text corpus as input data, and use the word vector plus the position vector encoding to represent the order of the sequence; the position encoding adopts the calculation method of sin and cos, and the final addition result is sent to the simplified Transformer model to extract text features;

[0167] The probability prediction subunit is used to send the text feature extraction results to the conditional random field CRF classifier and output the probability score of the label sequence:

[0168]

[0169] Where W = (w1, w2, ..., w n ) is the given text; y=(y1,y2,…,y n ) is the predicted label sequence; M is the state transfer matrix; Indicates that from label y i Move to label y i+1 probability; Indicates that the i-th word is labeled with label yi The probability of ; S(W, y) is the probability score of the input text W being predicted as the label sequence y;

[0170] The probability acquisition subunit is used to train the model using the Adam optimization algorithm during the back propagation process, continuously update the parameters, and finally obtain the prediction model of the label; preferably, the negative log-likelihood function is used as the model loss function, as follows:

[0171]

[0172] in, The input text W is predicted as a label sequence The probability score of ; S(W, y) is the probability score of the input text W being predicted as the true label sequence y.

[0173] To sum up, the technical solution of the present invention proposes a solution for extracting basic audit business rules, constructs a set of professional basic dictionaries and word vector generation models in the field of power audit, fully considers the characteristics of business rules in the audit field, and the importance differences between words, and converts the relationship between entities into an entity category. It is not limited to known audit business relationships, and the relationship is directly extracted from the text as an entity.

[0174] The embodiment of the present invention effectively solves the problem of extracting business rules in the audit field, and fully considers the semantic information of the text and the difference in importance between words. It effectively avoids the limitation that the classification model in the traditional relationship extraction method must pre-plan the relationship type through a triple sequential extraction model of power audit business rules that entity labels the relationship and is based on pattern matching, thereby improving the accuracy of entity relationship extraction.

[0175] The scheme of the present invention constructs a set of professional basic dictionaries in the field of power audit, and in the text cleaning process, according to the characteristics and rules of the text corpus in the specific field of power audit, the cleaning rules of retaining the commonly used punctuation marks (including "" [】()〔〕():-) in official documents and retaining the unchanged format of numbers, letters and uppercase letters are adopted. When calculating the weight of a word in a text, the tfidf values ​​of all non-stop words are first calculated, and the tfidf value of the stop word is initialized to 0, and then the tfidf value of all words in the text is added by 1 to obtain the weight of the word, and the weight of all words in the text is multiplied by its Word2Vec word embedding representation, so as to integrate the semantic information and importance of the word. Compared with directly calculating the tfidf values ​​of all words in the text and then weighting them, this method takes into account the influence of stop words, reduces its tfidf value, that is, the importance, to the minimum, and adds 1 to the tfidf values ​​of all words to avoid the situation where the word vectors of all stop words become zero vectors after weighted processing.

[0176] The present invention has built a set of word vector generation models in the field of power audit that integrates semantic information and importance. By inputting a text, its word vector matrix can be automatically mapped, which is the first in the field of power audit. According to the characteristics of the power audit business rule description text being relatively standardized, having strong regularity and unfixed relationships, the present invention proposes a method for entity labeling of relationships, which effectively avoids the limitation that the classification model in the traditional relationship extraction method must pre-plan the relationship type, and constructs a model for sequential extraction of power audit business rule triples based on pattern matching, so as to achieve effective extraction of business rule triples.

[0177] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0178] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0181] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for extracting basic rules for audit business, characterized in that: The following steps are involved: Performing text preprocessing on the input data to obtain the input corpus; the input data includes: ordinary text corpus in the audit field, annotated training text corpus and text corpus for extracting rules; Construct a word vector generation model in the field of power auditing; obtain the word vector of each word in the input corpus text based on Word2vec and the improved word frequency-inverse document frequency TF-IDF model, and obtain the word vector matrix of each text; Construct a rule extraction model based on a multi-head self-attention mechanism, use the word vector matrix of the annotated training text corpus as input data, and continuously iterate the model to complete the model establishment and parameter tuning; According to the rule extraction model after training, the word vector matrix of a text in the text corpus to be extracted rules is used as input data, and the entity relationship label of each word in the text is output; According to the entity relationship labels, a set of triples of basic audit business rules corresponding to the text of each rule to be extracted is obtained; The word vector of each word in the input corpus text is obtained by weighting based on Word2vec and the improved word frequency-inverse document frequency method, and the word vector matrix of each text is obtained, including: Copy a copy of the input familiar corpus D and save it as corpus D′; The corpus D′ is used to train the Word2Vec word embedding model based on CBOW, and the word embedding representation E={e1, e2, …, e N }; Use the constructed domain stop word dictionary to filter the stop words in corpus D and remove the stop words in the corpus; Use the corpus D that has completed stop word filtering to train the TF-IDF model and get the tfidf values ​​of all words in corpus D: tfidf i，j =tf i，j ×idf i Among them, tfidf i，j Representation word t i For text d in corpus D j The importance of tf i，j Representation word t i In text d j The frequency of occurrence in i Representation word t i A measure of the general importance of i，j Representation word t i In text d j The number of occurrences in ∑kn k，j Indicates that in the text j The sum of the occurrence times of all words in the corpus; |D| represents the total number of texts in the corpus, |{d:t i ∈d j }| means containing word t i The total number of texts; For a text d in corpus D′ j The stop words in the text are initialized to 0 with tfidf values; the tfidf values ​​of all words in the text are added by 1 to obtain the current text d j The weight of all words where w i，j =tfidf i，j +1, indicating the weight of the i-th word; The text d j Input the trained Word2Vec word embedding model and map the word embedding representations of all words in the text. i , i = 1, 2, 3...n; embed the obtained words to represent the weights w corresponding to the words i，j Multiply them together to get the final word vector matrix of the text: V j ={v 1，j ,v 2，j ,…,v n，j } v i，j =w i，j ×e i Among them, v i，j Represents text d j The final word vector of the i-th word in ; w i，j Represents text d j The weight of the i-th word in e i Represents text d j The Word2Vec word embedding representation of the i-th word in; Repeat the above steps for all texts in corpus D′ to obtain the word vector matrix corresponding to each text; The construction method of the rule extraction model based on the multi-head self-attention mechanism is as follows: The word vector matrix of the labeled training text corpus is used as input data, and the word vector plus the position vector code is used to represent the order of the sequence; the position code adopts the calculation method of sin and cos, and the final addition result is sent to the simplified Transforme r Model, extract text features; The text feature extraction result is sent to the conditional random field CRF classifier to output the probability score of the label sequence: Where W = (w1, w2, ..., w n ) is the given text; y=(y1,y2,…,y n ) is the predicted label sequence; M is the state transfer matrix; Indicates that from label y i Move to label y i+1 probability; Indicates that the i-th word is labeled with label y i The probability of ; S(W, y) is the probability score of the input text W being predicted as the label sequence y; In the back propagation process, the Adam optimization algorithm is used to train the model, and the parameters are continuously updated to finally obtain the label prediction model; the negative log-likelihood function is used as the model loss function, as follows: in, The input text W is predicted as a label sequence The probability score of ; S(W, y) is the probability score of the input text W being predicted as the true label sequence y.

2. A method for extracting basic audit business rules according to claim 1, characterized in that: The text preprocessing includes: Remove the null characters in the input data text; Replace common periods in sentences and at the end of sentences with spaces; Keep the punctuation marks commonly used in official documents, keep numbers, letters, and uppercase and lowercase letters unchanged, and remove other punctuation marks and special characters; Based on general dictionaries and independently constructed professional basic dictionaries in the audit field, Chinese word segmentation tools are used to segment the input text to obtain input idioms.

3. A method for extracting basic rules for audit business according to claim 1, characterized in that: The word vector matrix of the labeled training text corpus is used as input data, and the model is continuously iterated to complete the establishment and parameter tuning of the model, including: According to the constructed rule extraction model, the text corpus of the rules to be extracted is used as input data, and the entity relationship label of each word in each text in the text corpus is output; Obtain the word vector matrix corresponding to all texts in the text corpus to be extracted rules one by one; The word vector matrix is ​​used as input data and sent to the trained rule extraction model based on the multi-head self-attention mechanism for prediction, and the entity relationship label of each word in the text is output to obtain the label sequence result of the current text.

4. A method for extracting basic audit business rules according to claim 1, characterized in that: The method of obtaining a set of three-tuples of basic audit business rules corresponding to the text of each rule to be extracted according to the entity relationship label includes: An audit business rule triple extraction model based on rule expressions is established, and the relationship triples in the text are extracted in the order of "entity-relationship-entity" in the label sequence. The audit business basic rule triple set corresponding to the text of each rule to be extracted is output to generate the basic audit business rules.

5. A device for extracting basic rules of audit business, characterized in that: The device comprises: The preprocessing unit is used to perform text preprocessing on the input data to obtain the input corpus; the input data includes: ordinary text corpus in the audit field, annotated training text corpus and text corpus for extracting rules; A word vector generation unit is used to construct a word vector generation model in the field of power auditing; based on Word2vec and the improved word frequency-inverse document frequency TF-IDF model, the word vector of each word in the input corpus text is obtained to obtain a word vector matrix for each text; A business rule learning unit is used to build a rule extraction model based on a multi-head self-attention mechanism, using the word vector matrix of the annotated training text corpus as input data, and continuously iterating the model to complete the model establishment and parameter tuning; An entity relationship extraction unit is used to extract the entity relationship label of each word in the text according to the rule extraction model after training, taking the word vector matrix of a text in the text corpus of the rule to be extracted as input data; An audit business output unit, used to obtain a set of audit business basic rule triples corresponding to the text of each rule to be extracted according to the entity relationship label; The word vector generating unit specifically includes: The stop word processing subunit is used to copy a copy of the input familiar corpus D and save it as corpus D′, and use the corpus D′ to train the Word2Vec word embedding model based on CBOW to obtain a word embedding representation E with semantic information = {e1, e2, ..., e N }; Use the constructed domain stop word dictionary to filter the stop words in corpus D and remove the stop words in the corpus; The model training subunit is used to train the TF-IDF model using the corpus D that has been filtered for stop words, and obtain the tfidf values ​​of all words in the corpus D: tfidf i，j =tf i，j ×idf i Among them, tfidf i，j Representation word t i For text d in corpus D j The importance of tf i，j Representation word t i In text d j The frequency of occurrence in i Representation word t i A measure of the general importance of i，j Representation word t i In text d j The number of occurrences in ;∑ k n k，j Indicates that in the text j The sum of the occurrence times of all words in the corpus; |D| represents the total number of texts in the corpus, |{d:t i ∈d j }| means containing word t i The total number of texts; For a text d in corpus D′ j The stop words in the text are initialized to 0 with tfidf values; the tfidf values ​​of all words in the text are added by 1 to obtain the current text d j The weight of all words where w i，j =tfidf i，j +1, indicating the weight of the i-th word; The text d j Input the trained Word2Vec word embedding model and map the word embedding representations of all words in the text. i , i = 1, 2, 3...n; embed the obtained words to represent the weights w corresponding to the words i，j Multiply them together to get the final word vector matrix of the text: V j ={v 1，j ,v 2，j ,…,v n，j } v i，j =w i，j ×e i Among them, v i，j Represents text d j The final word vector of the i-th word in ; w i，j Represents text d j The weight of the i-th word in e i Represents text d j The Word2Vec word embedding representation of the i-th word in; The word vector matrix processing subunit is used to repeatedly perform the above steps on all the texts in the corpus D′ to obtain the word vector matrix corresponding to each text; The business rule learning unit is specifically used for: The text feature extraction subunit is used to use the word vector matrix of the annotated training text corpus as input data, and use the word vector plus the position vector encoding to represent the order of the sequence; the position encoding adopts the calculation method of sin and cos, and the final addition result is sent to the simplified Transformer model to extract text features; The probability prediction subunit is used to send the text feature extraction results to the conditional random field CRF classifier and output the probability score of the label sequence: Where W = (w1, w2, ..., w n ) is the given text; y=(y1,y2,…,y n ) is the predicted label sequence; M is the state transfer matrix; Indicates that from label y i Move to label y i+1 probability; Indicates that the i-th word is labeled with label y i The probability of ; S(W, y) is the probability score of the input text W being predicted as the label sequence y; The probability acquisition subunit is used to train the model using the Adam optimization algorithm during the back-propagation process, continuously update the parameters, and finally obtain the label prediction model; the negative log-likelihood function is used as the model loss function, as follows: in, The input text W is predicted as a label sequence The probability score of ; S(W, y) is the probability score of the input text W being predicted as the true label sequence y.

6. A device for extracting basic rules for audit business according to claim 5, characterized in that: The pre-processing unit specifically comprises: The character processing subunit is used to remove the null characters in the input data text; The period processing subunit is used to replace the common period in the middle of a sentence and the period at the end of a sentence with a space; The special character processing subunit is used to retain the punctuation marks commonly used in official documents, keep numbers, letters, and uppercase and lowercase letter formats unchanged, and remove other punctuation marks and special characters; The word segmentation processing subunit is used to segment the input text using Chinese word segmentation tools based on the general dictionary and the independently constructed professional basic dictionary in the audit field to obtain the input idiom.

Images