A method for entity extraction from audit knowledge graph based on deep learning algorithm

Abstract

The present invention relates to an audit knowledge graph entity extraction method based on a deep learning algorithm, comprising the following steps: step 1, inputting a text sequence describing the review process in an audit record; step 2, based on the text sequence describing the review process in the audit record input in step 1, generating an encoder integrating audit dictionary knowledge, and calculating the encoder output feature vector; step 3, using a single-layer LSTM network to decode the text feature vector output by the encoder, and obtaining a fine-grained problem clue triple sequence containing latent entity relationships. The present invention adopts a problem clue triple extraction based on deep learning to extract fine-grained entity relationships based on detailed audit process details, better utilizes the interaction and constraints between entity models and relationship models, realizes structured extraction and storage of audit items, and improves the performance of unstructured long text entity relationship extraction models.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent auditing technology and relates to an audit knowledge graph entity extraction method, in particular, an audit knowledge graph entity extraction method based on a deep learning algorithm. Background Art

[0002] Audit records are first-hand materials that detail review logic and problem examples. The entity relationships contained in them, such as business rules, regulations, and business overviews, are a form of expression of domain knowledge. By extracting entity relationships, the abstract semantic data in audit records is converted into visual knowledge graph information, which not only facilitates audit professionals to learn and analyze the types, frequencies, and manifestations of problems, but also enables the reasonable integration and cross-domain analysis of audit information, providing valuable reference for digital auditing.

[0003] The wide application of knowledge graphs in various industries provides a new means for data mining. Its essence is a data network based on graphs, where nodes are entities and edges are relationships. The data texts and information such as domain entities, relationships, and events in different fields are very different. Information extraction technology needs to be improved and optimized for the audit field. Directly applying information extraction technology in a certain field will cause a double decline in accuracy and efficiency. Common methods for extracting text relationships in audit records include entity co-occurrence and machine learning methods based on feature vectors. The basis of the entity co-occurrence assumption is that if two entities often appear together, there is a relationship when the entities appear. The co-occurrence frequency score is used to eliminate the situation of misidentification caused by occasional co-occurrence. This solution can obtain a high recall rate, but the accuracy is not high and it cannot show the hidden relationship. The machine learning method based on feature vectors focuses on constructing feature vectors. The more feature values ​​there are, the higher the accuracy of the extracted relationship, but the greater the computational complexity, and the feature generalization ability is weak. It is necessary to redesign the corresponding features according to different fields such as engineering, marketing, finance, human resources and other majors.

[0004] As the depth and breadth of auditing extend, the acquisition of auditing knowledge faces difficulties such as diverse text sources, different structures, diverse entity relationship types, rapid update of relationship instances, sparse relationship features, and widespread reference. In particular, the description of the auditing process uses declarative sentences to record various related information. Different auditors have different writing styles, and manually combing auditing knowledge will consume a lot of labor costs. Through knowledge graphs, the text data involved in auditing records can be analyzed from a relational perspective to analyze the business development situation, which is more intuitive and efficient. Accurately mining and depicting key information in records will also greatly help cross-time domain and cross-professional domain analysis.

[0005] Therefore, the present invention proposes an entity extraction method for audit knowledge graph based on deep learning algorithm, and applies deep learning technology to realize the fine-grained extraction method of problem triples in audit records, providing a data basis for entity recognition and potential logical relationship analysis of complex sentences in audit records.

[0006] After searching, no public documents of the prior art identical or similar to the present invention were found. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and propose an audit knowledge graph entity extraction method based on a deep learning algorithm. For the detailed details of the audit process, the method uses the problem clue triple extraction based on deep learning that integrates domain dictionary knowledge to extract fine-grained entity relationship joint extraction, which better utilizes the interaction and constraints between the entity model and the relationship model, realizes structured extraction and storage of audit matters, and improves the performance of the unstructured long text entity relationship extraction model.

[0008] The present invention solves the practical problem by adopting the following technical solutions:

[0009] A method for extracting entities from an audit knowledge graph based on a deep learning algorithm comprises the following steps:

[0010] Step 1: Enter the text sequence describing the review process in the audit record;

[0011] Step 2: Based on the text sequence describing the review process in the audit record input in step 1, an encoder integrating the audit dictionary knowledge is generated, and the encoder output feature vector is calculated;

[0012] Step 3: Use a single-layer LSTM network to decode the text feature vector output by the encoder to obtain a sequence of fine-grained question clue triples containing latent entity relationships.

[0013] Moreover, the specific method of step 1 is:

[0014] The audit record contains the review process description text W = {w1,w2,…,w n}, where w i Represents the i-th word in the text, and n is the length of the input sequence.

[0015] Moreover, the specific steps of step 2 include:

[0016] Step 2.1) Extract the common features H of the text W :Use the BiLSTM model to learn the forward and backward features of the description text W at each moment, and concatenate the two outputs as the output vector of the BiLSTM. The output vector of the BiLSTM is encoded as the input vector of the BERTa model, and the hidden state of the last layer is used as the general semantic feature H describing the text WW ;

[0017] Step 2.2) Construct the audit feature professional dictionary L A : Find all possible subsequences of the input text that may form professional expressions through dictionary matching, represented by W[ij], where i and j are the starting and ending positions of the subsequence in the input text respectively;

[0018] Step 2.3) Construct the mask matrix M: where the element in the i-th row and j-th column is

[0019] Step 2.4) Calculate the text vector H with audit features A :Use Transformer encoding layer to general semantic features H W Perform vectorized calculations;

[0020] Step 2.5) Calculate the encoder output feature vector H E =λH W +(1-λ)H A .

[0021] Moreover, the specific steps of step 3 include:

[0022] Step 3.1) Calculate the accumulated triple vector decoded at the i-th moment

[0023] Step 3.2) Expand the current decoder vector For the matrix With the encoder output vector H E Concatenate into a new matrix H L ;

[0024] Step 3.3) Calculate the starting position of the head and tail entities:

[0025] in and is the probability distribution of the starting position of the head entity;

[0026] in and is the probability distribution of the starting position of the head entity, and the symbol: represents matrix concatenation;

[0027] Step 3.4) Head and tail entity vectorization: in and for and Hidden layer vector;

[0028] Step 3.5) Calculate the probability distribution of the relationship prediction at the i-th moment.

[0029] Step 3.6) Calculate the triplet vector at the i-th moment.

[0030] Step 3.7) The decoder terminates after N time steps or when the decoding at the current moment is "no relationship";

[0031] Step 3.8) Finally, we get a fine-grained question clue triple sequence T={t1 t2 …t n}.

[0032] Advantages and beneficial effects of the present invention:

[0033] 1. The present invention can realize structured extraction and storage of audit items. By extracting and structured storing various contents in the audit records, the retrieval efficiency of the audit records is improved in a triple manner, which helps to mine the deep semantic information of the records.

[0034] 2. The present invention can realize structured and fine-grained display of latent relationships between matters. The encoder-decoder structure is adopted, and the interactive information between entities and their relationships in the audit records is considered. The entities and relationships existing in the plot text are taken as a whole, first through the BiLSTM model and then through the BERTa model to obtain common features. After integrating the audit professional dictionary, the mask matrix is ​​used to obtain the encoder output vector with audit knowledge characteristics. The single-layer LSTM network decodes the encoder output vector with audit knowledge characteristics, and finally obtains a fine-grained problem clue triple sequence with a two-stage learning relationship, which effectively displays the latent relationship between matter entities. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a process flow chart of the present invention. DETAILED DESCRIPTION

[0036] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings:

[0037] A method for extracting entities from audit knowledge graphs based on deep learning algorithms, such as Figure 1 As shown, the following steps are included:

[0038] Step 1: Enter the text sequence describing the review process in the audit record;

[0039] The specific method of step 1 is:

[0040] The audit record contains the review process description text W = {w1,w2,…,w n}, where w i Represents the i-th word in the text, and n is the length of the input sequence.

[0041] Step 2: Based on the text sequence describing the review process in the audit record input in step 1, an encoder integrating the audit dictionary knowledge is generated, and the encoder output feature vector is calculated;

[0042] The specific steps of step 2 include:

[0043] Step 2.1) Extract the common features H of the text W :Use the BiLSTM model to learn the forward and backward features of the description text W at each moment, and concatenate the two outputs as the output vector of the BiLSTM. The output vector of the BiLSTM is encoded as the input vector of the BERTa model, and the hidden state of the last layer is used as the general semantic feature H describing the text W W ;

[0044] Step 2.2) Construct the audit feature professional dictionary L A : Find all possible subsequences of the input text that may form professional expressions through dictionary matching, represented by W[ij], where i and j are the starting and ending positions of the subsequence in the input text respectively;

[0045] Step 2.3) Construct the mask matrix M: where the element in the i-th row and j-th column is

[0046] Step 2.4) Calculate the text vector H with audit features A :Use Transformer encoding layer to general semantic features H W Perform vectorized calculations;

[0047] Step 2.5) Calculate the encoder output feature vector H E =λH W +(1-λ)H A .

[0048] Step 3: Use a single-layer LSTM network to decode the text feature vector H output by the encoder E Decoding is performed to obtain a sequence of triples of fine-grained question clues containing latent relations between entities.

[0049] Input is H E , the output is a question clue triple sequence T = {t1 t2 … t n}, where t i It represents the triplet decoded at the i-th time step in the decoding process, and n is the total number of triplets.

[0050] The specific steps of step 3 include:

[0051] Step 3.1) Calculate the accumulated triple vector decoded at the i-th moment

[0052] Step 3.2) Expand the current decoder vector For the matrix With the encoder output vector H E Concatenate into a new matrix H L ;

[0053] Step 3.3) Calculate the starting position of the head and tail entities:

[0054] in and is the probability distribution of the starting position of the head entity;

[0055] in and is the probability distribution of the starting position of the head entity, and the symbol: represents matrix concatenation.

[0056] Step 3.4) Head and tail entity vectorization: in and for and Hidden layer vector;

[0057] Step 3.5) Calculate the probability distribution of the relationship prediction at the i-th moment.

[0058] Step 3.6) Calculate the triplet vector at the i-th moment.

[0059] Step 3.7) The decoder terminates after N time steps or when the decoding at the current moment is "no relationship";

[0060] Step 3.8) Finally, we get a fine-grained question clue triple sequence T={t1 t2 …t n}.

[0061] The working principle of the present invention is:

[0062] The present invention uses the audit records of State Grid Tianjin Electric Power as the corpus source, designs a method for extracting key information from the audit records and a structured representation method, extracts problem clue entity relationships from unstructured or semi-structured texts, and realizes the discovery of entity hidden relationships.

[0063] The present invention provides an entity relationship extraction method for audit records. According to the information of the problem description of the audit process, the problem clue triples are extracted based on deep learning by integrating domain dictionary knowledge to jointly extract fine-grained entity relationships. The review process in the audit record records the business situation during the entire term in a narrative way, which involves various events such as funds, projects, contracts, etc. It is necessary to decompose the entire business plot description text into multiple triples to form a triple set of <entity, relationship, entity>. Each triple is used to represent the implementation process of a single event, forming a structured representation of the event plot, improving the efficiency of audit analysis and trial, and can also provide basic data for professional work such as comprehensive research and judgment, management opinions, and typical problem recommendations.

[0064] The fine-grained approach refers to the division of entity categories into two levels according to the central word and the expanded word. The central word refers to the smallest unit of entity naming and cannot be separated any further. The expanded word refers to the extension of the central word's meaning through professional terminology, and a mapping relationship can be established with the central word. For example, "project" is the central word, and "marketing investment project" is the expanded word. Fine-grained matching prioritizes the expanded word as a conceptual entity, greatly reducing the number of misrecognitions caused by the generalization of conceptual entities.

[0065] The fusion of domain dictionary knowledge refers to constructing a dictionary based on business professional names as audit knowledge features, collecting various professional vocabulary and common expressions in audit records, and the constructed professional name dictionary includes written and conventional expressions as much as possible.

[0066] It should be emphasized that the embodiments of the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation modes. Any other implementation modes derived by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.

Claims

1. A method for extracting entities from an audit knowledge graph based on a deep learning algorithm, characterized in that: The following steps are involved: Step 1: Enter the text sequence describing the review process in the audit record; Step 2: Based on the text sequence describing the review process in the audit record input in step 1, an encoder integrating the audit dictionary knowledge is generated, and the encoder output feature vector is calculated; Step 3: Use a single-layer LSTM network to decode the text feature vector output by the encoder to obtain a fine-grained question clue triple sequence containing the latent relationship of the entity; The specific steps of step 2 include: Step 2.1) Extract the common features H of the text W :Use the BiLSTM model to learn the forward and backward features of the description text W at each moment, concatenate the two outputs as the output vector of the BiLSTM, encode the output vector of the BiLSTM as the input vector of the BERTa model, and use the hidden state of the last layer as the universal semantic feature H describing the text W W ; Step 2.2) Construct the audit feature professional dictionary L A : Find all possible subsequences of the input text that may form professional expressions through dictionary matching, represented by W[ij], where i and j are the starting and ending positions of the subsequence in the input text respectively; Step 2.3) Construct the mask matrix M: where the element in the i-th row and j-th column is Step 2.4) Calculate the text vector H with audit features A :Use Transformer encoding layer to general semantic features H W Perform vectorized calculations; Step 2.5) Calculate the encoder output feature vector H E =λH W +(1-λ)H A ; The specific steps of step 3 include: Step 3.1) Calculate the accumulated triple vector decoded at the i-th moment Step 3.2) Expand the current decoder vector For the matrix With the encoder output vector H E Concatenate into a new matrix H L ; Step 3.3) Calculate the starting position of the head and tail entities: in and is the probability distribution of the starting position of the head entity; in and is the probability distribution of the starting position of the head entity, and the symbol: represents matrix concatenation; Step 3.4) Head and tail entity vectorization: in and for and Hidden layer vector; Step 3.5) Calculate the probability distribution of the relationship prediction at the i-th moment: Step 3.6) Calculate the triple vector at the i-th moment: Step 3.7) The decoder terminates after N time steps or when the decoding at the current moment is "no relationship"; Step 3.8) Finally, we get a fine-grained question clue triple sequence T = {t1t2…t n }.

2. According to the audit knowledge graph entity extraction method based on deep learning algorithm according to claim 1, it is characterized by: The specific method of step 1 is: The audit record contains the review process description text W = {w1,w2,…,w n }, where w i Represents the i-th word in the text, and n is the length of the input sequence.

Images