Audit problem and rectification measure classification method and system based on neural network model

Abstract

The invention provides an audit problem and rectification measure classification method and system based on a neural network model. The method comprises the following steps: obtaining a problem description field, marking a text, loading a pre-trained BERT model, and matching to a problem first-level classification standard; a keyword matching technology is introduced to be combined with a BERT model to realize a final-stage classification standard of the problem; obtaining a problem classification field, matching rectification measure classification according to the problem classification, and formulating rectification measures; and obtaining a rectification measure description field, realizing a rectification measure classification standard, and updating a rectification measure classification weight corresponding to the problem classification. According to the method, auditing personnel can be helped to judge auditing problem classification more accurately and quickly, common rectification measure types can be recommended according to description of rectification problems, and the auditing work efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a method and system for classifying audit issues and rectification measures based on a neural network model. Background Art

[0002] Internal audit refers to an independent and objective supervision and evaluation activity within an organization. It promotes the achievement of organizational goals by reviewing and evaluating the appropriateness, legality and effectiveness of business activities and internal controls. Audit problem classification is the process of grouping and categorizing the problems found during the audit according to certain standards and dimensions. This classification helps auditors quickly identify key areas and risk points of the audit, so as to focus their energy and resources on in-depth review. It also provides a standardized method to record and analyze audit findings to ensure the consistency and systematicness of the audit process.

[0003] Audit rectification refers to a series of measures taken to correct problems found during the audit process and prevent them from happening again in the future. The formulation of rectification measures is a process in which the responsible person for rectification refines measures, implements responsibilities, and establishes a long-term mechanism based on the description of the problem. A clear classification of rectification measures helps the responsible person of the rectification unit to formulate effective rectification measures, and also helps auditors determine the rationality of rectification measures.

[0004] In the prior art, auditors classify audit issues based on their personal experience, and the person responsible for rectification fills in rectification measures based on their personal experience. Manual classification of audit issues and manual filling in rectification measures is a meticulous and professional job, which usually relies on the relevant personnel to have corresponding professional knowledge and experience. This solution has the following problems:

[0005] 1. It may be difficult to form a unified standard for classification and rectification measures based on experience, which will affect the standardization and systematization of audit work;

[0006] 2. The classification of audit issues and manual filling of rectification measures that rely on manual experience are inaccurate, which affects the subsequent work;

[0007] 3. It takes a lot of time to recall and judge the problem categories, think about and select corrective measures, which will reduce the efficiency of audit work;

[0008] 4. Personal experience is difficult to fully record and pass on, which may lead to the loss of knowledge and skills.

[0009] Patent document CN115640402B discloses a multi-task artificial intelligence audit opinion generation method, device and readable medium, which relates to the technical field of auditing. The method includes: obtaining a second audit description and a second reference provision, the second audit description is the processed first audit description, the second reference provision is the processed first reference provision, and the first audit description and the first reference provision both have identifiers; introducing parameters to the second audit description and the second reference provision using an attention mechanism to generate an attention mechanism output; using a neural network to process the attention mechanism output to simultaneously classify audit issues and generate audit opinions. The present invention can realize automatic classification of audit issues and simultaneous generation of audit opinions, further improve the correlation between audit issues and audit opinions, and enhance the analytical capabilities of the audit model.

[0010] Comparison of technical points: This invention uses the encoder-decoder BART to classify audit issues and generate audit opinions. The BART model performs well in text generation, but may not be as good as other models such as BERT in classification tasks. It should also be noted that the BART model is usually large, so the computational cost may need to be considered when resources are limited. Based on the BERT algorithm model, this two-way training model can capture richer contextual information and is more efficient when processing shorter text sequences because the autoregressive generation step of the decoder is not required. In addition, the application scenarios of this invention do not include the classification of corrective measures. The corresponding type of corrective measures cannot be recommended according to the type of corrective problem, and the application scenarios are limited. The classification evaluation results of this invention are not accurate enough. Summary of the invention

[0011] In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for classifying audit issues and corrective measures based on a neural network model.

[0012] According to the present invention, a method for classifying audit issues and rectification measures based on a neural network model includes:

[0013] Step S1: Get the question description field, tokenize the text, load the pre-trained BERT model, and match it to the first-level classification standard of the question;

[0014] Step S2: Introduce keyword matching technology and combine it with the BERT model to implement the final classification standard of the problem;

[0015] Step S3: Obtain the problem classification field, match the rectification measure classification according to the problem classification, and formulate rectification measures;

[0016] Step S4: Obtain the corrective measure description field, implement the corrective measure classification standard, and update the corrective measure classification weight corresponding to the problem classification.

[0017] Preferably, in step S1:

[0018] Get the problem description field:

[0019] The BERT model is based on the Transformer architecture, which is completely based on the attention mechanism. It processes sequence data through the self-attention layer. The model preprocesses the question description text, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values.

[0020] The pre-training tasks of the BERT model include the masked language model and Next Sentence Prediction, and its mathematical formula is as follows:

[0021] Masked Language Model:

[0022]

[0023] Among them, P(x) represents the target probability distribution, m i Indicates whether the i-th token is blocked, x ＜i represents all tokens before the i-th token; n represents the number of words in the sentence, the length of the sequence; x i represents the i-th word; θ represents the parameters of the model;

[0024] Next Sentence Prediction:

[0025]

[0026] Among them, s and t represent two sentences respectively, φ(s, t) represents the vector representation of the sentence pair, and t rand represents a random sentence from a different document than s;

[0027] In the Transformer encoder, the data first passes through the self-attention module to obtain a weighted feature vector Z:

[0028]

[0029] Among them, d k is the number of elements in each column of the K matrix;

[0030] After obtaining the feature vector, it is sent to the next fully connected network module of the encoder, expressed as:

[0031] FFN(Z)=max(0,ZW1+b1)W2+b2

[0032] Among them, W1 is the weight matrix of the first fully connected layer, b1 is the bias term of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer;

[0033] In self-attention, each word has 3 different vectors, namely Query vector Q, Key vector K, and Value vector V, all of length 64, which are respectively obtained by multiplying the embedding vector X by 3 different weight matrices W. Q , W K , W V It turns out that the sizes of the three matrices are the same, all 512x64;

[0034] The calculation process of Attention:

[0035] Convert the input word into an embedding vector; obtain Q, K, and V vectors based on the embedding vector; calculate a score for each vector, score = q·k; normalize the score, and divide the score by Apply softmax activation function to score:

[0036]

[0037] in, represents the exponential form of a scoring function between input x and candidate output y; the scoring function measures the degree of match between input x and candidate output y, j represents the index of the candidate output, ranging from 1 to n, where n is the number of all possible candidate outputs;

[0038] Softmax multiplies the Value value v to get the weighted score of each input vector; after adding them together, the final output result is: z = ∑v;

[0039] In the decoder, Q comes from the previous output of the decoder, and K and V come from the output of the encoder. The decoding process is a sequential operation process. When decoding the kth feature vector, the decoding results of the k-1th and previous ones are seen;

[0040] After decoding by the decoder, the decoded feature vector passes through a fully connected layer with a softmax activation function to obtain an output vector reflecting the probability of each word;

[0041] Position encoding is introduced when encoding word vectors. The position information of words is added to the word vectors to distinguish different words. Position encoding is a d model The specific encoding formula is as follows:

[0042]

[0043] Among them, pos represents the position of the word, and i represents the dimension of the word;

[0044] In order to make the model converge faster, the model uses Layer Normalization, the calculation formula is as follows:

[0045]

[0046] Among them, μ i is the mean of the ith sample, χ ij represents the jth eigenvalue of the ith sample, which is an element in the input data matrix, where i is the index of the sample and j is the index of the feature, σ is the standard deviation of all samples, m represents the number of features, and ∈ is a small positive number to avoid division errors caused by the standard deviation being zero;

[0047] Automatically match to the first level classification criteria of the problem:

[0048] Use the BERT model's tokenizer to tokenize the text and convert it into the format preset by the model, including adding tags to distinguish the beginning and end of sentences; load the pre-trained BERT model and build a classification head based on the BERT model. The first-level classification is a fully connected layer used to map the output of the BERT model to preset specific categories. The first-level classification standard is based on the eighteen internal control guidelines of the enterprise.

[0049] Preferably, in step S2:

[0050] When the sample size is lower than the preset standard, keyword matching technology is introduced in combination with the BERT model to achieve the final classification standard of the problem; when the amount of classified training data is less than the preset standard, keyword matching technology is introduced for pre-classification;

[0051] When a question is received, a regular expression is used to match the number of keywords in each final category in the question description;

[0052] The final classification is a subclass of the first-level classification standard, and the standard is formulated according to different situations of the enterprise; if there is an untrained classification in the final label, and the keyword corresponding to this untrained classification exists in the problem description, the final classification with the most keywords in the untrained classification label is directly returned as the final result; if there is no keyword corresponding to this untrained classification, no result is returned and the model is used to make probability predictions for the trained classification;

[0053] Automatically match the final classification criteria of the problem:

[0054] First, use the keyword list maintained in the background to match, and return the result directly when the keyword is matched. If there is no result, use the BERT fine-tuned model to predict the classification;

[0055] For the trained categories, after matching keywords, the final label model predicts the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned.

[0056] Preferably, in step S3:

[0057] The backend business locates and obtains the problem classification field through API calls to ensure that the classification label or code corresponding to each problem is captured;

[0058] Match the rectification measures classification according to the problem classification. According to business needs, pre-classify the rectification measures, analyze the historical data of problem classification and rectification measures matching, match the problem classification results with the corresponding rectification measures types according to the principle of maximum probability, predict the most appropriate rectification measures type for specific problem categories, perform further feature engineering on the problem text, extract preset information for rectification measures type matching, and the model selects or recommends several most appropriate rectification measures types from the rectification measures library based on the problem classification results; formulate corresponding rectification measures for different types of problems through the classification of rectification measures.

[0059] Preferably, in step S4:

[0060] Get the corrective action description field:

[0061] BERT is based on the Transformer architecture, which is completely based on the attention mechanism and processes sequence data through the self-attention layer. The model pre-processes the text describing the rectification measures, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values.

[0062] Introducing keyword matching technology combined with the BERT model to implement the classification standards for rectification measures:

[0063] Use regular expressions to match the number of keywords for each first-level label category in the corrective measure description; if there is an untrained category in the label, and the corrective measure description contains the keyword corresponding to the untrained category, then the final classification result needs to include all corrective measure categories with untrained category labels; if the corrective measure description does not contain the keyword corresponding to the untrained category, then the final classification result does not include the corrective measure category with untrained category labels;

[0064] For the trained categories, after keyword matching, the label model is used to predict the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned;

[0065] Update the classification weight of the corrective measure corresponding to the problem classification, and corresponding to the result of the corrective measure classification, re-predict and update the corresponding corrective measure type weight, i.e., probability, and re-predict the weight of the corrective measure type to update the corresponding corrective measure type weight.

[0066] According to the present invention, a classification system for audit issues and rectification measures based on a neural network model is provided, comprising:

[0067] Module M1: Get the question description field, tokenize the text, load the pre-trained BERT model, and match it to the first-level classification standard of the question;

[0068] Module M2: Introducing keyword matching technology combined with the BERT model to implement the final classification standard of the problem;

[0069] Module M3: Obtain the problem classification field, match the rectification measures classification according to the problem classification, and formulate rectification measures;

[0070] Module M4: Obtain the corrective measure description field, implement the corrective measure classification standard, and update the corrective measure classification weight corresponding to the problem classification.

[0071] Preferably, in the module M1:

[0072] Get the problem description field:

[0073] The BERT model is based on the Transformer architecture, which is completely based on the attention mechanism. It processes sequence data through the self-attention layer. The model preprocesses the question description text, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values.

[0074] The pre-training tasks of the BERT model include the masked language model and Next Sentence Prediction, and its mathematical formula is as follows:

[0075] Masked Language Model:

[0076]

[0077] Among them, P(x) represents the target probability distribution, m iIndicates whether the i-th token is blocked, x ＜i represents all tokens before the i-th token; n represents the number of words in the sentence, the length of the sequence; x i represents the i-th word; θ represents the parameters of the model;

[0078] Next Sentence Prediction:

[0079]

[0080] Among them, s and t represent two sentences respectively, φ(s, t) represents the vector representation of the sentence pair, and t rand represents a random sentence from a different document than s;

[0081] In the Transformer encoder, the data first passes through the self-attention module to obtain a weighted feature vector Z:

[0082]

[0083] Among them, d k is the number of elements in each column of the K matrix;

[0084] After obtaining the feature vector, it is sent to the next fully connected network module of the encoder, expressed as:

[0085] FFN(Z)=max(0,ZW1+b1)W2+b2

[0086] Among them, W1 is the weight matrix of the first fully connected layer, b1 is the bias term of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer;

[0087] In self-attention, each word has 3 different vectors, namely Query vector Q, Key vector K, and Value vector V, all of length 64, which are respectively obtained by multiplying the embedding vector X by 3 different weight matrices W. Q , W K , W V It turns out that the sizes of the three matrices are the same, all 512x64;

[0088] The calculation process of Attention:

[0089] Convert the input word into an embedding vector; obtain Q, K, and V vectors based on the embedding vector; calculate a score for each vector, score = q·k; normalize the score, and divide the score by Apply softmax activation function to score:

[0090]

[0091] in, represents the exponential form of a scoring function between input x and candidate output y; the scoring function measures the degree of match between input x and candidate output y, j represents the index of the candidate output, ranging from 1 to n, where n is the number of all possible candidate outputs;

[0092] Softmax multiplies the Value value v to get the weighted score of each input vector; after adding, the final output result is: z = ∑υ;

[0093] In the decoder, Q comes from the previous output of the decoder, and K and V come from the output of the encoder. The decoding process is a sequential operation process. When decoding the kth feature vector, the decoding results of the k-1th and previous ones are seen;

[0094] After decoding by the decoder, the decoded feature vector passes through a fully connected layer with a softmax activation function to obtain an output vector reflecting the probability of each word;

[0095] Position encoding is introduced when encoding word vectors. The position information of words is added to the word vectors to distinguish different words. Position encoding is a d model The specific encoding formula is as follows:

[0096]

[0097] Among them, pos represents the position of the word, and i represents the dimension of the word;

[0098] In order to make the model converge faster, the model uses Layer Normalization, the calculation formula is as follows:

[0099]

[0100] Among them, μ i is the mean of the ith sample, χ ij represents the jth eigenvalue of the ith sample, which is an element in the input data matrix, where i is the index of the sample and j is the index of the feature, σ is the standard deviation of all samples, m represents the number of features, and ∈ is a small positive number to avoid division errors caused by the standard deviation being zero;

[0101] Automatically match to the first level classification criteria of the problem:

[0102] Use the BERT model's tokenizer to tokenize the text and convert it into the format preset by the model, including adding tags to distinguish the beginning and end of sentences; load the pre-trained BERT model and build a classification head based on the BERT model. The first-level classification is a fully connected layer used to map the output of the BERT model to preset specific categories. The first-level classification standard is based on the eighteen internal control guidelines of the enterprise.

[0103] Preferably, in the module M2:

[0104] When the sample size is lower than the preset standard, keyword matching technology is introduced in combination with the BERT model to achieve the final classification standard of the problem; when the amount of classified training data is less than the preset standard, keyword matching technology is introduced for pre-classification;

[0105] When a question is received, a regular expression is used to match the number of keywords in each final category in the question description;

[0106] The final classification is a subclass of the first-level classification standard, and the standard is formulated according to different situations of the enterprise; if there is an untrained classification in the final label, and the keyword corresponding to this untrained classification exists in the problem description, the final classification with the most keywords in the untrained classification label is directly returned as the final result; if there is no keyword corresponding to this untrained classification, no result is returned and the model is used to make probability predictions for the trained classification;

[0107] Automatically match the final classification criteria of the problem:

[0108] First, use the keyword list maintained in the background to match, and return the result directly when the keyword is matched. If there is no result, use the BERT fine-tuned model to predict the classification;

[0109] For the trained categories, after matching keywords, the final label model predicts the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned.

[0110] Preferably, in the module M3:

[0111] The backend business locates and obtains the problem classification field through API calls to ensure that the classification label or code corresponding to each problem is captured;

[0112] Match the rectification measures classification according to the problem classification. According to business needs, pre-classify the rectification measures, analyze the historical data of problem classification and rectification measures matching, match the problem classification results with the corresponding rectification measures types according to the principle of maximum probability, predict the most appropriate rectification measures type for specific problem categories, perform further feature engineering on the problem text, extract preset information for rectification measures type matching, and the model selects or recommends several most appropriate rectification measures types from the rectification measures library based on the problem classification results; formulate corresponding rectification measures for different types of problems through the classification of rectification measures.

[0113] Preferably, in the module M4:

[0114] Get the corrective action description field:

[0115] BERT is based on the Transformer architecture, which is completely based on the attention mechanism and processes sequence data through the self-attention layer. The model pre-processes the text describing the rectification measures, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values.

[0116] Introducing keyword matching technology combined with the BERT model to implement the classification standards for rectification measures:

[0117] Use regular expressions to match the number of keywords for each first-level label category in the corrective measure description; if there is an untrained category in the label, and the corrective measure description contains the keyword corresponding to the untrained category, then the final classification result needs to include all corrective measure categories with untrained category labels; if the corrective measure description does not contain the keyword corresponding to the untrained category, then the final classification result does not include the corrective measure category with untrained category labels;

[0118] For the trained categories, after keyword matching, the label model is used to predict the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned;

[0119] Update the classification weight of the corrective measure corresponding to the problem classification, and corresponding to the result of the corrective measure classification, re-predict and update the corresponding corrective measure type weight, i.e., probability, and re-predict the weight of the corrective measure type to update the corresponding corrective measure type weight.

[0120] Compared with the prior art, the present invention has the following beneficial effects:

[0121] 1. The present invention relates to an intelligent classification device for improving audit efficiency and accuracy, which is applicable to an internal audit digital system and can help auditors to more accurately and quickly determine the classification of audit issues. It can also recommend commonly used rectification measures based on the description of the rectification issues, thereby improving the efficiency of audit work.

[0122] 2. The present invention enables auditors to input audit working papers and corresponding audit questions. The system will automatically recommend the problem type according to the problem description. Auditors can also modify the type. Intelligent devices can help auditors determine the problem classification more accurately and quickly, and realize the sedimentation and reuse of audit knowledge.

[0123] 3. When the person responsible for rectification is providing rectification feedback, the present invention can recommend the types of rectification measures needed based on the description of the rectification problem, help the rectification unit to formulate more effective measures, and promote the efficient implementation of the rectification work. BRIEF DESCRIPTION OF THE DRAWINGS

[0124] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0125] Figure 1 This is a schematic diagram of the implementation structure of the intelligent classification application of audit issues of the present invention;

[0126] Figure 2 It is a schematic diagram of the implementation structure of the intelligent classification application of the rectification measures of the present invention;

[0127] Figure 3 It is a flow chart;

[0128] Figure 4 A schematic diagram of the audit question classification standard;

[0129] Figure 5 This is a schematic diagram of the classification standards for corrective measures;

[0130] Figure 6 Network structure diagram for AI-based audit problem classification and corrective action classification. DETAILED DESCRIPTION

[0131] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0132] Embodiment 1:

[0133] The present invention is based on the bidirectional encoder model Bert, and performs deep semantic understanding through model pre-training and fine-tuning to process complex text data. The audit problem classification application can automatically classify the text description into the corresponding event; the rectification measure classification application can automatically classify the text description into the corresponding rectification measure category, and use the historical audit rectification data to perform secondary development and fine-tuning, so that the model can automatically give how to classify the audit description.

[0134] According to a neural network model-based audit problem and rectification measure classification method provided by the present invention, Figure 1-Figure 3 As shown, including:

[0135] Step S1: Get the question description field, tokenize the text, load the pre-trained BERT model, and match it to the first-level classification standard of the question;

[0136] Specifically, in step S1:

[0137] Get the problem description field:

[0138] The BERT model is based on the Transformer architecture, which is completely based on the attention mechanism. It processes sequence data through the self-attention layer. The model preprocesses the question description text, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values.

[0139] The pre-training tasks of the BERT model include the masked language model and Next Sentence Prediction, and its mathematical formula is as follows:

[0140] Masked Language Model:

[0141]

[0142] Among them, P(x) represents the target probability distribution, m i Indicates whether the i-th token is blocked, x ＜i represents all tokens before the i-th token; n represents the number of words in the sentence, the length of the sequence; x i represents the i-th word; θ represents the parameters of the model;

[0143] Next Sentence Prediction:

[0144]

[0145] Among them, s and t represent two sentences respectively, φ(s, t) represents the vector representation of the sentence pair, and trand represents a random sentence from a different document than s;

[0146] In the Transformer encoder, the data first passes through the self-attention module to obtain a weighted feature vector Z:

[0147]

[0148] Among them, d k is the number of elements in each column of the K matrix;

[0149] After obtaining the feature vector, it is sent to the next fully connected network module of the encoder, expressed as:

[0150] FFN(Z)=max(0,ZW1+b1)W2+b2

[0151] Among them, W1 is the weight matrix of the first fully connected layer, b1 is the bias term of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer;

[0152] In self-attention, each word has 3 different vectors, namely Query vector Q, Key vector K, and Value vector V, all of length 64, which are respectively obtained by multiplying the embedding vector X by 3 different weight matrices W. Q , W K , W V It turns out that the sizes of the three matrices are the same, all 512x64;

[0153] The calculation process of Attention:

[0154] Convert the input word into an embedding vector; obtain Q, K, and V vectors based on the embedding vector; calculate a score for each vector, score = q·k; normalize the score, and divide the score by Apply softmax activation function to score:

[0155]

[0156] in, represents the exponential form of a scoring function between input x and candidate output y; the scoring function measures the degree of match between input x and candidate output y, j represents the index of the candidate output, ranging from 1 to n, where n is the number of all possible candidate outputs;

[0157] Softmax multiplies the Value value v to get the weighted score of each input vector; after adding, the final output result is: z = Σv;

[0158] In the decoder, Q comes from the previous output of the decoder, and K and V come from the output of the encoder. The decoding process is a sequential operation process. When decoding the kth feature vector, the decoding results of the k-1th and previous ones are seen;

[0159] After decoding by the decoder, the decoded feature vector passes through a fully connected layer with a softmax activation function to obtain an output vector reflecting the probability of each word;

[0160] Position encoding is introduced when encoding word vectors. The position information of words is added to the word vectors to distinguish different words. Position encoding is a d model The specific encoding formula is as follows:

[0161]

[0162] Among them, pos represents the position of the word, and i represents the dimension of the word;

[0163] In order to make the model converge faster, the model uses Layer Normalization, the calculation formula is as follows:

[0164]

[0165] Among them, μ i is the mean of the ith sample, χ ij represents the jth eigenvalue of the ith sample, which is an element in the input data matrix, where i is the index of the sample and j is the index of the feature, σ is the standard deviation of all samples, m represents the number of features, and ∈ is a small positive number to avoid division errors caused by the standard deviation being zero;

[0166] Automatically match to the first level classification criteria of the problem:

[0167] Use the BERT model's tokenizer to tokenize the text and convert it into the format preset by the model, including adding tags to distinguish the beginning and end of sentences; load the pre-trained BERT model and build a classification head based on the BERT model. The first-level classification is a fully connected layer used to map the output of the BERT model to preset specific categories. The first-level classification standard is based on the eighteen internal control guidelines of the enterprise.

[0168] Step S2: Introduce keyword matching technology and combine it with the BERT model to implement the final classification standard of the problem;

[0169] Specifically, in step S2:

[0170] When the sample size is lower than the preset standard, keyword matching technology is introduced in combination with the BERT model to achieve the final classification standard of the problem; when the amount of classified training data is less than the preset standard, keyword matching technology is introduced for pre-classification;

[0171] When a question is received, a regular expression is used to match the number of keywords in each final category in the question description;

[0172] The final classification is a subclass of the first-level classification standard, and the standard is formulated according to different situations of the enterprise; if there is an untrained classification in the final label, and the keyword corresponding to this untrained classification exists in the problem description, the final classification with the most keywords in the untrained classification label is directly returned as the final result; if there is no keyword corresponding to this untrained classification, no result is returned and the model is used to make probability predictions for the trained classification;

[0173] Automatically match the final classification criteria of the problem:

[0174] First, use the keyword list maintained in the background to match, and return the result directly when the keyword is matched. If there is no result, use the BERT fine-tuned model to predict the classification;

[0175] For the trained categories, after matching keywords, the final label model predicts the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned.

[0176] Step S3: Obtain the problem classification field, match the rectification measure classification according to the problem classification, and formulate rectification measures;

[0177] Specifically, in step S3:

[0178] The backend business locates and obtains the problem classification field through API calls to ensure that the classification label or code corresponding to each problem is captured;

[0179] Match the rectification measures classification according to the problem classification. According to business needs, pre-classify the rectification measures, analyze the historical data of problem classification and rectification measures matching, match the problem classification results with the corresponding rectification measures types according to the principle of maximum probability, predict the most appropriate rectification measures type for specific problem categories, perform further feature engineering on the problem text, extract preset information for rectification measures type matching, and the model selects or recommends several most appropriate rectification measures types from the rectification measures library based on the problem classification results; formulate corresponding rectification measures for different types of problems through the classification of rectification measures.

[0180] Step S4: Obtain the corrective measure description field, implement the corrective measure classification standard, and update the corrective measure classification weight corresponding to the problem classification.

[0181] Specifically, in step S4:

[0182] Get the corrective action description field:

[0183] BERT is based on the Transformer architecture, which is completely based on the attention mechanism and processes sequence data through the self-attention layer. The model pre-processes the text describing the rectification measures, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values.

[0184] Introducing keyword matching technology combined with the BERT model to implement the classification standards for rectification measures:

[0185] Use regular expressions to match the number of keywords for each first-level label category in the corrective measure description; if there is an untrained category in the label, and the corrective measure description contains the keyword corresponding to the untrained category, then the final classification result needs to include all corrective measure categories with untrained category labels; if the corrective measure description does not contain the keyword corresponding to the untrained category, then the final classification result does not include the corrective measure category with untrained category labels;

[0186] For the trained categories, after keyword matching, the label model is used to predict the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned;

[0187] Update the classification weight of the corrective measure corresponding to the problem classification, and corresponding to the result of the corrective measure classification, re-predict and update the corresponding corrective measure type weight, i.e., probability, and re-predict the weight of the corrective measure type to update the corresponding corrective measure type weight.

[0188] Embodiment 2:

[0189] Embodiment 2 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0190] The present invention also provides an audit issue and corrective measures classification system based on a neural network model. The audit issue and corrective measures classification system based on a neural network model can be implemented by executing the process steps of the audit issue and corrective measures classification method based on the neural network model, that is, those skilled in the art can understand the audit issue and corrective measures classification method based on the neural network model as a preferred implementation of the audit issue and corrective measures classification system based on the neural network model.

[0191] According to the present invention, a classification system for audit issues and rectification measures based on a neural network model is provided, comprising:

[0192] Module M1: Get the question description field, tokenize the text, load the pre-trained BERT model, and match it to the first-level classification standard of the question;

[0193] Specifically, in the module M1:

[0194] Get the problem description field:

[0195] The BERT model is based on the Transformer architecture, which is completely based on the attention mechanism. It processes sequence data through the self-attention layer. The model preprocesses the question description text, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values.

[0196] The pre-training tasks of the BERT model include the masked language model and Next Sentence Prediction, and its mathematical formula is as follows:

[0197] Masked Language Model:

[0198]

[0199] Among them, P(x) represents the target probability distribution, m i Indicates whether the i-th token is blocked, x ＜i represents all tokens before the i-th token; n represents the number of words in the sentence, the length of the sequence; x i represents the i-th word; θ represents the parameters of the model;

[0200] Next Sentence Prediction:

[0201]

[0202] Among them, s and t represent two sentences respectively, φ(s, t) represents the vector representation of the sentence pair, and t randrepresents a random sentence from a different document than s;

[0203] In the Transformer encoder, the data first passes through the self-attention module to obtain a weighted feature vector Z:

[0204]

[0205] Among them, d k is the number of elements in each column of the K matrix;

[0206] After obtaining the feature vector, it is sent to the next fully connected network module of the encoder, expressed as:

[0207] FFN(Z)=max(0,ZW1+b1)W2+b2

[0208] Among them, W1 is the weight matrix of the first fully connected layer, b1 is the bias term of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer;

[0209] In self-attention, each word has 3 different vectors, namely Query vector Q, Key vector K, and Value vector V, all of length 64, which are respectively obtained by multiplying the embedding vector X by 3 different weight matrices W. Q , W K , W V It turns out that the sizes of the three matrices are the same, all 512x64;

[0210] The calculation process of Attention:

[0211] Convert the input word into an embedding vector; obtain Q, K, and V vectors based on the embedding vector; calculate a score for each vector, score = q·k; normalize the score, and divide the score by Apply softmax activation function to score:

[0212]

[0213] in, represents the exponential form of a scoring function between input x and candidate output y; the scoring function measures the degree of match between input x and candidate output y, j represents the index of the candidate output, ranging from 1 to n, where n is the number of all possible candidate outputs;

[0214] Softmax multiplies the Value value v to get the weighted score of each input vector; after adding them together, the final output result is: z = ∑v;

[0215] In the decoder, Q comes from the previous output of the decoder, and K and V come from the output of the encoder. The decoding process is a sequential operation process. When decoding the kth feature vector, the decoding results of the k-1th and previous ones are seen;

[0216] After decoding by the decoder, the decoded feature vector passes through a fully connected layer with a softmax activation function to obtain an output vector reflecting the probability of each word;

[0217] Position encoding is introduced when encoding word vectors. The position information of words is added to the word vectors to distinguish different words. Position encoding is a d model The specific encoding formula is as follows:

[0218]

[0219] Among them, pos represents the position of the word, and i represents the dimension of the word;

[0220] In order to make the model converge faster, the model uses Layer Normalization, the calculation formula is as follows:

[0221]

[0222] Among them, μ i is the mean of the ith sample, χ ij represents the jth eigenvalue of the ith sample, which is an element in the input data matrix, where i is the index of the sample and j is the index of the feature, σ is the standard deviation of all samples, m represents the number of features, and ∈ is a small positive number to avoid division errors caused by the standard deviation being zero;

[0223] Automatically match to the first level classification criteria of the problem:

[0224] Use the BERT model's tokenizer to tokenize the text and convert it into the format preset by the model, including adding tags to distinguish the beginning and end of sentences; load the pre-trained BERT model and build a classification head based on the BERT model. The first-level classification is a fully connected layer used to map the output of the BERT model to preset specific categories. The first-level classification standard is based on the eighteen internal control guidelines of the enterprise.

[0225] Module M2: Introducing keyword matching technology combined with the BERT model to implement the final classification standard of the problem;

[0226] Specifically, in the module M2:

[0227] When the sample size is lower than the preset standard, keyword matching technology is introduced in combination with the BERT model to achieve the final classification standard of the problem; when the amount of classified training data is less than the preset standard, keyword matching technology is introduced for pre-classification;

[0228] When a question is received, a regular expression is used to match the number of keywords in each final category in the question description;

[0229] The final classification is a subclass of the first-level classification standard, and the standard is formulated according to different situations of the enterprise; if there is an untrained classification in the final label, and the keyword corresponding to this untrained classification exists in the problem description, the final classification with the most keywords in the untrained classification label is directly returned as the final result; if there is no keyword corresponding to this untrained classification, no result is returned and the model is used to make probability predictions for the trained classification;

[0230] Automatically match the final classification criteria of the problem:

[0231] First, use the keyword list maintained in the background to match, and return the result directly when the keyword is matched. If there is no result, use the BERT fine-tuned model to predict the classification;

[0232] For the trained categories, after matching keywords, the final label model predicts the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned.

[0233] Module M3: Obtain the problem classification field, match the rectification measures classification according to the problem classification, and formulate rectification measures;

[0234] Specifically, in the module M3:

[0235] The backend business locates and obtains the problem classification field through API calls to ensure that the classification label or code corresponding to each problem is captured;

[0236] Match the rectification measures classification according to the problem classification. According to business needs, pre-classify the rectification measures, analyze the historical data of problem classification and rectification measures matching, match the problem classification results with the corresponding rectification measures types according to the principle of maximum probability, predict the most appropriate rectification measures type for specific problem categories, perform further feature engineering on the problem text, extract preset information for rectification measures type matching, and the model selects or recommends several most appropriate rectification measures types from the rectification measures library based on the problem classification results; formulate corresponding rectification measures for different types of problems through the classification of rectification measures.

[0237] Module M4: Obtain the corrective measure description field, implement the corrective measure classification standard, and update the corrective measure classification weight corresponding to the problem classification.

[0238] Specifically, in the module M4:

[0239] Get the corrective action description field:

[0240] BERT is based on the Transformer architecture, which is completely based on the attention mechanism and processes sequence data through the self-attention layer. The model pre-processes the text describing the rectification measures, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values.

[0241] Introducing keyword matching technology combined with the BERT model to implement the classification standards for rectification measures:

[0242] Use regular expressions to match the number of keywords for each first-level label category in the corrective measure description; if there is an untrained category in the label, and the corrective measure description contains the keyword corresponding to the untrained category, then the final classification result needs to include all corrective measure categories with untrained category labels; if the corrective measure description does not contain the keyword corresponding to the untrained category, then the final classification result does not include the corrective measure category with untrained category labels;

[0243] For the trained categories, after keyword matching, the label model is used to predict the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned;

[0244] Update the classification weight of the corrective measure corresponding to the problem classification, and corresponding to the result of the corrective measure classification, re-predict and update the corresponding corrective measure type weight, i.e., probability, and re-predict the weight of the corrective measure type to update the corresponding corrective measure type weight.

[0245] Embodiment 3:

[0246] Embodiment 3 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0247] The present invention provides an intelligent application to assist in the classification of audit issues and the formulation of corrective measures. The implementation structure diagram is as follows: Figure 1 as well as Figure 2 As shown, the steps and processes are shown in Figure 3 .

[0248] 101 Get the audit question description field

[0249] BERT is based on the Transformer architecture, which is completely based on the attention mechanism and processes sequence data through self-attention layers. When understanding and processing large amounts of data, the concept of attention mechanism can be used to improve efficiency and accuracy. The model preprocesses the audit problem description text, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding special tags, padding, and truncation, so that the model can distinguish between actual input and padding values.

[0250] The pre-training task of the BERT model mainly includes two parts: Masked Language Model and Next SentencePrediction. Its mathematical formula is as follows:

[0251] (1) Masked Language Model:

[0252]

[0253] Among them, P(x) represents the target probability distribution, m i Indicates whether the i-th token is masked, X ＜i Represents all tokens before the i-th token.

[0254] (2)Next Sentence Prediction:

[0255]

[0256] Among them, s and t represent two sentences respectively, φ(s, t) represents the vector representation of the sentence pair, and t rand represents a random sentence from a different document than s.

[0257] In the Transformer encoder, the data first passes through the self-attention module to obtain a weighted feature vector Z:

[0258]

[0259] After obtaining the feature vector, it will be sent to the next fully connected network module of the encoder, which can be expressed as:

[0260] FFN(Z)=max(0,ZW1+b1)W2+b2

[0261] In self-attention, each word has 3 different vectors, namely Query vector (Q), Key vector (K), and Value vector (V), all of length 64. They are obtained by multiplying the embedding vector X by 3 different weight matrices W. Q , W K , Q V It turns out that the sizes of the three matrices are the same, all 512x 64.

[0262] The calculation process of Attention is divided into 7 steps:

[0263] 1. Convert the input word into an embedding vector;

[0264] 2. Obtain three vectors q, k, and v based on the embedded vector;

[0265] 3. Calculate a score for each vector, score = q·k;

[0266] 4. In order to stabilize the gradient, normalize the score, that is, divide it by

[0267] 5. Apply softmax activation function to score:

[0268] 6. Softmax point multiplication of Value value v to obtain the weighted score of each input vector;

[0269] 7. After adding, we get the final output result: z = Σv

[0270] In the decoder, Q comes from the previous output of the decoder, and K and V come from the output of the encoder. The decoding process is a sequential operation process, that is, when decoding the kth feature vector, only the decoding results of the k-1th and before can be seen.

[0271] After decoding by the decoder, the decoded feature vector passes through a fully connected layer with a softmax activation function to obtain an output vector reflecting the probability of each word.

[0272] When encoding word vectors, position encoding is introduced, which adds the position information of the word to the word vector so that different words can be distinguished. Position encoding is usually a d model This makes it easier to perform unit addition operations with word vectors. The specific encoding formula is as follows:

[0273]

[0274] In the above formula, pos represents the position of the word, and i represents the dimension of the word.

[0275] In order to make the model converge faster, the model uses Layer Normalization. The calculation formula is as follows:

[0276]

[0277] 102 Automatically matched to the first level classification standard of audit issues

[0278] Use BERT's tokenizer to tokenize the text and convert it into a format that the model can understand, including adding special tags to distinguish the beginning and end of sentences. Load the pre-trained BERT model, which has been trained on a large amount of text data and can capture the deep characteristics of the language. Build a classification head (Classifier Head) based on the BERT model, that is, the first-level classification, which is usually a fully connected layer used to map BERT's output to a specific category. The first-level classification standard is based on the eighteen internal control guidelines of the enterprise. Figure 6 shown.

[0279] 103 When the sample size is lower than the preset standard, keyword matching technology is introduced in combination with the BERT model to achieve the final classification standard of audit issues

[0280] When the amount of classified training data is small, traditional machine learning methods may face the risk of overfitting because the model may learn the noise in the data rather than the underlying data distribution. In this case, keyword matching technology can be introduced to do a pre-classification.

[0281] When an audit question is received, a regular expression is first used to match the number of keywords in each final-level category in the audit question description.

[0282] The final classification is a subclass of the first-level classification standard, and personalized standards can be formulated according to different situations of the enterprise. If there is an untrained classification in the final label, and there are keywords corresponding to this untrained classification in the audit problem description, the final classification with the most keywords in the untrained classification label is directly returned as the final result; if there is no keyword corresponding to this untrained classification, no results are returned temporarily, but the model is used to make probability predictions for the trained classification.

[0283] 104 Automatically match the final classification standard of audit issues

[0284] The sub-categories of the eighteen control guidelines within the enterprise, for example, the first-level classification is asset management issues, then the last-level classification is the sub-categories under it, such as: irregular real estate title deeds, irregular inventory management, irregular inventory check and count, irregular capital transfer management, etc.

[0285] First, use the keyword list maintained in the background to match, and when the match is found, return it directly. If there is no result for keyword matching, use the BERT fine-tuned model to predict the classification, which effectively improves the accuracy and generalization ability of the classification.

[0286] For the trained categories, after matching keywords, the final label model predicts the two most likely categories. If the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned.

[0287] 105 Get the problem classification field

[0288] Once a clear understanding of the audit question classification system is established, the backend business can locate and obtain these classification fields through API calls to ensure that the classification label or code corresponding to each question is accurately captured.

[0289] 106 Match the corrective measures according to the problem classification

[0290] According to business needs, corrective measures are classified in advance. For example, a group formulates corrective measures for amount-based corrections, such as adjusting accounting records, recovering funds or recovering losses, and activating funds to tap potential. These standards can be different according to the company's different business needs. Secondly, analyze the historical data of problem classification and correction measures matching, match the results of problem classification with the corresponding correction measures type according to the principle of maximum probability, and predict the most appropriate correction measures type for a specific problem category. In order to improve the accuracy of the matching, it is necessary to further feature engineer the problem text to extract key information that helps match the correction measures type. Finally, the model will select or recommend several most appropriate correction measures types from the correction measures library based on the results of the problem classification.

[0291] 107 Formulate corrective measures according to the classification of measures

[0292] By classifying corrective measures, auditors can more accurately formulate corresponding corrective measures for different types of problems. For example, for financial issues, it may be necessary to strengthen internal controls and standardize financial processes; for compliance issues, it may be necessary to update relevant policies and procedures and improve employees' awareness of the importance of compliance. It effectively helps auditors determine the priority of corrective measures, formulate specific implementation plans for each measure, and clarify the allocation of responsibilities.

[0293] 108 Get the corrective measures description field

[0294] BERT is based on the Transformer architecture, which is completely based on the attention mechanism and processes sequence data through self-attention layers. The concept of attention mechanism can be used to improve efficiency and accuracy when understanding and processing large amounts of data. The model preprocesses the text of the rectification measures description, decomposing the text into words or subwords, and generates an attention mask to mark the padding position by adding special tags, padding, and truncation, so that the model can distinguish between actual input and padding values.

[0295] 109 Introducing keyword matching technology combined with the BERT model to implement classification standards for rectification measures

[0296] First, use regular expressions to match the number of keywords in each first-level label category in the description of the rectification measures.

[0297] If there is an untrained category in the label, and there is a keyword corresponding to the untrained category in the corrective measure description, the final classification result needs to include all corrective measure categories with untrained category labels; if there is no keyword corresponding to the untrained category in the corrective measure description, the final classification result does not include the corrective measure category with untrained category labels.

[0298] For the trained categories, after keyword matching, the label model is used to predict the two categories with the highest probability. If the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned.

[0299] 110 Update the classification weights of corrective measures corresponding to problem classifications

[0300] Corresponding to the above rectification measures classification results, the corresponding rectification measures type weights, ie, probabilities, are re-predicted and updated. After steps 106-109, the rectification measures type weights are re-predicted, thereby updating the corresponding rectification measures type weights.

[0301] Due to the adoption of the above technical scheme, the beneficial effect of the present invention is to create an artificial intelligence classification device, which can automatically classify audit problems according to the audit problem description and apply them to future audit projects; it can classify corrective measures and automatically recommend corrective measures according to the problem classification, so that the audit work is more targeted and the audit process is transparent and traceable; at the same time, the professionalism of the audit work is improved, the authority of the audit results is enhanced, and the audit project execution process is standardized, thereby improving the audit quality, and changing the reliance on the personal ability level of auditors to the reliance on the knowledge accumulation of the audit system.

[0302] From a quantitative perspective, the experimental results in Table 1 show that the F1-score of the present application is 0.9654, which demonstrates that the achievements of the present invention are significant.

[0303] Table 1 Model learning effect table

[0304] Model precision recall f1-score This application 0.9625 0.9805 0.9654

[0305] First of all, the audit problem classification standard is an important part of the audit work, which helps to clarify the audit objectives, improve audit efficiency and ensure audit quality. According to business needs, we divide audit problems into four levels. The first-level classification standard is based on 18 areas of internal control, including contract management issues, asset management issues, guarantee business issues, procurement business issues, capital activities, R&D activities, business outsourcing issues, internal information transmission, sales business issues, financial reporting issues, social responsibility issues, engineering project issues, information system issues, development strategy issues, comprehensive budget issues, human resources, organizational structure issues, and corporate culture issues; the fourth-level standard is based on the statistical caliber of each unit, such as procurement business including the lack of system policies, material supply inconsistent with contract agreement, irregular procurement acceptance management, irregular bidding process, price comparison is not asked, contract signing is not standardized, price comparison process is not standardized, supplier evaluation management is not standardized, supplier access management is not standardized, procurement contract terms have obvious omissions, settlement payment is not standardized, other management issues, bidding is not done, suppliers do not have specific qualifications or business scope is not consistent, accounting is not standardized, procurement approval is not standardized, such as Figure 4 shown.

[0306] Secondly, based on historical business data, we classified the rectification measures into recovering capital losses, strengthening business management, adjusting account statements, optimizing system processes, revitalizing capital assets, and optimizing information systems, such as Figure 5 shown.

[0307] Doing business with companies not listed in the supplier list:

[0308] 1. Get the audit question field description

[0309] Description of audit problem: Some suppliers for price comparison are not in the list of qualified suppliers. The purchasing department of Company A provided price comparison information of four suppliers for the maintenance project of Project XX. These four suppliers are Company A, Company B, Company C, and Company D. Among them, Company B and Company C are not in the list of suppliers. The effectiveness of the price comparison work cannot be ensured.

[0310] 2. Automatically match the primary classification standard

[0311] The model obtains the field description of the audit question, the text is embedded and vectorized, and the final probability vector of each category is obtained through forward transmission of the BERT model. The one with the largest value in the probability vector is the first-level classification standard, that is, the procurement business issue.

[0312] 3. Introduce keyword matching technology Introduce keyword matching technology combined with the BERT model to achieve the final classification standard for audit problem description

[0313] The system maintains a set of keyword dictionaries. One label corresponds to multiple keyword lists of the label. The keyword list is scanned for the input audit description, and the label corresponding to the keyword is returned if it is matched. The keyword matching situation for the final classification in this case is shown in Table 2:

[0314] Table 2 Procurement business keyword matching

[0315]

[0316] 4. Model matching to the final classification of audit issues

[0317] 1) Results of the final classification keyword matching of procurement business issues: as shown in Table 2;

[0318] 2) The procurement business problem is a trained model, and it is necessary to combine the model for probability prediction to determine the final classification result;

[0319] 3) The final label model predicts two categories with the highest probability: business with companies outside the supplier list and irregular procurement contract approval, with probabilities of 93.88% and 95.12% respectively, and the probability difference is less than 0.2;

[0320] 4) Return categories with a large number of keywords, i.e., those doing business with companies outside the supplier list (8 keywords).

[0321] 5. Get the problem classification field

[0322] The API is called based on the backend business to obtain the final classification of 18 areas of audit issues. In this case, the problem classification field obtained is business with companies outside the supplier list.

[0323] 6. Match the corrective measures according to the problem classification

[0324] The model matches data based on historical corrective measures for the audit issue of doing business with companies outside the supplier list, and recommends the two corresponding corrective measures with the highest weights: strengthening business management and optimizing institutional processes.

[0325] 7. Formulate corrective measures according to the classification of measures

[0326] According to the classification of the two corrective measures of strengthening business management and optimizing system processes, auditors can formulate corrective measures based on audit experience:

[0327] Description of corrective measures to strengthen business management:

[0328] 1) Immediately update and improve the existing list of qualified suppliers to ensure that all suppliers participating in the price inquiry have undergone strict qualification review and evaluation to meet the company's procurement standards and policies;

[0329] 2) Establish and implement a standardized supplier selection and evaluation process, including supplier pre-screening, selection, evaluation and regular review mechanisms to ensure supplier quality and reliability;

[0330] 3) Strengthen the training of procurement department staff to improve their understanding of procurement policies and procedures, ensure that they can strictly abide by the regulations during the price inquiry process, and avoid the recurrence of similar problems;

[0331] 4) Use information technology tools, such as procurement management systems, to automate supplier management and price comparison processes, reduce human errors, and improve efficiency and transparency;

[0332] 5) Establish a supervision and feedback mechanism, conduct regular audits of the procurement process, promptly identify and correct operations that do not comply with regulations, ensure the compliance and effectiveness of procurement activities, and improve the standardization and transparency of procurement work.

[0333] Description of corrective measures to optimize institutional processes:

[0334] 1) A comprehensive review of the existing supplier management system is needed to ensure that the system clearly defines the entry criteria, review process and continuous evaluation mechanism for qualified suppliers;

[0335] 2) Establish a transparent supplier information database to record supplier qualifications, historical performance and evaluation results, so as to quickly verify supplier qualifications when inquiring for price comparisons;

[0336] 3) Establish a strict price inquiry process, requiring procurement personnel to screen qualified suppliers from the database before inquiring and record the reasons for selecting suppliers and the evaluation process during the price inquiry process to improve the transparency and traceability of decision-making;

[0337] 4) Strengthen internal control mechanisms, such as setting up approval procedures to ensure that all price inquiry and comparison activities are reviewed by relevant departments before submission to prevent the participation of unqualified suppliers;

[0338] 5) Strengthen staff training to improve their understanding and execution of system processes, and ensure that procurement personnel can accurately understand and comply with relevant regulations on price inquiries and comparisons;

[0339] 6) Establish a continuous monitoring and evaluation system, regularly audit supplier management and price inquiry and comparison processes, promptly identify and correct defects in the process, and ensure the compliance and effectiveness of procurement activities.

[0340] 8. Get the corrective measures description field

[0341] Because business needs are not static, the types of problems and the requirements for corrective measures will also change accordingly, so the classification of corrective measures needs to be optimized and updated. Through feedback from actual applications, the matching logic and corrective measure types of the model are continuously evaluated and optimized to adapt to the changing business needs.

[0342] Update the classification of the corrective action after obtaining the corrective action description field.

[0343] 9. Introduce keyword matching technology combined with the BERT model to implement the classification standard for corrective measures description

[0344] The system also maintains a set of corrective measures keyword dictionaries. One label corresponds to multiple keyword lists for the label. The keyword list is scanned for the input corrective measures text, and the label corresponding to the keyword is returned if a match is found. The keyword matching situation for the corrective measures classification in this case is shown in Table 3:

[0345] Table 3 Corrective measures keyword matching

[0346]

[0347]

[0348] 1) Results of keyword matching for the seven rectification measures: As shown in Table 3, the number of keywords for strengthening business management and optimizing system management is the largest, 10 and 8 respectively;

[0349] 2) The optimized information system is an untrained model, and the number of keywords is 2, so the final classification result needs to include the optimized information system;

[0350] 3) Strengthening business management and optimizing institutional processes is a trained model. After keyword matching, the model predicts two categories with the highest probability: optimizing institutional processes and strengthening business management. The probabilities of the two are 91.12% and 90.73% respectively. If the probability difference is less than 0.2, the category with more keywords is returned, that is, strengthening business management. (Note: The business requirements in this case have not been updated).

[0351] 4) Based on the above, the final classification criteria for corrective measures are to strengthen business management and optimize information systems.

[0352] 10. Update the classification weights of corrective measures corresponding to problem classifications

[0353] According to the probability prediction results of the corrective measures, the model updates the weights of the corrective measures (strengthening business management and optimizing information systems) corresponding to the audit issue (doing business with companies outside the supplier list), which are 92.32% and 88.14% respectively.

[0354] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0355] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for classifying audit issues and corrective measures based on a neural network model, characterized in that: include: Step S1: Get the question description field, tokenize the text, load the pre-trained BERT model, and match it to the first-level classification standard of the question; Step S2: Introduce keyword matching technology combined with the BERT model to implement the final classification standard of the problem; Step S3: Obtain the problem classification field, match the rectification measure classification according to the problem classification, and formulate rectification measures; Step S4: Obtain the corrective measure description field, implement the corrective measure classification standard, and update the corrective measure classification weight corresponding to the problem classification.

2. The method for classifying audit issues and corrective measures based on a neural network model according to claim 1 is characterized in that: In step S1: Get the problem description field: The BERT model is based on the Transformer architecture, which is completely based on the attention mechanism. It processes sequence data through the self-attention layer. The model preprocesses the question description text, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values. The pre-training tasks of the BERT model include the masked language model and Next Sentence Prediction, and its mathematical formula is as follows: Masked Language Model: Among them, P(x) represents the target probability distribution, m i Indicates whether the i-th token is blocked, x ＜i represents all tokens before the i-th token; n represents the number of words in the sentence, the length of the sequence; x i represents the i-th word; θ represents the parameters of the model; Next Sentence Prediction: Among them, s and t represent two sentences respectively, φ(s, t) represents the vector representation of the sentence pair, and t rand represents a random sentence from a different document than s; In the Transformer encoder, the data first passes through the self-attention module to obtain a weighted feature vector Z: Among them, d k is the number of elements in each column of the K matrix; After obtaining the feature vector, it is sent to the next fully connected network module of the encoder, expressed as: FFN(Z)=max(0,ZW1+b1)W2+b2 Among them, W1 is the weight matrix of the first fully connected layer, b1 is the bias term of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer; In self-attention, each word has 3 different vectors, namely Query vector Q, Key vector K, and Value vector V, all of length 64, which are respectively obtained by multiplying the embedding vector X by 3 different weight matrices W. Q , W K , W V It turns out that the sizes of the three matrices are the same, all 512x64; The calculation process of Attention: Convert the input word into an embedding vector; obtain Q, K, and V vectors based on the embedding vector; calculate a score for each vector, score = q·k; normalize the score, and divide the score by Apply softmax activation function to score: in, represents the exponential form of a scoring function between input x and candidate output y; the scoring function measures the degree of match between input x and candidate output y, j represents the index of the candidate output, ranging from 1 to n, where n is the number of all possible candidate outputs; Softmax multiplies the Value value v to get the weighted score of each input vector; after adding, the final output result is: z = ∑υ; In the decoder, Q comes from the previous output of the decoder, and K and V come from the output of the encoder. The decoding process is a sequential operation process. When decoding the kth feature vector, the decoding results of the k-1th and previous ones are seen; After decoding by the decoder, the decoded feature vector passes through a fully connected layer with a softmax activation function to obtain an output vector reflecting the probability of each word; Position encoding is introduced when encoding word vectors. The position information of words is added to the word vectors to distinguish different words. Position encoding is a d model The specific encoding formula is as follows: Among them, pos represents the position of the word, and i represents the dimension of the word; In order to make the model converge faster, the model uses Layer Normalization, the calculation formula is as follows: Among them, μ i is the mean of the ith sample, χ ij represents the jth eigenvalue of the ith sample, which is an element in the input data matrix, where i is the index of the sample and j is the index of the feature, σ is the standard deviation of all samples, m represents the number of features, and ∈ is a small positive number to avoid division errors caused by the standard deviation being zero; Automatically match to the first level classification criteria of the problem: Use the BERT model's tokenizer to tokenize the text and convert it into the format preset by the model, including adding tags to distinguish the beginning and end of sentences; load the pre-trained BERT model and build a classification head based on the BERT model. The first-level classification is a fully connected layer used to map the output of the BERT model to preset specific categories. The first-level classification standard is based on the eighteen internal control guidelines of the enterprise.

3. The method for classifying audit issues and corrective measures based on a neural network model according to claim 1 is characterized in that: In step S2: When the sample size is lower than the preset standard, keyword matching technology is introduced in combination with the BERT model to achieve the final classification standard of the problem; when the amount of classified training data is less than the preset standard, keyword matching technology is introduced for pre-classification; When a question is received, a regular expression is used to match the number of keywords in each final category in the question description; The final classification is a subclass of the first-level classification standard, and the standard is formulated according to the different situations of the enterprise. If there is an untrained classification in the final-level label, and the keyword corresponding to this untrained classification exists in the problem description, the final-level classification with the most keywords in the untrained classification label is directly returned as the final result. If there is no keyword corresponding to the untrained category, no results are returned and the model is used to make probability predictions for the trained category. Automatically match the final classification criteria of the problem: First, use the keyword list maintained in the background to match, and return the result directly when the keyword is matched. If there is no result, use the BERT fine-tuned model to predict the classification; For the trained categories, after matching keywords, the final label model predicts the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned.

4. The method for classifying audit issues and corrective measures based on a neural network model according to claim 1 is characterized in that: In step S3: The backend business locates and obtains the problem classification field through API calls to ensure that the classification label or code corresponding to each problem is captured; Match the rectification measures classification according to the problem classification. According to business needs, pre-classify the rectification measures, analyze the historical data of problem classification and rectification measures matching, match the problem classification results with the corresponding rectification measures types according to the principle of maximum probability, predict the most appropriate rectification measures type for specific problem categories, perform further feature engineering on the problem text, extract preset information for rectification measures type matching, and the model selects or recommends several most appropriate rectification measures types from the rectification measures library based on the problem classification results; formulate corresponding rectification measures for different types of problems through the classification of rectification measures.

5. The method for classifying audit issues and corrective measures based on a neural network model according to claim 1 is characterized in that: In step S4: Get the corrective action description field: BERT is based on the Transformer architecture, which is completely based on the attention mechanism and processes sequence data through the self-attention layer. The model pre-processes the text describing the rectification measures, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values. Introducing keyword matching technology combined with the BERT model to implement the classification standards for rectification measures: Use regular expressions to match the number of keywords for each first-level label category in the corrective measure description; if there is an untrained category in the label, and the corrective measure description contains the keyword corresponding to the untrained category, then the final classification result needs to include all corrective measure categories with untrained category labels; if the corrective measure description does not contain the keyword corresponding to the untrained category, then the final classification result does not include the corrective measure category with untrained category labels; For the trained categories, after keyword matching, the label model is used to predict the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned; Update the classification weight of the corrective measure corresponding to the problem classification, and corresponding to the result of the corrective measure classification, re-predict and update the corresponding corrective measure type weight, i.e., probability, and re-predict the weight of the corrective measure type to update the corresponding corrective measure type weight.

6. A classification system for audit issues and corrective measures based on a neural network model, characterized in that: include: Module M1: Get the question description field, tokenize the text, load the pre-trained BERT model, and match it to the first-level classification standard of the question; Module M2: Introducing keyword matching technology combined with the BERT model to implement the final classification standard of the problem; Module M3: Obtain the problem classification field, match the rectification measures classification according to the problem classification, and formulate rectification measures; Module M4: Obtain the corrective measure description field, implement the corrective measure classification standard, and update the corrective measure classification weight corresponding to the problem classification.

7. The audit problem and corrective measures classification system based on the neural network model according to claim 6 is characterized in that: In the module M1: Get the problem description field: The BERT model is based on the Transformer architecture, which is completely based on the attention mechanism. It processes sequence data through the self-attention layer. The model preprocesses the question description text, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values. The pre-training tasks of the BERT model include the masked language model and Next Sentence Prediction, and its mathematical formula is as follows: Masked Language Model: Among them, P(x) represents the target probability distribution, m i Indicates whether the i-th token is blocked, x <i represents all tokens before the i-th token; n represents the number of words in the sentence, the length of the sequence; x i represents the i-th word; θ represents the parameters of the model; Next Sentence Prediction: Among them, s and t represent two sentences respectively, φ(s, t) represents the vector representation of the sentence pair, and t rand represents a random sentence from a different document than s; In the Transformer encoder, the data first passes through the self-attention module to obtain a weighted feature vector Z: Among them, d k is the number of elements in each column of the K matrix; After obtaining the feature vector, it is sent to the next fully connected network module of the encoder, expressed as: FFN(Z)=max(0,ZW1+b1)W2+b2 Among them, W1 is the weight matrix of the first fully connected layer, b1 is the bias term of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term of the second fully connected layer; In self-attention, each word has 3 different vectors, namely Query vector Q, Key vector K, and Value vector V, all of length 64, which are respectively obtained by multiplying the embedding vector X by 3 different weight matrices W. Q , W K , W V It turns out that the sizes of the three matrices are the same, all 512x64; The calculation process of Attention: Convert the input word into an embedding vector; obtain Q, K, and V vectors based on the embedding vector; calculate a score for each vector, score = q·k; normalize the score, and divide the score by Apply softmax activation function to score: in, represents the exponential form of a scoring function between input x and candidate output y; the scoring function measures the degree of match between input x and candidate output y, j represents the index of the candidate output, ranging from 1 to n, where n is the number of all possible candidate outputs; Softmax multiplies the Value value v to get the weighted score of each input vector; after adding them together, the final output result is: z = ∑v; In the decoder, Q comes from the previous output of the decoder, and K and V come from the output of the encoder. The decoding process is a sequential operation process. When decoding the kth feature vector, the decoding results of the k-1th and previous ones are seen; After decoding by the decoder, the decoded feature vector passes through a fully connected layer with a softmax activation function to obtain an output vector reflecting the probability of each word; Position encoding is introduced when encoding word vectors. The position information of words is added to the word vectors to distinguish different words. Position encoding is a d model The specific encoding formula is as follows: Among them, pos represents the position of the word, and i represents the dimension of the word; In order to make the model converge faster, the model uses Layer Normalization, the calculation formula is as follows: Among them, μ i is the mean of the i-th sample, x ij represents the jth eigenvalue of the ith sample, which is an element in the input data matrix, where i is the index of the sample and j is the index of the feature, σ is the standard deviation of all samples, m represents the number of features, and ∈ is a small positive number to avoid division errors caused by the standard deviation being zero; Automatically match to the first level classification criteria of the problem: Use the BERT model's tokenizer to tokenize the text and convert it into the format preset by the model, including adding tags to distinguish the beginning and end of sentences; load the pre-trained BERT model and build a classification head based on the BERT model. The first-level classification is a fully connected layer used to map the output of the BERT model to preset specific categories. The first-level classification standard is based on the eighteen internal control guidelines of the enterprise.

8. The audit problem and corrective measures classification system based on the neural network model according to claim 6 is characterized in that: In the module M2: When the sample size is lower than the preset standard, keyword matching technology is introduced in combination with the BERT model to achieve the final classification standard of the problem; when the amount of classified training data is less than the preset standard, keyword matching technology is introduced for pre-classification; When a question is received, a regular expression is used to match the number of keywords in each final category in the question description; The final classification is a subclass of the first-level classification standard, and the standard is formulated according to the different situations of the enterprise. If there is an untrained classification in the final-level label, and the keyword corresponding to this untrained classification exists in the problem description, the final-level classification with the most keywords in the untrained classification label is directly returned as the final result. If there is no keyword corresponding to the untrained category, no results are returned and the model is used to make probability predictions for the trained category. Automatically match the final classification criteria of the problem: First, use the keyword list maintained in the background to match, and return the result directly when the keyword is matched. If there is no result, use the BERT fine-tuned model to predict the classification; For the trained categories, after matching keywords, the final label model predicts the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned.

9. The audit problem and corrective measures classification system based on the neural network model according to claim 6 is characterized in that: In the module M3: The backend business locates and obtains the problem classification field through API calls to ensure that the classification label or code corresponding to each problem is captured; Match the rectification measures classification according to the problem classification. According to business needs, pre-classify the rectification measures, analyze the historical data of problem classification and rectification measures matching, match the problem classification results with the corresponding rectification measures types according to the principle of maximum probability, predict the most appropriate rectification measures type for specific problem categories, perform further feature engineering on the problem text, extract preset information for rectification measures type matching, and the model selects or recommends several most appropriate rectification measures types from the rectification measures library based on the problem classification results; formulate corresponding rectification measures for different types of problems through the classification of rectification measures.

10. The audit problem and corrective measures classification system based on the neural network model according to claim 6 is characterized in that: In the module M4: Get the corrective action description field: BERT is based on the Transformer architecture, which is completely based on the attention mechanism and processes sequence data through the self-attention layer. The model pre-processes the text describing the rectification measures, decomposes the text into words or subwords, and generates an attention mask to mark the padding position by adding tags, padding, and truncating, so that the model can distinguish between actual input and padding values. Introducing keyword matching technology combined with the BERT model to implement the classification standards for rectification measures: Use regular expressions to match the number of keywords for each first-level label category in the corrective measure description; if there is an untrained category in the label, and the corrective measure description contains the keyword corresponding to the untrained category, then the final classification result needs to include all corrective measure categories with untrained category labels; if the corrective measure description does not contain the keyword corresponding to the untrained category, then the final classification result does not include the corrective measure category with untrained category labels; For the trained categories, after keyword matching, the label model is used to predict the two categories with the highest probability; if the probability difference between the two categories is less than 0.2, the category with more keywords is returned; if the probability difference between the two categories is greater than 0.2, the category with higher probability is returned; Update the classification weight of the corrective measure corresponding to the problem classification, and corresponding to the result of the corrective measure classification, re-predict and update the corresponding corrective measure type weight, i.e., probability, and re-predict the weight of the corrective measure type to update the corresponding corrective measure type weight.

Images