Audit manuscript recommendation method and device and readable storage medium

Abstract

The invention provides an audit manuscript recommendation method and device and a readable storage medium. The method comprises the following steps: acquiring an original text of an audit manuscript; inputting the original text into a first model, zooming an embedded vector of each keyword in the original text through a dynamic keyword suppression module in the first model, and obtaining a first classification label of the original text according to the zoomed embedded vector of the original text; inputting the original text, a preset prompt word and information related to the original text and retrieved from the historical knowledge base into a second model, so that the second model generates a corresponding second classification tag; fusing the first classification label and the second classification label; and displaying the fused classification label. According to the method, the device and the medium, the problems that an existing audit manuscript recommendation method is insufficient in recommendation accuracy and difficult to adapt to dynamic evolution characteristics of audit services, and a final result source is obviously limited due to excessive dependence on a single model can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital audit, in particular to an audit document recommendation method and device and a readable storage medium. BACKGROUND

[0002] In recent years, the rapid development of digital audit technology has promoted the improvement of the automation level of the audit process, but the existing system still has bottlenecks in intelligent decision support and dynamic adaptability. As a core link, the accuracy of audit document recommendation directly affects the problem positioning efficiency and the quality of document writing. The current mainstream method mainly relies on artificial rule construction or general machine learning model, which is difficult to effectively handle the complex semantic association and dynamic changing knowledge system in the audit scene. Specifically, the existing audit document recommendation method at least has the problems of insufficient recommendation accuracy, difficulty in adapting to the dynamic evolution characteristics of audit business, and excessive dependence on a single model leading to obvious limitations in the source of the final result. SUMMARY

[0003] The technical problem to be solved by the present application is to solve the above-mentioned problems of the prior art, and to provide an audit document recommendation method, device and readable storage medium, which can solve the problems of insufficient recommendation accuracy, difficulty in adapting to the dynamic evolution characteristics of audit business, and excessive dependence on a single model leading to obvious limitations in the source of the final result.

[0004] In a first aspect, the present application provides an audit document recommendation method, which comprises:

[0005] obtaining an audit document original text;

[0006] inputting the original text into a first model, scaling the embedding vector of each keyword in the original text through a dynamic keyword suppression module in the first model, and obtaining a first classification label of the original text according to the embedding vector of the original text after scaling;

[0007] inputting the original text, a preset prompt word, and information related to the original text retrieved from a historical knowledge base into a second model to make the second model generate a corresponding second classification label, wherein the second model is a large language model;

[0008] fusing the first classification label and the second classification label;

[0009] displaying the fused classification label to make a user modify or confirm the fused classification label to obtain a final audit document classification label;

[0010] based on the final audit document classification label, recommending an audit document.

[0011] Furthermore, the scaling of the embedding vector of each keyword in the original text through the dynamic keyword suppression module in the first model specifically includes:

[0012] The dynamic keyword suppression module performs the following operations:

[0013] Based on a preset keyword database, keywords in the original text are identified, and the position and category of the keywords are marked in the original text.

[0014] The context-aware vector of each word in the original text after being marked is generated using a bidirectional gated recurrent unit (BiGRU) encoding layer.

[0015] The context-aware vector of each word is input into a gated inhibition network. The effectiveness of each word in the context is dynamically evaluated using the gated inhibition network, and corresponding gating coefficients are generated. The embedding vectors of the corresponding keywords are scaled according to the gating coefficients, while the embedding vectors of non-keywords remain unchanged.

[0016] Furthermore, the formula for calculating the gating coefficient is as follows:

[0017] g i =σ(W g ·[h i m i ]+b g )

[0018] Among them, g i Let W represent the gating coefficient of the i-th word, σ represent the Sigmoid activation function, and W represent the gating coefficient of the i-th word. g For the weight matrix, h i Let m represent the context-aware vector of the i-th word. i This represents the keyword mask for the i-th word. When the i-th word is a keyword, m i The value of m is 1 when the i-th word is not a keyword. i The value is 0, [h i m i ] represents h i and m i The concatenated vector, b g This is a bias term.

[0019] Furthermore, scaling the embedding vector of the corresponding keyword according to the gating coefficient specifically includes:

[0020] The embedding vector of each keyword is multiplied by the corresponding gating coefficient to achieve scaling of the keyword embedding vector.

[0021] Further, the first classification label of the original text is obtained according to the embedding vector of the original text after scaling, and specifically comprises:

[0022] The embedding vector of the original text after scaling is segmented, and the segmented embedding vectors are respectively input into a bidirectional encoder representation from transformers (BERT) model for semantic coding.

[0023] The semantic coding results corresponding to each segment are spliced or fused to generate a corresponding semantic vector.

[0024] The semantic vector is input into a long short-term memory (LSTM) model, and the LSTM model is used to calculate based on the semantic vector to capture sequence dependency and long-term context information in the text, and output a corresponding feature vector.

[0025] The feature vector is transmitted to a fully connected layer to obtain the first classification label of the original text.

[0026] Further, before the original text, the preset prompt word, and the information related to the original text retrieved from the historical knowledge base are input into the second model to make the second model generate a corresponding second classification label, the method further comprises:

[0027] Information related to the original text is retrieved from the historical knowledge base by using a retrieval-augmentation-generation (RAG) technology, wherein the historical knowledge base stores historical audit draft texts and corresponding classification labels.

[0028] Further, the method further comprises at least one of the following:

[0029] The original text and the corresponding audit draft classification label are uploaded to a system log, and information that has not been synchronized to the historical knowledge base is extracted from the system log regularly or irregularly to update the historical knowledge base.

[0030] The keyword library is updated according to the keywords fed back by the user.

[0031] In a second aspect, the present application provides an audit draft recommendation device, which comprises:

[0032] An original text acquisition module is configured to acquire an audit draft original text.

[0033] A first classification module is connected to the original text acquisition module and configured to input the original text into a first model, scale the embedding vector of each keyword in the original text by using a dynamic keyword suppression module in the first model, and obtain a first classification label of the original text according to the embedding vector of the original text after scaling.

[0034] The second classification module is connected with the original text acquisition module, and is configured to input the original text, a preset prompt word, and information related to the original text retrieved from a historical knowledge base into a second model, so that the second model generates a corresponding second classification label, wherein the second model is a large language model;

[0035] The classification label fusion module is connected with the first classification module and the second classification module, and is configured to fuse the first classification label and the second classification label;

[0036] The classification label determination module is connected with the classification label fusion module, and is configured to display the fused classification label, so that a user modifies or confirms the fused classification label to obtain a final audit document classification label;

[0037] The audit document recommendation module is connected with the classification label determination module, and is configured to recommend an audit document based on the final audit document classification label.

[0038] In a third aspect, the present application provides an audit document recommendation device, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to realize the audit document recommendation method of the first aspect.

[0039] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the audit document recommendation method of the first aspect.

[0040] The application provides an audit document recommendation method, device and readable storage medium. First, an audit document original text is obtained; then the original text is input into a first model, the embedding vector of each keyword in the original text is scaled by a dynamic keyword suppression module in the first model, and a first classification label of the original text is obtained according to the embedding vector of the original text after scaling; at the same time, the original text, a preset prompt word and information related to the original text retrieved from a historical knowledge base are input into a second model to enable the second model to generate a corresponding second classification label, wherein the second model is a large language model; the first classification label and the second classification label are fused; and the fused classification label is displayed to enable a user to modify or confirm the fused classification label to obtain a final audit document classification label; finally, audit document recommendation is performed based on the final audit document classification label. Through the fusion strategy and dynamic keyword suppression technology, the application can more accurately grasp the core business semantics of the audit text and realize more accurate multi-label classification, which not only improves the accuracy of classification, but also effectively reduces the recommendation range and significantly reduces the workload of manual secondary screening. Specifically, the application scales the keyword embedding vector in the original text through the dynamic keyword suppression module, which can more accurately capture the core semantics of the text, enhance the adaptability of the model to dynamically changing audit business and improve the accuracy of the first classification label. At the same time, in combination with the preset prompt word and the related information in the historical knowledge base, the second classification label is generated by using the large language model, which further enriches the basis for classification. In addition, by fusing the first and second classification labels and allowing the user to modify or confirm, the application not only further improves the classification accuracy, but also avoids over-reliance on a single model through the cooperation of the two models, thereby solving the problems of the existing audit document recommendation method, such as insufficient recommendation accuracy, difficulty in adapting to the dynamic evolution characteristics of the audit business, and obvious limitations in the source of the final result due to over-reliance on a single model. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A flowchart of an audit document recommendation method according to Embodiment 1 of the application;

[0042] Figure 2 An architectural diagram of an audit document recommendation method according to an embodiment of the application;

[0043] Figure 3 A whole architectural diagram of a model M1 according to an embodiment of the application;

[0044] Figure 4 An architectural diagram of a dynamic keyword suppression module according to an embodiment of the application;

[0045] Figure 5The overall architecture diagram of the model M2 of the embodiment of the present application;

[0046] Figure 6 The structural schematic diagram of the audit document recommendation device of the embodiment 2 of the present application;

[0047] Figure 7 The structural schematic diagram of the audit document recommendation device of the embodiment 3 of the present application. DETAILED DESCRIPTION

[0048] In order to make the skilled in the art better understand the technical solutions of the present application, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0049] It can be understood that the specific embodiments and drawings described herein are only used to explain the present application, but not to limit the present application.

[0050] It can be understood that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0051] It can be understood that, for the convenience of description, only the parts related to the present application are shown in the drawings of the present application, and the parts unrelated to the present application are not shown in the drawings.

[0052] It can be understood that each unit and module involved in the embodiments of the present application can only correspond to one entity structure, or can be composed of multiple entity structures, or multiple units and modules can be integrated into one entity structure.

[0053] It can be understood that the terms "first", "second" and the like in the embodiments of the present application are used to distinguish different objects, or to distinguish different treatments of the same object, but not to describe a specific order of the object.

[0054] It can be understood that the functions and steps marked in the flowchart and block diagram of the present application can occur in an order different from that marked in the drawings without conflict.

[0055] It can be understood that in the flowchart and block diagram of the present application, the system, device, equipment, method according to the embodiments of the present application are shown as possible implementation architecture, function and operation. Each block in the flowchart or block diagram can represent a unit, module, program segment, code, which contains executable instructions for realizing the specified function. Moreover, each block or combination of blocks in the block diagram and flowchart can be realized by a hardware-based system for realizing the specified function, or by a combination of hardware and computer instructions.

[0056] It can be understood that the units and modules involved in the embodiments of the application can be implemented in software or hardware, for example, the units and modules can be located in a processor.

[0057] SUMMARY

[0058] In audit document writing, document recommendation is a core link, and its accuracy directly affects the efficiency of problem positioning and the quality of document writing. However, the existing audit document recommendation method has the following shortcomings and deficiencies:

[0059] 1) Insufficient recommendation accuracy: Some audit systems use rule engines or keyword matching technology, but rule engines or simple keyword matching cannot adapt to multi-level problem classification, the recommended range is wide, and manual secondary screening is required.

[0060] 2) Low classification accuracy: In the audit text classification task, the existing common text classification model has obvious deficiencies, and it is difficult to accurately grasp the complex relationship between the key words in the audit field and the final output label, which further leads to the classification result being significantly disturbed by the key words, so that the accuracy of the final classification label is greatly affected.

[0061] 3) The existing audit document classification technology has significant limitations, and the model output result is static and fixed, which is difficult to adapt to the dynamic evolution characteristics of the audit business. In addition, this technology cannot be self-optimized and iteratively upgraded based on the continuously accumulated audit data and feedback information. Therefore, in the long-term application process, it is difficult to achieve stable improvement of classification accuracy, and it cannot meet the increasing requirements of audit work for the accuracy of document classification.

[0062] 4) In the actual long text multi-label classification task, the existing method often relies on a single model for training and generating classification results. However, in the business scenario with high precision requirements, there are obvious limitations in using the output of a single model as the source of the final result. The main problems are the lack of multi-dimensional result reference, and the existing neural network model is difficult to realize automatic iterative optimization, so it cannot fully meet the high-precision requirements of the business.

[0063] To solve the above technical problems, the present application provides an audit document recommendation method, device and readable storage medium, which can more accurately grasp the core business semantics of the audit text by fusing the strategy and dynamic keyword suppression technology, and realize more accurate multi-label classification. This not only improves the accuracy of classification, but also effectively reduces the recommended range and significantly reduces the workload of manual secondary screening, so as to at least solve the problems of insufficient recommendation accuracy of the existing audit document recommendation method, difficulty in adapting to the dynamic evolution characteristics of the audit business, and the obvious limitations of the source of the final result caused by excessive reliance on a single model.

[0064] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be specifically introduced below with reference to the accompanying drawings.

[0065] Embodiment 1

[0066] The present embodiment provides an audit document recommendation method, as shown in the figure, the method comprises: Figure 1

[0067] Step S101: obtaining an original text of an audit document;

[0068] In the present embodiment, the original text of the audit document can be obtained from the audit system.

[0069] Step S102: inputting the original text into a first model, scaling the embedding vector of each keyword in the original text through a dynamic keyword suppression module in the first model, and obtaining a first classification label of the original text according to the embedding vector of the original text after scaling.

[0070] In the present embodiment, the first model comprises a dynamic keyword suppression module (DKSM), which realizes dynamic adjustment of the influence degree of different keywords in the classification task by scaling the embedding vector of each keyword in the original text, effectively avoiding the limitations of static keyword matching. In the classification process, this dynamic adjustment mechanism enables the first model to focus on the most relevant keywords and semantic features for the current task, thereby improving the classification accuracy and making it better adapt to the multi-level problem classification requirements.

[0071] Optionally, the scaling of the embedding vector of each keyword in the original text through the dynamic keyword suppression module in the first model specifically comprises:

[0072] The dynamic keyword suppression module performs the following operations:

[0073] identifying the keywords in the original text based on a preset keyword library, and marking the positions and categories of the keywords in the original text;

[0074] generating a context-aware vector of each word in the marked original text using a BiGRU (Bidirectional Gated Recurrent Unit) encoding layer;

[0075] ​The context-aware vector of each word is input into a gated inhibition network. The effectiveness of each word in the context is dynamically evaluated using the gated inhibition network, and corresponding gating coefficients are generated. The embedding vectors of the corresponding keywords are scaled according to the gating coefficients, while the embedding vectors of non-keywords remain unchanged.

[0076] In this embodiment, the dynamic keyword suppression module includes a keyword detection module, a context-aware control module, and a gating network. The keyword detection module first identifies keywords in the original text, matches them against a pre-defined keyword database, and marks the position and category of the keywords. Then, the marked original text is input into the context-aware control module, which includes a BiGRU encoding layer to generate a context-aware vector for each word, capturing semantic information from the surrounding context. Next, the context-aware vector of each word is input into the gating network to dynamically evaluate the effectiveness of the keyword in the context, generate gating coefficients (i.e., suppression coefficients), and scale the keyword embedding vectors.

[0077] It should be noted that keywords are categorized into three types: positive, negative, and ineffective. Specifically: positive keywords are those that positively contribute to understanding and classifying the original manuscript; negative keywords are those that negatively contribute to understanding and classifying the original manuscript; and ineffective keywords are those that have no impact on understanding and classifying the original manuscript.

[0078] Optionally, the formula for calculating the gating coefficient is:

[0079] g i =σ(W g ·[h i m i ]+b g )

[0080] Among them, g i Let W represent the gating coefficient of the i-th word, σ represent the Sigmoid activation function, and W represent the gating coefficient of the i-th word. g For the weight matrix, h i Let m represent the context-aware vector of the i-th word. i This represents the keyword mask for the i-th word. When the i-th word is a keyword, m i The value of m is 1 when the i-th word is not a keyword. i The value is 0, [h i m i ] represents h i and m i The concatenated vector, b g This is a bias term.

[0081] In this embodiment, the gating coefficient is calculated for each word, where W g ∈R d+1×1 , b g ∈R, and sigma represents a Sigmoid activation function that compresses the output to [0, 1].

[0082] Optionally, the scaling of the embedding vector of the corresponding keyword according to the gating coefficient comprises:

[0083] Multiplying the embedding vector of each keyword with the corresponding gating coefficient to achieve scaling of the keyword embedding vector.

[0084] In this embodiment, keyword suppression only adjusts the weight of words marked as keywords (m i = 1), and the suppression formula of keywords is as follows:

[0085]

[0086] wherein, represents the adjusted embedding vector, e i represents the embedding vector of the i-th word, and the embedding vector of the keyword is dynamically scaled to enhance the weight of important keywords and suppress the weight of unimportant keywords.

[0087] Optionally, the first classification label of the original text is obtained according to the scaled embedding vector of the original text, and specifically comprises:

[0088] Segmenting the scaled embedding vector of the original text, and inputting the segmented embedding vector into a BERT (Bidirectional Encoder Representations from Transformers) model for semantic encoding;

[0089] Splicing or fusing the semantic encoding results corresponding to each segment to generate a corresponding semantic vector;

[0090] Inputting the semantic vector into an LSTM (Long Short-Term Memory) model, and calculating based on the semantic vector through the LSTM model to capture sequence dependency and long-term context information in the text, and outputting a corresponding feature vector;

[0091] Transferring the feature vector to a fully connected layer to obtain the first classification label of the original text.

[0092] In this embodiment, the embedding vector of the original text after scaling may need to be segmented due to the length of the text exceeding the maximum limit of BERT (such as 512 tokens). After segmenting the text as required, the BERT model is input for deep semantic coding. The BERT model processes each segment of text and extracts semantic information. Then, the semantic coding results of each segment of text output by BERT are spliced or fused to generate a corresponding semantic vector. Since the LSTM model can capture sequence dependency and long-term context information in the text, the semantic vector is input into the LSTM model, and the LSTM model outputs a feature vector. The feature vector is transmitted to the full connection layer, which performs linear transformation and nonlinear activation operation on the feature vector output by the LSTM to integrate key information in the sequence features. By designing the number of neurons and activation functions (ReLU, Sigmoid) of the full connection layer, it can map the sequence features to the classification label space.

[0093] Step S103: inputting the original text, the preset prompt word, and the information related to the original text retrieved from the historical knowledge base into a second model to enable the second model to generate a corresponding second classification label, wherein the second model is a large language model.

[0094] In this embodiment, the large language model can be DeepSeek, GPT series (such as GPT-3.5, GPT-4), etc. The original text, the preset prompt word, and the information related to the original text retrieved from the historical knowledge base (including the historical audit draft text segment and the corresponding classification label retrieved from the historical knowledge base) are input into the second model to enable the second model to generate a corresponding second classification label.

[0095] Optionally, before the inputting the original text, the preset prompt word, and the information related to the original text retrieved from the historical knowledge base into the second model to enable the second model to generate a corresponding second classification label, the method further comprises:

[0096] The information related to the original text is retrieved from the historical knowledge base by RAG (Retrieval-Augmented Generation) technology, wherein the historical knowledge base stores historical audit draft texts and corresponding classification labels.

[0097] In this embodiment, the original text is first vectorized to form an embedding vector, and then relevant content is matched in the historical knowledge base. The matched historical content is input into the second model (such as DeepSeek large model). The second model uses the prompt word, the input original text, and the retrieved relevant historical content to output a second classification label in combination with the general large model capability of the second model.

[0098] Step S104: fuse the first classification label and the second classification label.

[0099] In this embodiment, fusing the first classification label and the second classification label means performing a union process on the first classification label and the second classification label.

[0100] Step S105: display the fused classification label, so that a user modifies or confirms the fused classification label to obtain a final audit working paper classification label.

[0101] In this embodiment, the user can feed back the classification label: delete the classification label that does not conform, or modify the classification label (the classification label is generally fixed in the pre-training model stage, and the classification label library formed by summarizing historical working papers and company specifications, and the user can only select the fixed label to modify the label). Of course, the user can also not edit the label, but only mark the classification label as identified or not identified.

[0102] Step S106: recommend an audit working paper based on the final audit working paper classification label.

[0103] In this embodiment, through the working paper classification, the actual needs of the user and the audit scene can be more accurately matched, the accurate recommendation of the audit working paper is realized, thereby reducing invalid information interference and improving the pertinence and efficiency of the audit work.

[0104] Optionally, the method further comprises at least one of the following:

[0105] uploading the original text and the corresponding audit working paper classification label to a system log, and regularly or irregularly extracting information that has not been synchronized to the historical knowledge base from the system log to complete the update of the historical knowledge base;

[0106] receiving a keyword fed back by a user, and updating the keyword library according to the keyword fed back by the user.

[0107] In this embodiment, by regularly analyzing the system log, the user's modification opinions on the classification result, the newly added audit scene requirements and other potential information can be mined; combined with the keywords actively fed back by the user, these dynamic information is synchronized to the historical knowledge base and the keyword library in a timely manner, forming a closed-loop feedback mechanism, so that the accuracy of the classification label and the adaptability of the keyword library can be continuously optimized, and the dynamic response capability of the system to the audit scene can be improved.

[0108] It should be noted that, in order to solve the problems of the existing audit working paper recommendation method, such as insufficient recommendation accuracy, difficulty in adapting to the dynamic evolution characteristics of the audit business, excessive dependence on a single model, and obvious limitations in the source of the final result, the present application makes the following design:

[0109] 1. The fusion model has the advantage of precise classification, breaking the limitations of single model.

[0110] (1) BERT+LSTM model: BERT is good at capturing the deep semantic information of text, and can understand the complex semantic relationship and subtle difference in the text; LSTM performs well in processing sequence data, and can capture the long and short term dependence relationship in the text, and grasp the overall semantic structure of the sentence. The combination of the two can accurately understand the semantics and extract features of the audit text (i.e. the original text of the audit draft), so as to more accurately judge the classification of the text.

[0111] (2) DeepSeek large model combined with prompt and RAG technology: DeepSeek large model has strong language understanding and generation ability, and through the design of prompt, it can guide the model to focus on the key information in the audit text. RAG technology further enhances the ability of the model, which can extract key information from a large amount of text and make reasoning and judgment based on a rich knowledge base. This makes the model not only be able to rely on the surface semantics of the text, but also be able to combine background knowledge for in-depth analysis, improving the accuracy and rationality of classification.

[0112] This design breaks the limitations of single model. Compared with traditional static rules or simple keyword matching method, this fusion strategy can more accurately grasp the core business semantics of the audit text and achieve more accurate multi-label classification. This not only improves the accuracy of classification, but also effectively reduces the recommended range, greatly reducing the workload of manual secondary screening.

[0113] 2. Dynamic keyword weight adjustment and semantic understanding

[0114] (1) Dynamic keyword module suppression (DKSM) can dynamically adjust the influence of keywords according to different classification tasks, avoiding the limitations of static keyword matching. In the classification process, the model can focus on the most relevant keywords and semantic features to the current task, thereby improving the classification accuracy and making it better adapt to the needs of multi-level problem classification.

[0115] (2) Context-aware gating module, through the fusion of multi-head attention mechanism and dependency syntax features, deeply understands the context of keywords, accurately judges the effectiveness of keywords, and avoids misjudgment caused by keyword ambiguity or negative context. For example, in a negative sentence, the module can accurately identify and suppress irrelevant keywords, thereby narrowing the recommended range and improving the recommendation accuracy.

[0116] 3. Self-optimization and iterative upgrade of recommendation system

[0117] (1) The designed feedback mechanism enables users to edit the classification results and record the edited information in the log. By regularly analyzing the log, the users' modification opinions and new classification requirements for the classification results are mined, and these feedback information is updated to the RAG knowledge base in a timely manner, forming a closed-loop feedback mechanism.

[0118] (2) By regularly collecting user feedback information on the classification results, the keyword library is updated and the model parameters are optimized, realizing self-optimization and iterative upgrading of the DKSM module.

[0119] In a specific embodiment, an audit document recommendation method is provided as an innovative audit text processing technology, which has the core advantage of accurately classifying audit documents with multiple labels. This method combines the BERT+LSTM model, the dynamic keyword suppression module, and the DeepSeek large model + prompt word + RAG technology. First, the BERT deeply understands the semantic information of the text, captures the long and short term dependencies through LSTM, and dynamically adjusts the influence of keywords with the help of the dynamic keyword suppression module (DKSM) to achieve preliminary accurate classification. Then, the DeepSeek large model is used in combination with the carefully designed prompt word and the RAG technology that retrieves relevant information from the historical document database to further optimize the classification results. The classification accuracy is improved through the fusion of the two models, and a closed-loop feedback mechanism is established. The feedback log of the users on the classification results is regularly analyzed, and the feedback information is updated to the RAG knowledge base in a timely manner to form a dynamically updated knowledge base. The user feedback content is analyzed, and the keyword library is updated to the keyword library. This effectively solves the limitations of traditional solutions, improves the automation and accuracy of audit document classification, and realizes efficient and accurate multi-label classification of audit documents and continuous optimization of the subsequent knowledge base. Figure 2 The architecture diagram of the audit document recommendation method is shown, which specifically includes:

[0120] Step one: input the original text of the document;

[0121] Step two: process the input original text of the document through the capabilities of two models (model M1 and model M2), and finally output the classification labels corresponding to the original text of the document.

[0122] Step three: users feedback on the accuracy of the classification labels to form a feedback log. The feedback log is updated to the RAG knowledge base in a timely manner to form a dynamically updated knowledge base; at the same time, the user feedback log is analyzed, the keyword library is mined, and the keyword library is updated.

[0123] Specifically, the steps of the model M1 are one of the dual-model architectures of the present embodiment. With the dynamic keyword suppression module (DKSM), the effectiveness of keywords in the context is dynamically evaluated, and by generating a suppression coefficient g e [0, 1], the embedding vector of the keyword is scaled to control its influence on the model prediction; BERT is good at capturing the deep semantic information of the text and can understand the complex semantic relationship and subtle differences in the text; LSTM performs well in processing sequence data and can capture the long and short term dependencies in the text and grasp the overall semantic structure of the sentence. Finally, the fully connected layer is connected to form the overall neural network classification model M1, and the overall architecture of the model M1 is as shown in Figure 3 The steps of the model M1 are as follows:

[0124] Step S1, input the original text of the audit working papers (such as the paragraphs of the audit report), and output the embedding vector of the keyword generation context-aware vector generation suppression coefficient through the dynamic keyword suppression module DKSM.

[0125] The dynamic keyword suppression module DKSM is a module specially designed for the classification of the audit field in the M1 model, and plays a key role in the classification of the audit. By accurately identifying and adjusting the weight of the keyword, the accuracy and adaptability of the classification are effectively improved. It first identifies the keyword in the text, and then analyzes its context using the gating suppression network module to dynamically adjust the weight, suppress the influence of the keyword in the ambiguous or negative context, and at the same time enhance the semantic strength of the core keyword. The architecture diagram of the dynamic keyword suppression module is as shown in Figure 4 The steps of the dynamic keyword suppression module are as follows:

[0126] (1) Input the original text data (such as the paragraphs of the audit report), and provide it to the keyword detection module. The keyword detection module identifies the keywords in the text, matches them with the pre-defined classification label library (i.e. the keyword library), and marks the position (i.e. the position in the original text) and category of the keywords.

[0127] Among them, the label library pre-stores the keywords, including some keywords with negative context and ambiguous modification. The keywords here can be divided into three categories: keywords that have a positive factor on understanding the original text of the working papers and classifying the original text; keywords that have a negative factor on understanding the original text of the working papers and classifying the original text; and keywords that have no influence on understanding the original text of the working papers and classifying the original text. The formation of this keyword library needs to be dynamically formed by analyzing historical audit working papers to form a classification label library.

[0128] (2) Input the marked text into the context-aware control module, which contains a BiGRU encoding layer for generating a context-aware vector for each word to capture the semantic information of the preceding and following text.

[0129] Where, the marked text refers to the marked original text and the keyword position and the above-mentioned keyword category (positive, negative, no effect).

[0130] (3) Gating Network. Dynamically evaluate the effectiveness of keywords in the context, generate inhibition coefficient, scale the embedding vector of keywords.

[0131] ① Input definition: context-aware vector: h i ∈R d , that is, the context-aware vector generated by BiGRU. Keyword mask: m i ∈{0,1}, mark whether the ith word is a keyword (1 means it is a keyword).

[0132] ② Gating coefficient calculation: g i =σ(W g ·[h i ;m i ]+b g ), where,

[0133] W g ∈R d+1×1 : weight matrix;

[0134] [h i ;m i ]: represents the concatenated vector of context-aware vector h i and keyword mask m i .

[0135] b g ∈R: bias term;

[0136] σ: Sigmoid activation function, compresses the output to [0,1].

[0137] ③ Dynamic inhibition rule. Keyword inhibition: only adjust the weight of the word marked as a keyword (m i =1), and the inhibition formula of the keyword is as follows:

[0138]

[0139] Where, represents the adjusted embedding vector, e i represents the embedding vector of the ith word.

[0140] It should be noted that the gating coefficient (i.e. inhibition coefficient) is calculated for each word, but only the weight of the keyword is adjusted (the m i =0 of non-keywords, output the original vector). The inhibition coefficient is used to inhibit the effect of the keyword.

[0141] where the suppression effect is explained as follows:

[0142] g i → 0: indicates complete suppression of the semantic contribution of the keyword;

[0143] g i → 1: indicates preservation of the original semantic intensity;

[0144] g i > 0: indicates enhancement of the keyword's impact (requires the design of a special activation function).

[0145] Step S2, the processed embedding vector (referring to the embedding vector of the original text data, but in this embedding vector, the keyword is suppressed and contains the suppression information of the keyword) may need to be segmented due to the length of the text exceeding the maximum limit of BERT (such as 512 tokens). After segmenting the text as required, input each segment into the BERT model for deep semantic encoding. The BERT model processes each segment of text and extracts semantic information.

[0146] Step S3, the semantic encoding results of each segment of text output by BERT are spliced or fused. BERT model uses its powerful Transformer architecture to encode these embedding vectors, capturing complex semantic relationships and context dependencies in the text. After processing by multiple layers of self-attention mechanism and feedforward neural network, the BERT model generates new semantic representation vectors. In particular, the vector corresponding to the [CLS] token output by BERT is used as the global semantic representation of the entire text. This vector not only integrates the word features adjusted by the gating suppression network, but also condenses the deep semantic understanding of the text by the BERT model, providing high-quality input features for the downstream LSTM model.

[0147] Step S4, the LSTM model captures the sequence dependency and long-term context information in the text, and the output features are passed to the fully connected layer. The fully connected layer performs linear transformation and nonlinear activation operation on the feature vectors output by LSTM, integrating key information in the sequence features. By designing the number of neurons and activation functions (ReLU, Sigmoid) of the fully connected layer, it can map the sequence features to the classification label space. In the model training phase, using the label information in the training data set, through the optimization algorithm (Adam), the weight parameters of the fully connected layer are constantly adjusted to minimize the difference between the predicted label and the true label.

[0148] It should be noted that in step S3, the text encoding results output by the BERT model are spliced or fused to generate new semantic representation vectors. In step S4, the semantic vectors output by BERT are input into the LSTM model. The LSTM model has the ability to capture the dependency relationship and long-term context information of the text sequence, and it will calculate based on these semantic vectors to output corresponding feature vectors, which integrate the key information of the text in the sequence aspect. The feature vectors output by the LSTM are passed to the fully connected layer. The fully connected layer further integrates and processes these feature vectors through linear transformation and nonlinear activation operations, so that the final output can correspond to the classification label of the draft (i.e., the first classification label). The classification label may be, for example: income management throughout the process, cost problem management, asset management, fund management, system control, engineering project control, agent and agent commission control, etc.

[0149] Specifically, the model M2 is another model of the dual-model architecture of the present embodiment, which integrates the application of the knowledge base module, the DeepSeek large model, and the prompt word engineering. In the knowledge base module, structured "text-classification label" data is formed by auditing historical records, which can help the model quickly locate key information points in the text. The DeepSeek large model uses its own capabilities to capture complex semantic relationships and potential meanings in the input text, and outputs classification labels in combination with knowledge base search content. The overall architecture of the model M2 is shown in Figure 5 as follows: specifically including the following steps:

[0150] Step S1, input the original text data and submit it to the DeepSeek large model.

[0151] Step S2, simultaneously vectorize the original text to form an embedding vector, and match related content in the historical knowledge base. Input the matched historical content into the DeepSeek large model.

[0152] Step S3, the DeepSeek large model uses the prompt word, the input original text, and the searched related historical content, in combination with its general large model capabilities, to output classification labels (i.e., the second classification label).

[0153] Specifically, in the dual-model architecture, after the models M1 and M2 output classification labels, they enter the dynamic feedback layer F. The user modifies and confirms these classification labels, and uploads the feedback "text-classification label" to the system log. The system regularly extracts records from the log to update the historical knowledge base. At the same time, key words are extracted from the feedback classification label feedback log to automatically update the historical knowledge base and the keyword library, realizing the dynamic optimization of the historical knowledge base and the keyword library.

[0154] It should be noted that the fusion of the classification label means that the output classification label is processed by set operation. The user can feed back the classification label: delete the classification label that does not conform, or modify the classification label (the classification label is generally fixed in the pre-training model stage, and the classification label library formed by summarizing historical manuscripts and company specifications, and the user can only select the fixed label to modify the label). Of course, the user can also not edit the label, but only mark the classification label as recognized or not recognized; at the same time, the user can also mark the keywords in the manuscript text. The above feedback information is not mandatory, and the user can directly adopt the classification label generated by the model to reduce the complexity of user operation. Finally, these feedback information is summarized into a feedback information library.

[0155] It should be noted that M1 is a self-training model, which is more suitable for the audit habits and audit rules of the use unit; the M2 model is a general large model with the ability and industry standardization specification, and the two models are complementary, which can improve the accuracy and professionalism of classification.

[0156] It should be noted that the historical knowledge base is used to store classification labels and full texts of manuscripts, and the keyword library refers to the keywords extracted above, and does not include classification labels. The classification label fed back by the user is placed in the historical knowledge base in combination with the full text of the manuscript.

[0157] It should be noted that the audit manuscript recommendation method provided by the application has the following characteristics:

[0158] a) Contextual adaptive weight adjustment of dynamic keyword suppression module (DKSM): According to the characteristics of the audit text, the joint input design of keyword mask and context semantic vector (BiGRU output) is used to realize fine-grained weight adjustment. Based on the gating mechanism of dynamically suppressing or enhancing the weight of the keyword in the context, especially for the accurate processing of negative context and ambiguous modification in the audit text.

[0159] b) Classification architecture of double model cooperation (M1 and M2): Model M1: fusion architecture of DKSM+BERT+LSTM module, focusing on local semantic and long-range dependency analysis of audit text, improving field adaptability through dynamic keyword suppression, and forming classification labels. Model M2: based on DeepSeek large model+RAG retrieval enhancement+prompt word engineering, using historical knowledge base (text-classification label) to optimize global classification decision, outputting classification label.

[0160] c) Double dynamic feedback driven knowledge base closed loop optimization: User feedback log automatic analysis technology: update keyword library through user marked keywords, provide training data source for subsequent M1 model update. Knowledge base incremental update strategy: update the original text to the historical knowledge base combined with the classification tag feedback information of the user to the original text, improve the accuracy of the knowledge base retrieval content; at the same time, the original text and the classification tag are used as the training data source for updating the subsequent M1 model.

[0161] The audit document recommendation method provided by the embodiment of the application first acquires an audit document original text; then inputs the original text into a first model, scales the embedding vector of each keyword in the original text through a dynamic keyword suppression module in the first model, and obtains a first classification label of the original text according to the embedding vector of the original text obtained after scaling; at the same time, inputs the original text, a preset prompt word, and information related to the original text retrieved from a historical knowledge base into a second model, so that the second model generates a corresponding second classification label, wherein the second model is a large language model; then fuses the first classification label and the second classification label; and displays the fused classification label, so that a user modifies or confirms the fused classification label to obtain a final audit document classification label; finally, performs audit document recommendation based on the final audit document classification label. Through the fusion strategy and the dynamic keyword suppression technology, the application can more accurately grasp the core business semantics of the audit text and realize more accurate multi-label classification, which not only improves the accuracy of classification, but also effectively reduces the recommendation range and significantly reduces the workload of manual secondary screening. Specifically, the application scales the keyword embedding vector in the original text through the dynamic keyword suppression module, which can more accurately capture the core semantics of the text, enhance the adaptability of the model to dynamically changing audit business, and improve the accuracy of the first classification label. At the same time, in combination with the preset prompt word and the related information in the historical knowledge base, the second classification label is generated by using the large language model, which further enriches the basis for classification. In addition, by fusing the first and second classification labels and allowing the user to modify or confirm, the application not only further improves the classification accuracy, but also avoids over-reliance on a single model through double model cooperation, thereby solving the problems of the existing audit document recommendation method, such as insufficient recommendation accuracy, difficulty in adapting to the dynamic evolution characteristics of audit business, and obvious limitations in the source of the final result caused by over-reliance on a single model.

[0162] Embodiment 2:

[0163] As shown in Figure 6 The embodiment provides an audit document recommendation device for executing the above-mentioned audit document recommendation method, comprising:

[0164] An original text acquisition module 11 is configured to acquire an original text of an audit working paper;

[0165] A first classification module 12 is connected with the original text acquisition module 11 and configured to input the original text into a first model, scale an embedding vector of each keyword in the original text by a dynamic keyword suppression module in the first model, and obtain a first classification label of the original text according to an embedding vector of the original text after scaling.

[0166] A second classification module 13 is connected with the original text acquisition module 11 and configured to input the original text, a preset prompt word, and information related to the original text retrieved from a historical knowledge base into a second model to enable the second model to generate a corresponding second classification label, wherein the second model is a large language model.

[0167] A classification label fusion module 14 is connected with the first classification module 12 and the second classification module 13 and configured to fuse the first classification label and the second classification label.

[0168] A classification label determination module 15 is connected with the classification label fusion module 14 and configured to display the fused classification label to enable a user to modify or confirm the fused classification label to obtain a final audit working paper classification label.

[0169] An audit working paper recommendation module 16 is connected with the classification label determination module 15 and configured to recommend an audit working paper based on the final audit working paper classification label.

[0170] Optionally, the first classification module 12 includes:

[0171] A keyword suppression unit is configured to perform the following operations by the dynamic keyword suppression module:

[0172] Identify keywords in the original text based on a preset keyword library and mark positions and categories of the keywords in the original text;

[0173] Generate a context-aware vector of each word in the marked original text by using a bidirectional gated recurrent unit (BiGRU) encoding layer;

[0174] Input the context-aware vector of each word into a gated suppression network, dynamically evaluate the effectiveness of each word in the context by using the gated suppression network, generate a corresponding gating coefficient, and scale an embedding vector of a corresponding keyword according to the gating coefficient, while an embedding vector of a non-keyword remains unchanged.

[0175] Optionally, a calculation formula of the gating coefficient is:

[0176] g i =σ(W g ·[h i ;m i ]+b g )

[0177] wherein g i represents the gating coefficient of the i-th word, σ represents the Sigmoid activation function, W g is a weight matrix, h i represents the context-aware vector of the i-th word, m i represents the keyword mask of the i-th word, the value of m i is 1 when the i-th word is a keyword, and the value of m i is 0 when the i-th word is not a keyword, [h i ; m i ] represents the vector obtained by splicing h i and m i , and b g is a bias term.

[0178] Optionally, the keyword suppression unit comprises:

[0179] an embedding vector scaling unit configured to multiply the embedding vector of each keyword with the corresponding gating coefficient to achieve scaling of the keyword embedding vector.

[0180] Optionally, the first classification module 12 further comprises:

[0181] a segmentation unit configured to segment the scaled embedding vector of the original text, and input the segmented embedding vector into a bidirectional encoder representation from Transformer (BERT) model for semantic encoding;

[0182] a semantic vector generation unit configured to splice or fuse the semantic encoding results corresponding to each segment to generate a corresponding semantic vector;

[0183] a feature vector acquisition unit configured to input the semantic vector into a long short-term memory (LSTM) model, and capture sequence dependency and long-term context information in the text by calculating based on the semantic vector through the LSTM model, and output a corresponding feature vector;

[0184] a first classification label acquisition unit configured to input the feature vector into a fully connected layer to obtain a first classification label of the original text.

[0185] Optionally, the apparatus further comprises:

[0186] The retrieval matching module is configured to retrieve information related to the original text from a historical knowledge base by a retrieval augmented generation (RAG) technique, wherein the historical knowledge base stores historical audit document texts and corresponding classification labels.

[0187] Optionally, the device further comprises at least one of the following:

[0188] The knowledge base updating module is configured to upload the original text and the corresponding audit document classification label to a system log, and periodically or non-periodically extract information from the system log that has not been synchronized to the historical knowledge base to complete updating of the historical knowledge base.

[0189] The keyword library updating module is configured to receive a user feedback keyword and update the keyword library according to the user feedback keyword.

[0190] Embodiment 3

[0191] Reference Figure 7 The embodiment provides an audit document recommendation device, which comprises a memory 21 and a processor 22. The memory 21 stores a computer program. The processor 22 is configured to run the computer program to perform the audit document recommendation method in Embodiment 1.

[0192] The memory 21 is connected with the processor 22. The memory 21 can be a flash memory or a read-only memory or other memories. The processor 22 can be a central processing unit or a single-chip microcomputer.

[0193] Embodiment 4

[0194] The embodiment provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the audit document recommendation method in Embodiment 1.

[0195] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, computer program modules or other data. Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile discs (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer.

[0196] In summary, the audit document recommendation method, device and readable storage medium provided by the embodiment of the present application first acquire an original text of an audit document; then input the original text into a first model, scale the embedding vector of each keyword in the original text through a dynamic keyword suppression module in the first model, and obtain a first classification label of the original text according to the embedding vector of the original text obtained after scaling; at the same time, input the original text, a preset prompt word, and information related to the original text retrieved from a historical knowledge base into a second model to enable the second model to generate a corresponding second classification label, wherein the second model is a large language model; then fuse the first classification label and the second classification label; and display the fused classification label to enable a user to modify or confirm the fused classification label to obtain a final audit document classification label; finally, recommend an audit document based on the final audit document classification label. Through the fusion strategy and dynamic keyword suppression technology, the present application can more accurately grasp the core business semantics of the audit text and achieve more accurate multi-label classification, which not only improves the accuracy of classification, but also effectively reduces the recommended range and significantly reduces the workload of manual secondary screening. Specifically, the present application can more accurately capture the core semantics of the text by scaling the keyword embedding vector in the original text through the dynamic keyword suppression module, thereby enhancing the adaptability of the model to dynamically changing audit businesses and improving the accuracy of the first classification label. At the same time, in combination with the preset prompt word and the related information in the historical knowledge base, the second classification label is generated by using the large language model, which further enriches the basis for classification. In addition, by fusing the first and second classification labels and allowing the user to modify or confirm, the present application not only further improves the classification accuracy, but also avoids over-reliance on a single model through the cooperation of the two models, thereby solving the problems of the existing audit document recommendation method, such as insufficient recommendation accuracy, difficulty in adapting to the dynamic evolution characteristics of the audit business, and obvious limitations in the source of the final result due to over-reliance on a single model.

[0197] It can be understood that the above embodiments are only exemplary embodiments adopted for illustrating the principles of the present application, and the present application is not limited thereto. Various modifications and improvements can be made by those of ordinary skill in the art without departing from the spirit and essence of the present application, and these modifications and improvements are also considered to be within the protection scope of the present application.

Claims

1. A method for recommending audit working papers, characterized in that, The method includes: Obtain the original text of the audit working papers; The original text is input into the first model, and the embedding vector of each keyword in the original text is scaled by the dynamic keyword suppression module in the first model. The first classification label of the original text is obtained based on the embedding vector of the original text obtained after scaling. The original text, preset prompt words, and information related to the original text retrieved from the historical knowledge base are input into the second model so that the second model generates the corresponding second category label, wherein the second model is a large language model; Merge the first category label and the second category label; The merged category labels are displayed so that users can modify or confirm them, resulting in the final audit working paper category labels. Audit working paper recommendations are made based on the final audit working paper classification tags.

2. The method according to claim 1, characterized in that, The scaling of the embedding vector of each keyword in the original text through the dynamic keyword suppression module in the first model specifically includes: The dynamic keyword suppression module performs the following operations: Based on a preset keyword database, keywords in the original text are identified, and the position and category of the keywords are marked in the original text. The context-aware vector of each word in the original text after being marked is generated using a bidirectional gated recurrent unit (BiGRU) encoding layer. The context-aware vector of each word is input into a gated inhibition network. The effectiveness of each word in the context is dynamically evaluated using the gated inhibition network, and corresponding gating coefficients are generated. The embedding vectors of the corresponding keywords are scaled according to the gating coefficients, while the embedding vectors of non-keywords remain unchanged.

3. The method according to claim 2, characterized in that, The formula for calculating the gating coefficient is as follows: g i =σ(W g ·[h i ;m i ]+b g ) Among them, g i Let W represent the gating coefficient of the i-th word, σ represent the Sigmoid activation function, and W represent the gating coefficient of the i-th word. g For the weight matrix, h i Let m represent the context-aware vector of the i-th word. i This represents the keyword mask for the i-th word. When the i-th word is a keyword, m i The value of m is 1 when the i-th word is not a keyword. i The value is 0, [h i ;m i ] represents h i and m i The concatenated vector, b g This is a bias term.

4. The method according to claim 2, characterized in that, The scaling of the embedding vector of the corresponding keyword according to the gating coefficient specifically includes: The embedding vector of each keyword is multiplied by the corresponding gating coefficient to achieve scaling of the keyword embedding vector.

5. The method according to claim 1, characterized in that, The step of obtaining the first classification label of the original text based on the embedding vector of the scaled original text specifically includes: The original text embedding vector obtained after scaling is segmented, and the segmented embedding vectors are input into the BERT model based on Transformer bidirectional encoder for semantic encoding. The semantic encoding results of each segment are concatenated or merged to generate the corresponding semantic vector. The semantic vector is input into the Long Short-Term Memory (LSTM) network model. The LSTM model calculates based on the semantic vector to capture sequence dependencies and long-term contextual information in the text and outputs the corresponding feature vector. The feature vector is passed to a fully connected layer to obtain the first classification label of the original text.

6. The method according to claim 1, characterized in that, Before inputting the original text, preset prompt words, and information related to the original text retrieved from the historical knowledge base into the second model so that the second model generates the corresponding second classification label, the method further includes: Information related to the original text is retrieved from a historical knowledge base by using the Retrieval Enhancement Generative Grading (RAG) technique, wherein the historical knowledge base stores historical audit working papers and corresponding classification tags.

7. The method according to claim 2, characterized in that, The method further includes at least one of the following: The original text and corresponding audit working paper classification tags are uploaded to the system log. Information that has not yet been synchronized to the historical knowledge base is extracted from the system log periodically or irregularly to complete the update of the historical knowledge base. Receive keywords from users and update the keyword database based on those keywords.

8. An audit working paper recommendation device, characterized in that, The device includes: The raw text acquisition module is used to acquire the raw text of audit working papers; The first classification module is connected to the original text acquisition module and is used to input the original text into the first model, scale the embedding vector of each keyword in the original text through the dynamic keyword suppression module in the first model, and obtain the first classification label of the original text based on the scaled embedding vector of the original text. The second classification module is connected to the original text acquisition module and is used to input the original text, preset prompt words, and information related to the original text retrieved from the historical knowledge base into the second model so that the second model generates corresponding second classification labels. The second model is a large language model. A classification label fusion module, connected to the first classification module and the second classification module, is used to fuse the first classification label and the second classification label; The category label determination module is connected to the category label fusion module and is used to display the fused category labels so that the user can modify or confirm the fused category labels to obtain the final audit working paper category labels. The audit working paper recommendation module, connected to the classification label determination module, is used to recommend audit working papers based on the final audit working paper classification labels.

9. An audit working paper recommendation device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to implement the audit working paper recommendation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the audit working paper recommendation method as described in any one of claims 1-7.

Images