Cross-domain secure interaction method and system for public service system based on universal large model
By combining natural language processing, graph neural network and deep learning technology, the problem of security knowledge acquisition and interaction consistency of general big models in the accounting field is solved, intelligent interaction and data security in the accounting field is realized, and the system is enhanced Internal control management capabilities.
Patent Information
- Application Number
- CN202411244353.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-06
AI Technical Summary
When applying general models to the field of system security, it is difficult to learn sufficient comprehensive and accurate security knowledge from public corpus, and the generated security instructions or configuration suggestions lack interpretability and credibility, making it difficult to ensure that many Global consistency and logical coherence of the round-robin security interaction process.
By receiving business query requests from accounting user terminals, using natural language processing technology for identity verification and semantic analysis, combining the request legitimacy detection model of graph neural network and the pre-trained accounting field general model, multi-level representation learning and deep semantics are performed Inferencing, generating preliminary response results, and processing sensitive information through deep learning named entity recognition and privacy computing technology to ensure the data security of reply.
It realizes reliable, secure and intelligent interaction in the accounting field that has multiple rounds of dialogue capabilities, enhances the system's internal control management and prevents financial fraud risks, and ensures the interpretability and security of responses.
Smart Images

Figure CN118761852B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to security technology, and in particular to a cross-domain security interaction method and system for a public service system based on a general large model. Background Art
[0002] The general large model can automatically learn semantic representations and logical relationships from massive unstructured data, and has multiple intelligent interaction capabilities such as language understanding, knowledge reasoning, and multi-round dialogues, providing technical support for introducing more flexible and intelligent human-computer interaction methods into traditional information systems.
[0003] However, there are still many technical challenges when applying general large models to the field of system security. On the one hand, due to the high sensitivity and professionalism of system security data, it is difficult for general large models to directly learn sufficiently comprehensive and accurate security knowledge from public corpora, and it is difficult to perform professional security question-answering, security decision-making assistance and other tasks. On the other hand, due to the black box characteristics of general large models, the security instructions or configuration suggestions they generate lack interpretability and credibility, and the risks are uncontrollable. In addition, general large models for open scenarios are difficult to ensure the global consistency and logical coherence of multiple rounds of security interaction processes.
[0004] In the existing technology, security expert systems and rule-based intelligent security assistants built for specific security scenarios have achieved automated system security management to a certain extent. However, such methods usually rely on predefined rule knowledge bases, lack flexibility and scalability, and are difficult to adapt to the ever-changing security threat situation. At the same time, rule-matching interaction methods are difficult to support multi-round dialogues and personalized interactions. Summary of the invention
[0005] The embodiments of the present invention provide a method and system for cross-domain secure interaction of a public service system based on a universal large model, which can solve the problems in the prior art.
[0006] According to a first aspect of the embodiments of the present invention,
[0007] Provide a cross-domain secure interaction method for public service systems based on a general large model, including:
[0008] Receive business query requests from accounting user terminals, use the request identification model to authenticate the source of the business query request, and compare it with the accounting personnel information database to determine whether the business query request has the corresponding accounting business authority; for business query requests that have passed identity authentication, use natural language processing technology to perform semantic analysis, extract accounting professional terms contained in the business query request, and accounting features of business scenarios; use a request legitimacy detection model based on a graph neural network, using the knowledge graph of violation cases in the accounting field as a training sample, to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations;
[0009] If there is no compliance risk, the accounting features are input into the pre-trained general model in the accounting field, and the general model in the accounting field is used to perform multi-level representation learning on the accounting features. The attention mechanism is used to focus on the key information points of the business query request, and deep semantic reasoning is carried out in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request;
[0010] The preliminary reply result is scanned for sensitive information, and a named entity recognition model based on deep learning is used to identify various types of sensitive information in the preliminary reply result; a model is generated according to a self-learning desensitization strategy, and corresponding desensitization processing is performed on sensitive information of different types and importance. After the desensitization processing, privacy computing technology is used to evaluate the privacy leakage risk of the remaining reply content. If it is lower than the preset security threshold, the preliminary reply result is determined to comply with the data security guidelines and is used as the final reply result.
[0011] In an optional embodiment,
[0012] For business query requests that have passed identity authentication, natural language processing technology is used for semantic analysis to extract accounting terms and accounting features of business scenarios contained in the business query requests, including:
[0013] Develop a financial term extraction and semantic disambiguation tool based on the conditional random field algorithm, introduce a financial dictionary to adapt the conditional random field algorithm to the domain, so that it can accurately identify professional terms in business query requests and determine the actual semantics of professional terms based on the context;
[0014] Use the pre-trained Chinese dependency syntactic analysis model to perform syntactic dependency analysis on business query requests, reveal the inherent logical association of accounting business elements in the query text of business query requests, and provide structured clues for semantic reasoning;
[0015] Construct a financial accounting structured knowledge graph for financial accounting business scenarios. When semantically parsing business query requests, link and match the concept nodes of the financial accounting structured knowledge graph in real time based on the syntactic dependency relationship corresponding to the business query request, so as to achieve semantic information fusion of query statements and domain knowledge.
[0016] Comprehensively use natural language processing technology based on the bag-of-words model to extract key information from business query requests after semantic disambiguation, and automatically extract accounting professional terms contained in business query requests and accounting features of business scenarios.
[0017] In an optional embodiment,
[0018] Using a request legitimacy detection model based on a graph neural network and taking the knowledge graph of accounting violation cases as training samples, we can determine whether a business query request contains compliance risks such as financial fraud or illegal or irregular operations. The following are the risks:
[0019] Based on the collected and sorted accounting violation cases, we construct a knowledge graph of accounting violation risk, which includes violation concept nodes and relationship edges. The relationship edges describe the correlation and evolution law between different violation behaviors.
[0020] A heterogeneous graph neural network model is designed that integrates text feature learning and multi-relation reasoning. The heterogeneous graph neural network model can learn the semantic embedding representation of different types of violation concept nodes and dynamically adjust the message transmission and aggregation strategies between violation concept nodes based on the relationship type.
[0021] The pre-trained heterogeneous graph neural network model is used to learn the node semantic embedding representation on the accounting violation risk knowledge graph, and the accounting professional terms and accounting features of the business scenarios extracted from the semantic analysis of the business query request are mapped to the semantic space aligned with the accounting violation risk knowledge graph. The similarity between the accounting professional terms, accounting features of the business scenarios and the violation concept nodes is calculated to determine whether the business query request triggers a known violation risk scenario.
[0022] In the message transmission process of the heterogeneous graph neural network model, the accounting violation risk knowledge graph is used as a path to infer the violation risk associations implicit in the semantic environment of the business query request, and the overall compliance of the business query request is evaluated from the two dimensions of local risk and global risk to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations.
[0023] In an optional embodiment,
[0024] The accounting features are input into the pre-trained general model in the accounting field, and the general model in the accounting field is used to perform multi-level representation learning on the accounting features. The attention mechanism is used to focus on the key information points of the business query request, and deep semantic reasoning is carried out in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request, including:
[0025] Construct a general accounting model for accounting business. Through pre-training with large-scale accounting corpus, the general accounting model can master the vocabulary, grammar and semantic features of accounting language, introduce accounting business characteristic training tasks, and strengthen the model's understanding and application ability of accounting knowledge.
[0026] Use the pre-trained general accounting model to perform multi-layer Transformer encoding on query requests, dynamically adjust the interaction weights between words through the Self-Attention mechanism, and introduce Attention at the top layer of the network to focus on key words and sentences, forming a hierarchical query semantic representation that integrates the attention mechanism;
[0027] Build a knowledge graph in the accounting field covering accounting standards and tax policies, map knowledge graph entities to the pre-trained language model embedding space, identify the starting nodes related to the query semantics, and expand semantically related entities and relationships from the starting nodes through knowledge graph reasoning;
[0028] Identify entity combinations that can represent specific business scenarios in the entity set expanded by the knowledge graph, analyze the grammatical dependencies of entities in the query text to extract the corresponding business logic, and restore the contextual semantics by combining the entity category and attribute modification to generate a business logic context information network that matches the query request;
[0029] The semantic features in the business logic context information network are integrated as the structured generation model input. After semantic interaction through multiple layers of Transformer networks, the Attention mechanism is used to focus on the key semantics. Combined with the language knowledge in the accounting field, the replication mechanism and Beam Search strategy are used to autoregressively decode and generate the response text that matches the query.
[0030] In an optional embodiment,
[0031] The semantic features in the business logic context information network are integrated into the structured generation model input. After the semantic interaction of the multi-layer Transformer network, the Attention mechanism is used to focus on the key semantics. Combined with the language knowledge in the accounting field, the replication mechanism and the Beam Search strategy are used to autoregressively decode and generate the response text that matches the query, including:
[0032] Integrate the multi-granularity heterogeneous semantic fragments identified in the business logic context information network into a unified structured input, wherein the multi-granularity heterogeneous semantic fragments include keywords, phrases, sentences, entities, and relations;
[0033] Through the multi-layer Transformer encoder, the structured semantic input is hierarchically modeled, the distributed representation of semantic units of different granularity is learned, and the local semantics and global semantics are interactively modeled to form a query representation that integrates semantics at different levels.
[0034] The decoder also uses the Transformer architecture to iteratively generate responses in an autoregressive manner. Each decoding step focuses on the semantic fragment in the encoded representation that is most relevant to the generated content, dynamically references key semantic information, and feeds back the decoding status to the encoder to guide subsequent encoding.
[0035] When predicting decoded words, the model vocabulary and query text are combined to flexibly generate and reuse keywords and professional terms through a copy mechanism;
[0036] The Beam Search decoding strategy is adopted to comprehensively consider the generation quality and efficiency to obtain a smooth and coherent response sequence;
[0037] Based on the pre-trained language model, targeted fine-tuning is performed using accounting domain corpus to learn domain text features. At the same time, external structured accounting knowledge is used as an additional supervision signal to guide the model to generate text.
[0038] In an optional embodiment,
[0039] Based on the self-learning desensitization strategy generation model, different types and different levels of importance of sensitive information are desensitized accordingly. After desensitization, privacy computing technology is used to evaluate the privacy leakage risks of the remaining reply content, including:
[0040] Through transfer learning, the pre-trained language model is fine-tuned on large-scale accounting text data, so that it can adaptively learn the diverse characteristics of private information in the accounting context, and build a private information identifier for accounting scenarios, which can accurately detect various explicit or implicit private fragments from accounting texts;
[0041] Collect desensitization examples of different types of financial privacy, build a privacy-desensitization strategy pairing dataset, and train a seq2seq self-learning desensitization strategy generation model on the dataset, which can automatically generate the optimal desensitization strategy for each identified privacy fragment based on the privacy type, importance, and contextual factors;
[0042] The self-learning desensitization model is applied to the reply text of accounting business inquiries. For the identified direct privacy, strong desensitization processing such as encryption and random replacement is adopted, and for the indirect privacy, weak desensitization processing such as generalization and synonym replacement is adopted, so as to protect the privacy while maximally retaining the availability of the reply content.
[0043] Apply homomorphic encryption privacy computing technology to the desensitized reply text, match it with the predefined privacy mode in the ciphertext domain to calculate the overall privacy leakage probability. If the probability exceeds the security threshold, further desensitization is performed iteratively on high-risk text fragments until the overall privacy risk is controlled within an acceptable range.
[0044] The privacy leakage probability calculation is carried out through the federated learning method with multi-party participation, ensuring that the privacy calculation evaluation process itself will not cause additional privacy leakage;
[0045] The output is a secure accounting business response text that has undergone dual privacy protection through self-learning desensitization and privacy computing. While retaining the original meaning well, sensitive privacy information is effectively shielded and the overall privacy leakage risk is reliably controlled.
[0046] According to a second aspect of the embodiments of the present invention,
[0047] Provide a cross-domain secure interaction system for public service systems based on a general large model, including:
[0048] The first unit is used to receive business query requests from accounting user terminals, use the request identification model to authenticate the source of the business query request, and compare it with the accounting personnel information database to determine whether the business query request has the corresponding accounting business authority; for business query requests that have passed identity authentication, use natural language processing technology to perform semantic analysis, extract accounting professional terms contained in the business query request, and accounting features of business scenarios; use a request legitimacy detection model based on a graph neural network, and use the knowledge graph of violation cases in the accounting field as a training sample to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations;
[0049] The second unit is used to input the accounting features into the pre-trained general model in the accounting field if there is no compliance risk, use the general model in the accounting field to perform multi-level representation learning on the accounting features, focus on the key information points of the business query request through the attention mechanism, and conduct deep semantic reasoning in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request;
[0050] The third unit is used to scan the sensitive information of the preliminary reply result, and use the named entity recognition model based on deep learning to identify various types of sensitive information in the preliminary reply result; generate a model according to the self-learning desensitization strategy, and take corresponding desensitization treatment for sensitive information of different types and different levels of importance. After the desensitization treatment, the privacy computing technology is used to evaluate the privacy leakage risk of the remaining reply content. If it is lower than the preset security threshold, it is determined that the preliminary reply result meets the data security criteria and is used as the final reply result.
[0051] According to a third aspect of the embodiments of the present invention,
[0052] An electronic device is provided, comprising:
[0053] processor;
[0054] a memory for storing processor-executable instructions;
[0055] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0056] A fourth aspect of the embodiments of the present invention is:
[0057] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0058] For accounting business query requests that have passed identity authentication, this paper proposes a query semantic parsing method that integrates multiple natural language processing technologies to automatically extract accounting professional terms and business scenario features contained in the query text. This method comprehensively uses technologies such as conditional random field algorithm, dependency syntax analysis, knowledge graph, bag-of-words model, etc., and through domain-adaptive term extraction, semantic disambiguation, syntax analysis, semantic reasoning and other links, it realizes the semantic fusion of query text and accounting domain knowledge, and provides structured semantic representation for subsequent retrieval services.
[0059] This paper innovatively applies graph neural network and knowledge graph reasoning to the detection of violation risks in the accounting field, making full use of the correlation characteristics of accounting violations, and can judge the compliance risk of query requests from both local similarity and global correlation, which is more comprehensive and accurate than the traditional rule matching method. This is of great value for strengthening the internal control management of accounting business systems and preventing financial fraud risks. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 A schematic diagram of a process flow of a cross-domain secure interaction method of a public service system based on a general large model according to an embodiment of the present invention;
[0061] Figure 2 It is a structural diagram of a cross-domain secure interaction system of a public service system based on a universal large model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0063] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0064] Figure 1 FIG. 1 is a flow chart of a cross-domain secure interaction method of a public service system based on a general large model according to an embodiment of the present invention. Figure 1 As shown, the method includes:
[0065] S101. Receive business query requests from accounting user terminals, use the request identification model to authenticate the source of the business query request, and compare it with the accounting personnel information database to determine whether the business query request has the corresponding accounting business authority; for business query requests that have passed identity authentication, use natural language processing technology to perform semantic analysis, extract accounting professional terms contained in the business query request, and accounting features of business scenarios; use a request legitimacy detection model based on a graph neural network, using the knowledge graph of violation cases in the accounting field as a training sample, to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations;
[0066] S102. If there is no compliance risk, the accounting features are input into the pre-trained general model in the accounting field, and the general model in the accounting field is used to perform multi-level representation learning on the accounting features. The attention mechanism is used to focus on the key information points of the business query request, and deep semantic reasoning is carried out in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request;
[0067] S103. Scan the preliminary reply result for sensitive information, and use a named entity recognition model based on deep learning to identify various types of sensitive information in the preliminary reply result; generate a model based on a self-learning desensitization strategy, and take corresponding desensitization treatments for sensitive information of different types and importance. After the desensitization treatment, use privacy computing technology to evaluate the privacy leakage risk of the remaining reply content. If it is lower than the preset security threshold, it is determined that the preliminary reply result meets the data security criteria and is used as the final reply result.
[0068] In an optional embodiment,
[0069] For business query requests that have passed identity authentication, natural language processing technology is used for semantic analysis to extract accounting terms and accounting features of business scenarios contained in the business query requests, including:
[0070] Develop a financial term extraction and semantic disambiguation tool based on the conditional random field algorithm, introduce a financial dictionary to adapt the conditional random field algorithm to the domain, so that it can accurately identify professional terms in business query requests and determine the actual semantics of professional terms based on the context;
[0071] Use the pre-trained Chinese dependency syntactic analysis model to perform syntactic dependency analysis on business query requests, reveal the inherent logical association of accounting business elements in the query text of business query requests, and provide structured clues for semantic reasoning;
[0072] Construct a financial accounting structured knowledge graph for financial accounting business scenarios. When semantically parsing business query requests, link and match the concept nodes of the financial accounting structured knowledge graph in real time based on the syntactic dependency relationship corresponding to the business query request, so as to achieve semantic information fusion of query statements and domain knowledge.
[0073] Comprehensively use natural language processing technology based on the bag-of-words model to extract key information from business query requests after semantic disambiguation, and automatically extract accounting professional terms contained in business query requests and accounting features of business scenarios.
[0074] For example, for accounting business query requests that have passed identity authentication, this paper proposes a query semantic parsing method that integrates multiple natural language processing technologies to automatically extract accounting professional terms and business scenario features contained in the query text. This method uses a combination of conditional random field algorithms, dependency syntax analysis, knowledge graphs, bag-of-words models and other technologies, and through domain-adaptive term extraction, semantic disambiguation, syntax analysis, semantic reasoning and other links, it achieves the semantic fusion of query text and accounting domain knowledge, and provides structured semantic representation for subsequent retrieval services.
[0075] First, for the recognition and semantic disambiguation of professional terms in the accounting field, this paper develops an accounting term extraction tool based on the conditional random field (CRF) algorithm. Traditional CRF has achieved widespread success in sequence labeling tasks, but for professional fields such as accounting, the generic CRF model often has difficulty in accurately identifying professional terms in a specific context. To this end, this paper introduces an external financial dictionary to adapt the CRF model to the field.
[0076] Specifically, when constructing the feature function of the CRF model, whether the vocabulary matches the financial dictionary is taken as an important feature and given a higher weight. At the same time, in the model training corpus, the professional vocabulary that matches the dictionary is given a specific label to guide the model to focus on financial terms. In addition, when decoding the CRF, for the identified professional terms, by analyzing their context, using word embedding models such as Word2Vec to calculate the similarity between the terms and multiple semantics, the semantic disambiguation of professional terms is achieved.
[0077] For example, for the query "How to perform accelerated depreciation of fixed assets?", after CRF extraction, two professional terms "fixed assets" and "accelerated depreciation" were identified. Further context analysis determined that "depreciation" here is the depreciation concept in accounting, rather than other semantics.
[0078] On the basis of professional terminology recognition, in order to deeply understand the semantic structure of query sentences, this paper uses a pre-trained Chinese dependency syntactic analysis model to analyze the syntactic dependency relationship of the query text. Dependency syntactic analysis reveals the modification, domination and other relationships between words in a sentence, and depicts the internal logical structure of semantic elements. It is an important intermediate representation for semantic understanding.
[0079] Taking the above query request as an example, through dependency analysis, we can find that: "fixed assets" and "depreciation" have a definite relationship, "acceleration" modifies "depreciation", and "conduct" dominates "depreciation". These structured syntactic relationships provide important clues for semantic reasoning.
[0080] In order to further realize the semantic integration of query text and accounting knowledge, this paper constructs a structured accounting knowledge graph for accounting business scenarios. The knowledge graph adopts the form of ontology, defines the core concepts, attributes and relationships in the accounting field, and represents accounting knowledge in a framework and structured manner.
[0081] When performing semantic analysis on the query request, the syntactic relationship obtained by the above dependency analysis is used to link and match the corresponding concept nodes in the knowledge graph in real time. Through this semantic mapping of query text to knowledge graph, unstructured natural language queries are embedded into the structured knowledge system, achieving a more robust and accurate semantic representation.
[0082] For example, by performing graph matching on "accelerated depreciation of fixed assets", "fixed assets" can be linked to the asset concept of the knowledge graph, "depreciation" can be linked to the financial processing behavior, and "acceleration" corresponds to an attribute of the depreciation method. This knowledge-enhanced matching further eliminates the ambiguity of accounting concepts and provides rich semantic information for query understanding.
[0083] Finally, based on semantic disambiguation and graph matching, this paper uses NLP technology represented by the Bag of Words (BoW) model to extract key information from business query requests. The Bag of Words model ignores word order and grammar, treats the text as a set of words, and focuses on the co-occurrence statistics of words.
[0084] Based on the BoW representation, the words in the query text are weighted by algorithms such as TF-IDF to mine the keywords that contribute most to the financial and accounting semantics. At the same time, the topic model, keyword extraction and other technologies are used to automatically extract the structural features of the query text, such as the financial and accounting business theme, financial and accounting elements, and financial and accounting report types.
[0085] For example, by extracting keywords for "how to accelerate depreciation of fixed assets", we obtain keywords such as "fixed assets", "depreciation", and "acceleration"; based on the LDA topic model inference, we find that the query belongs to the topic of "asset management"; further matching of financial accounting elements reveals that it involves elements such as "assets" and "depreciation methods".
[0086] By deeply integrating the above NLP technologies, this paper realizes the comprehensive semantic analysis of accounting business query requests. It is summarized as follows in the form of a process:
[0087] CRF professional terminology extraction is performed on accounting business query requests, and financial and economic dictionaries are introduced for domain adaptation and semantic disambiguation to accurately identify accounting professional concepts in query texts.
[0088] Using the pre-trained Chinese dependency syntax analysis model, the syntactic dependencies of accounting elements in the query text are revealed to obtain structured semantic clues.
[0089] Build a structured knowledge graph for accounting scenarios, and match knowledge graph concepts in real time based on syntactic analysis to achieve semantic fusion of query text and domain knowledge.
[0090] The bag-of-words model, TF-IDF, topic model, keyword extraction and other technologies are used to extract key information from the query text after semantic disambiguation and automatically extract the accounting professional features contained therein.
[0091] The output integrates structured query semantic representation of accounting terms, dependency structures, knowledge graphs, and keywords, providing high-quality semantic input for subsequent retrieval services.
[0092] The semantic parsing method proposed in this paper makes full use of the applicability of NLP technology in the field of accounting, and improves the accuracy and comprehensiveness of accounting semantic understanding through the integration of multiple technologies. This paradigm of embedding domain knowledge into the language understanding process provides a solid semantic foundation for intelligent question answering in complex accounting query scenarios, and has important reference significance for building a new generation of intelligent accounting service platform.
[0093] In an optional embodiment,
[0094] Using a request legitimacy detection model based on a graph neural network and taking the knowledge graph of accounting violation cases as training samples, we can determine whether a business query request contains compliance risks such as financial fraud or illegal or irregular operations. The following are the risks:
[0095] Based on the collected and sorted accounting violation cases, we construct a knowledge graph of accounting violation risk, which includes violation concept nodes and relationship edges. The relationship edges describe the correlation and evolution law between different violation behaviors.
[0096] A heterogeneous graph neural network model is designed that integrates text feature learning and multi-relation reasoning. The heterogeneous graph neural network model can learn the semantic embedding representation of different types of violation concept nodes and dynamically adjust the message transmission and aggregation strategies between violation concept nodes based on the relationship type.
[0097] The pre-trained heterogeneous graph neural network model is used to learn the node semantic embedding representation on the accounting violation risk knowledge graph, and the accounting professional terms and accounting features of the business scenarios extracted from the semantic analysis of the business query request are mapped to the semantic space aligned with the accounting violation risk knowledge graph. The similarity between the accounting professional terms, accounting features of the business scenarios and the violation concept nodes is calculated to determine whether the business query request triggers a known violation risk scenario.
[0098] In the message transmission process of the heterogeneous graph neural network model, the accounting violation risk knowledge graph is used as a path to infer the violation risk associations implicit in the semantic environment of the business query request, and the overall compliance of the business query request is evaluated from the two dimensions of local risk and global risk to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations.
[0099] For example, in order to prevent the potential risk of illegal operations in accounting business query requests, this paper proposes a request legitimacy detection model based on graph neural network (GNN). The model uses the knowledge graph of illegal cases in the accounting field as training samples, uses heterogeneous graph neural networks to perform semantic embedding learning on illegal concept nodes, and combines semantic similarity matching and graph reasoning technology to judge whether the business query request has compliance risks of financial fraud and illegal operations from both local and global dimensions.
[0100] First, for the structured representation of accounting violation risk knowledge, this paper constructs a violation risk knowledge graph containing violation concept nodes and relationship edges based on the collected and collated violation cases in the accounting field. The knowledge graph intuitively depicts the correlation and evolution law between different violation behaviors in the form of a graph, laying an important foundation for the accurate identification of violation risks.
[0101] In the process of constructing the knowledge graph, each violation case is extracted into a violation concept node in the graph, and the attributes of the node describe the type, method, consequences and other information of the violation. In addition, the association between different violation concept nodes is represented by directed edges, such as the "association" relationship between "falsely inflated profits" and "falsified financial statements", and the "cause" relationship between "forged invoices" and "tax evasion". These heterogeneous associations reveal the inherent evolutionary logic of complex violations and help to infer violation risks from the perspective of behavioral association.
[0102] For the complex violation risk knowledge graph containing heterogeneous nodes and multiple associated edges, this paper designs a heterogeneous graph neural network model that integrates text feature learning and multi-relation reasoning. Different from traditional GNNs on homogeneous graphs, heterogeneous GNNs can learn differentiated semantic representations of different types of nodes and dynamically adjust the node message transmission and aggregation strategies according to the semantics of the relationship between nodes, better adapting to the characteristics of heterogeneous graphs.
[0103] Specifically, the heterogeneous GNN model adopts a meta-path-based heterogeneous neighbor aggregation mechanism. With the violation concept node as the center, the meta-path guided by various relationship edges is extracted from the knowledge graph, and the meta-path defines the channel for message transmission between nodes. For example, the "false profit" node may have two meta-paths: "false profit-accompanying-faking financial statements-leading to fraudulent listing" and "false profit-means-forged invoices-violation-tax evasion", which respectively reflect the two risk dimensions of financial reporting and taxation of profit fraud.
[0104] On each meta-path, according to the type and direction of the relationship edge, the heterogeneous GNN uses different transformation functions to transform the messages of neighbor nodes, and uses a type-aware attention mechanism to adaptively weight the importance of different neighbors, and finally updates the node semantic representation through the aggregation function. By repeatedly iterating the above propagation aggregation process on the heterogeneous graph, the model can learn the node embedding representation that integrates the semantics of the offending node itself and the graph structure information.
[0105] Based on the trained heterogeneous GNN model, this paper further uses the semantic parsing results of the aforementioned business query requests to realize automatic detection of the compliance of query requests. The key idea is to embed the accounting professional terms and business scenario features extracted from the query request into the semantic space aligned with the violation risk knowledge graph, and judge whether the query request triggers a known violation risk scenario through semantic similarity matching.
[0106] In order to achieve the mapping of cross-domain semantic space, this paper adopts a distance-based semantic matching method. First, the professional terms and business features extracted from the query request are converted into dense vectors using pre-trained word embedding models such as Word2Vec. Then, the similarity between the query semantic vector and the embedding vector of the violation concept node is measured by Euclidean distance, and the violation concept nodes whose similarity exceeds the preset threshold are activated. This matching process essentially determines the possibility of the query request triggering a known violation risk locally.
[0107] On the basis of activating the violation concept node, this paper further implements the violation risk reasoning based on the knowledge graph in the heterogeneous GNN model. Starting from the activated violation concept node, the heterogeneous GNN model infers the violation risk associated with the query request in the global business environment along the relationship edge in the knowledge graph. This process comprehensively utilizes the correlation and evolution law between accounting violations, and can discover the chain violation risk implied in the query request.
[0108] Take the query request of "manual adjustment of financial data" as an example. Through semantic matching, this request may activate the "false profit" node in the knowledge graph. On this basis, graph reasoning finds that "false profit" may be associated with "embellishing financial statements" and then lead to "fraudulent listing", revealing the deep-level violation risk of data adjustment behavior. Through multi-hop relational reasoning, heterogeneous GNN can comprehensively evaluate the overall compliance of query requests.
[0109] In summary, the process of the request legitimacy detection method proposed in this paper is as follows:
[0110] An accounting violation risk knowledge graph containing heterogeneous violation concept nodes and relationship edges is constructed offline to characterize the correlation and evolution law between violation behaviors.
[0111] A heterogeneous GNN model is designed that integrates meta-path feature extraction and type-aware neighbor aggregation, and is trained to learn the semantic embedding representation of violation concept nodes on the knowledge graph.
[0112] The accounting professional terms and business features extracted from the semantic analysis of business query requests are mapped to a semantic space aligned with the knowledge graph, and the possibility of the request triggering a violation risk locally is determined through similarity matching.
[0113] In heterogeneous GNN, starting from the activated violation concept node, multi-hop reasoning is performed along the relationship edge to evaluate the violation risk associated with the request in the global business environment.
[0114] Based on the results of local similarity matching and global graph reasoning, it is determined whether the business query request has an overall compliance risk of financial fraud or illegal and irregular operations.
[0115] This paper innovatively applies graph neural network and knowledge graph reasoning to the detection of violation risks in the accounting field, making full use of the correlation characteristics of accounting violations, and can judge the compliance risk of query requests from both local similarity and global correlation, which is more comprehensive and accurate than the traditional rule matching method. This is of great value for strengthening the internal control management of accounting business systems and preventing financial fraud risks.
[0116] In an optional embodiment,
[0117] The accounting features are input into the pre-trained general model in the accounting field, and the general model in the accounting field is used to perform multi-level representation learning on the accounting features. The attention mechanism is used to focus on the key information points of the business query request, and deep semantic reasoning is carried out in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request, including:
[0118] Construct a general accounting model for accounting business. Through pre-training with large-scale accounting corpus, the general accounting model can master the vocabulary, grammar and semantic features of accounting language, introduce accounting business characteristic training tasks, and strengthen the model's understanding and application ability of accounting knowledge.
[0119] Use the pre-trained general accounting model to perform multi-layer Transformer encoding on query requests, dynamically adjust the interaction weights between words through the Self-Attention mechanism, and introduce Attention at the top layer of the network to focus on key words and sentences, forming a hierarchical query semantic representation that integrates the attention mechanism;
[0120] Build a knowledge graph in the accounting field covering accounting standards and tax policies, map knowledge graph entities to the pre-trained language model embedding space, identify the starting nodes related to the query semantics, and expand semantically related entities and relationships from the starting nodes through knowledge graph reasoning;
[0121] Identify entity combinations that can represent specific business scenarios in the entity set expanded by the knowledge graph, analyze the grammatical dependencies of entities in the query text to extract the corresponding business logic, and restore the contextual semantics by combining the entity category and attribute modification to generate a business logic context information network that matches the query request;
[0122] The semantic features in the business logic context information network are integrated as the structured generation model input. After semantic interaction through multiple layers of Transformer networks, the Attention mechanism is used to focus on the key semantics. Combined with the language knowledge in the accounting field, the replication mechanism and Beam Search strategy are used to autoregressively decode and generate the response text that matches the query.
[0123] For example, in response to complex accounting business query needs, this paper proposes an accounting business question-answering method that integrates domain knowledge and semantic reasoning. This method is based on a pre-trained general accounting model, uses multi-level Transformer encoding and attention mechanisms to learn the semantic representation of query requests, and combines external accounting knowledge graphs to achieve knowledge-based semantic expansion and reasoning, generates business logic context information that matches the query request, and finally decodes and outputs targeted query reply text.
[0124] First, in order to enhance the language model's ability to understand the language and knowledge in the accounting field, this paper constructs a general large model for accounting business. Through unsupervised pre-training on massive accounting corpus, the model can master the vocabulary, grammar and semantic features of accounting language. The pre-training corpus covers various structured and unstructured accounting text data such as accounting standards, tax policies, corporate financial reports, etc., covering the general language knowledge in the accounting field.
[0125] On the basis of general corpus pre-training, this paper further introduces supervised training tasks with accounting business characteristics, such as key accounting policy classification, economic business matters judgment, financial indicator anomaly detection, etc., to strengthen the model's understanding and application of accounting knowledge. For example, by training the model to judge the compliance of accounting policy choices such as "revenue recognition" and "inventory valuation", it can deeply understand the internal logic of accounting standards.
[0126] Using the general accounting model obtained by pre-training, this paper further constructs a hierarchical query semantic encoder. The encoder receives the text input of the accounting business query request and realizes the deep representation learning of the query semantics through multi-layer stacked Transformer modules. The self-attention mechanism of Transformer can model the long-distance semantic dependency between words in the query text, dynamically adjust the interaction weights between different words through training, and highlight the keywords that are important for understanding the overall semantics.
[0127] At the top level of Transformer encoding, this paper introduces an additional query focusing module based on Attention to identify the key words and key sentences that best express the core semantics from the entire query text. The focusing module trains a binary classifier to determine whether each phrase or sentence is the query focus, and uses the classification probability as the Attention weight to aggregate the output of the top-level Transformer in a weighted average manner to extract a vector representation representing the query focus semantics.
[0128] Through the semantic encoding of multi-layer Transformer and the query focusing of the top-level Attention, a hierarchical query semantic representation integrating word-level, sentence-level, and focus-level granularity is finally formed. Compared with the semantic representation of a single granularity, the hierarchical semantic representation can take into account both local details and global themes, providing more accurate and comprehensive semantic clues for subsequent knowledge reasoning.
[0129] In order to introduce external structured knowledge to expand and reason query semantics, this paper constructs a knowledge graph in the accounting field covering accounting standards and tax policies. The knowledge graph constructs a directed graph network with entities as nodes and relationships between entities as edges, formally representing concepts in the accounting field and their semantic associations. For example, accounting element entities such as "income", "cost", and "profit" are connected by business relationship edges such as "carry-over", "collection", and "accrual", and tax entities such as "value-added tax" and "income tax" are connected by tax relationship edges such as "payment" and "exemption".
[0130] In order to achieve seamless connection between knowledge graph and query semantics, this paper adopts a knowledge representation learning method based on the Transm model to map entities in the knowledge graph to the same embedding space as the pre-trained language model. By minimizing the distance between each entity embedding vector and the semantic vector encoded by the language model of its text description, the knowledge graph entity can obtain a semantic distributed representation, laying the foundation for cross-domain semantic matching.
[0131] On this basis, this paper designs a query semantic expansion and reasoning method combined with knowledge graph. First, the query focus semantic vector and the embedding vector of the knowledge graph entity are similarly calculated, and a batch of graph entities most relevant to the query semantics are identified as the starting nodes. With the starting node as the center, the relationship edges of the knowledge graph are expanded to connect a subgraph containing the neighbors of the starting node and its multi-hop related entities. The expanded entities carry the background knowledge related to the query.
[0132] Next, this paper uses a pattern matching-based reasoning method in the extended subgraph to find entity combinations that can represent specific business scenarios. Using the predefined key entity combination patterns in the accounting field, we identify entity sequences that reflect business scenarios such as accounting processing procedures and tax-related matters. By analyzing the grammatical dependencies of these entities in the original query, we further refine the corresponding business processing logic. For example, for entities such as "revenue recognition" and "invoice issuance", by analyzing their subject-predicate-object relationships in the query, we can extract the logical constraints of "at what point in time to recognize revenue" and "what types of invoices to issue".
[0133] After obtaining the entity combination and its logical constraints representing the business scenario, this paper further uses the entity's category information and attribute modification in the knowledge graph to restore the contextual semantics of the business scenario. The entity category defines the role of the scenario elements, such as "asset" and "liability" represent accounting elements, and "tax basis" and "tax obligation" represent tax-related matters. The entity attributes reflect the limiting conditions of the scenario, such as "asset-liquidity: flow" represents the liquidity attribute of the asset, and "income tax-tax rate: 25%" limits the applicable tax rate of income tax. Combined with the structured semantic information of the entity, this paper organizes the relevant entities into a hierarchical business logic network to characterize the contextual semantic environment involved in the query.
[0134] Finally, this paper linearizes the business logic context information network into a structured text sequence as the input of the subsequent generation model. By splicing semantic units of different granularities such as entity words, category words, attribute words, and relation words, a structured representation of the query semantics is formed. Due to the integration of external knowledge, this representation can make up for the incompleteness of the query text's own semantics.
[0135] This paper uses a pre-trained text generation model to automatically generate reply text that matches the query based on structured semantic representations. The generation model also uses the Transformer encoder-decoder architecture to deeply interact with structured semantics through the self-attention mechanism and explore the implicit associations between semantic units. In the decoding stage, each step of the model combines the attention distribution of the generated text and the encoded semantics to search for the most probable generation path in the form of beam search. In addition, this paper also introduces the copy mechanism to improve the accuracy of the generated text, that is, to give priority to copying from the input semantics rather than generating professional vocabulary that does not appear in the vocabulary.
[0136] By introducing accounting domain knowledge to enhance query semantic representation, the business question-answering method in this paper can generate targeted responses for complex accounting professional queries. Taking the query "How to pay value-added tax" as an example, the traditional method may only match general expressions related to "value-added tax" and "payment", such as "value-added tax taxpayers pay value-added tax in accordance with regulations". However, the method in this paper obtains background information such as "value-added tax prepayment", "value-added tax rate", and "confirmation of taxable items" through knowledge reasoning, and generates targeted responses such as "value-added tax general taxpayers calculate the taxable amount according to the confirmed taxable sales income according to the applicable tax rate, and pay the previous value-added tax during the declaration period of the next month". It can be seen that the introduction of structured knowledge enables the question-answering system to obtain the necessary professional background and business logic, and can provide operational business explanations for specific business scenarios.
[0137] In summary, the main process of the accounting business question-and-answer method proposed in this article is as follows:
[0138] 1. Build a general large model in the accounting field, and enable the model to acquire language knowledge in the accounting field through pre-training with massive accounting corpus and training with accounting business-specific tasks.
[0139] 2. Use the pre-trained general large model in the accounting field to perform multi-layer Transformer encoding on business query requests, and obtain a hierarchical query semantic representation that integrates word-level, sentence-level, and focus-level granularity through self-attention mechanism and query focusing.
[0140] 3. Construct a knowledge graph in the accounting field, and map the knowledge graph entities to the same semantic space as the pre-trained model through knowledge representation learning.
[0141] 4. In the knowledge graph, the entity most relevant to the query semantics is used as the starting node, and the related entity subgraph is expanded through relationship edges as the background knowledge of the query.
[0142] 5. Identify entity combinations that represent specific business scenarios based on pattern matching in related entity subgraphs, and refine the business logic constraints of entities in queries.
[0143] 6. Combine the entity’s categories, attributes and other structured information to restore the contextual semantics of the business scenario and build a business logic context information network.
[0144] 7. Serialize the business logic context information network into structured text, input the pre-trained text generation model, and generate the reply text matching the query through the Attention mechanism and Copy strategy decoding.
[0145] The innovation of this paper is to introduce knowledge graph into accounting question answering, and enhance the knowledge understanding and semantic reasoning ability of language model by integrating external structured knowledge into query semantic representation. Knowledge-driven accounting business question answering can break through the limitations of manually written rules and fully utilize financial knowledge to automatically respond to complex and changing business scenarios. It is an important exploration of the intelligentization of traditional accounting business systems.
[0146] In an optional embodiment,
[0147] The semantic features in the business logic context information network are integrated into the structured generation model input. After the semantic interaction of the multi-layer Transformer network, the Attention mechanism is used to focus on the key semantics. Combined with the language knowledge in the accounting field, the replication mechanism and the Beam Search strategy are used to autoregressively decode and generate the response text that matches the query, including:
[0148] Integrate the multi-granularity heterogeneous semantic fragments identified in the business logic context information network into a unified structured input, wherein the multi-granularity heterogeneous semantic fragments include keywords, phrases, sentences, entities, and relations;
[0149] Through the multi-layer Transformer encoder, the structured semantic input is hierarchically modeled, the distributed representation of semantic units of different granularity is learned, and the local semantics and global semantics are interactively modeled to form a query representation that integrates semantics at different levels.
[0150] The decoder also uses the Transformer architecture to iteratively generate responses in an autoregressive manner. Each decoding step focuses on the semantic fragment in the encoded representation that is most relevant to the generated content, dynamically references key semantic information, and feeds back the decoding status to the encoder to guide subsequent encoding.
[0151] When predicting decoded words, the model vocabulary and query text are combined to flexibly generate and reuse keywords and professional terms through a copy mechanism;
[0152] The Beam Search decoding strategy is adopted to comprehensively consider the generation quality and efficiency to obtain a smooth and coherent response sequence;
[0153] Based on the pre-trained language model, targeted fine-tuning is performed using accounting domain corpus to learn domain text features. At the same time, external structured accounting knowledge is used as an additional supervision signal to guide the model to generate text.
[0154] For example, the business logic context information network constructed through the above steps has integrated the expansion and reasoning of the original query semantics by accounting domain knowledge. In order to further use this network to guide the generation of reply text, this paper proposes a Transformer-based encoding-decoding framework to semantically model the business logic network and decode the output query matching reply.
[0155] First, it is necessary to integrate the multi-granularity heterogeneous semantics in the business logic network into a unified structured input. The network contains text fragments of different lengths such as keywords, phrases, and sentences, as well as structured semantic units such as entities and relations. This paper adopts an identifier-based serialization method to uniformly represent them. For each text fragment, the corresponding semantic identifier is added before and after it, for example, "[KW]Sales Revenue[ / KW]" indicates that "Sales Revenue" is a keyword, and "[PH]VAT Rate[ / PH]" indicates that "VAT Rate" is a phrase. For structured units, they are represented in the form of triples, for example, "[ENT]VAT[ / ENT][REL]Applicable Tax Rate[ / REL][ENT]6%[ / ENT]" indicates that there is an "Applicable Tax Rate" relationship between the entity "VAT" and the entity "6%". By setting identifiers for different semantics, the heterogeneity of semantics is retained and unified into a structured sequence that can be processed sequentially.
[0156] After obtaining the structured semantic input, this paper uses a multi-layer Transformer encoder to learn its hierarchical semantic representation. The Transformer's self-attention mechanism can model the association between any two semantic units, no matter how far apart they are in the sequence. The first layer of the encoder first learns the short-distance dependencies between words within each semantic segment to capture local semantics. Then, the upper encoder takes the output of the lower layer as input, and interactively models the long-distance dependencies between different semantic segments through cross-segment attention calculation to obtain global semantics. Through multi-layer hierarchical modeling, a query semantic representation of different granularity and structure is finally formed, which integrates local and global information.
[0157] At the Transformer decoding end, this paper uses autoregression to iteratively generate reply text. Specifically, the decoder predicts the next word based on the previously generated words at each step. When predicting, the decoder's self-attention mechanism first performs semantic modeling on the generated sequence, capturing the dependencies between words to ensure the coherence of the generated content. Then, through the interactive attention between the decoder and the encoder, the generated content is matched with the query semantic representation on the encoder end to identify the key semantic fragments that are most helpful for the current generation. For example, after "according to the Provisional Regulations on Value-Added Tax" has been generated, the interactive attention will be highly focused on the semantic fragments containing "value-added tax" and "applicable tax rate" on the encoder end. By dynamically referring to these key semantic information, the decoder can flexibly adjust the generation strategy of subsequent words according to different contexts. In addition, after each interactive attention calculation, the decoder also feeds back its own state to the encoder, guiding the encoding representation to be updated in time to adapt to the new decoding context, forming a two-way information interaction between encoding and decoding.
[0158] When predicting the next word to be decoded, in order to avoid errors in the generation of professional terms or specific expressions, this paper not only considers words in the Transformer vocabulary, but also considers copying words directly from the query text. The model uses a gating mechanism to balance generation and copying, that is, dynamically adjusts the probabilities of the two prediction methods according to the current decoding state. When there are highly matched words in the query text, the model is more inclined to copy; otherwise, it relies more on vocabulary generation. The gating mechanism enables the model to flexibly generate new words and accurately use keywords and professional terms that have already appeared in the query.
[0159] In the specific decoding implementation, this paper adopts the Beam Search strategy for approximate optimal search. Unlike greedily selecting the word with the highest probability each time, Beam Search maintains multiple generation paths at the same time and selects the first few paths with the highest comprehensive scores as candidates each time. Beam Search achieves a good balance between prediction quality and decoding efficiency, avoiding local optimality and not bringing excessive computational complexity. It is a widely used text generation strategy in engineering practice.
[0160] In addition to the model structure, pre-training and fine-tuning strategies are also crucial to the effectiveness of generative question answering. This paper pre-trains the Transformer model on a large-scale corpus in a general domain, giving it basic language understanding and generation capabilities. Next, the model is fine-tuned on the corpus in the accounting field, introducing various structured and unstructured accounting texts including accounting standards and tax policies, so that the model can further learn the characteristics of the domain language. During fine-tuning, the structured accounting knowledge in the business knowledge base is also used as an additional supervisory signal. For example, the accounting subject system in the knowledge base can be used as a classification target, and the tax preferential policy can be used as a generation target. Through multi-task learning, the model can inherently establish a connection between domain vocabulary and structured knowledge.
[0161] The following is an example of a query on "how to determine the time when the corporate income tax obligation occurs" to illustrate the application process and effect of this method:
[0162] 1. Build a business logic context network of query semantics. Through knowledge graph reasoning, we obtain entities related to the query, such as "corporate income tax", "tax obligation occurrence time", "taxable income", etc., together with the relationship between entities ("corporate income tax" - "tax obligation time" "taxable income" - "tax calculation basis") to form a logical network.
[0163] 2. Sequence the business logic network to generate structured input. After semantic annotation, the text in the logic network is "[KW] Enterprise income tax [ / KW] [PH] Time when tax obligation occurs [ / PH] [KW] Taxable income [ / KW]", and the structured triples are "[ENT] Enterprise income tax [ / ENT] [REL] Time of tax obligation [ / REL] [ENT] Time point for determining taxable income [ / ENT]", "[ENT] Taxable income [ / ENT] [REL] Tax calculation basis [ / REL] [ENT] Total income minus costs and expenses [ / ENT]".
[0164] 3. Input the Transformer encoder for multi-layer semantic modeling. The first layer of attention captures the intrinsic connection between "corporate income tax" and "tax liability occurrence time". The high-level interactive modeling phrase "tax liability occurrence time" and the entity pair "corporate income tax - taxable income determination time point" global association.
[0165] 4. The decoder generates the reply text by autoregression. First, the "tax liability time of corporate income tax" is generated based on the encoded semantics, and then the "taxable income amount is determined at the time point" is generated by focusing on the relevant semantic fragments of "taxable income". Then, referring to the tax calculation basis of "taxable income amount", the "current or next period is determined by the taxable income amount after the enterprise obtains income and deducts related costs and expenses during the accounting period" is supplemented.
[0166] 5. The output result is "The time when the corporate income tax obligation occurs is based on the time when the taxable income is determined, that is, the current or next period determined by the taxable income after the company obtains income and deducts related costs and expenses during the accounting period." It can be seen that under the guidance of accounting knowledge, the response generated by this method covers the key basis for judging the time of corporate income tax obligations, with accurate explanations and clear logic.
[0167] In summary, this paper proposes a generative question-answering method that integrates domain knowledge. It introduces structured knowledge information by building a business logic network, and designs a corresponding Transformer encoding-decoding model to achieve knowledge-based query semantic representation and reply text generation. This method can effectively use accounting domain knowledge to enhance the semantic association between queries and replies, has good interpretability and applicability in complex professional scenarios, and provides new ideas for intelligent information services in the accounting field.
[0168] In an optional embodiment,
[0169] Based on the self-learning desensitization strategy generation model, different types and different levels of importance of sensitive information are desensitized accordingly. After desensitization, privacy computing technology is used to evaluate the privacy leakage risks of the remaining reply content, including:
[0170] Through transfer learning, the pre-trained language model is fine-tuned on large-scale accounting text data, so that it can adaptively learn the diverse characteristics of private information in the accounting context, and build a private information identifier for accounting scenarios, which can accurately detect various explicit or implicit private fragments from accounting texts;
[0171] Collect desensitization examples of different types of financial privacy, build a privacy-desensitization strategy pairing dataset, and train a seq2seq self-learning desensitization strategy generation model on the dataset, which can automatically generate the optimal desensitization strategy for each identified privacy fragment based on the privacy type, importance, and contextual factors;
[0172] The self-learning desensitization model is applied to the reply text of accounting business inquiries. For the identified direct privacy, strong desensitization processing such as encryption and random replacement is adopted, and for the indirect privacy, weak desensitization processing such as generalization and synonym replacement is adopted, so as to protect the privacy while maximally retaining the availability of the reply content.
[0173] Apply homomorphic encryption privacy computing technology to the desensitized reply text, match it with the predefined privacy mode in the ciphertext domain to calculate the overall privacy leakage probability. If the probability exceeds the security threshold, further desensitization is performed iteratively on high-risk text fragments until the overall privacy risk is controlled within an acceptable range.
[0174] The privacy leakage probability calculation is carried out through the federated learning method with multi-party participation, ensuring that the privacy calculation evaluation process itself will not cause additional privacy leakage;
[0175] The output is a secure accounting business response text that has undergone dual privacy protection through self-learning desensitization and privacy computing. While retaining the original meaning well, sensitive privacy information is effectively shielded and the overall privacy leakage risk is reliably controlled.
[0176] By way of example, this paper proposes a privacy protection method for accounting business that integrates self-learning desensitization and privacy computing, aiming to perform active privacy detection and dynamic desensitization processing on the reply texts generated by business questions and answers in accounting scenarios, minimizing the risk of privacy leakage while ensuring the availability of replies.
[0177] First, in order to accurately identify the private information in the accounting reply text, it is necessary to build a private information identifier for accounting scenarios. Traditional rule or dictionary matching methods are difficult to fully cover the diverse forms of private information in the accounting context. For example, "Xiao Wang's year-end bonus is 100,000 yuan" is an explicit privacy, while "Company A achieved a profit of 50 million this year, an increase of 20% over last year" is an implicit privacy that indirectly discloses the company's business secrets. In order to overcome the above difficulties, this paper uses transfer learning to fine-tune the pre-trained language model on large-scale accounting text data, so that it can adaptively learn the deep semantic features of different types of private information in accounting scenarios. For example, the contextual expression patterns of sensitive information such as salary, tax, and assets are learned from massive accounting reports, business process documents and other data. On this basis, fine-tuning is performed on the labeled accounting privacy identification dataset to form a private information identifier. This identifier can not only detect explicit privacy, but also discover implicit privacy hidden in the context, realizing fine-grained privacy detection in the accounting field.
[0178] After detecting private information, desensitization is a key step in protecting privacy. Different types and different levels of private information require desensitization of corresponding strengths. To achieve this goal, this paper collects and constructs a privacy-desensitization strategy paired dataset. Each sample in the dataset is a binary pair of (private information, desensitization strategy). Private information includes multi-dimensional attributes such as privacy type (such as identity information, financial information), importance (core privacy, general privacy), and context. Desensitization strategies include encryption, randomization, generalization, character masking and other technical means. On the paired dataset, this paper trains a seq2seq type desensitization strategy generation model. The model can automatically generate targeted optimal desensitization strategies based on the input private fragments and their attributes. For example, for the sensitive personal information "Zhang San's ID number is 1234567890", the desensitization strategy generated by the model may be "randomly replace '1234567890' with '*********'". The desensitization model has self-learning ability. By continuously learning the latest desensitization examples in the industry, the generation quality of desensitization strategies is continuously improved.
[0179] By applying the recognition and desensitization models to the accounting business response text to be published, adaptive privacy protection can be achieved. For the response content "Mr. Li Si, your salary income in November 2021 is 15,000 yuan, and your personal income tax is 1,000 yuan", the privacy identifier first detects three direct privacy fragments of "Li Si", "15,000 yuan", and "1,000 yuan". Subsequently, the desensitization model generates corresponding strategies for each privacy fragment. Since "Li Si" is sensitive identity information, encrypted strong desensitization means are adopted to replace it with an irreversible random string "#USR102". Although salary income and tax amount are also private, their importance is relatively low, so a generalized weak desensitization method is adopted to replace them with "10,000-20,000 yuan" and "500-1,500 yuan" with a larger semantic range respectively. After the above-mentioned differentiated desensitization processing, the reply text becomes "Mr. #USR102, your salary income in November 2021 is 10,000-20,000 yuan, and your personal income tax is 500-1,500 yuan". While better retaining the original meaning, the privacy information is effectively blurred.
[0180] In order to further evaluate the residual privacy leakage risk of the reply content after desensitization, this paper introduces privacy computing technology. First, the desensitized text is encrypted into ciphertext using the homomorphic encryption algorithm, and then the ciphertext is matched with a series of predefined privacy patterns. The privacy pattern library contains common privacy combinations in the accounting field, such as "personal name + position + income", "company name + asset size + profitability", etc. The matching calculation is implemented through ciphertext domain operations such as Bloom filtering, which can obtain the similarity probability between the ciphertext and each privacy pattern, and then estimate the overall privacy leakage probability of the desensitized text. If the probability exceeds the preset security threshold, another round of desensitization is performed on the high-risk text fragment until the privacy leakage probability is reduced to an acceptable range. Considering that privacy computing itself may introduce new privacy leakage risks, this paper adopts the federated learning paradigm of multi-party secure computing to carry out privacy leakage probability calculation. Under the premise that different participants each hold a local fragment of the desensitized text, cryptographic primitives such as secret sharing and multi-party computing are used to collaboratively complete the probability calculation in a decentralized and third-party-free manner, eliminating privacy leakage in the evaluation process.
[0181] After self-learning desensitization and privacy calculation, the output accounting business response text has obtained privacy and security guarantees in two aspects: first, sensitive privacy information is fine-grainedly detected and customized desensitized, forming a de-privacy security version based on the original text; second, the probability of privacy leakage is quantitatively evaluated and dynamically controlled for the desensitized text, so that the potential risk of privacy leakage is controlled within an acceptable threshold.
[0182] The following takes the question-and-answer scenario of "A Company's 2021 Financial Status Consultation" as an example to intuitively demonstrate the operation process and privacy protection effect of this method:
[0183] Original business inquiry:
[0184] Xiao Wang: What is the financial status of Company A in 2021? What are the main financial data? Thank you!
[0185] Original reply text:
[0186] Financial Manager of Company A: Hello, in 2021, our company's total operating income was 5.028 billion yuan, a year-on-year increase of 20.5%, and its net profit reached 812 million yuan, an increase of 25%. At the end of the year, the company's total assets were 8.82 billion yuan, net assets were 5.8 billion yuan, and the asset-liability ratio was 34.2%. All indicators remained stable. The above are preliminary accounting data, and the final audit report shall prevail.
[0187] In the first step, the privacy information identifier based on the pre-trained language model was applied to automatically detect five sensitive information fragments from the reply text: "Company A", "Total operating income of 5.028 billion yuan", "Net profit of 812 million yuan", "Total assets of 8.82 billion yuan", and "Net assets of 5.8 billion yuan". Among them, "Company A" is indirect privacy, and financial data such as "total operating income" are direct privacy.
[0188] In the second step, the seq2seq desensitization strategy generation model is used to self-learn and generate the best desensitization strategy for each privacy fragment according to the privacy importance and the context. Considering the semantic importance, "Company A" is generalized by weak desensitization and replaced with "a listed company". For core financial privacy such as "net profit" and "total assets", strong desensitization numerical randomization is used to replace them with numerical ranges of "600-1000 million yuan" and "800-1000 million yuan", respectively.
[0189] In the third step, the desensitization strategy obtained through self-learning is applied to the original reply text to obtain the following desensitized version:
[0190] "A financial manager of a listed company: Hello, our company's total operating revenue in 2021 was 4-6 billion yuan, a year-on-year increase of 20-30%, and net profit reached 600-100 million yuan, an increase of 20-30%. At the end of the year, the company's total assets were 8-10 billion yuan, net assets were 5-7 billion yuan, and the debt-to-asset ratio was 30-40%. All indicators remained stable. The above are preliminary accounting data, and the final audit report shall prevail." It can be seen that the desensitized text is highly consistent with the original text in structure and content, and the macro-semantics of financial information are basically retained, but sensitive privacy details such as company identity and specific financial figures are hidden, which initially achieves a balance between privacy protection and availability.
[0191] The fourth step is to quantify the remaining privacy leakage risk of the desensitized text, encrypt it into ciphertext, and perform a secure similarity matching calculation with the predefined privacy pattern combination. If the desensitized text matches the high-risk privacy pattern of "company name + profit data + total assets" by more than 60%, secondary desensitization will be triggered, further generalizing "total operating income" to "tens of billions of yuan" and generalizing "increase rate" to "fast growth" until the overall privacy leakage probability is reduced to below 30%.
[0192] Step 5: Output the reply text after self-learning desensitization and privacy computing double privacy security protection, and deliver it for use:
[0193] "Hello, financial manager of a listed company in a certain industry: In 2021, our company's total operating income was several billion yuan, with a rapid year-on-year growth, and net profit reached several hundred million yuan. At the end of the year, the company's total assets and net assets remained stable, and the debt-to-asset ratio was controlled at a reasonable level. The above is preliminary accounting data, and the final audit report shall prevail."
[0194] In summary, this paper integrates self-learning desensitization and privacy computing technology to construct a text privacy protection framework for accounting business question-and-answer scenarios. The framework first uses AI technologies such as pre-trained language models to accurately identify diverse privacy information in the accounting context, and then adaptively generates the optimal desensitization strategy for different privacy. Privacy computing technology is then applied to the desensitized text to quantitatively evaluate the remaining privacy leakage risk, and the risk is dynamically controlled within an acceptable threshold through iterative desensitization. This method can realize accounting text privacy detection and protection in an automated, fine-grained, and multi-level manner, making it possible to carry out accounting intelligent information services based on privacy big data. In the next step, it is planned to verify the method on a larger scale of real accounting data, refine the technology, and evaluate the performance, so as to further enhance its industrial application value.
[0195] Figure 2 FIG. 1 is a schematic diagram of the structure of a cross-domain secure interaction system of a public service system based on a general large model according to an embodiment of the present invention. Figure 2 As shown, the system comprises:
[0196] The first unit is used to receive business query requests from accounting user terminals, use the request identification model to authenticate the source of the business query request, and compare it with the accounting personnel information database to determine whether the business query request has the corresponding accounting business authority; for business query requests that have passed identity authentication, use natural language processing technology to perform semantic analysis, extract accounting professional terms contained in the business query request, and accounting features of business scenarios; use a request legitimacy detection model based on a graph neural network, and use the knowledge graph of violation cases in the accounting field as a training sample to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations;
[0197] The second unit is used to input the accounting features into the pre-trained general model in the accounting field if there is no compliance risk, use the general model in the accounting field to perform multi-level representation learning on the accounting features, focus on the key information points of the business query request through the attention mechanism, and conduct deep semantic reasoning in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request;
[0198] The third unit is used to scan the sensitive information of the preliminary reply result, and use the named entity recognition model based on deep learning to identify various types of sensitive information in the preliminary reply result; generate a model according to the self-learning desensitization strategy, and take corresponding desensitization treatment for sensitive information of different types and different levels of importance. After the desensitization treatment, the privacy computing technology is used to evaluate the privacy leakage risk of the remaining reply content. If it is lower than the preset security threshold, it is determined that the preliminary reply result meets the data security criteria and is used as the final reply result.
[0199] According to a third aspect of the embodiments of the present invention,
[0200] An electronic device is provided, comprising:
[0201] processor;
[0202] a memory for storing processor-executable instructions;
[0203] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0204] A fourth aspect of the embodiments of the present invention is:
[0205] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0206] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A cross-domain secure interaction method for a public service system based on a general large model, characterized in that: include: Receive business query requests from accounting user terminals, use the request identification model to authenticate the source of the business query request, and compare it with the accounting personnel information database to determine whether the business query request has the corresponding accounting business authority; for business query requests that have passed identity authentication, use natural language processing technology to perform semantic analysis, extract accounting professional terms contained in the business query request, and accounting features of business scenarios; use a request legitimacy detection model based on a graph neural network, using the knowledge graph of violation cases in the accounting field as a training sample, to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations; If there is no compliance risk, the accounting features are input into the pre-trained general model in the accounting field, and the general model in the accounting field is used to perform multi-level representation learning on the accounting features. The attention mechanism is used to focus on the key information points of the business query request, and deep semantic reasoning is carried out in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request; Scan the preliminary response results for sensitive information, and use a named entity recognition model based on deep learning to identify various types of sensitive information in the preliminary response results; generate a model based on a self-learning desensitization strategy, and take corresponding desensitization treatments for sensitive information of different types and importance levels. After the desensitization treatment, use privacy computing technology to evaluate the privacy leakage risk of the remaining response content. If it is lower than the preset security threshold, it is determined that the preliminary response results meet the data security criteria and are used as the final response results; The accounting features are input into the pre-trained general model in the accounting field, and the general model in the accounting field is used to perform multi-level representation learning on the accounting features. The attention mechanism is used to focus on the key information points of the business query request, and deep semantic reasoning is carried out in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request, including: Construct a general accounting model for accounting business. Through pre-training with large-scale accounting corpus, the general accounting model can master the vocabulary, grammar and semantic features of accounting language, introduce accounting business characteristic training tasks, and strengthen the model's understanding and application ability of accounting knowledge. Use the pre-trained general accounting model to perform multi-layer Transformer encoding on query requests, dynamically adjust the interaction weights between words through the Self-Attention mechanism, and introduce Attention at the top layer of the network to focus on key words and sentences, forming a hierarchical query semantic representation that integrates the attention mechanism; Build a knowledge graph in the accounting field covering accounting standards and tax policies, map knowledge graph entities to the pre-trained language model embedding space, identify the starting nodes related to the query semantics, and expand semantically related entities and relationships from the starting nodes through knowledge graph reasoning; Identify entity combinations that can represent specific business scenarios in the entity set expanded by the knowledge graph, analyze the grammatical dependencies of entities in the query text to extract the corresponding business logic, and restore the contextual semantics by combining the entity category and attribute modification to generate a business logic context information network that matches the query request; The semantic features in the business logic context information network are integrated into the structured generation model input. After the semantic interaction of multiple layers of Transformer network, the Attention mechanism is used to focus on the key semantics. Combined with the language knowledge in the accounting field, the replication mechanism and Beam Search strategy are used to autoregressively decode and generate the reply text that matches the query. The semantic features in the business logic context information network are integrated into the structured generation model input. After the semantic interaction of the multi-layer Transformer network, the Attention mechanism is used to focus on the key semantics. Combined with the language knowledge in the accounting field, the replication mechanism and the Beam Search strategy are used to autoregressively decode and generate the response text that matches the query, including: Integrate the multi-granularity heterogeneous semantic fragments identified in the business logic context information network into a unified structured input, wherein the multi-granularity heterogeneous semantic fragments include keywords, phrases, sentences, entities and relations; Through the multi-layer Transformer encoder, the structured semantic input is hierarchically modeled, the distributed representation of semantic units of different granularity is learned, and the local semantics and global semantics are interactively modeled to form a query representation that integrates semantics at different levels. The decoder also uses the Transformer architecture to iteratively generate responses in an autoregressive manner. Each decoding step focuses on the semantic fragment in the encoded representation that is most relevant to the generated content, dynamically references key semantic information, and feeds back the decoding status to the encoder to guide subsequent encoding. When predicting decoded words, the model vocabulary and query text are combined to flexibly generate and reuse keywords and professional terms through a copy mechanism; The Beam Search decoding strategy is adopted to comprehensively consider the generation quality and efficiency to obtain a smooth and coherent response sequence; Based on the pre-trained language model, we use accounting domain corpus for targeted fine-tuning to learn domain text features. At the same time, we use external structured accounting knowledge as additional supervision signals to guide the model to generate text. Using a request legitimacy detection model based on a graph neural network and taking the knowledge graph of accounting violation cases as training samples, we can determine whether a business query request contains compliance risks such as financial fraud or illegal or irregular operations. The following are the risks: Based on the collected and sorted accounting violation cases, we construct a knowledge graph of accounting violation risk, which includes violation concept nodes and relationship edges. The relationship edges describe the correlation and evolution law between different violation behaviors. A heterogeneous graph neural network model is designed that integrates text feature learning and multi-relation reasoning. The heterogeneous graph neural network model can learn the semantic embedding representation of different types of violation concept nodes and dynamically adjust the message transmission and aggregation strategies between violation concept nodes based on the relationship type. The pre-trained heterogeneous graph neural network model is used to learn the node semantic embedding representation on the accounting violation risk knowledge graph, and the accounting professional terms and accounting features of the business scenarios extracted from the semantic analysis of the business query request are mapped to the semantic space aligned with the accounting violation risk knowledge graph. The similarity between the accounting professional terms, accounting features of the business scenarios and the violation concept nodes is calculated to determine whether the business query request triggers a known violation risk scenario. In the message transmission process of the heterogeneous graph neural network model, the accounting violation risk knowledge graph is used as a path to infer the violation risk associations implicit in the semantic environment of the business query request, and the overall compliance of the business query request is evaluated from the two dimensions of local risk and global risk to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations.
2. The method according to claim 1, characterized in that For business query requests that have passed identity authentication, natural language processing technology is used for semantic analysis to extract accounting terms and accounting features of business scenarios contained in the business query requests, including: Develop a financial term extraction and semantic disambiguation tool based on the conditional random field algorithm, introduce a financial dictionary to adapt the conditional random field algorithm to the domain, so that it can accurately identify professional terms in business query requests and determine the actual semantics of professional terms based on the context; Use the pre-trained Chinese dependency syntactic analysis model to perform syntactic dependency analysis on business query requests, reveal the inherent logical association of accounting business elements in the query text of business query requests, and provide structured clues for semantic reasoning; Construct a financial accounting structured knowledge graph for financial accounting business scenarios. When semantically parsing business query requests, link and match the concept nodes of the financial accounting structured knowledge graph in real time based on the syntactic dependency relationship corresponding to the business query request, so as to achieve semantic information fusion of query statements and domain knowledge. Comprehensively use natural language processing technology based on the bag-of-words model to extract key information from business query requests after semantic disambiguation, and automatically extract accounting professional terms contained in business query requests and accounting features of business scenarios.
3. The method according to claim 1, characterized in that Based on the self-learning desensitization strategy generation model, different types and different levels of importance of sensitive information are desensitized accordingly. After desensitization, privacy computing technology is used to evaluate the privacy leakage risks of the remaining reply content, including: Through transfer learning, the pre-trained language model is fine-tuned on large-scale accounting text data, so that it can adaptively learn the diverse characteristics of private information in the accounting context, and build a private information identifier for accounting scenarios, which can accurately detect various explicit or implicit private fragments from accounting texts; Collect desensitization examples of different types of financial privacy, build a privacy-desensitization strategy pairing dataset, and train a seq2seq self-learning desensitization strategy generation model on the dataset, which can automatically generate the optimal desensitization strategy for each identified privacy fragment based on the privacy type, importance, and contextual factors; The self-learning desensitization model is applied to the reply text of accounting business inquiries. For the identified direct privacy, strong desensitization processing such as encryption and random replacement is adopted, and for the indirect privacy, weak desensitization processing such as generalization and synonym replacement is adopted, so as to protect the privacy while maximally retaining the availability of the reply content. Apply homomorphic encryption privacy computing technology to the desensitized reply text, match it with the predefined privacy mode in the ciphertext domain to calculate the overall privacy leakage probability. If the probability exceeds the security threshold, further desensitization is performed iteratively on high-risk text fragments until the overall privacy risk is controlled within an acceptable range. The privacy leakage probability calculation is carried out through the federated learning method with multi-party participation, ensuring that the privacy calculation evaluation process itself will not cause additional privacy leakage; The output is a secure accounting business response text that has undergone dual privacy protection through self-learning desensitization and privacy computing. While retaining the original meaning well, sensitive privacy information is effectively shielded and the overall privacy leakage risk is reliably controlled.
4. A cross-domain secure interaction system for a public service system based on a general large model, used to implement the method described in any one of claims 1 to 3, characterized in that: include: The first unit is used to receive business query requests from accounting user terminals, use the request identification model to authenticate the source of the business query request, and compare it with the accounting personnel information database to determine whether the business query request has the corresponding accounting business authority; for business query requests that have passed identity authentication, use natural language processing technology to perform semantic analysis, extract accounting professional terms contained in the business query request, and accounting features of business scenarios; use a request legitimacy detection model based on a graph neural network, and use the knowledge graph of violation cases in the accounting field as a training sample to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations; Using a request legitimacy detection model based on a graph neural network and taking the knowledge graph of accounting violation cases as training samples, we can determine whether a business query request contains compliance risks such as financial fraud or illegal or irregular operations. The following are the risks: Based on the collected and sorted accounting violation cases, we construct a knowledge graph of accounting violation risk, which includes violation concept nodes and relationship edges. The relationship edges describe the correlation and evolution law between different violation behaviors. A heterogeneous graph neural network model is designed that integrates text feature learning and multi-relation reasoning. The heterogeneous graph neural network model can learn the semantic embedding representation of different types of violation concept nodes and dynamically adjust the message transmission and aggregation strategies between violation concept nodes based on the relationship type. The pre-trained heterogeneous graph neural network model is used to learn the node semantic embedding representation on the accounting violation risk knowledge graph, and the accounting professional terms and accounting features of the business scenarios extracted from the semantic analysis of the business query request are mapped to the semantic space aligned with the accounting violation risk knowledge graph. The similarity between the accounting professional terms, accounting features of the business scenarios and the violation concept nodes is calculated to determine whether the business query request triggers a known violation risk scenario. In the message transmission process of the heterogeneous graph neural network model, the accounting violation risk knowledge graph is used as the path to infer the violation risk association implicit in the semantic environment of the business query request, and the overall compliance of the business query request is evaluated from the two dimensions of local risk and global risk to determine whether the business query request has compliance risks of financial fraud and illegal and irregular operations; The second unit is used to input the accounting features into the pre-trained general model in the accounting field if there is no compliance risk, use the general model in the accounting field to perform multi-level representation learning on the accounting features, focus on the key information points of the business query request through the attention mechanism, and conduct deep semantic reasoning in combination with the knowledge graph in the accounting field to build a business logic relationship network that meets the business query request, generate multi-dimensional context information, and output the preliminary response results for the business query request, including: Construct a general accounting model for accounting business. Through pre-training with large-scale accounting corpus, the general accounting model can master the vocabulary, grammar and semantic features of accounting language, introduce accounting business characteristic training tasks, and strengthen the model's understanding and application ability of accounting knowledge. Use the pre-trained general accounting model to perform multi-layer Transformer encoding on query requests, dynamically adjust the interaction weights between words through the Self-Attention mechanism, and introduce Attention at the top layer of the network to focus on key words and sentences, forming a hierarchical query semantic representation that integrates the attention mechanism; Build a knowledge graph in the accounting field covering accounting standards and tax policies, map knowledge graph entities to the pre-trained language model embedding space, identify the starting nodes related to the query semantics, and expand semantically related entities and relationships from the starting nodes through knowledge graph reasoning; Identify entity combinations that can represent specific business scenarios in the entity set expanded by the knowledge graph, analyze the grammatical dependencies of entities in the query text to extract the corresponding business logic, and restore the contextual semantics by combining the entity category and attribute modification to generate a business logic context information network that matches the query request; The semantic features in the business logic context information network are integrated into the structured generation model input. After the semantic interaction of multiple layers of Transformer network, the Attention mechanism is used to focus on the key semantics. Combined with the language knowledge in the accounting field, the replication mechanism and Beam Search strategy are used to autoregressively decode and generate the reply text that matches the query. The semantic features in the business logic context information network are integrated into the structured generation model input. After the semantic interaction of the multi-layer Transformer network, the Attention mechanism is used to focus on the key semantics. Combined with the language knowledge in the accounting field, the replication mechanism and the Beam Search strategy are used to autoregressively decode and generate the response text that matches the query, including: Integrate the multi-granularity heterogeneous semantic fragments identified in the business logic context information network into a unified structured input, wherein the multi-granularity heterogeneous semantic fragments include keywords, phrases, sentences, entities and relations; Through the multi-layer Transformer encoder, the structured semantic input is hierarchically modeled, the distributed representation of semantic units of different granularity is learned, and the local semantics and global semantics are interactively modeled to form a query representation that integrates semantics at different levels. The decoder also uses the Transformer architecture to iteratively generate responses in an autoregressive manner. Each decoding step focuses on the semantic fragment in the encoded representation that is most relevant to the generated content, dynamically references key semantic information, and feeds back the decoding status to the encoder to guide subsequent encoding. When predicting decoded words, the model vocabulary and query text are combined to flexibly generate and reuse keywords and professional terms through a copy mechanism; The Beam Search decoding strategy is adopted to comprehensively consider the generation quality and efficiency to obtain a smooth and coherent response sequence; Based on the pre-trained language model, we use accounting domain corpus for targeted fine-tuning to learn domain text features. At the same time, we use external structured accounting knowledge as additional supervision signals to guide the model to generate text. The third unit is used to scan the sensitive information of the preliminary reply result, and use the named entity recognition model based on deep learning to identify various types of sensitive information in the preliminary reply result; generate a model according to the self-learning desensitization strategy, and take corresponding desensitization treatment for sensitive information of different types and different levels of importance. After the desensitization treatment, the privacy computing technology is used to evaluate the privacy leakage risk of the remaining reply content. If it is lower than the preset security threshold, it is determined that the preliminary reply result meets the data security criteria and is used as the final reply result.
5. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 3.
6. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 3 is implemented.
